Use of case-based exams as an instructional teaching tool to teach clinical reasoning

Abstract

Background: It is a very well-known fact that examinations drive learning to a great extent. The examination program is actually the ‘hidden curriculum’ for the students. In order to improve teaching and learning one option is to strategically use of exams.

Aims: This report of the strategic use of an innovative assessment tool in clinical problem solving domain, presents the design, format, content, students’ results and evaluation of one year test results of instructive case-based exams for 6th year medical students.

Method: Using a hybrid form of the OSCE, PMP and KFE formats, we developed a case-based stationary exam. Students were treated as advanced beginners in medical career and forced to an inquiry to use their clinical knowledge in the cases. Case discussions and question-answer sessions followed the exams. Six exams were held in 2000–2001 and 382 students participated in the study. One or two problems were used for each exam and the mean duration was 27 minutes for 7–11 stations. 17–19 observers contributed to each exam. Exams were evaluated by questionnaire based feedbacks of the students and oral feedbacks of the staff members.

Results: The exams were well received and rated ‘fair’ by the students and the format was found highly ‘relevant for learning’ while the content was ‘instructive’ and ‘not difficult’. The total non-satisfactory performance rate was 2.36%. Students asked to take a similar test weekly. Although it was labor intensive, staff members appreciated the collaborative working process.

Conclusions: Instructive case-based exams and the following case discussions seemed a high potential and motivating teaching tool in the clinical problem solving domain for 6th year students.

Introduction

One of the most important domains in clinical competence is the ability to think critically about diagnosis and management, which is usually referred to as ‘clinical problem solving’ (Schuwirth. 1998; Boshuizen & Schmidt. 2000), and its importance is recognized (GMC 1993). It is an ability that requires “reasoning”, which involves a combination of thinking and decision making processes to judge the best course of action to take in the clinical context. The process itself is an intellectual activity which requires obtaining information, critical thinking and integration with prior knowledge. All the activities in the process aim to reduce uncertainity of the clinical condition, and synthesis is the main feature (Rimoldi & Raimondo 1998).

Clinical problem solving skills differ between novices and experts. Due to the expert's well organized knowledge structure (or semantic network) and great deal of practice, the higher the level of expertise, the faster the problem solving and the more efficacious the results. The number and connections of ‘Illness Scripts’ (Charlin. 2002) in memory, based on previous practice, determine the level of expertise in problem solving. In this context, ‘scripts’ and ‘pattern recognition’ concepts are relatively new developments for the explanation of the cognitive process of clinical problem solving as a content specific ability. Carefully looking up a patient's initial information is very important in recognizing patterns and increased expertise, associated with the use of less information, speed and the selection of data, reduces uncertainty in the process (Derry. 1990; Schmidt et al. 1990; Vleuten & Newble. 1995; Custers & Boshuizen. 2002).

Table

Practice points
•The examinations drive learning to a great extent and the examination program is actually the ‘hidden curriculum’ for the students.
•As an experimental way of teaching clinical problem solving, instructive case-based assessment at short intervals and following case discussions is a potential learning tool for clerkships.

In other words, meaningful construction of content specific knowledge and matching the problem with previous experience relevant to the current case are the key points of clinical problem solving ability, and have implications in teaching and learning in medicine using a heuristic approach.

Meanwhile, it is a well-known fact that examinations drive learning to a great extent. The examination program is actually the hidden curriculum for the students. In order to improve teaching and learning, the strategic use of exams is very important (Van der Vleuten et al. 2001).

With this innovative use of case-based exams we aimed to help students to improve their case experience through facing them to the sliced scripts and asking to use their clinical problem solving ability while providing written answers and feedbacks during the exams. In the next part of this article, we aim to explain how we use case-based exams for teaching clinical problem solving, and the development process of the exams, their format, content, student results and feedback of the first six examinations.

Method

At the Ege University Faculty of Medicine, the medical doctor program lasts 6 years. The last three years are called ‘clinical years’ and clinical teaching methods vary among departments. In the past, no attention was generally given to students’ clinical problem solving ability.

We developed case-based stationary exams. Clinical problems selected based on the following three criteria: firstly, the case should be one of the most important health priorities of the community, secondly, it should have handling capability by the general practitioner in a primary healthcare setting, and thirdly, it should meet the integrated educational objectives of the division.

Later exam cases were created which each representing a priority clinical problem. The best clinical approach model for each case was developed by the discussion of 2–5 faculty members. The procedure was, (1) to select one clinical presentation of one patient and decide the key features as well as the initial information (age, sex, occupation, symptoms), (2) to list the tentative diagnosis options according to the initial information, (3) to address the diagnostic differentiation procedures, (4) to specify the required medication and patient education, (5) to state the prognosis and patient follow up strategy.

Each case was divided into the clinical sub problems (student tasks) such as attaining correct diagnosis, deciding correct treatment. Next, the cases were divided into key decisions and each element was structured as a single task in the stations. All the relevant patient material, such as X-rays, laboratory results and hospital forms was provided. In the questions, the team established all decision options (suitable/unsuitable for given situation) or the correct answer (such as a short essay). They provided evidence based explanations (why the option was suitable or not). Stations were weighted according to the complexity of the tasks, based on 100 points. For instance, reaching the correct tentative diagnosis option was 20 while writing the prescription was 25.

At some of the task stations, a list of diagnosis or treatment options was presented which were plausible given the initial situation. The correctness or incorrectness of the options was weighted and explanations were provided on the back of the option card. According to the degree of correctness or incorrectness of the options, points were written on the observer checklist. Except at some stations, it was not suggested how many of the proposed decisions should be made. In contrast to the patient management problems assessment procedure, the option(s) selected did not dictate the way the exam would proceed. Because the tasks were linked to each other and accurate knowledge or performance was required to perform correctly at the next station, corrective feedback was provided in the format of the best answer in the initial presentation of the next task (e.g. the most plausible tentative diagnosis at the previous station was diabetes mellitus for the following reasons … . At this station, please look carefully at the patient x-ray and write further laboratory orders in the given forms). Gradually more information was revealed, which might require students to reorient their subsequent decisions. Following is the main structure of each case in the exam.

1st station

• Initial information about the patient and key features of clinical presentation (age, profession, sex, and history as independent variables and signs, symptoms and their durations as dependents). The student has to perceive and appreciate the meaning of the key features.

• Questions on the cards (what the possible tentative diagnoses are, what to do at the next step, what else she/he wants to know). The student has to identify the possible pathological process that is occurring, know how to differentiate one pathological process from another, and know from epidemiology the most likely causes of a particular pathological process.

• Options and/or place for writing the answer.

2nd station

• The best answer of the first task and further information on the card or patient record (correct tentative diagnosis, what is necessary for further clinical reasoning, laboratory test results, X-rays, etc.). The student has to evaluate all pieces of information available and decide on the likely cause and course of illness scripts and select an option or write.

3rd station

• The correct diagnosis and its explanations, further questions. The student has to select or write the best treatment and follow up strategy, indicate additional key features needed for solving the patient's problem and preventing further complications and write the correct prescription.

Checklists were used by observers in the exams. Observers only marked the students’ selections and monitored them through the stations. Every observer was trained how to use the checklist. In the checklist each option/decision has previously attained scores. If short written answers were required at a station (e.g. treatment), an answer key sheet was used.

We scheduled time to meet the students immediately after each exam. All the cases, answers and scores of each option/decions were presented on overheads and the correct problem solving process was discussed. In the session students have calculated their own exam scores. The post exam meeting finished with a question-answer session.

We included measures to assess students satisfaction (five point Likert Scale questionnaires (5 = completely agree, 1 = completely disagree) in the format of six items, and written comments) and faculty perceptions (oral feedback) for educational effects and sutainability of the exams.

Results

In the educational year of 2000–2001 we constructed 6 exams. Table 1 shows the clinical problems, number of stations, duration and number of observers for each one. Different departments played a part in exam preparation and provided between 17 and 19 observers for the exams. The mean duration was 27 minutes and the number of stations was a minimum of 7 and a maximum of 11.

Table 1.  Instructive examinations in 2000–2001

Exam number	Number of problems	Clinical problem(s)	Number of stations	Duration (Min.)	Number of observers
1	2	Irritable colon syndrome, diabetes mellitus	11	22	19
2	2	Allergic rhinitis- asthma, hypertension	8	24	18
3	2	Stomach pain, colon neoplasm	7	35	17
4	1	Acute rheumatic fever	7	28	19
5	2	Leucemia, hypertension	6	24	17
6	2	Spondylo arthropathy, diabetes mellitus	7	28	16
Total		13	29	161	106

382 students were assessed and 9 of them were found unsatisfactory (2.36 %), shown in Table 2. In the format The achievement and previous results of the students with unsatisfactory results were reviewed and it was found that a low level of achievement was not correlated with their past performance. As shown in Table 3, all the exams were well received by the students. The fourth exam was rated highest. The students rated all the exams ‘fair to everyone’.

Table 2.  Students’ scores

		Satisfactory		Non satisfactory
Exam number	Number of participating students	Number	(%)	Number	(%)	Mean	SD	(min.–max.)
1	44	44	100.00	–	–	65.4	11.8	(53–91)
2	46	46	100.00	–	–	72.7	10.6	(51–92)
3	84	77	91.67	7	8.33	68.8	14.6	(21–99)
4	73	73	100.00	–	–	86.1	7.5	(66–99)
5	63	61	87.06	2	2.94	76.7	13.8	(31–99)
6	72	72	100.00	–	–	80.4	6.8	(67–98)
Total	382	373	97.64	9	2.36

Table 3.  The students’ ratings

Items	N	Coverage %*	Mean	SD	F**	Sig.	Cause of the difference***
I liked the exam.	342	89.5	3.87	1.18	3.331	0.006	4th exam (4.39±0.58)
I found the exam fair to everyone.	337	88.2	3.95	1.14	1.592	0.162	–
There was not enough time.	324	84.8	1.91	1.31	9.127	0.000	1st exam (2.98±1.46)
The pre-exam explanations were not sufficient.	330	86.4	2.31	1.37	3.839	0.002	1st exam (2.78±1.34)
I could not adapt to the exam.	331	86.6	2.34	1.39	3.312	0.006	1st exam (3.00±1.35)
The exam was too difficult.	327	85.6	1.78	1.01	3.781	0.002	4th exam (1.48±0.75)

*The number of students assessed in 2000–2001 was 382. **One Way Anova ***The Bonferroni post hoc tests.

Discussion

Patient Management Problems (PMPs) are one way of testing clinical problem solving ability. PMPs were introduced to the assessment procedures of many medical schools and licensing bodies, particularly in North America (McGuire & Babbott. 1967). In PMPs, clinical reasoning was typically measured by asking students to respond to problems that could be presented in a standardized format. The main assumption of this attempt was that clinical problem solving is a generic ability.

By the end of 1970s, the Objective Structured Clinical Examination (OSCE) was introduced, with the aim of providing a better solution to the drawbacks of traditional clinical examinations. The OSCE has widely impacted the clinical assessment programs of many medical schools during the last twenty five years (Harden & Gleeson. 1979; Mennin & Kalishman. 1998; Fowel & Bligh. 2001). The OSCE format integrated the assessment of clinical problem solving ability with clinical competence in a structured fashion.

At the same time, more efficient assessment formats have been developed to replace the PMP, such as “key features”, based on the assertion that successful handling of a clinical case depends on a few critical elements. When testing students on problem-solving skills, one should focus on such key pieces of patient's information. Authors call this type of assessment a Key Features Examination (KFE) (Bordage & Page,. 1987; Bordage et al. 1995; Hatala & Norman,. 2002). KFE enabled the use of more cases and assessment of much more content with limited time and cost. This approach fitted the content specificity attribute of problem solving relatively well.

In order to utilize the above advantages of the OSCE, PMP and KFE, we used a hybrid method for our instructive exams.

Although option given assessment tasks are open to the “cueing” effect (Schuwirth. 1998) and must be avoided in summative exams, we tried to use this cueing effect for enhancing learning and reinforcement in exams and as motivation for further learning.

Giving the best possible answer of the previous station gave us a non-biased analysis of each subsequent task and an analysis of the weakness of the student in each particular domain. (e.g. tentative diagnosis and/or treatment). These analytic findings were converted into implications of instructional design.

The driving concepts of the post exam meetings were reinforcement, corrective feedback, elaboration, and metacognition. We observed a mutual agreement of the faculty staff members’ free comments, that the post exam meetings turned into highly motivating case-based teaching sessions. Students were challenged to reflect on their own performance, based on explained expert performance, which allowed them to understand what they knew and what they did not know (Bruning et al. 1999).

Both of the two positive items in the student evaluation questionnaire ranked above 3.00 points for each exam. These findings showed us that format was well received by students. According to the written comments, the fairness of any single clinical exam was the most important issue for our students. The format seems to have the potential to solve in large measure the problem of ‘fairness’, which they are still suffering in summative oral exams. Usage of this format for the first time in the curriculum showed both students and faculty members that ‘it is possible to be fair in exams'.

Pre-exam explanations were given by a combination of a ‘written exam manual’ and ‘oral questioning’ except in the first exam. Explanation in the first exam was given only in writing, and we agreed that students did not pay much attention to the written material because they were being assessed. We decided to support the written explanations with questioning in the 10 minutes before the exam. This worked, which was reflected in students’ ratings on the explanation and adaptation items.

Generally, when using a new exam format, adaptation problems tend to be the main cause of possible ‘too difficult’ ratings, instead of ‘real difficulty of content’. In our innovation, the results were the opposite. Students found the exams very easy, as can be seen in the students’ scores. We interpret this finding to mean that although there was no specific attention to teaching clinical problem solving, students’ have been achieved some degree of expertise for internal medicine cases.

Students enjoyed all the components of the exams and asked to repeat a similar exam as a drill every Friday. We believe instructive exams will enhance learning in the clinical phases of the medical curriculum and this format helped them to link concepts in their knowledge structure with focusing on enabling conditions, faults and consequences in sequences of medical problem solving (Boshuizen & Schmidt. 2000), which they had never been asked or shown how to use in any clinical examination they had taken before.

The process and results enhanced our discussion of clinical learning, teaching and the use of assessment methods. We had the chance to share our experience with other faculties (Istanbul University Medical Faculty and Ege University Faculty of Dentistry) as well as with our faculty members (in the clinical assessment faculty development program). At the end of 2000–2001 we started to construct computer based exams.

Conclusions

Instructive case-based assessment at short intervals (e.g. weekly) is a potential learning tool for clerkships. Post exam meetings are great challenges for both students and faculty members. Since 2002, we have moved the exam to a computer medium and now the Department of Internal Medicine is benefiting from these exams. We will continue to use and study instructive exams as a teaching tool and share our experience with other departments.

Additional information

Notes on contributors

Halil Ibrahim Durak

HALIL IBRAHIM DURAK provided the original idea for this study and wrote the manuscript.

Suleyman Ayhan Caliskan

SULEYMAN AYHAN CALISKAN contributed to exam development and analysis processes.

Serhat Bor

SERHAT BOR is clerkship director of the Internal Medicine Department and the content developer.

Cees Van Der Vleuten

CEPS VAN DER VLEUTEN provided support, guidance and amendments to the manuscript.