Effects of a supplementary final year curriculum on students’ clinical reasoning skills as assessed by key-feature examination

Abstract

Background: The final year of medical education is considered crucial in making students ‘fit for purpose’. Studies have shown that many students leave medical school without having experienced sufficient preparation for their upcoming professional life.

Aim: The aim of this study was to examine the effectiveness of a supplementary internal medicine final year curriculum on clinical reasoning skills.

Method: Final year internal medicine students from two universities participated in the study which was based on a static-group design. The experimental group (n = 49) took part in a final year student curriculum with interactive case-based seminars and skills training sessions. The comparison group (n = 25) did not receive any additional training beyond working on the ward. Clinical reasoning skills were assessed using a key-feature pre-post test.

Results: Prior to their clinical rotation, the two groups did not differ in the key-feature examination (p < 0.924). The experimental group performed significantly better than the comparison group (p < 0.028) in the post-intervention key-feature examination.

Conclusions: Supplementary interactive case-based seminars and skills training sessions are effective and significantly improve the clinical reasoning skills of final year students in internal medicine. Further study is warranted and should look to examine the effectiveness of a final year student curriculum on other performance measures.

Introduction

The final year of medical education is considered to be especially important in making students ‘fit for purpose’ (Wass 2005). It aims to aid the transition from students’ medical college years to their future professional career and to produce competent and reflective physicians by ensuring appropriate learning outcomes. Competencies such as clinical skills, practical clinical procedures, patient investigation, communication, information management, patient management and clinical reasoning form the core of final year medical education (Shumway & Harden 2003). Nevertheless, studies have shown that many students leave medical school without having experienced sufficient preparation for their upcoming professional life (Clack 1994; Fox et al. 2000; Goldacre et al. 2003; Daelmans et al. 2004).

The acquisition of clinical reasoning skills primarily results from dealing with multiple patient examples over a period of time. This facilitates the availability of concepts and conceptual knowledge (Norman 2005). Previous studies exploring clinical reasoning skills have predominantly focused on the problem-based learning approach with its constructive, self-directed, collaborative and contextual learning environment (Dolmans et al. 2005; Dolmans & Schmidt 2006). Compared to traditional, lecture-based sessions and curricula, problem-based learning has been shown to positively affect diagnostic competencies (Schmidt et al. 1996) and clerkship performance ratings in undergraduate education (Richards et al. 1996; Whitfield et al. 2002) as well as clinical reasoning skills in postgraduate education (Doucet et al. 1998). For the acquisition of procedural skills, contextual learning is extremely important (Kneebone et al. 2002; Nikendei et al. 2005; Nikendei et al., 2007b). The effectiveness of skills curricula has been demonstrated using self-reports of skills performed on the ward (Remmen et al. 1999), written skills tests (Remmen et al. 2001) and objective structured clinical examinations in pre-post (Bradley & Bligh 1999) and comparison designs (Jünger et al. 2005).

No investigation of the effects of a supplementary final year curriculum on clinical reasoning skills is to be found in the literature. The aim of this study was therefore to examine the effectiveness of such a curriculum within the field of internal medicine on students’ performance in a key-feature examination based on a pre-post group comparison design. We hypothesised that (1) key-feature examination results at the beginning of the study would be comparable for both experimental and comparison groups, (2) both groups would show significant improvement in key-feature examination results during the final year, although (3) the experimental group would show significantly more improvement in the post-intervention examination.

Table

Practice points

The final year of medical education is especially important in aiding the transition from students’ medical college years to their future professional career and in producing competent and reflective physicians by ensuring appropriate learning outcomes. A supplementary final year student curriculum in internal medicine including patient-centred learning and skills-training sessions significantly improved clinical reasoning skills as measured by a key-feature examination. Further research is needed to investigate the effects of a final year student curriculum on other measures assessing the performance of final year students on the ward.

Methods

Sample

Two groups of final year medical students belonging to natural cohorts voluntarily participated in the study. The experimental group consisted of 49 final year students from the University of Heidelberg (16 male, 33 female; age: 26.1 ± 1.77 years) and the comparison group consisted of 25 final year students from the University of Tübingen (9 male, 17 female; age: 26.4 ± 1.96 years). Differences in sex and age were not statistically significant (sex: p < 0.445; age: p < 0.510). Both universities demonstrate very good and comparable results in the second national medical licensing examination following the final year of medical education (German NBE part II; (IMPP 2007)) and are almost identical in size, department structure, and number and type of patients treated. For both universities, the structure of final year medical education is state-defined. All final year students are required to complete a three-month clerkship in internal medicine. Those final year students who commenced their internal medicine final year term between March 2006 and August 2006 were included in the study with a participation rate of 100% at both faculties.

Study design and final year curriculum

The study was performed using a static-group design. Focus groups were conducted with final year students from both universities (n = 65) (Schrauth et al. 2009) and medical representatives (n = 7) of the different internal medicine departments of the University of Heidelberg in order to explore the perceived needs of final year students. Learning objectives and educational strategies were defined by the medical representatives of the internal medicine departments, as proposed by Kern et al. (1998). The experimental group took part in an introduction week comprising four 2-hour sessions which focused on familiarising final year students with important working materials and procedures on the ward (Nikendei et al. 2006). The introduction week was followed by a curriculum consisting of four weekly sessions (Monday to Thursday), each lasting 1 hour. All four sessions in any given week were related to a specific topic (Table 1) and adopted an interactive case-based learning approach (Nikendei et al. 2007a, b; Srinivasan et al. 2007). Tuesday sessions encompassed skills training and special training sessions such as ward round training (Nikendei et al. 2007a), communication training (Kraus et al. 2006) and teaching rounds on the ward. Wednesday and Thursday sessions aimed to promote electrocardiogram interpretation and pharmacological skills based on patient cases. Both experimental and comparison groups took part in a key-feature examination before and after their 3-month clerkship in internal medicine.

Table 1.  Final year student curriculum at the University of Heidelberg

Topic of the week	Monday: Interactive seminar	Tuesday: Skills training session	Wednesday: Pharmacology seminar	Thursday: ECG seminar
Week 1: Introduction	Introduction week: ‘Essentials from admission to discharge’ (Nikendei et al. 2006)
Week 2: Ward round/frequently occurring problems	Basic principles: ward round	Ward round training (Nikendei et al. 2007a)	Sleep medication	Side-effects psychotropic drugs I
Week 3: Infections/antibiotics	Infections/focus search	Prescription and application of antibiotic drips	Antibiotics	Blocks
Week 4: Emergency medicine	Basic principles: Resuscitation	Mega-code training	Emergency medication	Cardiac arrhythmia
Week 5: Hypertension	Hypertensive derailment	I.V. application of antihypertensive drugs	Antihypertensive drugs	Cardiac hypertrophy
Week 6: Communication/depression and anxiety	Basic principles: Communication	Communication training (Kraus et al. 2006)	Psychotropic drugs	Side-effects psychotropic drugs II
Week 7: Kidney failure/dialysis	Acute kidney failure	Central venous catheter application	Dosing in chronic kidney failure	Electrolytes
Week 8: –	–	–	–	–

Notes: The curriculum commences with an introduction week and lasts for the duration of students’ 3-month clerkship. Examples of weekly topics and learning objectives for teaching sessions in weeks 1–7 for the experimental group of 49 final year students from the University of Heidelberg are presented below.

Design of key-feature assessment

The range of problems sampled by each key-feature assessment was selected based on the learning objectives for the internal medicine final year rotation with general internal medicine as content domain. The Swiss one-dimensional blueprint for internal medicine was used (Bloch & Burgi 2002) to weight sub-domains in internal medicine in order to ensure equivalent learning objectives for both experimental and comparison groups. The main body of the key-feature examination used in this study was derived and adapted from Fischer et al. (2005), whose key-feature items have already proven to be a reliable assessment tool in undergraduate internal medicine education. The development and construction of further key-feature problems was based on the steps recommended by Page et al. (1995). The final version of the key-feature examination included 16 patient cases with 75 questions written in ‘short-answer write-in’ formats and ‘short menu’ formats (Table 2). Pre- and post-intervention key-feature examinations were identical. Each key-feature question was scored using a dichotomous scoring system. Accordingly, participants were able to attain assessment scores between 0 and 75. A paper and pencil format was used and 90 minutes were allowed for the examination.

Table 2.  Topics of key-feature problems used in the examination and their distribution over defined sub-domains of internal medicine (%)

Sub-domain	Key-feature examination (%)	Swiss blueprint (%)
Rheumatology/immunology	6.7	13.0
General internal medicine/Psychosomatic medicine/alcoholism	16.0	20.0
Cardiology/angiology/hypertension	20.0	15.0
Endocrinology/nephrology	10.7	11.0
Gastroenterology	13.3	10.0
Oncology/infectiology	9.3	15.0
Pneumology/emergency medicine	24.0	16.0

Note: Swiss catalogue of learning objectives (Bloch & Burgi 2002) as reference for the weighting of internal medicine sub-domains.

Statistical analysis

In order to test for group differences in sex and age, a chi-square test and a Student's t-test for independent groups were conducted, respectively. Differences in key-feature assessment results were analysed using an ANOVA for attained scores with the between factor ‘Group’ (experimental vs. comparison) and the within factor ‘Time of Assessment’ (pre vs. post). Main effects were further investigated using post hoc tests. P values <0.050 (one-tailed) were considered to be statistically significant. The data are presented as mean ±SD. Cronbach's alpha was calculated as a test of key-feature examination reliability.

Results

Key-feature examination performance

Table 3 presents the key-feature examination scores attained by the experimental and comparison groups pre- and post-intervention. An ANOVA was conducted for the attained assessment scores with the between factor ‘Group’ (experimental vs. comparison) and the within factor ‘Time of Assessment’ (pre vs. post). No significant effect of ‘Group’ was found (F(1,72) = 1.63; p < 0.206). A significant effect of ‘Time of Assessment’ (F(1,72) = 75.97; p < 0.001) was due to higher assessment scores in the post-evaluation as compared to the pre-evaluation. Furthermore, a significant ‘Group’ × ‘Time of Assessment’ interaction (F(1.72) = 10.21; p < 0.002) was observed (Figure 1). Post hoc tests revealed a higher assessment score in the experimental group as compared to the comparison group in the post-intervention evaluation (p < 0.028) whereas the two groups did not differ prior to their clinical rotation (p < 0.924; Table 3).

Figure 1. Assessment scores for experimental (EG; n = 49) and comparison groups (CG; n = 25) pre- and post-intervention (0–75 points); means and 95% confidence interval. *Post hoc test revealed higher assessment sores in the experimental as compared to the comparison group in the post-intervention evaluation (p < 0.028), although the two groups showed no difference in pre-intervention assessment scores (p < 0.924).

Table 3.  Results of key-feature examination for experimental (n = 49) and comparison group (n = 25) pre- and post-intervention (0–75 points)

	Experimental group		Comparison group
Time of Assessment	M	SD	M	SD	p
Pre-curriculum evaluation	36.84	±6.91	36.64	±7.28	n.s.
Post-curriculum evaluation	44.00	±7.72	39.96	±6.43	<0.028

Notes: M = mean, SD = standard deviation, p = p-value of post hoc test results.

Reliability of key-feature examination

Reliability was calculated using Cronbach's alpha. In the pre-intervention assessment, Cronbach's alpha was 0.71 and in the post-intervention assessment 0.75.

Discussion

This study investigated the effects of a supplementary curriculum on clinical reasoning skills in internal medicine for final year medical students. Although the number of participants in our study was limited, the students were well-matched with respect to age, sex and pre-test outcome measures. Performance was assessed using a pre-post intervention key-feature examination. Both experimental and comparison groups showed improvement in clinical reasoning after completing their internal medicine rotation. However, the experimental group attained significantly higher examination scores as compared to comparison students who conducted their clinical rotation without the additional final year curriculum, despite identical assessment scores prior to rotation.

Results show that both groups profited from final year education in terms of clinical reasoning skills. However, structured, case-based teaching and skills training sessions during final year education resulted in incremental improvement in clinical reasoning outcomes as measured by the key features examination. This result was expected based on the findings of various studies indicating that the systematic supervision of students during clinical rotations and feedback in the context of workplace learning are infrequent (Remmen et al. 2000; Van der Vleuten et al. 2000; Van Der Hem-Stokroos et al. 2001; Daelmans et al. 2004; Howley & Wilson 2004). A structured final year student curriculum comprising interactive case-based seminars and skills training sessions would seem to at least guarantee a minimum of standardisation in final year medical education, to address important learning objectives, and to also foster clinical reasoning skills. We speculate that more sustained facilitation of clinical reasoning skills might reinforce student motivation and engagement on the ward and in turn increase the benefits of workplace learning gained by final year students. This assumption is supported by the observation that participation in problem-based learning curricula leads to improved ratings of student clerkship performance in undergraduate training (Richards et al. 1996; Whitfield et al. 2002).

We used the key-feature examination to assess the acquisition of reasoning skills (Bordage et al. 1995; Page & Bordage 1995; Page et al. 1995). Key-features are defined as critical steps in the resolution of a clinical problem and focus upon the step in which examinees are most likely to make errors in resolving the problem. They capture difficult aspects of practical problem-identification and management (Page & Bordage 1995). Key-feature examinations assess competencies at the ‘knows-how’ level according to Miller's learning pyramid (Miller 1990). They have proved to be a reliable and valid approach in assessing clinical reasoning skills (Bordage et al. 1995; Page & Bordage 1995; Fischer et al. 2005) and to represent a feasible tool in evaluating internal medicine clerkships (Hatala & Norman 2002). In this study, we were able to achieve reliabilities of 0.71 and 0.75 (Cronbach's alpha). This is close to the recommended examination reliability of 0.8 for summative assessment tools (Page & Bordage 1995). It should be kept in mind that although key-feature examinations represent a feasible tool for the assessment of students in internal medicine clerkships, they have been shown to only modestly correlate with other measures of knowledge and clinical performance (Hatala & Norman 2002). Hence, it remains unclear whether a statistically significant increase in key-feature test scores is related to a significant increase in clinical performance. Studies are therefore required that address the effects of a final year curriculum such as that applied in this study using performance measures such as the MINI-CEX (clinical examination exercise; Norcini et al. 1995, 2003) and OSCE (objective structured clinical examination; Harden & Gleeson 1979; Wass et al. 2001).

Although the experimental and comparison groups performed comparably well in the pre-intervention, some inherent differences might exist between the two groups, given that these came from different medical schools. A further potential confoundation is to be seen in the fact that a specific and structured educational intervention was compared with the non-specific and unstructured clinical education provided on the ward. This difference might be related to a greater level of attention having been paid to the experimental group. Nevertheless, the final year student curriculum presented here has since been integrated into the standard education programme offered to final year students at our school. Given that it has proven to be both practicable and on account of its generalisability, we hope to encourage a transfer of the curriculum to other institutions. Follow-up studies are required which assess the confidence and performance of final year students who have participated in such a curriculum programme upon becoming newly qualified doctors.

Conclusions

Supplementary, interactive, case-based seminars and skills training sessions are effective and significantly improve the clinical reasoning skills of final year students in the field of internal medicine as measured by key-feature written examinations. Further research is needed in order to confirm the effectiveness of such a curriculum using other performance measures.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

C. Nikendei

C. NIKENDEI, MD, the University of Heidelberg Medical Hospital, is responsible for skills-lab training, education of final year students, and the PAL programme at the Medical Hospital.

S. Mennin

S. MENNIN is a Professor Emeritus, Department of Cell Biology & Physiology Assistant Dean Emeritus, Educational Development and Research University of New Mexico School of Medicine.

P. Weyrich

P. WEYRICH, MD, the University of Tübingen Medical Hospital, is responsible for the medical education of 3–5th year medical students in Internal Medicine and the skills-lab training programme.

B. Kraus

B. KRAUS, MD, the University of Heidelberg Medical Hospital, is responsible for the medical education of final year students.

S. Zipfel

S. ZIPFEL, MD, is the professor and chairman, University of Tübingen Medical Hospital, Department of Psychosomatic Medicine and Psychotherapy.

M. Schrauth

M. SCHRAUTH, MD, the University of Tübingen Medical Hospital, is responsible for the standardized patient programme at the Faculty of Medicine and for the medical education of final year students.

J. Jünger

J. JÜNGER, MD, the University of Heidelberg Medical Hospital, is responsible for the medical education programme at the Medical Hospital.