A systematic review of the relationship between patient mix and learning in work-based clinical settings. A BEME systematic review: BEME Guide No. 24

Figures & data

Table

Practice points

There is a large semantic overlap between the terms ‘clinical experience’, ‘clinical exposure’, ‘clinical encounters’ or ‘patient encounters’ and ‘case mix’ or ‘patient mix’. ‘Patient mix’ is mostly presented without definition. Operationalisations of patient mix were fairly different, allowing very few inferences from the studies. This is particularly interesting in the light of the emphasis adequate patient mix has gained in the diverse accreditation standards of several countries. Educational research would benefit from a standardised approach in patient mix descriptions; volume can always be measured, but diversity should be explicated in relation to the learning outcome. A model for patient mix measurement is suggested. We found indications that patient mix is related to self-reported learning outcome, most evidently the experienced quality of the learning programme. A relationship between patient mix and formal assessment has not been demonstrated. Supervision quality and learning style might mediate the relationship between patient mix and learning. Future studies should aim at addressing which parts of patient mix contribute to learning and which parts do not. It would be interesting to see what the effect of tailoring the patient mix to the specific learning goals and needs of individual students would yield.

Figure 1. Two-dimensional approach of patient mix. Consider a training programme in a large group of patients with the same diagnoses (A). Because there is no diversity, this cannot be considered to be a ‘patient mix’, but merely the training of a single skill or restricted clinical problem. Situation C expresses a situation in which a large diversity of skills and/or symptoms and diagnoses are theoretically possible, but very few or even no patients are present. This can be regarded as a patient mix, but an extremely meagre one – simply because there are no or very few patients. The line labelled ‘D’ expresses (an arbitrary) ‘cutoff point’ where the diversity is rich enough to start calling the population ‘patient mix’. All points labelled B are considered to be a patient mix.

List 1 Inclusion and exclusion criteria

Inclusion criteria

• Empirical, educational studies with actual patient exposure (no simulations), reporting on quantitative data

• Study population: studies conducted with medical students/trainees at any level of the formal medical training/career

• Patient mix, clinical encounters, or clinical experience in workplace-based learning had to be described. Patient mix had to be described in some detail, thereby addressing the volume as well as the diversity. Studies on the exposure to a restricted clinical problem or skill were excluded. Medical subspecialties were not excluded beforehand, as long as the patient mix was diverse

• Learning outcome measures had to be described by self-reported measures, assessment by trainers, preceptors, or others, or by objective structured clinical examinations (OSCEs) either with real or with standardised patients, multiple choice or other written exercises

• The relationship between patient mix and learning had to be quantified by statistical analysis

• Studies in all languages were included

Exclusion criteria

• Studies on qualitative research

• Dental and veterinary curricula, any paramedical curricula, nursing curricula, physician assistant curricula, nurse practitioner curricula and dietetic curricula

• Theoretical medical curricula (not work-based)

• Complementary/alternative medicine

Table 1  Patient mix and learning outcome instruments and descriptions

Instruments used for measuring patient mix

Questionnaire

Hand written logbook

Reference cards

Electronic logbook

Curriculum description

Patient mix operationalisations

Volumes of patient encounters

‘Top 10’

Encounters per organ systems specified

Encounters per specialism specified

Encounters per predefined skills

Content of the course/rotation/apprenticeship

Encounters per diagnosis

Learning outcomes

1. Self-assessment

Questionnaire/otherwise

Self-efficacy

Self-perceived knowledge

Self-perceived clinical confidence

Self-perceived competence

2. Formal assessment:

Theoretical examination

Multiple-choice exam

Other written exams

Practical examination

OSCE/other simulation(s)

Bedside or other patient evaluation

Presentation

Learning outcomes content

Theoretical knowledge (specifiy)

Specific technical skills (id)

Clinical competence in practice (id)

Table 2  Summary of the included studies

Author, year of publication	Country	Design	Time span	Speciality	Number of participants analysed and educational level participants	PM instrument	PM description (volume/diversity)	Learning instrument (self-reported measure is specified, Cronbach's alpha if reported)	Relation patient mix and learning	MERSQI	Highest Kirkpatrick level
Ahmed &  Hughes 1999	UK	Single group, post test	Seven weeks	Paediatrics	Two-hundred and twenty-six students	Questionnaire	Change in exposure from year 1 to year 2 in 42 conditions and 20 skills	MCQ^† and other written exam (0.67–0.71) General and clinical competence assessment	Clinical experience did not relate to MCQ/written exam score	13	3
Boots et al.  2008	Australia	Pre-post test design with two groups	10 weeks	Internal medicine	Two-hundred and twenty students and 174 interns	Questionnaire	Encounters (ordinal scale) of 15 specified skills	Questionnaire and self-confidence	There may be a relation between skill exercise and confidence	9.5	2
Chatenay et al.  1996	Canada	Single cohort randomised to attend one out of four training sites. Post test only	10 weeks	Surgery	One-hundred and nine students	Hand written logbook	Pat volume of elective/ER admissions, operations scrubs, outpatient clinics and procedures.	MCQ^† (0.67), OSCE* (0.48), and clinical performance	Skills were enhanced by increased volume of some, but not all, clinical experience. Complex relation between feedback and OSCE* performance. Quality of feedback seems to mediate this relationship	14	3
Dolmans et al.  2002	The Netherlands (NL)	Observational cohort study	Three to 12 weeks per rotation	Eight disciplines	One thousand two-hundred and eight residents	Questionnaire	One variable composed of three questionnaire items (sufficient -patients, -diagnostic variety and -patients independently dealt with)	Self-reported effectiveness questionnaire	Self-perceived effectiveness depends on patient mix and supervision. Supervision more strongly influences effectiveness when patient mix is limited	10	1
Duke et al.  2011	Canada	Non equivalent control group, pre-post test	Four weeks	Family medicine	Seventy-nine residents	(Unspecified) logbook	Percentage of students who attended clinics of 18 subspecialty Number of outpatient clinics attended	MCQ^† knowledge test Self-confidence questionnaire Oral and written feedback assessment	No difference between ambulatory and inpatient sites	12.5	2
Fung et al.  2007	USA	Retrospective single group, post test	Not specified (three clerkships)	Inpatient internal, ambulatory and family medicine	One-hundred and sixty-six third-year students	Electronic (PDA**) logbook	Number of patients in six diagnostic categories	OSCE* (0.34–0.65)	No relation between patient exposure and OSCE* score	12	3
Greenberg &  Getson 1999	USA	Single group, post test	Eight weeks	Paediatrics	One hundred and eighteen students	Hand written logbook	Number of patients in four diagnostic domains	Clinical performance, case presentation and NBME^‡	No relation between volume and exam score	11.5	2
Gruppen et al.  1993	USA	Single group, pre-post test	One month	Internal medicine	Forty-three third-year students	Hand written logbook	– Number of patients, – Percentage of students who encountered 20 diagnostic categories –Top 19	Written exam	No correlation between the students levels of experience and knowledge	12	2
Sorensen Hoifoidt 2004	Norway	Cross-sectional	Four months	Psychiatry	Eighty-five preregistration house officers	Questionnaire	Number of subjects having experience in 12 psychiatric skills	Questionnaire, subjective learning benefit (0.84)	Subjective learning benefit was related to amount of experience, competence and formal teaching programme. Supervision and previous experience had no impact on subjective learning	11.5	3
Jacobson et al.  1998	USA	Observational cohort study	Twelve weeks	Internal medicine	Forty-three students	Hand written logbook	–Top 10 –Percentage of encounters of six diagnostic categories and one skill	Categorisation of self-reported learning points	In and outpatient encounters differed. Learning differences between in- and outpatient apply to pathophysiology, evaluation/work-up and patient education/counselling.	8.5	2
Jolly et al.  1996	UK	Single group, post test	4  +  16 weeks	Pathology, then medicine and surgery	One-hundred and fifty-two clinical students	Questionnaire	Questionnaire responses handling patient volume and skills	OSCE* (0.69)	No relation between clinical experience and educational outcome	13.5	3
Lampe et al.  2008	USA	Non-randomised trial	Six months	Emergency medicine	Thirty-seven senior medical students	Electronic (PDA**) logbook	No of students that met the pre-specified target complaints	MCQ^† and written exam	Group seeing a required number of representative patients showed better knowledge	12.5	2
Martin et al.  2000	UK	Single group, post test	One year	Medicine and surgery	One-hundred and ninety-four (150 returned learning style form) students	Questionnaire	Total number of patients, outpatients and emergencies	OSCE* (0.70)	No association between clinical experience and OSCE* score. Positive association clinical experience with learning style.	12	3
Mcleod et al.  1997	Canada	Observational cohort study	Eight week	Internal medicine	Forty residents and 29 clinical clerks	Interview (unspecified), logbook, and questionnaire	Patient numbers of 20 diagnoses, ordered as top 20. – (Top) 16 skills, % patient encounters in which skill was relevant	Relevance for learning questionnaire/interview	Inpatient-based experience is better than ambulatory care experience for learning	8.5	2
McManus et al.  1998	UK	Prospective study of two cohorts assessed at application to med school and at the end of their final year	About five years	Undergraduate curriculum, not otherwise specified	Six-hundred and eighty-four students (first cohort 301/second 383)	Questionnaire	Single experience score based on 15/20 (first/second cohort) unspecified acute conditions, 18/18 surgical operations and 17/29 practical procedures	MCQ^† and other written exam Clinical performance (0.87, 0.88)	No association between clinical experience and exam score. Study habits predict examination performance	14.5	3
Nomura et al.  2008	Japan	Pre-post test design with non-equivalent control group	Two years	Multidisciplinary	Two-thousand four-hundred and seventy-four before +1166 after postgraduate ‘residents’ without clinical experience	Questionnaire	Number of students with ‘no experience’ with specified diagnosis (grouped) in 22 (sub) specialisms	Questionnaire, self confidence	Clinical experience and confidence levels improved, especially at university hospitals	10.5	2
O’Hara et al.  2002	USA	Single group, post test	Four weeks  +  Four days	Women healthcare in a family medicine clerkship	Four-hundred and forty-five students	Hand written logbook	– Volume of patient by ages – Volume and percentage of top 10 diagnosis by age group	Questionnaire, comfort level	Relationship between experience and comfort level between some diagnostic categories	8	1
Saywell et al.  2002	USA	Single group, post test	Four weeks	Musculoskeletal medicine in a family medicine clerkship	Four-hundred and forty-five third-year students	Hand written logbook	– Volume of patient by ages – Volume and percentage of top 10 diagnosis by age group	Questionnaire comfort level	Relationship between experience and comfort level between some diagnostic categories	8	1
Schwiebert  et al.1993	USA	Single group, post test	One month	Family medicine	One-hundred and eighty-five third-year students	Hand written logbook	– Proportion of students who encountered each of 20 diagnoses ordered as a top 20. – Number of patients	Written exam, oral exam	Slight differences between university and private practice in patient mix, but no difference in results on oral and written exam.	11.5	2
Wimmers 2006	NL	Single group, post test	Twelve weeks	Internal medicine	One-hundred and fifty-two students	Hand written logbook	– Number of patients – Number of different diseases encountered	Combined clinical performance assessment and oral examination (0.67)	An increased number of patient encounters did not (directly) lead to improved competence. Quality of supervision indirectly had impact on student learning and the number of patient encounters	13	3
Yu et al. 2011	New Zealand	Single group, post test	Six weeks	Surgery	One-hundred and sixty-six fourth-year students	Questionnaire and hand written logbook	Number of patients in seven diagnostic categories	Clinical assessment, critical appraised topic and OSCE* (0.69–0.74)	Heterogeneity of clinical experiment from sites did not translate into heterogeneity of learning outcomes	13.5	2
Van der Zwet  et al. 2010	NL	Single group, post test	Ten weeks	General practice	Two-hundred and eighty four fifth-year students	Questionnaire	Based on factor analysis; patient mix variable composed of three questionnaire items: number of patients, patient variety and quality of patient contacts	Questionnaire, instructional quality	Supervision and patient mix load on instructional quality	10	1

*Objective structured clinical examination.

^†Multiple-choice questions.

^‡ National board of medical examiners.

**Personal digital assistant.

Figure 2. Flow chart.

Figure 3. Model for reporting patient mix.

Ahmed SF, Hughes IA. Clinical experience during the paediatric undergraduate course. J R Soc Med 1999; 92(6)293–298

 PubMedGoogle Scholar

Boots RJ, Egerton W, McKeering H, Winter H. They just don't get enough! Variable intern experience in bedside procedural skills. Intern Med J 2009; 39(4)222–227

 PubMedGoogle Scholar

Chatenay M, Maguire T, Skakun E, Chang G, Cook D, Warnock GL. Does volume of clinical experience affect performance of clinical clerks on surgery exit examinations?. Am J Surg 1996; 172(4)366–372

 PubMedGoogle Scholar

Dolmans DH, Wolfhagen IH, Essed GG, Scherpbier AJ, Van der Vleuten CP. The impacts of supervision, patient mix, and numbers of students on the effectiveness of clinical rotations. Acad Med 2002; 77: 332–335

 PubMed Web of Science ®Google Scholar

Duke P, Curran V, Hollett A. Training family medicine residents to care for children: What is the best approach?. Can Fam Physician 2011; 57(2)e46–e50

 PubMedGoogle Scholar

Fung CC, Relan A, Wilkerson L. Demystifying “learning” in clinical rotations: Do immersive patient encounters predict achievement on the clinical performance examination (CPX)?. Acad Med 2007; 82: S97–S100

 PubMedGoogle Scholar

Greenberg LW, Getson P. Clinical experiences of medical students in a pediatric clerkship and performance on the NBME subject test: Are they related?. Med Teach 1999; 21: 420–423

 Google Scholar

Gruppen LD, Wisdom K, Anderson DS, Woolliscroft JO. Assessing the consistency and educational benefits of students' clinical experiences during an ambulatory care internal medicine rotation. Acad Med 1993; 68(9)674–680

 PubMedGoogle Scholar

Jacobson EW, Keough WL, Dalton BE, Giansiracusa DF. A comparison of inpatient and outpatient experiences during an internal medicine clerkship. Am J Med 1998; 104(2)159–162

 PubMedGoogle Scholar

Jolly BC, Jones A, Dacre JE, Elzubeir M, Kopelman P, Hitman G. Relationships between students' clinical experiences in introductory clinical courses and their performances on an objective structured clinical examination (OSCE). Acad Med 1996; 71(8)909–916

 PubMed Web of Science ®Google Scholar

Lampe CJ, Coates WC, Gill AM. Emergency medicine subinternship: Does a standard clinical experience improve performance outcomes?. Acad Emerg Med 2008; 15(1)82–85

 PubMedGoogle Scholar

Martin IG, Stark P, Jolly B. Benefiting from clinical experience: The influence of learning style and clinical experience on performance in an undergraduate objective structured clinical examination. Med Educ 2000; 34(7)530–534

 PubMedGoogle Scholar

McLeod PJ, Meagher T, Tamblyn RM, Zakarian R. Are ambulatory care-based learning experiences different from those on hospital clinical teaching units?. Teach Learn Med 1997; 9(2)125–130

 Web of Science ®Google Scholar

McManus IC, Richards P, Winder BC, Sproston KA. Clinical experience, performance in final examinations, and learning style in medical students: Prospective study. BMJ 1998; 316(7128)345–350

 PubMed Web of Science ®Google Scholar

Nomura K, Yano E, Aoki M, Kawaminami K, Endo H, Fukui T. Improvement of residents' clinical competency after the introduction of new postgraduate medical education program in Japan. Med Teach 2008; 30(6)e161–e169

 PubMed Web of Science ®Google Scholar

O'Hara BS, Saywell RM, Zollinger TW, Wooldridge JS, Kelton GM, Ries JS, Burba JL. Students' experience with women's health care in a family medicine clerkship. Med Educ 2002; 36(5)456–465

 PubMedGoogle Scholar

Saywell RM, O'Hara BS, Zollinger TW, Wooldridge JS, Burba JL, Mckeag DB. Medical students' experience with musculoskeletal diagnoses in a family medicine clerkship. Med Teach 2002; 24(2)186–192

 PubMedGoogle Scholar

Schwiebert LP, Ramsey CN, Jr, Davis A. Standardizing the clinical content of a third-year family medicine clerkship. Fam Med 1993; 25(4)257–261

 PubMedGoogle Scholar

Wimmers PF, Schmidt HG, Splinter TA. Influence of clerkship experiences on clinical competence. Med Educ 2006; 40(5)450–458

 PubMed Web of Science ®Google Scholar

Yu TC, Wheeler BR, Hill AG. Effectiveness of standardized clerkship teaching across multiple sites. J Surg Res 2011; 168(1)e17–e23

 PubMedGoogle Scholar

van der Zwet J, Hanssen VG, Zwietering PJ, Muijtjens AM, van der Vleuten CP, Metsemakers JF, Scherpbier AJ. Workplace learning in general practice: Supervision, patient mix and independence emerge from the black box once again. Med Teach 2010; 32(7)e294–e299

 PubMed Web of Science ®Google Scholar