Evaluating the effectiveness of undergraduate clinical education programs

Figures & data

Table 1. Essential measures to use in evaluating the effectiveness of undergraduate clinical education programs

Categories		Specific Measures	Corresponding LCME Substandard(s)	Kirkpatrick Level[^Citation4,Citation5]
Outcome Measures	Performance on national assessments	USMLE Step 2 CK	8.4	2
		USMLE Step 2 CS	8.4	2, 3
		NBME Clinical Subject Examinations	9.0	2
	Performance on local assessments	Local medical knowledge assessments	8.4, 9.0, 9.4, 9.8	2
		Objective Structured Clinical Examinations (OSCEs)	8.4, 9.0, 9.4	2, 3
		Clinical evaluations	9.0, 9.4	2, 3
		Other measures of competency attainment or entrustment (e.g., mini-CEX)	6.1, 9.0, 9.4	2, 3
	National measures of student satisfaction	AAMC Graduation Questionnaire	Throughout	1, 2
	Local measures of student satisfaction	Clerkship/course evaluations	8.3, 8.5	1, 2
		Faculty teaching evaluations	8.3, 8.5	1, 2
		Resident teaching evaluations	8.5	1, 2
	Career outcomes	Specialty selection	8.4	1, 2, 3
		Residency matching results	8.4	2
		Performance in the first year of residency (self and program director evaluations)	8.4	3
Process Measures	Curriculum design	Clerkship and session learning objectives	6.1, 8.3	1, 2
		Instructional formats and quality of instruction	6.0, 7.2, 8.3	1
		Assessments aligned to learning objectives	8.3, 9.0	1, 2
		Grade distribution	9.0, 9.6, 9.8	2
		Formative feedback, including mid-clerkship feedback	9.0, 9.7	2, 3
		Direct observation	9.4	2, 3
	Clinical experiences	Clinical settings/sites	5.5, 6.4	1
		Clinical roles and expectations	3.5, 6.1	1
		Required clinical experiences	6.2, 8.6	1, 2
		Patient volumes	5.5	1
	Learning environment	Physical spaces and resources	5.4, 5.6, 5.8, 5.9, 5.11	1, 4
		Supervision	9.3	1, 4
		Security and safety	5.7	1, 4
		Mistreatment	3.6	1, 4
	Preparation of instructors	Faculty	4.5, 6.1	1, 2, 4
		Residents	6.1, 9.1	1, 2, 4
	Compliance	Grade submission	9.8	-
		Duty hours	8.8	-
		Other university/college policies	4.6	-
Comparability across sites/campuses is relevant to most of the above measures (if applicable)			8.7	1, 2, 3

Kirkpatrick DL, Kirkpatrick JD. Kirkpatrick’s four levels of training evaluation. Alexandria, VA: ATD Press; 2016.

 Google Scholar

Hammick M, Dornan T, Steinert Y. Conducting a best evidence systematic review. Part 1: from idea to data coding. BEME Guide No. 13. Med Teach. 2010;32(1):3–15.

 PubMed Web of Science ®Google Scholar