Using systematically observed clinical encounters (SOCEs) to assess medical students’ skills in clinical settings

Figures & data

Figure 1 A schematic of the two-step systematically observed clinical encounter (SOCE) process. The student first interviews and examines a patient while being observed by a standardized observer (SO). The student then presents their findings to a faculty physician who is supervising learners in the General Pediatrics Clinic.

Table 1 Generalizability analysis using urGENOVA® for estimating the sources of skills score variance

Facets	Degrees of freedom	Variance component	Percentage of variance
Student	51	2.1886	30.5%
Encounter:student	146	4.9985	69.5%

Notes: Data were collected by observing 51 third-year medical students on the pediatrics clerkship at the University of Iowa carver college of Medicine during the 2006–2007 academic year

Table 2 Predicted systematically observed clinical encounter (SOCE) score reliability as a function of the number of student– patient encounters completed by students

Number of SOCE cases	Score reliability
2	0.467
3	0.568
4	0.637
5	0.686
6	0.724
7	0.754
8	0.778
9	0.798
10	0.814
11	0.828

Notes: The generalizabilty study was undertaken with urGenova® using all available data collected by observing 51 third-year medical students on the pediatrics clerkship at the University of Iowa carver college of Medicine during the 2006–2007 academic year

Appendix 1 Clinical skills rating scale completed by faculty physicians after each SOCE. This instrument has six rating scales about the student’s data collection and clinical reasoning skills. A student’s score can range between 6 and 30 points

Domain scores
History	1	2	3	4	5
Completeness of HPI:	Few	Some	Many	Most	All
Did the student present all of the major facts of the HPI?
Development of HPI:	Little	Limited	Partial	Consistent	Complete
Did the student develop the HPI by focus on onset of symptoms, site, time course, severity, setting, and aggravating-relieving factors?
Other appropriate history:	Little	Some	Many	Most	All
Did the student provide other history (past medical history, family history, social history, ROS) appropriate to the presenting problem?
Accuracy of history:	Large amount	Some	Little	Rare	None
Did the student present any incorrect information?
Physical exam	1	2	3	4	5
Completeness of PE:	Little	Some	Many	Most	All
Did the presentation include appropriate detail about the important physical findings?
Clinical reasoning	1	2	3	4	5
Soundness of reasoning:	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Development of Diff DX and problem list demonstrated sound clinical reasoning.

Abbreviations: Diff DX, differential diagnosis; HPI, history of the present illness; PE, physical exam; ROS, review of systems; SOCE, systematically observed clinical encounter.