Using an objective structured clinical examination (OSCE) to assess multiple physician competencies in postgraduate training

Abstract

Background: Competency-based models of medical education require reliable and valid assessment of multiple physician roles.

Aims: To develop and evaluate an objective structured clinical examination (OSCE) designed to assess 7 physician competencies (CanMEDS Roles).

Methods: Twenty four candidates from 4 neonatal–perinatal medicine training programs participated in a 10-station OSCE. Ten 5-point rating scales were developed and used to assess the CanMEDS Roles of Medical Expert, Communicator, Collaborator, Manager, Health Advocate, Scholar and Professional. Three descriptors of performance anchored the ratings. For each station, examiners completed appropriate CanMEDS ratings, a station-specific binary checklist and an overall process-related global rating. Trained standardized patients (SP) and standardized health professionals (SHP) completed rating scales that assessed verbal and non-verbal expression, empathy and coherence as well as the overall global rating.

Results: Each station incorporated 3–5 physician Roles. Interstation alpha was 0.80 for checklist scores and 0.88 for examiners' overall global rating. Median interstation alpha for individual CanMEDS ratings was 0.72 (range 0.08–0.91). There were significant correlations between examiner Medical Expert scores and SP/SHP overall global scores and between examiner Communicator scores and 4 SP/SHP assessments of communication skills. Second year trainees' CanMEDS scores for each competency were significantly higher than those of first year trainees (p < 0.05).

Conclusions: The OSCE may be useful as a reliable and valid method of simultaneously assessing multiple physician competencies.

Additional information

Notes on contributors

Ann Jefferies

ANN JEFFERIES MD, MEd is a staff neonatologist and associate professor, Dept. of Paediatrics, University of Toronto. She chairs the Neonatal – Perinatal Specialty Committee, Royal College of Physicians and Surgeons of Canada.

Brian Simmons

BRIAN SIMMONS BM is a staff neonatologist and associate professor, Dept. of Paediatrics, University of Toronto. He is a member of the Nucleus Committee, Neonatal – Perinatal Specialty Committee, Royal College of Physicians and Surgeons of Canada.

Diana Tabak

DIANA TABAK is the Associate Director, Standardized Patient Program, at the Donald Wilson Centre for Research in Education, University of Toronto.

Jodi Herold Mcilroy

JODI MCILROY BHSc (PT), MA, PhD is a psychometrician and Educational Scientist at the Donald R. Wilson Centre for Research in Education, University of Toronto, Ontario.

Kyong-Soon Lee

KYONG-SOON LEE MD, MSc is a staff neonatologist and associate professor, Dept. of Paediatrics, University of Toronto. Previously she was Program Director, Neonatal – Perinatal Medicine Subspecialty Training Program at McMaster University, Hamilton, Ontario.

Henry Roukema

HENRY ROUKEMA MD, MSc is a staff neonatologist and Director of Newborn Nurseries, Dept. of Paediatrics, University of Western Ontario. Previously he was Program Director, Neonatal – Perinatal Medicine Subspecialty Training Program at University of Western Ontario, London, Ontario.

Martin Skidmore

MARTIN SKIDMORE MB, ChB is a staff neonatologist and assistant professor, Dept. of Paediatrics, University of Toronto. He is the Program Director, Integrated Fellowship Training Program in Neonatal-Perinatal Medicine at University of Toronto.