Implementation of competence by design in Canadian neurosurgery residency programs*

* Previous presentations

• American Association of Neurological Surgery (AANS), May 2020, Boston, MA, USA, E-poster (preliminary results from first survey).

• AMEE 2020: The Virtual Experience, September 2020, Glasgow, UK, E-poster Virtual (mixed-methods results).

• CBME Program Evaluation Summit, September 2020, Royal College of Physicians and Surgeons of Canada, Oral presentation.

Abstract

Objective

The Royal College of Physicians and Surgeons of Canada (RCPSC) recently redesigned the Canadian neurosurgery residency training curriculum by implementing a competency-based model of training known as Competence by Design (CBD) centered around the assessment of Entrustable Professional Activities (EPAs). This sequential explanatory mixed-methods study evaluated potential benefits and pitfalls of CBD in Canadian neurosurgery residency education.

Methods

Two four-month interval surveys were distributed to all Canadian neurosurgery residents participating in CBD. The surveys assessed important educational components: CBD knowledge of key stakeholders, potential system barriers, and educational/psychological impacts on residents. Paired t-tests were done to assess changes over time. Based on longitudinal survey responses, semi-structured interviews were conducted to investigate in-depth residents’ experience with CBD in neurosurgery. The qualitative analysis followed an explanatory approach, and a thematic analysis was performed.

Results

Surveys had 82% average response rate (n = 25). Over time, most residents self-reported that they retrospectively understood concepts around CBD intentions (p = 0.02). Perceived benefits included faculty evaluations with more feedback that was clearer and more objective (53% and 51%). Pitfalls included the amount of time needed to navigate through EPAs (90%) and residents forgetting to initiate EPA forms (71%). There was no significant change over time. During interviews, five key themes were found. Potential solutions identified by residents to enhance their experience included learning analytics data availability, mobile app refinement, and dedicated time to integrate EPAs in the workflow.

Conclusion

This study was the first to assess resident-perceived benefits and pitfalls of the neurosurgery CBD training program in an educational framework context. In general, residents believed that theoretical principles behind CBD were valuable, but that technological ability and having enough time to request EPA assessments were significant barriers to success. Long-term studies are required to determine the definitive outcomes of CBD on residents’ performance and ultimately, on patient care.

Keywords:

• Medical education research
• international medical education
• postgraduate
• surgery

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Competence by Design (CBD): The Royal College of Physicians and Surgeons of Canada (RCPSC) developed a competency-based medical education system (CBME) known as Competence by Design (CBD). It is a hybrid model that combines different entrustable professional activities (EPAs) and milestones.

Additional information

Notes on contributors

Magalie Cadieux

Magalie Cadieux, MD, MMSc, is a Neurosurgery resident at the University of Calgary and completed a Medical Education master’s degree at Harvard Medical School.

Michael Healy

Michael Healy, Ed.D, is a Surgery research fellow at Massachusetts General Hospital.

Emil Petrusa

Emil Petrusa, PhD, is affiliated faculty in the Master of Science in Health Professions Education program in the Center for Interprofessional Studies and Innovation at MGH Institute of Health Professions.

Lara Cooke

Lara Cooke, MD, FRCPC, is Associate Professor of Neurology and Head of Neurology section at the University of Calgary.

Lara Traeger

Lara Traeger, PhD, is a psychologist in the Cancer Center and the Behavioral Medicine Service at MGH, and an Assistant Professor of Psychology at Harvard Medical School.

Jennifer C. Kesselheim

Jennifer C. Kesselheim, MD, MEd, is a pediatric hemato-oncologist at Dana Farber Cancer Institute and director of the Master of Medical Sciences in Medical Education program at Harvard Medical School.

Jay Riva-Cambrin

Jay Riva-Cambrin, MD, MSc, is a pediatric neurosurgeon and the Director of the Neurosurgery residency training program at the University of Calgary.

Roy Phitayakorn

Roy Phitayakorn, MD, MHPE, FACS, is a general and endocrine surgeon at MGH and is the Surgery Director of Medical Student Education and Surgical Education Research.