Abstract
Introduction
Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of the implementation of the 12 principles of programmatic assessment and to gain insight into enablers and barriers across different institutions and contexts.
Methods
After the 2020 Ottawa conference, we surveyed 15 Health Profession Education programmes from six different countries about the implementation of the 12 principles of programmatic assessment. Survey responses were analysed using a deductive thematic analysis.
Results and Discussion
A wide range of implementations were reported although the principles remained, for the most part, faithful to the original enunciation and rationale. Enablers included strong leadership support, ongoing faculty development, providing students with clear expectations about assessment, simultaneous curriculum renewal and organisational commitment to change. Most barriers were related to the need for a paradigm shift in the culture of assessment. Descriptions of implementations in relation to the theoretical principles, across multiple educational contexts, coupled with explanations of enablers and barriers, provided new insights and a clearer understanding of the strategic and operational considerations in the implementation of programmatic assessment. Future research is needed to further explore how contextual and cultural factors affect implementation.
Keywords:
Correction Statement
This article has been corrected with minor changes. These changes do not impact the academic content of the article.
Acknowledgements
This work was made possible thanks to the efforts of individuals, including the authors, who took the time to complete the survey questionnaire describing implementation experiences in their programmes. As such, the authors would like to acknowledge and thank the following: Marjan Govaerts, (Maastricht, the Netherlands) Lambert Schuwirth (Flinders University, Australia), Teresa Chan (McMaster, Canada), Paul Dilena (General Practice, Parkside, Australia), Vincent Arockiasamy (University of British Columbia, Canada) and Kent Hecker (University of Calgary, Canada).
Disclosure statement
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.
Additional information
Notes on contributors
Dario Torre
Dario Torre, MD, MPH, PhD, Professor of Medicine and Associate Director for Program Evaluation and Long Term Outcomes at Uniformed Services University of Health Sciences in Bethesda, Maryland, USA.
Neil E. Rice
Neil Rice, Head of Psychometrics and Informatics, University of Exeter, College of Medicine and Health, Exeter, UK.
Anna Ryan
Anna Ryan, MBBS, PhD, Associate Professor, Director of Assessment, Department of Medical Education, Melbourne Medical School, University of Melbourne, Australia.
Harold Bok
Harold Bok, Faculty for Veterinary Medicine, Centre for Quality Improvement in Veterinary education, Department of Veterinary Medicine, Utrecht University, Utrecht, The Netherlands.
Luke J. Dawson
Luke J. Dawson, BSc, BDS, PhD, FDSRCS(Eng), FHEA, MA (TLHE) NTF, Professor of Dental Education, Director of Undergraduate Education, School of Dentistry, Liverpool, UK.
Beth Bierer
Beth Beirer, MD, Director of Assesmsnent and Evaluation, Cleveland Clinic Learner College of Medicine of Case Western University, Cleveland, OH, USA.
Tim J. Wilkinson
Tim J. Wilkinson, MB, ChB, MD, PhD(Otago), M Clin, Ed(UNSW), FRACP FRCP(London), FANZAHPE, Director of the University of Otago MBChB programme, Education unit, University of Otago, Christchurch, New Zealand.
Glendon R. Tait
Tom Laughlin, MD, Associate Professor of Family Medicine, Department of Family Medicine, Dalhousie University, Halifax, Canada.
Tom Laughlin
Kieran Veerapen, MD, Associate Professor of medicine, Assistant Dean for Faculty Development, University of British Columbia, Vancouver, BC, Canada.
Kiran Veerapen
Glendon R. Tait, MD, MSc, Director of Student Assessment, MD Program at the University of Toronto; Associate Professor Department of Psychiatry and practices Consultation-Liaison Psychiatry with Sinai Health System, and The Wilson Centre, University of Toronto, Canada.
Sylvia Heeneman
Sylvia Heeneman, PhD, Professor of Health Profession Education at the School of Health Profession Education, Department of Pathology, Faculty of Health, Medicine and Life Sciences, Maastricht University, the Netherlands.
Adrian Freeman
Adrian Freeman, MD, MMedSci, FRCGP, FAcadMed, Professor of Medical Education at University of Exeter Medical School, examiner for the Royal College of General Practitioners, President of the European Board of Medical Assessors and Deputy chair of the GMC Panel for Tests of Competence, University of Exeter, College of Medicine and Health, Exeter, UK.
Cees van der Vleuten
Cees van der Vleuten, PhD, Professor of Education, Faculty of Health Medicine and Life Sciences, Maastricht University, The Netherlands.