The effect of testing and feedback on the forgetting curves for radiograph interpretation skills

Abstract

Objectives: Forgetting curves plot skill decay over time. After exposure to a simulation-based radiograph interpretation learning system, we determined the rate of learning decay and how this was impacted by testing (with and without feedback). Further, we examined the association of initial learning parameters on the forgetting curve.

Methods: This was a multicenter, four-arm randomized control trial. Medical trainees completed 80 elbow radiographs and a 20-case post-test. Group 1 had no testing until 12 months; Groups 2–4 had testing every 2 months until 12 months. At 6 months, Group 3 testing was feedback-enhanced, while Group 4 had feedback-enhanced testing at 2, 6, and 10 months.

Results: There were 106 participants (n = 42 Group 1; n = 22 Groups 2 and 3; n = 20 Group 4). Group 1 showed an –8.1% learning decay at 12-months relative to other groups. In Groups 2, 3, and 4, there was no significant learning decay (+0.8%), and there were no differences in skill decay between these groups. Initial score and learning curve slope were predictive of retained skill.

Conclusions: Learning decay was mitigated by exposure to 20 test cases (with and without feedback) every two months. Initial learning parameters predicted learning retention and may inform refresher education scheduling.

Acknowledgements

We wish to acknowledge the recruitment efforts of Dr. Shannon McPhee and Ms. Eleanor Fitzpatrick of IWK Health Centre, Dr. Brian Magwood of Rady Faculty of Health Sciences, Dr. Eric Koelink of McMaster Children’s Hospital, and Ms. Maryse Lagacé of CHU Sainte Justine. Further, we thank Ms. Caseita Dewar for her administrative support in the preparation of the manuscript. We also thank Dr. Heather Carnahan for helpful study design suggestions and PERC/INSPIRE collaboratives for support in participant enrollment from their networks.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Kathy Boutis

Kathy Boutis, MD, MSc, Department of Pediatrics, Division of Emergency Medicine, Hospital for Sick Children and the University of Toronto, Toronto, Canada.

Martin Pecaric

Martin Pecaric, PhD, Contrail Consulting Services, Toronto, Canada.

Benoit Carrière

Benoit Carrière, MD, MHPE, Department of Pediatrics, Division of Emergency Medicine, CHU Sainte-Justine and Université de Montréal, Montreal, Canada.

Jennifer Stimec

Jennifer Stimec, MD, Department of Diagnostic Imaging, Hospital for Sick Children and the University of Toronto, Toronto, Canada.

Andrew Willan

Andrew Willan, PhD, Hospital for Sick Children, University of Toronto, Research Institute and Dalla Lana School of Public Health, University of Toronto, Toronto, Canada.

Julie Chan

Julie Chan, MD, Department of Surgery, Division of Orthopedic Surgery, Queen’s University, Kingston, Canada.

Martin Pusic

Martin Pusic, MD PhD, Department of Emergency Medicine and Division of Learning Analytics at the NYU School of Medicine, New York, USA.