Exploring students’ [pre-pandemic] use and the impact of commercial-off-the-shelf learning platforms on students’ national licensing exam performance: A focused review – BEME Guide No. 72

Abstract

Background

Commercial-off-the-shelf learning platforms developed for medical education (herein referred to as MedED-COTS) have emerged as a resource used by a majority of medical students to prepare for licensing examinations. As MedED-COTS proliferate and include more functions and features, there is a need for an up-to-date review to inform medical educators on (a) students’ use of MedED-COTS outside the formal medical school curriculum, (b) the integration of MedED-COTS into the formal curriculum, and (c) the potential effects of MedED-COTS usage on students’ national licensing exam scores in the USA.

Methods

Due to the limited number of studies published on either the use or integration of MedED-COTS, a focused review of literature was conducted to guide future research and practice. Data extraction and quality appraisal were conducted independently by three reviewers; with disagreements resolved by a fourth reviewer. A narrative synthesis was completed to answer research questions, contextualize results, and identify trends and issues in the findings reported by the studies included in the review.

Results

Results revealed consistent positive correlations between students’ use of question banks and their licensing exam performance. The limited number of integration studies, combined with a number of methodological issues, makes it impossible to isolate specific effects or associations of integrated commercial resources on standardized test or course outcomes. However, consistent positive correlations, along with students’ pervasive use and strong theoretical foundations explaining the results, provide evidence for integrating MedED-COTS into medical school curricula and highlight the need for further research.

Conclusions

Based on findings, we conclude that students use exam preparation materials broadly and they have a positive impact on exam results; the literature on integration of MedED-COTS into formal curriculum and the use by students of resources outside of exam preparation is scant.

Keywords:

• Commercial learning platforms
• national board exams
• focused review
• BEME

Disclosure statement

David Harris serves as an uncompensated basic science lead on the Aquifer Sciences Initiative, part of Aquifer (non-profit organization). The authors alone are responsible for the content and writing of the article.

Glossary

MedED-COTS: Commercially distributed (Commercial-Off-The-Shelf) learning resources that are designed to help medical students prepare for national board exams and may also be used to facilitate Medical EDucation.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes on contributors

Atsusi Hirumi

Atsusi Hirumi, PhD, Professor of Instructional Design and Technology, Department of Learning Sciences and Educational Research, College of Community Innovation and Education, University of Central Florida, Orlando, FL.

Luke Horger

Luke Horger, MA, Instructional Design and Technology, College of Community Innovation and Education, University of Central Florida, Orlando, FL.

David M. Harris

David Harris, PhD, Associate Professor of Physiology, Department of Medical Education, University of Central Florida College of Medicine, Orlando, FL.

Andrea Berry

Andrea Berry, MPA, Executive Director, Faculty Life Office and Instructor, Department of Medical Education, University of Central Florida College of Medicine, Orlando, FL.

Feroza Daroowalla

Feroza Daroowalla, MD, MPH, Associate Professor of Medicine, Department of Medical Education, University of Central Florida College of Medicine, Orlando, FL.

Shalu Gillum

Shalu Gillum, JD, MLS, AHIP, Head of Public Services for the Harriet F. Ginsburg Health Sciences Library and Assistant Medical Librarian, Department of Medical Education, University of Central Florida College of Medicine, Orlando, FL.

Nyla Dil

Nyla Dil, DVM, MSc, PhD, Assistant Professor of Medicine, Department of Medical Education, University of Central Florida College of Medicine, Orlando, FL.

Juan C. Cendán

Juan C. Cendan, MD, Professor of Surgery, Interim Dean, Herbert Wertheim College of Medicine, Florida International University, Miami, FL.