Progress testing in clinical science education: Results of a pilot project between the National Board of Medical Examiners and a US Medical School

Abstract

Background: Though progress tests have been used for several decades in various medical education settings, a few studies have offered analytic frameworks that could be used by practitioners to model growth of knowledge as a function of curricular and other variables of interest.

Aim: To explore the use of one form of progress testing in clinical education by modeling growth of knowledge in various disciplines as well as by assessing the impact of recent training (core rotation order) on performance using hierarchical linear modeling (HLM) and analysis of variance (ANOVA) frameworks.

Methods: This study included performances across four test administrations occurring between July 2006 and July 2007 for 130 students from a US medical school who graduated in 2008. Measures-nested-in-examinees HLM growth curve analyses were run to estimate clinical science knowledge growth over time and repeated measures ANOVAs were run to assess the effect of recent training on performance.

Results: Core rotation order was related to growth rates for total and pediatrics scores only. Additionally, scores were higher in a given discipline if training had occurred immediately prior to the test administration.

Conclusions: This study provides a useful progress testing framework for assessing medical students’ growth of knowledge across their clinical science education and the related impact of training.

Additional information

Notes on contributors

Andrė F. De Champlain

Andrė F. De Champlain, PhD, is a principal measurement scientist in the Professional Services Unit of the National Board of Medical Examiners. His research interests relate primarily to the application of psychometric models in medical education.

Monica M. Cuddy

Monica M. Cuddy, MA, is a measurement associate III in the Professional Services Unit of the National Board of Medical Examiners. Her research interests center on the use of HLM in medical education.

Peter V. Scoles

Peter V. Scoles, MD, is a senior vice president in the Assessment Programs Unit of the National Board of Medical Examiners.

Marie Brown

Marie Brown, MA, is a manager, measurement analysis in the Professional Services Unit of the National Board of Medical Examiners.

David B. Swanson

David B. Swanson, PhD, is a vice president in the Professional Services Unit of the National Board of Medical Examiners.

Kathleen Holtzman

Kathleen Holtzman, MA, is an assistant vice president, Test Development Operations and Developmental Programs, in the Professional Services Unit of the National Board of Medical Examiners.

Aggie Butler

Aggie Butler, PhD, is a vice president, Medical Education and Health Profession Services, in the Assessment Programs Unit of the National Board of Medical Examiners.