Early predictors of summative assessment performance in general practice post-graduate training: A retrospective cohort study

Abstract

Background: Success in summative general practice (GP) training assessments is one indicator of competence for practice. Early-training factors predictive of outcomes would facilitate targeted interventions aimed at preventing candidate failures.

Methods: We undertook a retrospective cohort study of Australian GP trainees in two training organizations over five years. Associations of pre-training and early-training predictors with summative examination scores in an Applied Knowledge Test (AKT), Key Features Paper (KFP), and Objective Structured Clinical Examination (OSCE), plus failure on any one of these, were tested via univariate and multivariable regression. Predictors were program-entry selection-score decile, pre-training-commencement multiple choice assessment (MCQA), direct observation of practice performance, and clinical supervisor reports.

Results: On univariate analyses, selection decile and MCQA were associated with all outcomes except AKT. There were no associations of other predictors with any outcomes. On multivariable analysis, selection decile and MCQA performance were predictive of OSCE performance. MCQA performance was also predictive of KFP performance. On multivariable analysis, no predictors were associated with AKT performance or failing any examination.

Conclusions: Selection decile and pre-commencement MCQA performance were predictive of performance in some summative assessments. These findings could inform selection policies and targeted early interventions for trainees at most risk for exam failure.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Remediation: A purposeful intervention including “any form of additional training, supervision, or assistance above that required for a typical resident” (Riebschleger and Haftel 2013). It involves three steps (1) the identification of deficiencies in an individual’s performance; (2) an attempt to provide remedial education to that individual; and (3) reassessment to determine the impact of the education (Hauer et al. 2009).

Hauer KE, Ciccone A, Henzel TR, Katsufrakis P, Miller SH, Norcross WA, Papadakis MA, Irby DM. 2009. Remediation of the deficiencies of physicians across the continuum from medical school to practice: a thematic review of the literature. Acad Med. 84:1822–1832.

Riebschleger MP, Haftel HM. 2013. Remediation in the context of the competencies: a survey of pediatrics residency program directors. J Grad Med Educ. 5:60–63.

Additional information

Funding

The authors would like to acknowledge that this project was funded by an Education Research Grant by the Australian Government Department of Health.

Notes on contributors

Rebecca Stewart

Rebecca Stewart, MBBS, FRACGP, FANZAHPE, GCTertEd, MClinEd, PCEval, is a Medical Education Consultant and General Practitioner working in private practice in Townsville, Australia.

Nick Cooling

Nick Cooling, BSc(Hons), Dip Nut Diet, BMBS, FRACGP, DRACOG, MEd, is the Director of Internationalisation & Global Health in the School of Medicine, University of Tasmania, Australia.

Graham Emblen

Graham Emblen, MBBS, FRACGP, DRANZCOG, MasFamMed, is the Senior Associate Director Medical Education at General Practice Training Queensland, Australia.

Allison Turnock

Allison Turnock, BMedSci MBBS(Hons), DCH, FRACGP, is the Director – Rural Pathways and Medical Director, GP & Primary Care, Department of Health and Human Services, Tasmania, Australia. She is also an Adjunct Senior Lecturer at the University of Tasmania.

Amanda Tapley

Amanda Tapley, M Med Stat, B Biomed Sci, is the Senior Research Officer and Statistician, GP Synergy, New South Wales, Australia.

Elizabeth Holliday

Elizabeth Holliday, PhD, BSc, MSc, is an Associate Professor of Biostatistics in the School of Medicine and Public Health at the University of Newcastle, and senior statistical affiliate at the Hunter Medical Research Institute, New South Wales, Australia.

Jean Ball

Jean Ball, B.Math, Grad Dip Med Stats, is a statistical programmer with the Hunter Medical Research Institute, Newcastle, Australia.

Jennifer Juckel

Jenny Juckel, BPsych Sci (Hons), PhD, was the Research Assistant for this research project at General Practice Training Queensland.

Parker Magin

Parker Magin, MBBS (Hons), PhD, MFM (Clin), MGPP, GDipClinEpi, DPD, DipPaed, FRACGP, is Conjoint Professor, School of Medicine and Public Health, University of Newcastle, and the Director, NSW & ACT Research and Evaluation Unit, GP Synergy, New South Wales, Australia.