Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45

Abstract

Background: Selection into specialty training is a high-stakes and resource-intensive process. While substantial literature exists on selection into medical schools, and there are individual studies in postgraduate settings, there seems to be paucity of evidence concerning selection systems and the utility of selection tools in postgraduate training environments.

Aim: To explore, analyze and synthesize the evidence related to selection into postgraduate medical specialty training.

Method: Core bibliographic databases including PubMed; Ovid Medline; Embase, CINAHL; ERIC and PsycINFO were searched, and a total of 2640 abstracts were retrieved. After removing duplicates and screening against the inclusion criteria, 202 full papers were coded, of which 116 were included.

Results: Gaps in underlying selection frameworks were illuminated. Frameworks defined by locally derived selection criteria, and heavily weighed on academic parameters seem to be giving way to the evidencing of competency-based selection approaches in some settings.

Regarding selection tools, we found favorable psychometric evidence for multiple mini-interviews, situational judgment tests and clinical problem-solving tests, although the bulk of evidence was mostly limited to the United Kingdom. The evidence around the robustness of curriculum vitae, letters of recommendation and personal statements was equivocal. The findings on the predictors of past performance were limited to academic criteria with paucity of long-term evaluations. The evidence around nonacademic criteria was inadequate to make an informed judgment.

Conclusions: While much has been gained in understanding the utility of individual selection methods, though the evidence around many of them is equivocal, the underlying theoretical and conceptual frameworks for designing holistic and equitable selection systems are yet to be developed.

Acknowledgements

The review team would sincerely like to thank the information scientist, Mr Lars Erikkson at the School of Medicine Library, University of Queensland, Australia for support in framing and executing the search strategy. The team would also like to thank the Royal Australasian College of Physicians for providing support and protected time for undertaking this review.

Disclosure statement

CR: consultancy for the RACP and the AGPT/DOH on matters of selection into postgraduate training.

Notes on contributors

Chris Roberts, MBChB, FRACGP, MMedSci, PhD, is an Associate Professor, Medical Education at the University of Sydney, New South Wales, Australia.

Priya Khanna, MSc, MEd, PhD, is a researcher at the Royal Australasian College of Physicians, New South Wales, Australia.

Louise Rigby, is a PhD candidate at University of Sydney is a manager at the Health Education and Training Institute, New South Wales, Australia.

Emma Bartle, PhD, is Teaching and Learning Chair, School of Dentistry at the University of Queensland, Australia.

Anthony Llewellyn, BMedSci, MBBS, FRANZCP, MHA, GAICD, is a senior staff specialist in psychiatry training, Hunter New England Local Health District, New South Wales, a senior lecturer, University of Newcastle, and a specialist lead in Rural, Health Education and Training Institute, New South Wales, Australia.

Julie Gustavs, PhD, is a manager at the Royal Australasian College of Physicians, New South Wales, Australia.

Libby Newton, BSc, is a researcher at the Royal Australasian College of Physicians, New South Wales, Australia.

James P. Newcombe, BMedSci (Hons), MPH (Hons), MBBS, GAICD, FRACP, FRCPA, is an infectious diseases physician and clinical microbiologist at the Royal North Shore Hospital, New South Wales, Australia.

Mark Davies, MBBS, FRACP, is a staff specialist in neonatology at the Royal Brisbane and Women's Hospital and an associate professor of neonatology at the University of Queensland, Australia.

Jill Thistlethwaite, BSc, MM, MS, PhD, MMEd, FRCGP, FRACGP, is an adjunct professor at University Technology Sydney, honorary professor in the School of Education at the University of Queensland, and a medical advisor to the NPS MedicineWise in Australia.

James Lynam, BSc (Hons), MBBS, MRCP, FRACP, is a practicing medical oncologist at the Calvary Mater Newcastle, the Network Director of Physician Training for the Hunter New England Network and a conjoint lecturer at the University of Newcastle, New South Wales, Australia.