The relative predictive value of undergraduate versus graduate selection tools in two Australian medical schools

Abstract

Context: Monash University and the University of Western Australia admit both school-leavers and graduates into their Bachelor of Medicine and Bachelor of Surgery (MBBS) courses. The Undergraduate Medicine and Health Sciences Admission Test (UMAT) and the Graduate Medical Schools Admissions Test (GAMSAT) are used for selection, along with an academic score and an interview score. The aim of this study was to compare the relative predictive validity of the selected components in the two entry streams, particularly UMAT versus GAMSAT.

Methods: Aggregated scores for course outcomes were calculated in the categories of knowledge, clinical and total scores, at four-time points. A path analysis was conducted based on multivariate regressions with model constraint parameters defined across the outcome variables to investigate change over time.

Results: Academic scores were the strongest predictors of knowledge scores and end of course results. Interview scores had a small positive increasing effect, being stronger for clinical than knowledge outcomes. The effect size for GAMSAT was greater than for UMAT.

Conclusions: Aptitude tests and interview scores added small but significant incremental predictive value to previous academic achievement. GAMSAT showed larger predictive value on outcomes than UMAT, for which one section (UMAT 3) had a negative effect.

Ethics approval and consent to participate

Ethics approval for this project was established at MU (CF12/3911 – 2012001871) and recognized at UWA (RA/4/1/6380).

Acknowledgments

Irene Lichtwark and Selina Tran from Monash University constructed the final data files and cleaned the data. Selina Tran conducted the MPlus analysis which forms the basis of the results.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article. In the course of their work, all authors are or have been members of boards or committees of the UMAT and GAMSAT Consortia.

Funding

This research project was funded jointly by the UMAT Consortium and the GAMSAT Consortium.

Glossary

Aptitude tests: Are a form of mental ability test, distinguished from achievement tests by the degree of uniformity of antecedent experience (cumulative effect of a multiplicity of everyday experiences) and a difference in purpose (prediction of the likelihood of success in an academic setting).

Anastasi A, Urbina S. 1997. Psychological testing. Upper Saddle River (NJ): Prentice Hall.

Notes on contributors

Annette Mercer, PhD, is an Associate Professor in Medical Education with an interest in the selection of medical students.

Margaret Hay, PhD, is the Director of Admissions and Director of the Monash Institute for Health and Clinical Education in the Faculty of Medicine, Nursing and Health Sciences.

Wayne C. Hodgson, PhD, is Deputy Dean (Education) in the Faculty of Medicine, Nursing and Health Sciences.

Ben J. Canny, PhD, is Head of the School of Medicine in the Faculty of Health at the University of Tasmania.

Ian B. Puddey, MD, is Emeritus Professor and formerly Dean in the Faculty of Health and Medical Sciences.