Predicting failure before it happens: A 5-year, 1042 participant prospective study

Abstract

Purpose of the article

Students who fail assessments are at risk of negative consequences, including emotional distress and cessation of studies. Identifying students at risk of failure before they experience difficulties may considerably improve their outcomes.

Methods

Using a prospective design, we collected simple measures of engagement (formative assessment scores, compliance with routine administrative tasks, and attendance) over the first 6 weeks of Year 1. These measures were combined to form an engagement score which was used to predict a summative examination sat 14 weeks after the start of medical school. The project was repeated for five cohorts, giving a total sample size of 1042.

Results

Simple linear regression showed engagement predicted performance (R²_adj = 0.03, F(1,1040) = 90.09, p < 0.001) with a small effect size. More than half of failing students had an engagement score in the lowest two deciles.

Conclusions

At-risk medical students can be identified with some accuracy immediately after starting medical school using routinely collected, easily analysed data, allowing for tailored interventions to support students. The toolkit provided here can reproduce the predictive model in any equivalent educational context. Medical educationalists must evaluate how the advantages of early detection are balanced against the potential invasiveness of using student data.

Keywords:

• Assessment
• psychometrics
• student support

Disclosure statement

The authors have no declarations of interest to report.

Glossary

Learning analytics: Is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.

SIEMENS, G. & GASEVIC, D. 2012. Guest Editorial - Learning and Knowledge Analytics. Journal of Educational Technology & Society. 15: 1-2.

Additional information

Funding

This work was supported by Principal's Teaching Award Scheme at the University of Edinburgh.

Notes on contributors

Avril Dewar

Avril Dewar, MSc, is a fellow in medical education at Edinburgh Medical School.

David Hope

David Hope, PhD, is a senior lecturer at Edinburgh Medical School.

Alan Jaap

Alan Jaap, MD, is Deputy Director of Teaching at Edinburgh Medical School.

Helen Cameron

Helen Cameron, MBChB, is Acting Head of School and Dean of Medical Education at Aston Medical School.