How does one assess the accuracy of academic success predictors? ROC analysis applied to university entrance factors

Abstract

This article attempts to present a novel application of a method of measuring accuracy for academic success predictors that could be used as a standard. This procedure is known as the receiver operating characteristic (ROC) curve, which comes from statistical decision techniques. The statistical prediction techniques provide predictor models and their goodness-of-fit; in addition, ROC analysis allows to assess the accuracy of the ability to discriminate from success and failures cases of a classifier or predictive model, and so it could be considered complementary to others more commonly used. Thus, the ROC curve is used to compare and interpret the relative contribution of each university entrance factor in the correct classification as success or failure of the academic performance, as well as to establish cut-off scores for admissions and counselling purposes. It is revealed that the ROC analysis allows us to identify the better university entrance factor for each subject in predicting students’ academic success.

Keywords:

• ROC curve
• accuracy measure
• academic performance
• undergraduate
• university entrance factors

Notes

†Note that we have neither explained the Spanish educational system, the university entrance method nor the university degrees, as our purpose is not to evaluate the Spanish system. We only study the utility and interest of the ROC analysis to evaluate the quality of the ability of a classifier to discriminate into two homogeneous groups of undergraduate students.