How to improve the predictive validity of a composite admission score? A case study from Hungary

Abstract

An essential task in higher education is to construct a fair admission procedure. A great deal of research has been conducted on a central aspect of admission: predictive validity. However, to the best of our knowledge, this is the first study that investigates how the predictive validity of a composite admission score could be improved without redesigning the tests and introducing new measures. In this study, relying on the existing instruments of the Hungarian nationally standardized university entrance score, we construct an alternative score that not only has higher predictive validity but also a lower variation across disciplines and a smaller under- and overprediction bias in various student groups. To measure the predictive validity, we use an advanced statistical framework. The analysis relies on data of 24,675 students enrolled in the undergraduate programs of the Budapest University of Technology and Economics. We find that while the current score is effective in predicting university success, its predictive validity can be improved by a few changes: lifting the branching nature of the admission, focusing on general rather than program-specific knowledge, and introducing a multiplicative rewarding scheme for advanced level examinations.

Keywords:

• Predictive validity
• academic performance
• college admission
• ROC analysis
• range restriction

Acknowledgements

We are grateful to Hung Nguyen for carefully reading the manuscript, moreover, we thank Bálint Csabay and Miály Szabó for their support in data collection.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was supported by the Ministry of Innovation and the National Research, Development and Innovation Office within the framework of the Artificial Intelligence National Laboratory Programme. The authors have been also supported by the New National Excellence Program of the Ministry for Innovation and Technology from the source of the National Research, Development and Innovation Fund (ÚNKP-21-4 and ÚNKP-20-3). This work was prepared with the professional support of the doctoral student scholarship program of the Co-operative Doctoral Program of the Ministry for Innovation and Technology from the source of the National Research, Development and Innovation Fund.