Comparability of examination standards between subjects: an international perspective

Abstract

Heated discussions about the comparability of standards between examination subjects have kept Qualification Authorities, Testing Services, independent researchers and academics around the world busy for many years. As a result, many countries have adopted statistical techniques which aspire to make aggregated scores based on different subjects comparable. This paper presents eight international case studies and highlights the criticisms against the statistical comparability methods. The side‐effects (problems) they may have sparked in various countries are discussed in length. The author maintains the view that the Gordian knot in this case is not statistical: it is one of perceived unfairness. The Qualification Authorities and the Testing Services should step forward and explain to the stakeholders what can be done and what cannot be done using statistical techniques. Once the stakeholders accept the limitations of any statistical technique, it will be easier for them to strike a balance between the desire for comparability and the need to democratically allow students to choose their own combination of subjects to study. The psychometricians should not be forced to apply statistical solutions to complex problems caused by political decisions; they are more likely to fail than to succeed.