Towards productive critique of large-scale comparisons in education

ABSTRACT

International large-scale assessments and comparisons (ILSAs) in education have become significant policy phenomena. How a country fares in these assessments has come to signify not only how a nation’s education system is performing, but also its future prospects in a global economic ‘race’. These assessments provoke passionate arguments at specialist conferences and in scholarly journals and they are just as passionately debated in the media. Within academe, ILSAs are researched by sociologists and psychometricians, policy experts and statisticians. This multidisciplinary, multi-voice discussion has not always served to highlight the complexity of the issues involved. Instead, discussions across various groups of actors have often led to a polarisation of views and a hardening of stances. Large-scale comparisons have deeply divided academic opinion with regard to their validity, usefulness and use. The divergence in ontological commitments, methodologies and paradigms of research makes discussions among one set of scholars almost incomprehensible to another. New theories, concepts and vocabularies are urgently required to engage productively with this important phenomenon. Borrowing concepts from Science and Technology Studies (STS) and the history and sociology of numbers, I argue that understanding such comparative exercises as socio-technical assemblages would move the critique of large-scale comparisons in education in more productive directions.

KEYWORDS:

• Global metrics
• international comparisons
• PISA
• socio-technical assemblages
• STS

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Notes on contributors

Radhika Gorur

Radhika Gorur is a Senior Lecturer at Deakin University, Australia, and a Director of the Laboratory for International Assessment Studies. Her research seeks to understand how some policy ideas cohere, stabilise, gain momentum and make their way in the world. Exploring contemporary practices of quantification and ‘evidence-based policy’ has been central to her research agenda. Using material-semiotic approaches and concepts from STS, she has been developing and contributing to a ‘sociology of measurement and numbers’ that makes explicit the instrumental and constitutive work of quantification, calculation and comparison in policy. She is currently engaged in a large, ARC-funded project focusing on global policy networks and accountability in low-income nations.