How to mess with PISA: Learning from Japanese kokugo curriculum experts

ABSTRACT

To remove cultural bias is critical for the legitimacy of Programme for International Student Assessment (PISA) as an internationally reliable academic assessment. Since its inception, PISA has made extensive effort to address this issue by putting in place a range of methodological and procedural measures to ensure its test fairness. This study attempts to disrupt the clean and reassuring accounts of PISA's technical solutions to cultural bias and exposes the irrationalities, tensions and messes that get muted in the rational and scientific discourse of test fairness. To this end, this article turns to Japanese kokugo (national language) curriculum experts as a source of critical insights. In particular, it looks at the way they responded to PISA's notion of reading literacy and then how those who were tasked to develop test items for reading literacy in PISA 2009 made sense of their item development experience. Their experience highlights the inherent cultural bias in the PISA's framework for reading literacy itself as well as the highly messy and coercive processes of PISA item development. In conclusion, I call for broadening the sources of critical insights beyond Anglo-American and select European countries to denaturalize the underpinning premises of PISA that remain unproblematized by Anglo-European critical education policy scholars.

KEYWORDS:

• PISA
• Japan
• reading literacy
• cultural bias
• performativity

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. Professor N, the interviewed Japanese representative at REG, believes that PISA's reading literacy and Japanese concept of dokukai (reading and comprehension) are so different that the use of the term dokukai in the translation of PISA reading literacy (PISA-gata dokukai) is inappropriate and misleading.

Additional information

Notes on contributors

Keita Takayama

Keita Takayama is an associate professor at the School of Education, the University of New England, Australia. He has published extensively in the areas of comparative education, sociology of education and curriculum studies. He has been instrumental in promoting a better understanding of the implications of globalization to education practice, policy and theory both within and beyond Australia. In particular, his work on OECD's PISA and its implication for national education policy making has been widely recognized.