The impact of large-scale assessments in education on education policy: evidence from around the world

Introduction

We would like to extend a very warm welcome to readers of this special issue on the impact of large-scale assessments (LSAs) in education on education policy.

Over the past 50 years and following the International Association for the Evaluation of Educational Achievement’s (IEA) study of mathematics in twelve countries (Foshay et al. 1962), LSAs have become ubiquitous. In a recent systematic review of the impact of LSAs on education policy (Best et al. 2013), it was estimated that learning assessments are now being undertaken in about 70% of the countries in the world. For example, the Programme for International Student Assessment (PISA) conducted by the Organisation for Economic Cooperation and Development (OECD) was implemented in 75 countries in 2015 with around 510,000 participating students and their schools. Similarly, the Trends in International Mathematics and Science Study (TIMSS), conducted by the IEA, collected information from schools and students in 59 countries in 2015.

Given their widespread implementation one would assume that learning assessments are well-designed, well-implemented and are having a profound policy impact. Hence, the aim of this special issue is to examine the link between LSAs and education policy through examples from the Americas, the Asia-Pacific region, Europe and the Middle East.

Articles

The first article by Cristián Cox and Lorena Meckes traces Chile’s long-standing involvement in LSAs from one of a small number of countries participating in the early assessments in the late 1960s to one of the many countries participating in TIMSS and PISA as well as the regional assessment in South America today. The authors illustrate how these studies have been used in an ongoing programme of reforms in various aspects of the education system, including curriculum, learning standards as well as national assessments and teacher education. They emphasise that governments of different persuasions have supported Chile’s involvement in these assessments as the international perspective provides an external vantage point which is helpful in a relatively unequal society with a history of internal conflict.

Next, Esther Sui Chu Ho provides evidence of the various effects that participation in PISA has had in Hong Kong. Thus, for example, the assessment results have been used to support education reform in Hong Kong that focuses on student outcomes not only in terms of academic achievement but also in terms of non-academic outcomes such as students’ social and communication skills, self-learning ability, and global outlook. As PISA collects information on these aspects assessment results can be used to legitimise and monitor this reform. The author also provides insights into the ongoing efforts to move the attention of the mass media and public away from international rankings and towards changes in Hong Kong’s performance over time.

In her article about Italy, Valeria Damiani illustrates that participation in LSAs has led to the development of regulatory interventions and the reorganisation of upper secondary schools. Overall, however, the author notes a relative lack of impact, mainly as a consequence of a disconnect between policy formulation and policy implementation. For this process to be more effective, the author illustrates that changes in education policy need to be accompanied by changes in policies that address the social, economic and cultural differences in Italy.

In Jordan, participation in LSAs over the last 25 years has had numerous effects as illustrated in the article by Emad Ababneh, Ahmad Al-Tweissi and Khattab Abulibdeh. Not only have these assessments influenced strongly the development of school curricula and teacher training guides but they have also been used by funding agencies to evaluate initiatives and programmes in the education sector in Jordan. Despite this widespread use of LSAs, the authors argue that their impact could be even greater if a number of crucial aspects were improved, including student motivation to respond to the tests, training to analyse data from the assessments and funding security for the implementation of these assessments.

In their article about the experience in Mexico, Felipe Martínez-Rizo and Juana E.Silva-Guerrero not only reflect on international but also national LSAs. They describe how the national assessment with its initial formative purposes became a high-stakes test that was used to rank schools and to evaluate individual teacher’s performance without appropriate consideration of the context of specific schools, classes and students. The authors discuss that changes which have sought to address these issues have included giving Mexico’s National Institute of Education Evaluation (INEE) full autonomy and the creation of a new teacher assessment system. However, the authors argue that more time is needed before it is known whether or not these changes have led to a more appropriate use of results from LSAs.

For the USA, William H. Schmidt and Nathan A. Burroughs differentiate between using LSAs to enable research which informs educational reform on the one hand and to implement standards and enforce accountability on the other hand. As one example of major impact, the authors describe how the Third International Mathematics and Science Study (TIMSS) has influenced the development of standards in mathematics, called the Common Core, which have been accepted by as many as 40 US states – a major harmonisation in a decentralised system. Like other authors in this special issue, Schmidt and Burroughs express reservations regarding the use of LSAs for accountability purposes of individual schools and teachers due to concerns regarding the alignment of tests to standards and the appropriateness of background measures that seek to take into account differences in school contexts and student background when evaluating performance.

The final article in this issue by Mollie Tobin, Dita Nugroho and Petra Lietz seeks to summarise the impact of LSAs on education policy. To this end, the authors synthesise evidence from two systematic reviews on the topic, one conducted across all economically developing countries worldwide and one conducted across all countries in the Asia-Pacific region. As reflected in the contributions to this special issue, results show that LSAs mainly impact curricular reforms and performance standards. The evidence also indicates that LSAs are used by governments for accountability purposes and as leverage for certain political priorities, although these purposes are far outweighed by the use of LSAs for the monitoring of education quality and equity. Finally, again as illustrated by examples from various countries in this special issue, the studies found that high quality of assessment programmes and integration into policy processes facilitated the impact of LSAs on education policy whereas weak assessment bodies as well as financial constraints and funding uncertainty hindered impact.

Conclusion

The articles in his special issue illustrate the relationship between LSAs and education policy and the way in which the political, economic and social contexts of different countries shape this relationship. Examples from Chile, Hong Kong, Italy, Jordan, Mexico, the United States of America as well as from economically developing countries and countries in the Asia-Pacific show that results of LSAs have influenced mainly curricular reforms and performance standards. They also show how different political priorities as well as government and education system structures have facilitated or hindered the impact of LSAs on education policy. Finally, the examples illustrate that integration into policy processes, increased training in the analysis and reporting of results from LSAs to inform policy and funding certainty would contribute to achieving the aim of realising the full potential of LSAs.

Petra Lietz
Australian Council for Educational Research, Australia
[email protected]
Mollie Tobin
Australian Council for Educational Research, Australia
[email protected]

Acknowledgements

The Guest Editors gratefully acknowledge the Australian Council for Educational Research (ACER) and the Australian Department of Foreign Affairs and Trade (DFAT) for their financial support of this special issue.