Changing times, changing assessments: International perspectives

Introduction

In 2019, when we drew up the call for papers for this special issue of Educational Research, we could have no idea that it would coincide with a pandemic leading to worldwide school closures and educational disruption. The ‘changing times’ of the title anticipated a focus on innovative assessment developments, assessment of non-traditional constructs, e-assessment and reappraisals of equity and assessment concerns, with an international perspective on how these are influencing educational policy and practice in different ways around the world.

At the point of writing this editorial, it remains to be seen how far the educational changes wrought by the pandemic – exponential growth in online teaching and, in some countries, the abandonment of some ‘high stakes’ examinations, for example – are short-term responses to the immediate situation, only for practices to revert to ‘business as usual’ when the health crisis subsides. What has become clear is that the impact of the pandemic is not restricted to 2020, with plans already underway for a modified delivery of the 2021 examination series in some countries and an understanding that ‘catch up’ activities for students who have been impacted by the pandemic may be required for several years. The word ‘unprecedented’ can be used accurately in this context, as we have no evidence of how school closures of the scale brought about by the pandemic impact attainment (Education Endowment Foundation 2020). The Global Education Coalition (2020a) lists the adverse consequences of school closures on the lives of children and young people internationally and notes (2020b) how connectivity has become the critical resource that now enables access to education for many communities. There are ramifications for all sectors of education: for instance, Volante, DeLuca, and Klinger (2020) look across the world at how higher education is addressing the issue of admission to post-secondary institutions and identify how changes may be accelerated due to the pandemic; UK NARIC (2020) has published a similar and detailed study. One of the effects of the disruption to summative assessment and examinations resulting from the pandemic might be greater soul-searching about the purposes of some established assessments.

There is no doubt that assessment now, and going forwards, has a critical role to play in measuring the impact of the pandemic on learning – providing much-needed evidence to inform the global response to educational disruption. Such research, currently underway, should and must form the basis of future publications offering insight into the specific challenges that have been raised by the pandemic in myriad different teaching and learning circumstances.

A long, deep view of assessment and change

It is evident that the pandemic has exposed, or brought into sharp relief, fundamental and complex issues to do with assessment and change that predate it and that will continue to be significant in the post-pandemic educational landscape. The international collection of papers that form this special issue gives the opportunity to take that longer and deeper view: this is not the place to focus prematurely on the direct or indirect effects of the pandemic. Several of the articles reference the idea of dislocation between the desire for change, and actual change itself. This sense of tension has long been felt: Broadfoot (2009) claims that despite widespread recognition of the need for changes in approaches to assessment, in assessment use, and in the content and conduct of assessment, actual change remains elusive: examination and assessment practices around the world ‘remain strikingly similar to those that prevailed a century or more ago’ (vii). However, she does suggest that the twentieth century’s focus on increasing accuracy in measurement has been replaced by attention to assessment’s ‘fitness for purpose’ and impact on learning. In practice, of course, these are not alternatives and consideration of all these aspects is integral to the evolution of assessment, but we can see how different authors give different emphases. The evidence from these papers is that there are healthy signs of development in approaches to assessment, at the student, teacher and system level.

We see across the collection of articles how the field of assessment responds to and is shaped by the requirements of the contexts in which the assessments will be used. Each situation presents unique issues and challenges, and the papers reveal how these impact on the approaches adopted and influence the changes described. The scale of these contexts ranges from the application of assessment rubrics in one school, through to a multi-country study looking at the conceptualisation of a relatively new ‘21^st century skill’, learning to learn, and its impact on teachers’ approaches to assessment. And, in addition to the range in the scale, the geographical range is fascinating, with authors based in ten countries across five continents. A feeling of dynamism that pervades the accounts of research in this special issue certainly challenges any perceptions of assessment as a field that is slow to change; instead, the impression is of both transformation and reflection.

Themes of assessment and change

Whilst each of the eight papers in this special issue contributes its own unique, contextualised research story of assessment and change, the thematic connections between the papers provide further illumination. Sometimes, the points of contact reveal how challenges faced in one situation may, nonetheless, resonate in other, geographically distant locations. The studies by Coombs, Ge and DeLuca – From sea to sea: The Canadian landscape of assessment education and Stringher et al., – Learning to learn and assessment: Complementary concepts or different worlds? – explore teachers’ assessment literacy and, in both cases, call for change. For Coombs et al., the focus is on the assessment education of teachers preparing to enter the profession in Canada. They describe the expansion, in both quantity and range, of classroom assessment geared towards improving learning and point out the increased challenge for the profession. In their in-depth survey of institutions involved in teacher preparation, collecting evidence from interviews and document review, the authors find considerable variability across programmes and a diversity of approaches. They conclude by calling for more time to be available in teacher education courses for teachers to develop their own theoretical understanding and practice of assessment. Stringher et al. have a wider lens, with a review of qualitative evidence drawn from six country contexts: Brazil, Ecuador, Italy, Mexico, Spain and Uruguay. Here, the focus is on the understanding and pedagogy of experienced teachers in relation to the key skill of learning to learn (‘L2L’), identified as the ‘hyper competence’ of improving or updating one’s own learning. The authors had a particular interest in exploring the relationship between teachers’ understanding of L2L and their own classroom practice. Overall, they found limited evidence of formative practices, identifying a lack in the connection between assessment and L2L. None of the six countries allocated resource for L2L teacher training and there was variability in how L2L was integrated in national curricula. These authors, too, call for greater attention to be given to developing teachers’ assessment literacy whilst in training and also as on-going professional development.

Over time, there have been persistent calls for greater investment in developing teachers’ knowledge of assessment theory and practice whilst in training and in the early part of their careers (Brookhart 2011; Department for Education 2015, 2019), thus enabling them to become confident in their knowledge of assessment approaches, whether this is identifying which strategy for questioning is most appropriate in a particular lesson, or as critical consumers of reports on interventions, or as users of standardised test data. Black and Wiliam’s oft-cited review of classroom assessment (1998) called for teachers ‘to be provoked and supported in trying to establish new practices in formative assessment, there being extensive evidence to show that the present levels of practice in this aspect of teaching are low’ (61). With a greater level of professional skill in this area comes the potential for teachers to engage more confidently with the process of assessment and have more agency in the use of assessment in their classrooms.

Notions of teacher agency in relation to assessment and change are investigated in several papers within this collection. In Australia’s national assessment programme rubrics: An impetus for self-assessment?, Fletcher takes one of the assessments used in the Australian accountability system, the writing marking rubrics from NAPLAN (National Assessment Program - Literacy and Numeracy) and focuses on their use as a formative instrument in the classroom. She recognises this is an unusual approach but, as considerable thought and expertise go into the development of rubrics and scoring guides, it seems a valuable opportunity to develop the resource further. Taking on the dual role of practitioner-researcher, in introducing the rubrics in the classroom, Fletcher avoids the pitfalls identified by Brookhart (2011) where activities are inappropriately labelled as formative. She describes how teachers and students have agency in using the rubrics: how a variety of approaches was adopted and how teachers were able to adjust their planning to spend much longer than anticipated on the work. Fletcher also notes the greater confidence that the more experienced teachers showed as they adapted their plans to the specific needs of their students, reinforcing the importance of developing skills in assessment as part of teachers’ professional competencies.

In taking hold of the assessment material developed as part of an accountability system and using it not to ‘teach to the test’ (as the use of test materials is sometimes described), but rather as a vehicle to explore the development of writing skills, Fletcher encourages us to think more creatively about the assessment materials available, which are perhaps here better described as potential teaching resources. Elsewhere, Baird et al. (2017) note that evidence of the efficacy of assessment for learning tends to be qualitative; in this case, the method that was adopted results in a detailed consideration of writing rubrics and enlightening evidence about student and teacher responses to the tasks. In her discussion of intelligent accountability, O’Neill (2013) distinguishes between the primary and secondary purposes of assessment: here, Fletcher takes an assessment designed for accountability (a secondary purpose, in O’Neill’s categorisation) and, rather than it being used to hold someone or something to account, she assigns it a primary purpose – to improve teaching and learning. Whilst recognising the constraints imposed on the use of assessment data by teachers as a result of many countries’ accountability systems, opportunities often remain for constructive use of the resources, as Fletcher demonstrates.

The rapid movement of educational ideas from one setting to another can be a powerful driver of assessment change. The paper by Heng, Song and Tan – Understanding the interaction of assessment, learning and context: Insights from Singapore – reminds us of the perils of ‘policy borrowing’, or identifying initiatives or features of the school system apparently successful in one context, introducing them to another and assuming they will have a similar impact. The influence of Singaporean educational practices in other cultures is widely recognised and is largely due to Singapore’s consistently high performance on the international large-scale assessments. The city state is often described as having been transformed in a single generation ‘from a developing country to a modern industrial economy’ (Schleicher 2018, 24). Heng et al. reflect on how assessment for learning approaches, often lauded, are operationalised in a limited way in Singaporean classrooms. They point out that the context in which the teachers work is much more nuanced than is sometimes suggested; that assumptions around assessment for learning as ‘best practice’, transferrable across diverse educational contexts, need to be cognisant of how practice is situated within a wider context. This paper also draws our attention to the way in which educational policy–making is endeavouring to drive cultural change, with the Singaporean Ministry of Education announcing plans to reduce the pervasive influence of assessment of learning and to develop students’ ‘joy of learning’ and ‘learning for life’. Schleicher (2018, 24) suggests this is an established and successful initiative with interest in ‘nurturing creative and critical thinking, social and emotional skills, and character qualities’ and with some success being manifested through, for example, Singapore’s high performance in PISA’s first foray into this area with the assessment of creative problem-solving skills in 2012. Reform is ongoing and, in 2019, the Ministry of Education announced that there would be a phased removal of streaming and the removal of some tests (The Straits Times 2019).

Several of the papers refer to the intended and unintended consequences of educational policy reform on assessment processes and practices. In How do teachers respond to assessment reform? Exploring decision-making processes, Walland and Darlington describe teachers’ responses to the reform of a part of the qualification system in England, a system in which a series of high stakes assessments is integral. This reform reduced the modularity of the system and focused on creating a linear approach where the examinations (A-levels, which are key to university admission) are taken at the end of a two-year course of study. The top-down reform was driven by a political perspective that ‘standards’ – the demand of the assessments – had fallen, but some decisions in the implementation of these changes were placed in the hands of teachers. Walland and Darlington’s evidence, from interviews with teachers who were involved in the decision-making around their schools’ transition to the new arrangements, reveals a divergence of responses. Teachers arrived at their different decisions by weighing up the relative benefits to different stakeholders but also often having limited choices in practice. The authors point out the constraints on teachers’ decision-making, where decisions resulting in reduced choice for students may be the result of budgetary considerations. Other teachers, who did not reduce student options, felt that the perceived benefits outweighed the financial cost. Another motivation to retain the modular approach was identified as resisting the perceived negative effect of the change on the school and the authors suggest this may be due to opposition to the way in which the reform was introduced. These different approaches reflect the dilemma of teachers working within the constraints imposed by both their schools’ internal structures and national assessment reform. Decisions which are ostensibly taken on pedagogical grounds may, in fact, be driven by factors outside the teachers’ control, such as the resourcing implications.

Throughout many different assessment contexts around the world, we see a growing interest in measurement that goes beyond subject-based assessment to areas variously described as life skills, 21^st century competencies, non-cognitive or soft skills (Broadfoot 2009). Ahonen and Kankaanranta (2015), for example, describe the situation in Finland, and Bujanda and Campos (2015) that in Costa Rica; PISA 2018 included a survey of students’ global competence (OECD 2020) and in 2015 a module on collaborative problem–solving was included (OECD 2017). Alongside specific ethical considerations related to the measurement of life skills, we can expect development and analytical methodologies to evolve as assessment of this diverse area becomes more prominent.

An awareness of the considerable methodological demands involved in the development of life skills assessments is reflected within this special issue. In ‘Information Literacy’: Japan’s challenge to measure skills beyond subjects, Shinohara and Horoiwa describe the design of a national assessment in a relatively novel domain. They give detailed insight into the conceptualisation and development of a computer-based test of information literacy for secondary pupils in Japan. This is conceived as cross-curricular and the foundation of learning, with three aspects: ‘practical skills of using information’, ‘scientific understanding of information’, and ‘attitude to participation in the information society’. The authors emphasise how this is clearly broader than the assessment of computer literacy skills. As with all assessment development that breaks new ground, considerable attention is given to defining the assessable domain and the analytical methodologies required. In this regard, Shinohara and Horoiwa note that the analyses undertaken for the international large-scale assessments (ILSAs) provide useful pointers, although there are specific challenges raised in ILSA development due to the diverse range of jurisdictions, and therefore cultures, in which the surveys collect data. The importance of developing sophisticated measures to assess these skills is shown in the results from the IEA’s International Computer and Information Literacy Study (ICILS) (Fraillon et al. 2020), which found a relationship between students’ socio-economic background and their computer and information literacy skills. In the results of that survey, it was evident that these skills were not developed just by being part of a generation that was exposed to technology unknown to previous generations.

Another challenge within the area of life skills assessment is how to measure the construct of wellbeing. In Broadening assessable domains: Lessons learnt from the development of a wellbeing survey, Kuhn, Fletcher and Bradshaw provide considerable detail about the steps undertaken in England to develop a robust survey in order to be able to evaluate young people’s perceptions of the impact on their wellbeing of participation in an out-of-school activity and award programme. As Shinohara and Horoiwa do, they build on previous measures and clearly describe the assessment domain of focus, which is particularly relevant when the constructs are less well defined than many subject-based domains. They recognise the strengths of self-report measures, widely used, but also their limitations. As interest in assessing these constructs continues to grow, rigorous development work building on a sound theoretical base and clearly defined assessment domains will be ever more important if we are to have confidence in the resulting measures and to ensure that the use of the results is valid.

An understanding that equity in assessment is inextricably linked to children’s right to access quality education is implicit in the accounts of assessment and change presented in this special issue. The final paper in the issue, Relationships between the assessment of school quality and social justice, deals with these concerns explicitly. In this study from Portugal, Sampaio and Leite look at how the means by which school effectiveness is evaluated can challenge fundamental issues of social justice. As such, it explores some of the tensions inherent in educational accountability where external factors influence behaviours within schools. The authors identify the influence of the wider European context on the evolution of educational policy in Portugal and its focus on equity and social justice. As both Sampaio et al. and Marôco (2020) note, the arrival of democracy in Portugal in 1974 led to a consensus, politically and socially, around the importance of education for the country’s social and economic development. In their paper, Sampaio and Leite go on to discuss whether the potentially contradictory demands of equity and accountability can be reconciled, focusing on two particular and influential policies, each including schools’ self-evaluation. Through interviews, the authors compare the perceptions of these policies between schools with high and low classifications of school quality (determined by a combination of analysis of outcomes and self-evaluation) and suggest a tension in the implementation of policies which are designed to contribute to social justice and which, in assessing school quality, have a focus on student attainment.

Assessment, change and the future

The pandemic, and its legacy, may accelerate the speed or direction of change in educational assessment in ways that are not easy to predict. The discussion of equity above brings us back to our opening considerations about the educational consequences of the pandemic. One of the impacts that many are very alert to is the likely widening of the attainment gap between those who are and who are not economically disadvantaged within a country – increasing inequity. In tackling these global problems, high-quality assessment will have an important role to play in providing robust data to shape, and challenge, the educational policy agendas within and across countries. In the months and years to come, the themes so cogently explored through the papers in this special issue – teacher agency, assessment literacy, formative assessment, life skills measurement and social justice – will continue to have relevance and resonance, as assessment researchers collaborate to support the educational opportunities of learners throughout the world.

Disclosure statement

No potential conflict of interest was reported by the author(s).