COVID-19 impact on high stakes assessment: a New Zealand journey of collaborative adaptation amidst disruption

ABSTRACT

New Zealand’s defined coastal boundaries, isolation and small population were favourable factors to minimise the spread of COVID-19. Decisive governmental leadership and a public willing to comply with high-level lockdown in the first phase, resulted in minimal disruption to assessment. But as the pandemic progressed through Delta and Omicron variants, concerns grew about equitable access to assessments, declining school attendance, and inequitable educational outcomes for students, especially of Māori and Pacific heritage. School and educational agency experiences of high stakes assessment in a period of uncertainty were examined through document analysis and research interviews. Using Gewirtz’s contextual analysis of the multi-dimensional and complex nature of justice, and Rogoff’s conceptual framework of three planes of socio-cultural analysis: the personal (learner), inter-personal (school) and institutional (educational agencies), revealed that though collaborative adaptations minimised assessment disruptions on wellbeing and equity of access, they did not transform high stakes assessment.

KEYWORDS:

• New Zealand
• COVID-19
• high stakes assessment
• senior secondary school
• interprofessional collaboration
• wellbeing

Introduction

In 2020–22 the COVID-19 pandemic spread rapidly around the world, causing most nations to close schools in attempts to limit infection rates. Consequential effects of closures and changes to learning and teaching, particularly in high stakes assessments for senior secondary school students, have been of international concern (e.g., OECD, 2020). High stakes assessments are commonly used across jurisdictions to serve several purposes: accountability of educational institutions, including their teachers and students; assurances of dependability of the assessments and any related certification; preparation of young people for the workforce and citizenship; and student selection for study or work (Woods et al., 2019). High achievement opens opportunities to desirable educational pathways and employment while failure and lower grades reduce eligibility or the range of opportunities for education and employment, often restricting renumeration, and leading to poorer working conditions and health outcomes (UNESCO, 2020). High stakes assessment is often thought to be reliable, dependable (devised by curriculum and testing experts), useful (yields publishable data for statistical analyses), and fair in rewarding achievers on merit, supposedly devoid of political influence (Cairns, 2020; Phelps, 2005). Governments and educational agencies invest in making the systems as resource and time efficient as possible.

Even prior to the global pandemic, there were increasing concerns about the deleterious effects on student learning and wellbeing (Rodway et al., 2016). Concerns about high stakes assessment arise because, ‘there is a direct link between the tests and rewards and sanctions for students, their teachers, or institutions’ (Elwood et al., 2017 p. 3). From a learner perspective, the higher the stakes of assessments seem, the more likely they are to ‘invest in more instrumental approaches to learning’ and ‘playing the [examination] game’ (Smyth & Banks, 2012, p. 293), with effects on student motivation for learning, as well as equity of access and outcomes. Of particular concern is the ‘backwash’ effect wherein narrow and superficial teaching and learning occur because teachers and students divert attention to improving test taking skills and test formulae (Cairns, 2020). Klenowski and Wyatt-Smith (2012) advocate for a move towards ‘a richer and comprehensive set of achievement indicators for student learning’ (p. 75), in which teachers’ assessment capabilities are enhanced and valued, for it is teachers who are most influential in improving learner outcomes. Yet high stakes assessments continue. Woods et al. (2019) acknowledge that ‘examination systems are established [and perpetuated] by those in power within government, examination regulators and awarding organisations’ (p. 89), with inherent tensions in the system to balance levers for ongoing credibility of the qualification and dependability of the processes (Elwood et al., 2017; Woods et al., 2019). The global pandemic may have altered the balance of these levers, so this paper asks what impact has the pandemic had on high stakes’ assessments in New Zealand?

A theoretical framework underpins analysis of the interplay amongst these levers, combining firstly the conceptual work of Gewirtz’s (2006) notions of social justice and, secondly, Rogoff’s (1995) socio-cultural planes of analysis: the personal, interpersonal, and institutional in the social practice of assessment. Notions of social justice concern educationalists because analyses of national and international assessment data reveal patterns of inequitable achievement for people across socioeconomic groups and other groupings such as race, ethnicity, gender, and disability (e.g., OECD, 2020). Acknowledging the intersectionality of group identity, whereby people belong to and derive their identities across affiliations with multiple groupings, and hence the need for caution in categorising individuals or families, those with a concern for social justice strive to address issues of exclusion or unfair treatment (Curtin et al., 2019).

Addressing social justice issues is complex because of embedded issues of power, distribution of resources and fundamental beliefs and values about human and economic capital. Gewirtz’s (2006), theoretical framework encompasses two concepts of social justice: distributive and relational justice. Distributive justice refers to the distribution of wealth, power, and privilege across society, while relational justice advocates for communal sharing of resources and valuing social relationships in a non-hierarchal manner. These concepts have relevance for assessment, since traditional assessment certifications (high stakes assessments) have ‘reproduced hierarchies in terms of race, class and gender intersections in twenty-first century education practice’ (Curtin et al., 2019, p. 84). Such patterns are discernible in international testing regimes, such as PISA (Schleider, 2019), and affect subsequent life chances for students in terms of eligibility for further education pathways and employment opportunities. From a relational justice perspective, there is an opportunity to confront established hierarchies and focus on valuing diversity (Curtin et al., 2019), interpersonal relationships and mutual or shared benefits, rather than competitive systems of winners and losers.

Rogoff’s (1995) conceptual framework offers a means of understanding and applying the goal of relational justice in educational assessment through three planes of analysis – the personal (the learner), inter-personal (relationships in the context of the school) and institutional (educational agencies and policymakers). Within and through the intersectionality of these planes, there may be possibilities for shifts in the understanding, and transformation, of high-stakes assessment in the uncertainties of the pandemic.

The paper begins with a brief background on the New Zealand education system and the secondary school qualification, before contextualising the NZ COVID-19 experience. The paper examines literature on the complexities arising from COVID-19 on high stakes assessments before turning to an analysis of NZ assessment experiences during a pandemic, doing something rarely brought together – the experiences and perspectives of the school, educational agency, and educational policy maker. Further analysis in relation to the theoretical framework that combines the work of Gewirtz and Rogoff, leads to the argument that though cross-sector collaboration creates opportunities for transformative actions and reduces the disruptive effects of a pandemic on high stakes assessment results, the capacity to transform assessment for distributive and relational justice is limited.

Brief explanation of New Zealand’s education system

New Zealand has a decentralised education system, in which the governance and management of schools is the responsibility of each school’s Board of Trustees (BOT; McMahon, 1996). The BOT is the staff employer, and is comprised of members elected from the community, the principal, student and staff elected representative and co-opted members where required, and a proprietors’ representative for state-integrated schools (e.g., Education Government, 2022). ‘The role of the state through the Ministry of Education is that of property owner, policymaker, funder and regulator’ (McMahon, 1996, p. 2). While the state stipulates curriculum requirements (such as the New Zealand Curriculum), schools have responsibility for delivery of it, and educational matters like student assessment processes and procedures. Schools do so in relation to the NZ Curriculum and the New Zealand Qualification Authority (NZQA). The senior secondary school qualification, the National Certificate of Educational Achievement (NCEA), aligns with the first three levels of a national 10-level framework, known as the New Zealand Qualifications Framework (New Zealand Qualifications Authority, 2022). NZQA administer the New Zealand Qualifications Framework (NZQF).

The education system has four levels: early childhood education (from birth to school entry age), primary (aged 5–11 years) and secondary education (from 12–19 years of age), and further education (higher and vocational education). Although school entry in NZ is compulsory from the child’s sixth birthday, commonly children start school on or soon after their fifth birthday. Important to understand is the range of provision offered from senior primary through to junior secondary. While most students finish primary school around age 11 (School Year 6), they have five options at Year 7: full primary (Years 0–8), intermediate school (Years 7–8), composite school (Years 7–13), junior secondary school (Years 7–10), or area school/kura (Māori immersion) (Years 0–13). Most secondary schools cater for students in Years 9–13, with exceptions being Junior High Schools (Years 7–10), and Senior High Schools (Years 11–13). Students may leave school prior to Year 13, but not until after their 16^th birthday.

Most schools in New Zealand are government-owned and fully state funded, and largely co-educational (MOE, 2022a). State-integrated schools (354 of the 2536 NZ schools) have a special character related to a specified religion, philosophy or set of values (such as Māori medium schools affiliated with a particular iwi – Māori tribal group). Kura Kaupapa Māori schools (72/2536) have a Māori philosophy and Māori is the language of instruction. While there are private schools, they are smaller in number (<5%), as are various other school types serving health, social or locational needs (e.g., Correspondence Schooling for students living in remote areas).

National Certificate of Educational Achievement (NCEA)

The NCEA is NZ’s education qualification, taught in the last three years of secondary schooling, though also available for foundational and vocational education and training. NCEA is comprised of three levels of increasing difficulty. Students are required to achieve a prescribed number of credits to gain an NCEA Certificate in the respective levels, varying from 80 credits in Level 1 to 60 credits at Level 3 (in addition to 20 credits from level 2 or above), including literacy and numeracy requirements (NZQA, 2021). There are two types of standards: Achievement Standards and Unit Standards. Achievement Standards assess skills and knowledge associated with the National Curriculum, while Unit Standards encompass broader knowledge and skill, often related to vocational education and training (Yates & Johnston, 2018). Credits are awarded once evidence is gathered on achievement of specified standards. Standards are descriptions of the knowledge or skills that students are expected to achieve to meet the standard. Credit values are given to each standard (typically around 3–5 credits), with ‘most curriculum courses assessing five to eight achievement standards’¹ (Yates & Johnston, 2018, p. 639).

There are two types of assessment: external and internal. External assessments include examinations and portfolios (the latter for performance type tasks and subjects) and are assessed by NZQA, either on ‘paper’ or, increasingly, digital form. Internal assessments include skills and knowledge that are performance based like speeches, projects, group work, physical activity, and performing Arts. ‘Internals’ are credited on documented assessment evidence and decisions made by teacher judgements on standards criteria. Students receive credits at differing grades of performance – achieve (pass), merit and excellence. Students who accumulate sufficient evidence at these varying grade levels may have their courses, and their Certificate, endorsed accordingly (NZQA, 2021b).²

Considerable professional development is invested in teachers to build assessment capability and adherence to the expected processes and standards, which NZQA monitors for approval (NZQA, 2020a). Informal and formal professional learning opportunities related to NCEA assessments are provided by a range of providers, including NZQA (2020, p. 76), who offer a variety of e-learning modules and in-person workshops from subject-specific topics to support for assessment and assessor moderation. These processes yielded a 91.3% agreement rate for internal credits in 2019, and 81.8% for grade (NZQA, 2020, p. 75).

NCEA is highly regarded (e.g., Yates & Johnston, 2018) for its high proportion of school-based assessment to national examinations, broader ways of assessing learning, opportunities for formative assessment and attention to learning compared with traditional assessment for qualifications. Nevertheless, tensions remain between formative and summative purposes. Nationally, striving for higher attainment rates pushes teachers towards summative purposes and students towards gaming credits, compromising attention to wellbeing and deeper learning – a point to which the concluding sections of the paper return.

International perspectives of COVID-19 impact on high stakes assessment

Internationally, school closures have resulted in learning gaps for students, reduced coverage of school curriculum and achievement (OECD, 2020). Nations, like those in the UK and Sweden (Wiberg et al., 2021) that delayed or cancelled external exams in the pandemic incurred considerable political and public angst (Cairns, 2020). Reliance on standardised assessments reduced responsiveness to students’ interests and the global realities of the pandemic (Cairns, 2020), but also revealed tensions in the policy-making space: multiple demands for quick decision-making, assurances of the credibility of the qualification system, equitable opportunities for entry to tertiary study in Sweden and across the UK, realism about fewer resources required for technical standardisation of student achievements than upskilling for dependable teacher judgements, and ‘caving in’ to public outcry to subsequently rely on teacher derived grades (Kippin & Cairney, 2021). Other nations that pivoted to online examinations, such as the US, encountered a range of measurement and proctoring challenges, but mostly user hesitancy (Camara, 2020). In short, entrenched reliance on standardised high-stakes assessment limited flexibility to respond to the COVID-19 crisis.

COVID-19 in New Zealand

New Zealand’s Covid experience differed across the, to date, three years of the pandemic, which the author characterises as:

• Phase 1–2020 (arrival of Covid-19) transformative and united

• Phase 2–2021 (Delta) – adaptive

• Phase 3–2022 (Omicron) – disruptive diversity

This section briefly describes the public health and political environment of each phase to provide critical contextual information before turning to the central focus of the paper – the Covid impact on senior secondary school assessment.

Phase 1: transformative and united response

Geographically isolated in the Southern Hemisphere and surrounded by sea, New Zealand was able to close its borders, the envy of many nations, and delay the arrival of the virus until 28 February 2020. This delay meant NZ had time to learn from other nations. It was summer – a season when people spent time outside which, combined with low population density (five million people on a similar land area to the UK, 268,838 sq. km), readily allowed for social distancing.

These location, temporal and seasonal factors coalesced with several political and economic factors to create a distinctive and short-lived experience of the virus in 2020. A centre-left government who sought information from epidemiologists, other medical experts, modelling by mathematicians, and support from the business community, enabled educative messaging to be conveyed to the public (Wilson, 2020). A NZ alert level system was devised which comprised four increasing levels of restriction in movement, social contact, and economic activity up to Level 4 ‘Lockdown’ (except essential workers, everyone remained at home in their ‘bubble’, only permitted to leave for essential groceries or medical supplies, and local exercise). The ‘bubble’ comprised only those people with whom one lived under the same roof. Visiting relatives, friends, etc., was strictly prohibited. Compliance was assisted by daily Prime Minister and Director General of Health messaging around, ‘a team of 5 million’, ‘be kind’, frequent reminders of the restrictions, and supportive economic packages (such as wage subsidies of up to 80%). NZ subsequently endured five weeks of Level Four national lockdown, followed by two weeks of Level Three restrictions.

Educationally, apart from children of essential workers, all students were required to learn from home until the country returned to Alert Level 2 (early June 2020).

Phase 2: adaptive response

The arrival of Delta in 2021 led to further national lockdowns (August 2021) and additional lockdowns of the northern regions of NZ. The government’s strategy had been one of the elimination until sufficient levels of vaccine coverage were reached (Dyer, 2021). After unsuccessfully trying to eliminate the virus, partly due to the virus circulating amongst marginalised populations and delayed national vaccination campaigns (second half of 2021), the nation moved to a response of suppression and reduction of spread. Mask wearing, contact tracing, scanning in to record movements so that virus spread could be monitored, and close contact isolation were strategies for adapting to the virus. Weariness, and wariness, grew around state health mandates that required vaccine passes for entry to public amenities and eligibility for professional practice in the health, education, and social services sectors. Community factions emerged that resisted central government restrictions, and tensions surfaced between collective rights to health safety and individual rights to freedom and choice.

Phase 3: disruptive and diverse response

Soon after the arrival of Omicron variants in NZ in December 2021, the government moved to a strategy of slowing, and learning to live with the virus. Mandates were gradually dropped, except for the wider health sector. While mask wearing and vaccination boosters were recommended, there was a lessening of government decrees and a shift to greater individual responsibility. But, as occurred internationally, the Omicron variants took hold in the community, and amidst a greater prevalence of flu and other respiratory illnesses, the NZ health sector came under enormous pressure, as did schools and businesses, with significant disruption to continuity of services and daily life until August 2022. The paper turns now to examine the impact on senior school assessments.

Materials and methods

The analysis in this paper is based primarily on a critical review of literature and supplemented by interviews conducted by the author of educational agency personnel in the Ministry of Education and New Zealand Qualification Authority, as well as teachers and students in a regional secondary school. The purpose of the interviews, based on the theoretical model previously described, was to provide explanatory insights into processes and experiences that were not evident in published works.

For the document analysis, the University library databases ‘Discover’, and ‘Google Scholar’ were searched, followed by ‘Education Counts’, ‘ERO Reports’, and ‘NZQA Reports’. Keywords included: COVID-19, high stakes assessments or examinations, New Zealand.

To increase validity of the small case study design (Yin, 2018), perspectives were sought from different types of participants: policymaker (MOE), policy enactor (NZQA) and school personnel (teachers and students). A regional, middle decile school was invited to participate because its students comprised a range of socio-economic home backgrounds, ethnic representation, a mix of rural and urban dwellers and students with a range of educational achievement. The school was a Year 9–13 school, with a roll of 1514, 68% European, 26% identified as Māori (Māori comprise 16% NZ population), 4% Asian and 2% Pacific, and included a hostel of 180 boarding students (ERO, 2017). Teachers and students who had been associated with NCEA Levels 2 or 3 in 2020 and 2021, were invited to participate in research interviews. Fourteen teachers were interviewed representing a range of experience (from <5 to >20 years), school roles (teacher, Year Level Deans, Heads of Department, Specialist Classroom), and subject areas (Mathematics, English, Te Reo Māori, Geography, Social Studies, Sport and Physical Education, Sciences, Media Studies and Hard Materials Construction). Semi-structured interviews were conducted, whereby the same questions were asked of all interviewees, with some flexibility to explore matters raised by interviewees in more-depth.

Two educational agency experts in senior secondary school assessment from the Ministry of Education (MOE) and NZ Qualifications Authority (NZQA) were invited and agreed to participate. Interviewees verified interview transcripts before the data were analysed. The researcher analysed the interview transcripts through three cycles of process coding, categorisation, and theme development (Saldana, 2011). Though the limits of generalisability from one school and two agency representatives are acknowledged, they nevertheless provided insights into the lived experiences of the ‘impact of COVID-19 on high stakes assessment in New Zealand’. Full university ethics approval (SOB: 21/38) was given, and voluntary, informed consent sought from participants.

Results

This section reports findings in relation to three phases of the pandemic:

Phase 1: Collaborative and transformative

Phase 2: Adaptative

Phase 3: Disruptive and diverse

Phase 1: collaborative and transformative assessments

Closing schools led to variations in access to digital capability with implications for assessment, and impacts on achievement. This section of the paper firstly considers the national impact with a brief analysis of literature followed by educational agency and school interview data to explain the national data and how collaborative efforts led to some tentative transformations.

Digital capability and access to learning and assessment

Access to digital devices and internet connectivity are fundamental components of equity for digital learning and assessment (Hunia et al., 2020), yet up to 50% of their 2,684 respondents ‘reported lack of access to digital devices and tools’ (p. 10), or restricted sharing during national lockdown:

I am currently using my parent’s digital devices, neither of which have the capability to allow me to use a mic or my headphones to engage in Zoom conferences with my teachers. This has been frustrating and interfering with my online classes. (Hunia et al., 2020, p. 11)

In the lockdown context, fewer than half of the NCEA students had limited or no access to devices other than smartphone, meaning reduced opportunities to access learning and assessment. These findings were validated by a nationwide Education Review Office survey (ERO, 2020) and a university study (Yates et al., 2021, p. 70).

Achievement

While reports of learning and assessment indicated general decline, in terms of attainment of NCEA, all Year levels surprisingly showed percentage increases of numbers attaining their respective levels in 2020 compared with 2019, [L1 71.8% c.f. 70.6%; L2 80.1% c.f. 77.5%; L3 72.1% c.f. 67.3%; UE 53.4% c.f. 49.3%] (NZQA, 2021, p. 10). However, merit endorsement levels dropped slightly in 2020 compared with 2019 for levels 1 and 2 [L1 32.3% c.f. 33.9%; L2 24.9% c.f.25.1%; L3 26.3% c.f. 26.2%], while excellence endorsement levels showed minor increases [L1 20.7% c.f. 19.9%; L2 17.9% c.f. 16.7%; L3 17.5% c.f. 14.8%]. Ethnically, there were some interesting patterns, as seen in Table 1 (adapted from NZQA, 2020, pp.13–14).

Table 1. New Zealand NCEA Attainment Data 2018-2021 (NZQA, 2022).

Ethnicity	NCEA Level 1				NCEA Level 2				NCEA Level 3
	2018%	2019%	2020%	2021%	2018%	2019%	2020%	2021%	2018%	2019%	2020%	2021%
Asian	76.9	73.9	73.1	70.0	77.3	78.3	80.0	81.1	70.5	71.3	76.5	76.2
European	78.0	76.0	75.8	74.0	81.5	81.1	83.2	81.2	69.9	70.8	74.6	73.2
Māori	58.4	57.7	60.8	57.7	68.6	68.9	71.9	68.3	52.9	55.1	60.7	58.5
Pacific Peoples	62.8	61.8	68.2	62.3	72.1	71.3	77.1	71.5	58.9	60.3	68.9	64.9

Table 2. Summary interview data.

Question area	MOE	NZQA	Teachers	Students
Experiences	Meetings Communications to/with schools Interventions; adaptable qualification system	School anxiety Frequent communications with schools; clarified standards; we encouraged evidence gathering; permission; re-assurance	Variable student engagement; 1/3 thrived, 1/3 survived, 1/3 ‘dived’. Teachers changed from usual class time availability to fewer synchronous & more asynchronous teaching sessions	Cool sleeping-in; missed mates; bored; Parent lost job; grandparent got sick;
Challenges or difficulties	Wellbeing Learning	Helping schools prioritize assessment tasks; tried to remove barriers; delayed issue of devices	Students with restricted learning - limited internet or no/shared device. Delivered printed copies & loaned devices. Adjusted teaching – difficult for practical subjects or complex concepts	Missed extended family; too much noise & distraction to focus on assessments; no device (could not afford it); students not going on-line; Teachers varied in helpfulness & interesting learning
Benefits	Opportunities for transformation. Permission to schools to innovate & adapt. Accelerated digital learning & assessment.	Opportunities to co-construct solutions; trial ideas, mutual support schools-NZQA; cross-agency collaboration; deepened school knowledge of assessment & curriculum	Advanced organization with resource preparation; improved digital pedagogical skills; greater student appreciation for teachers; personalized, active learning	Liked learning later in the day and when I felt like it. Could spend all day on one subject.
Issues	Equity/inclusion	Slower response & delivery; delayed arrival of devices;	Equity - internet connectivity, shared or no devices, family & financial distractions from learning. Wellbeing	Missed so much learning; hard to catch-up; missed friends and teacher explaining things beside us
What helped learning, teaching, assessment	Listening to sector; across sector conversations; resource provision; permission to do things differently	Learning Recognition Credits (LRCs); inter-agency discussions MOE, NZQA, Universities NZ; clarity about assessment for learning; quality conversations with schools	LRCs, fewer credits = more time for deeper tg & lg. Team teaching. Adaptive & responsive pedagogy. Supplementary resources, more flexibility with assessments.	Pushed out timing of assessments; a few received devices to use at home; kind or understanding teachers; a few had whanau members to help or explain learning
Decision making	Co-construction Multi-agency Across sector	MOE, NZQA, Universities NZ, teacher unions, experts, advisory group, students	Senior management team; some joint departmental decisions; pedagogy mostly by teacher	Teachers gave extra time to submit work
Resources	Home TV Print materials Device distribution Correspondence School	Ongoing online professional learning. Matrix to guide teacher judgements. Frequent communications to schools by SRMs	Increased access to e-books, or commercial product; device distribution, helpful NZQA, MOE links. Suggested future practical home kits	Home TV; teacher Zoom links; messages; school internet for learning
Changes	LRCs; credit limit changes; timing of Term 1 school holidays & exams	LCRs; reduced credit number to award qualification; adjustments for 2020/21 only;	Switched order of topics, internal & external assessments; adjusted pedagogical approaches	So much online learning – tired of that way of learning; boring; some teachers okay others not much help
Impact	LRCs increased motivation to stay learning/at school; similar distribution of NCEA attainment; resilient system with internal assessment flexibility – learning impact unknown	Improved outcomes for diverse groups; better results than anticipated though varied across school types & greater variability in Auckland; broadened teachers’ conceptions of assessment	LRCs reduced pressure & anxiety on students & teachers. Some focused on understanding or deeper learning; others on essential learning for credit counting. More students left school early.	Still got credits I needed, but not sure I understand the learning. Cool to get credits for real learning in the community helping whānau and with essential work

In brief, all ethnic groups improved their respective NCEA level attainment percentages from 2019 to 2020, except Asian students at L1 who dropped marginally from 73.9 to 73.1%, and European students at L1 minimally from 76.0 to 75.8%. However, comparisons across ethnic groups revealed discrepancies in attainment, especially for Māori and Pacific Peoples.

The paper now turns to agency and school interview data to explain how limited access to digital resources and reduced assessment opportunities resulted in increased attainment levels. One influential factor was extraordinary inter-agency/sector collaborative efforts to minimise the impact of COVID-19 on wellbeing and achievement. (Refer to Table 2 below for a summary of interview data)

The MOE as the policy agency, NCEA qualification owner and standard setting body, worked in partnership with NZQA and representative personnel across the education sector to ensure NCEA continued as a robust and dependable qualification. Their greatest concerns were about student wellbeing and continuation of learning and assessment, so the MOE resourced Home TV learning, additional digital devices, learning packages to schools, and ‘published hundreds of documents to support schools to give students recognition against actual assessments’ (MOE Int.). Furthermore, the NZQA had frequent communications with schools providing advice and permission for innovative adaptations and increased uptake of digital practice exams.

We tried to remove barriers for assessment and utilise the flexibility of the qualification to elicit evidence across classrooms without interrupting it for testing. Much of what we did in COVID-19 affected times was permission giving. Often it was clarifying requirements of standards –could they gather evidence over time for sufficiency and use that to assess students [instead of formal assessments](NZQA Int.).

The MOE and NZQA sought to acknowledge hardships, ensure students were not penalised, and ‘motivate students to remain engaged with school and to help them see that qualification attainment was doable’ (MOE Int.). They sought widespread input into creating viable adaptations. Collaborations across MOE and NZQA, and various sector representatives (teacher unions and principal/school associations – PPTA, SPANZ, Ngā Kura a Iwi and te Rūnanga nui o ngā Kura Kaupapa Māori, Universities NZ, advisory groups, and students), resulted in sector acceptance of co-constructed solutions of learning recognition credits (LRCs), and reduced entry requirements for University Entrance and degree entry. For every five credits a student accrued, they were eligible for an additional credit, up to a maximum of 10 LRCs for Level 1 and 8 LRCs for Levels 2–3. Similarly, there was a total credit reduction required for eligibility for Certificate (46 c.f. 50) and course endorsements (12 c.f. 14; PPTA, 2021). NZQA developed matrices to provide pedagogical guidance and to inform assessments e.g. recognition of extra-curricular learning during lockdown, and an employability skills framework (NZQA, 2020), with scoping examples to support evidence gathering and interpretation. The NZQA interviewee believed, ‘the system supported students to meet thresholds, and overall results were better than we anticipated, though varied across school types and regions’.

School-based experiences and impact

Teachers were asked about their experiences of teaching and assessing during COVID-19. A focus on student and staff wellbeing dominated interviews, as seen in two illustrative quotes:

Under our principal the focus was not on good grades but on the wellbeing of our kids, which took pressure off us. He contacted us weekly and wanted to know how we were. (T9)

Some faculties, mostly humanities, languages, and arts adapted their programmes to prioritise student wellbeing. (T13)

Teachers and students reported variable student engagement (about 1/3^rd ‘thrived’, 1/3 ‘survived’, and 1/3 ‘dived’).

About 6/15 students were engaged daily – asked for work, did their learning and assessments. Some did not engage at all, due to low income or no laptop until they got one through the school. (T9)

Two classes of top academics produced the best work I’ve seen. (T2)

Students appreciated teachers who kept in regular contact, chunked learning into manageable size, provided additional or flexible time for learning and assessment, explained, or gave feedback on learning, varied their teaching resources, and accordingly devoted more time and effort to assessments. However, other students were fatigued by home distractions, greater expectations for independent, online learning and fewer interpersonal interactions.

I struggled with assessments because there was too much distraction at home. (S3)

I found it boring and too hard to do assessments by myself. (S9)

Impact

Inequities occurred for students with little or no internet access, or digital devices, and home pressures for paid work to supplement family incomes or to care for siblings and older relatives. These students’ capacity to participate in and complete internal assessments was impacted:

Rural kids were isolated, with mobile and internet stability problems. Larger families had only one computer, or no space to work quietly (T5)

A lot of kids worked as essential workers in supermarkets, or on the farm. (T9)

Teachers believed decreased NCEA credit requirements reduced pressure and anxiety. Consequently, some teachers taught and assessed fewer topics to deepen student understanding, believing that student learning improved, while other teachers felt ‘although some students did the work, their level of understanding during lockdown was poor’.

I reduced the number of credits to help them succeed. (T9)

Some went through the motions of answering questions but did not have the understanding behind it. (T11)

Results in my subject were the same, if not better than last year. I think this was due to fewer standards addressed and the quality of the student cohort. (T10)

Students made comments like, ‘I still got the credits I needed, but I’m not sure I understand the learning’, ‘I worry about gaps in my learning and how I will cope at university’. Others talked of appreciating the LRCs, which enabled them to complete requirements early in the school year and leave to ‘earn money for university – you earn way more working those extra weeks than any scholarship will give you’. These students expressed strategic, instrumental views – ‘credit counting’ and ‘doing enough to get NCEA’ while other students ‘want[ed] to understand because I will need to know it for university’.

Adjustments to the school timetable (fewer assemblies, less sporting fixtures, etc.) allowed some catch up on missed assessments after lockdown. Most interviewed teachers believed while students had gaps in their learning, students benefited from opportunities to develop greater skills in self-directed learning or life experience.

Greater permission giving by NZQA to gather informal forms of evidence allowed students to personalize their assessments. They learned life skills from being involved in their iwi and helping their elders and could include those as evidence in their assessments. (T9)

In summary, inter-agency and cross sector collaborations enabled co-constructed, transformative solutions to pandemic-related assessment dilemmas which helped student and teacher wellbeing, mitigated equity concerns in terms of future pathways, and provided the sector with assessment certainty in the first phase of the pandemic.

Phase 2: adaptive assessment

Schools readily adapted to online learning and assessments when NZ entered its second year of pandemic lockdowns (Dowden, 2022). Such adaptation was evident in the increased uptake of online NCEA exams, across more subject areas and with increased numbers of schools involved. For comparative purposes, in 2019 35 NCEA digital examinations in 14 subjects were offered, in which 14,343 unique students participated from 197 schools. By 2021, 67 NCEA digital examinations in 24 subjects were offered, in which 25,812 unique students participated from 323 schools (NZQA, 2020, p.73; New Zealand Qualifications Authority, 2022, p. 73). Interestingly, 6.5% of all external assessed results were digital, up from 5.8% in 2020. The pandemic gave schools permission to be more innovative:

They can utilize the final school term to optimize learning time, advance digital ways of learning and assessing. (MOE Int.)

If the school can justify that they have enough data, they can collect evidence any way they like. (NZQA Int.)

Collaborative decisions were again made amongst the NZQA, MOE, Universities New Zealand, and sector reference groups (which included teacher unions and principal associations), to ensure senior secondary school student cohorts were not academically disadvantaged for future learning or employment pathways. Such decisions included 1) provision of learning recognition credits (LRCs) with higher ratios available for students in regions who endured additional lockdowns (1:4, c.f. 1:5), 2) reduced credit thresholds to be awarded excellent or merit endorsed course(s) (12 credits c.f. 14) and endorsed certificates (46 credits c.f. 50), and 3) adjusted credit requirements for University Entrance (12 credits, c.f. 14 credits in three University Entrance-approved subjects), though students “still needed to attain NCEA Level 3 and meet the literacy and numeracy requirement to be awarded University Entrance’(New Zealand Qualifications Authority, 2022, p. 5). A further adaptation was delayed dates for external examinations and portfolio submissions (for subjects like visual arts), to extend teaching and learning time in the last school term.

Teacher and student interview data reported smoother adaptations to digital forms of assessment and seeking natural evidence for achievement on standards internally assessed.

We just switched over to online teaching and assessing again – easier because we had resources from last year, and we were increasingly incorporating digital forms of assessing into our programmes anyway. (T2).

Because we had done it last year, it was familiar to us and not the big drama of last time. (S2).

Knowing what was important to assess helped everyone’s wellbeing; staff supported others, while the Principal and HODs gave permission to do things differently. Our top students were highly motivated to succeed, the middle were good, the bottom students less motivated than usual. (T1)

In summary, this phase was characterised by ongoing adaptations and efforts made to lessen workloads and stress, such as NZQA made external moderations optional for some regions during 2021 in recognition of ‘the disruption caused to teaching, learning and assessment as a result of COVID-19’ (New Zealand Qualifications Authority, 2022, p. 76).

Phase 3: disruptive assessment

The changed governmental approach in 2022 to not mandate school closures during the growing Omicron outbreak enabled schools to provide more in-person teaching opportunities than had been possible in the two previous years of the pandemic. The MOE passed responsibility to local schools to move to hybrid or offsite learning (Ministry of Education, 2022b), if the supply of relievers was insufficient to cover for staff absences due to illness. Nevertheless, ongoing disruptions to teaching programmes and internal assessment schedules in the first half of the school year caused sufficient distress for the Secondary School Principals Association to pen letters to the government,

The disruption to learning during this calendar school year is at levels we have never experienced before. Although affecting all students, these disruptions have disproportionately impacted our most vulnerable learners (Gerritsen, 2022, para 1).

Principals referred to unprecedented rates of student absence, concerns about levels of disengagement from students feeling too far behind or overwhelmed, and reporting to counsellors high rates of anxiety and depression. One principal reported about half the students had been absent in the first two school terms because of Covid and a further 25–30% for household Covid isolation requirements (Gerritsen, 2022, para 3).

The NZQA and MOE were in discussion, as this paper was submitted, seeking to support students and teachers, though noting, ‘any solutions would need to be equitable and proportionate to the disruption faced, credit actual learning that had occurred, be easy to understand, broadly accepted and minimize workloads’ (Gerritsen, 2022, para 4). Concerns were received seriously, with the Associate Minister of Education offering to meet with them to ‘deal with uncertainty and disruption in this Covid-19 environment’ (Gerritsen, 2022, para 6).

At the school level, teachers interviewed by the author indicated that variable student attendance made it difficult to know when and what to assess. Nevertheless, teachers spoke about: using more checkpoints (breaking assessment tasks into smaller chunks to keep students on track), extending deadlines for assessment tasks, being more flexible with students submitting part or whole assessments, using platforms like Google Classrooms so that students could complete assessments at home or in class, and in what counted as evidence.

However, departmentally some concerns were expressed about consistency/reliability, fairness, and tensions between assessing for learning and assessing for attainment. Some teachers spoke of focusing on fewer units, or critical parts of learning units, to ensure depth of understanding by their students to optimise success in future pathways, while other teachers were more focused on coverage to meet NCEA standards requirements. There was also variable practice with a few teachers using non-contact times to offer optional tutorials or catch-up assessment opportunities for students.

We are trying to get kids through – especially those on the cusp of not achieved to achieved. I think we need to focus more on helping them get merits and excellence. I realized after COVID why we torture kids with exams. It’s in studying for exams that students start to understand interconnections between topics and grasp concepts more deeply. (T7)

Student interviews revealed concerns about 1) disrupted learning (especially when relief staff were perceived as not competent in the topic area), 2) variable quality and quantity of assessment resources available on line, 3) feelings of being disadvantaged in completing assessments – some teachers expected students to complete assessments regardless of time at school; and 4) restrictions, whereby other teachers would not accept internal assessment submissions later than the set date or not allow students to submit work for assessment if they had been absent for parts of the teaching programme. Whilst acknowledging limits of teacher flexibility in times of undue illness and time pressures, this diversity of experience in terms of timing assessment submissions and indeed, what was important to assess and what counts as evidence raises equity concerns.

The high stakes we put on kids is unfair because it disregards the values and key competencies in the curriculum. Learning is change or growth over time, yet we focus on the measurement rather than the change itself. (T13)

This period of unprecedented disruption to learning, wherein different combinations of students were present in class, challenged the continuity of teaching and learning programmes – and therefore, assessment schedules and principles. Yet national qualifications have requirements associated with the robustness, dependability, and fairness of assessment processes. For instance, there are common assessment tasks for which NZQA specifies a date on which all students will complete the requirements – but students absent on those days miss the assessment opportunity. Similarly, there are nationally specified dates for submission of candidate work for practical/performing arts type subjects; and schools submit a timetable of internal moderation prior to the year starting (meaning, they have limited capacity to alter dates on which students may submit evidence of learning). Internal and external moderation is required to:

Ensure that student evidence meets the criteria of the standard on a national basis. Consequently, each school is required to have a process for randomly selecting student samples for moderation before submitting to NZQA moderators. (New Zealand Qualifications Authority, 2022, p. 76)

What this period of disruption reveals is that despite having a qualification that is seemingly learner-centred with its 1) inclusion of internal assessments, 2) capacity to nationally adjust credit thresholds for attainment of the qualification and recognition of varying qualities of attainment (i.e. merit and excellence endorsements), as well as entry requirements for university Entrance, 3) accelerating transition to digital tasks and examinations that can continue in the midst of a pandemic, there are limits to the extent of adaptation and transformation that can occur during a protracted pandemic.

Discussion

School and educational agency concern about wellbeing led to collaborative efforts and assessment adaptations for equity. This section examines these emerging themes in relation to literature and the theoretical framework.

Wellbeing through collaboration

Yates et al. (2021) argue the importance of a pedagogy of care, affection, and empathy during times of emergency. Such care flowed through NZ from the Prime Minister’s messaging ‘to be kind’, resourcing of $66 million to support wellbeing (OECD, 2020, p. 31), and through education sector collaboration. Extraordinary collaboration occurred across educational agencies, teacher and principal unions etc, to problem solve a means to reduce anxiety, and create hope, for more vulnerable students to complete their qualification. Collaborations resulted in the creation of LRCs and reduced credit requirements to meet qualification thresholds. Additionally, NZQA created new Unit Standards to credit wider life learning during lockdown, with guiding templates for teachers and associated ‘employability framework’ (NZQA, 2020b). Teachers and students’ views were sought, listened to, and acted upon, without political fallout – enacting Gewirtz’s (2006) relational justice whereby social relationships were valued in a non-hierarchal manner, and resulted in adjustments to high stakes assessment. Positive impact was seen in favourable national statistics of qualification attainment (ERO, 2020).

Sector-wide collaboration may be further understood in terms of the intersectionality of Rogoff’s (1995) socio-cultural planes of analysis, in the social practice of assessment. At the personal level, teachers and students negotiated extended times for assessment submission and co-constructed new ways of assessing learning (e.g. digital artefacts, community forms of evidence). At the interpersonal (school) plane of analysis, within and across departmental collaborations modified school timetabling that increased learning time and ensured manageable assessment workloads. Institutionally, policymaker and enactor collaborations extended to the involvement of students and school representatives, symbolising intersection of Rogoff’s (1995) vertical and horizontal planes. Significantly, intersectionality based on values of wellbeing and mutual benefit resulted in more just assessment practices for students during the collaborative transformative phase, demonstrating the power of collectivist action to reduce the pressures of high stakes assessments.

Adaptations for equity

Striving towards equity means mediating inequitable distribution of resources, seeking to recognise (hear) and respect the voice of the marginalised, and creating opportunities for their participation (Gewirtz, 2006). In this research study, educational agencies respected the concerns of sector representatives, teachers, and students by actively seeking their voice and their participation in decision making to mitigate effects of missed assessments and qualification thresholds at the national level. LRCs were an elegant mechanism for symbolising and mobilising assessment changes, for LRCs reduced teacher and student workload, retained integrity of NCEA, ensured students were not disadvantaged by the pandemic, and were widely accepted politically and educationally.

If assessment is the gatekeeper in determining life choices, then a system that can be sufficiently flexible to adjust itself in times of disruptive learning, could arguably be deemed to promote social justice – in terms of more equitable distribution of qualifications and valuing of relational justice through participation in decision-making by those affected. Successful impact of these efforts was evident in national qualification attainment percentages (Table 1), and student reactions in interview data. The relatively disadvantaged ethnicities, namely Pacific Peoples and Māori, and lower decile schools, fared better in 2020 than previous years due to LRCs and reduced credit thresholds, giving them access to socially valued opportunities. However, Table 1 shows percentage drops in 2021.

The combined effect of system level collaborative adaptations for wellbeing and equity led to adaptations of assessment timing, amounts of credits required to meet thresholds, digital ways of accessing and capturing assessments, and improved equitable access to qualification attainment. These efforts reduced the disruptive effects of COVID-19 on high stakes assessment at an assessment management level – but did little to transform assessment practices for students and teachers.

Assessment processes across the system were not consistent. For teachers’, whose conceptions of assessment related to improved learning, conditions of ‘strong[er] internal school support, supportive working conditions, and encouraging external policies’ (Yan et al., 2021, p. 21) enhanced their practice. Other teachers and learners, whose conceptions of assessment were more instrumental and related to qualification attainment, viewed LRCs and related changes more expediently, and focused on credit counting – seeking certification for mastery, rather than mastery itself.

Adjusting credit thresholds may appear to enhance equity in terms of access – allowing this cohort of students to proceed to higher learning and career pathways – but raises questions. The advantaged students essentially gained more LRCs, because they had more credits on which to be eligible for LRCs. How might the more disadvantaged have been allocated a compensatory higher ratio, without threatening public acceptability of LRCs and the integrity of the qualification? As Gewirtz (2006) reminds us, ‘In the real world … ’just practices can only ever meet with partial success’ (p. 79).

Further questions arise about the credibility and validity of the qualification. How arbitrary are the number of credits deemed necessary for each level to be awarded? How is qualification credibility maintained if the exceptional adjustments continue for more than a year or two? How valid is the award of certificates to students who had assessment gaps and only just met thresholds, compared with previous year cohorts who would not have been certificated?

Despite LRC adjustments, persistent inequities remained when attainment percentages of European groups and groups achieving excellence endorsements are compared across ethnicities (Table 3). It seems therefore that only fleeting, or superficial, transformation occurred in NZ’s high stakes assessment, despite the accelerated uptake of digital assessments and examinations, and expansion of what counts as evidence.

Table 3. New Zealand NCEA Certificate Excellence Endorsement Data 2019-2021 (NZQA, 2022).

Ethnicity	NCEA Level 1			NCEA Level 2			NCEA Level 3
	2019%	2020%	2021%	2019%	2020%	2021%	2019%	2020%	2021%
Asian	32.8	33.7	36.6	26.4	28.1	29.4	20.2	25.3	27.2
European	21.3	22.3	21.9	18.4	19.6	19.0	16.3	19.2	18.7
Māori	9.6	10.1	10.7	7.6	8.9	8.3	7.8	9.0	9.5
Pacific Peoples	6.7	8.6	9.9	5.5	6.1	7.1	4.6	5.6	6.3

Gewirtz (2006) warns that strides towards equity are mediated by contextual complexities. The digital divide experienced by learners, and the uneven impact on families, are internationally shared concerns, exasperated in COVID-19 times (OECD, 2020). Arguably, inequity is a by-product of political, economic, and educational structures and systems, and requires an inter-agency response across the financial, social welfare, health, and education sectors, though, as shown in this paper, collaborative efforts across the education system can lessen detrimental effects of injustices perpetuated through high stakes assessment. Nevertheless, in terms of Gewirtz’s (2006) distributive justice of equitable power in what is assessed and valued, and relational justice in confronting established hierarchies of assessment, valuing diversity of assessment and shared benefits for all is yet to be transformed.

Conclusion

New Zealand’s COVID-19 experience provides some new learning about high stakes assessment that may be relevant for other nations. Firstly, collaboration across policymaker, policy enablers and actors, including teachers and schools with a wellbeing priority, creates conditions for relational justice in which multiple voices are heard and included in co-constructing solutions that result in fairer access to educational assessment, as well as political, professional, and public acceptance. Secondly, establishing a national qualification with in-built flexibility of systems and structure that combines the higher validity and usability of teacher/student designed assessments with the higher reliability of national examinations, accommodates some adjustments in times of pandemic crises. Thirdly, in periods of upheaval, giving early assurance and certainty about qualification attainment relieves anxiety and ensures continuity of achievement. However, improving assessment practices, in terms of what is assessed, by whom, for what purposes and mutual benefits is a long term investment, which is not shifted by tweaking credit requirements. Equity concerns for attainment and entry to future learning pathways may be partially addressed by collaborative action and amelioration measures, like LRC and additional educational provision, though inter-agency commitment and investment across political, economic, health and education sectors are required for deeper effects.

High stakes assessments are situated within dynamically evolving systems, in which compromises are made among competing tensions, national policies, and community expectations. Opportunities for transformation are constrained by existing systems of knowledge, power, and practice. Where there is a collective will and combined action, inspired by wellbeing values and a qualification structure with inherent flexibility that includes teacher-designed and assessed elements, change can occur. But in times of continuing disruption, high-stakes assessment systems seemingly have limited capacity for transformation.

Acknowledgments

The author acknowledges insightful conversations with the MOE and NZQA officials, whose roles or names are not revealed for confidentiality purposes, as well as to the school – the assistant principal who helpfully assisted with organisational details as well as teachers and students who volunteered for interviews.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Jenny Poskitt

Jenny Poskitt is an Associate Professor in the College of Humanities and Social Sciences, Massey University, New Zealand. Her research focuses on improving the connection between assessment research, policy, and practice to enhance teachers’ professional learning and students’ learning and achievement.

Notes

1. Note, impending changes to NCEA will rebuild standards to consistent sizes.

2. University Entrance requirements include 14 credits in each of three approved Level 3 subjects, plus 10 Literacy credits at ≥ Level 2, and 10 Numeracy credits at ≥ Level 1, either from specified Achievement Standards or three prescribed Unit Standards (NZQA, 2021b).