Teacher education policy to improve teacher quality: Substantive reform or just another hurdle?

ABSTRACT

Mounting criticism suggests that the recent introduction of a ‘gatekeeping’ test to improve the quality of teachers in Australia—the Literacy and Numeracy Test for Initial Teacher Education (LANTITE)—has had limited success in achieving its aim. Shaped by a discourse of inputs on how teacher quality might be achieved (‘quality in, quality out’), LANTITE was to address the apparent decline in the quality of Australian teachers by reforming how candidates were being admitted into initial teacher education courses. This paper presents a quantitative analysis of 2,013 LANTITE scores, alongside the qualitative perspectives of 109 final year teacher candidates, to argue that LANTITE does little to change who gains admission into initial teacher education but is instead an ineffective mechanism for improving quality and a costly hurdle for candidates.

KEYWORDS:

• Australia
• teacher assessment
• teacher education
• mixed methods
• teacher quality

Introduction

In line with international policy trends among members of the Organisation for Economic Cooperation and Development (OECD) aimed at improving ‘core’ educational outcomes in literacy, numeracy, and science (Savage & O’Connor, 2019, p. 2), there has been increasing attention in the last decade on the role of ‘quality teachers’ in enhancing international competitiveness in the global education market (AGDET, 2016; Australian Government Department of Education and Training [AGDET], 2015; Cochran-Smith et al., 2017; Dinham, 2013; Rowe & Skourdoumbis, 2017; Sayed & Ahmed, 2015; Scholes et al., 2017). This has resulted in a flurry of teacher education reforms, within a number of countries, which focus on effectively selecting and preparing high-quality teacher candidates (Hutt et al., 2018; Ledwell & Oyler, 2016).

This paper examines the implementation and impact of one recent reform initiative aimed at improving teacher quality in the Australian context; namely, the Literacy and Numeracy Test for Initial Teacher Education (LANTITE), introduced by the federal government in 2016. Using qualitative and quantitative data comprising LANTITE test results, surveys and focus group interviews from one large Australian teacher education course provider, we consider whether the initiative has resulted in the kinds of substantive reform intended.

As a background to the study, this paper interrogates the use of gatekeeping tests used in teacher education programmes globally, as way to position how LANTITE has been employed within Australia as a policy solution to the ‘problem’ with teacher education. We then discuss the data, which suggests that rather than leading to the intended reform of a better-quality cohort than those currently accepted into the profession, LANTITE has resulted in the commodification of teacher education and a lack of clarity over who is responsible for addressing the ongoing problems.

We are cognizant that much of the research to date on the standardised testing of teachers—and of teacher candidates, in particular—has been the tendency for minorities to be over-represented in groups most adversely affected by such gatekeeping mechanisms (Goldhaber & Hansen, 2010; Graham, 2013; Petchauer & Baker-Doyle, 2016). However, our argument is not that LANTITE produces a similar effect, but that it seems to have no effect or impact at all regarding its claims to improve teacher quality. Rather, it simply creates an additional impost for all candidates to undertake the test, creating not only a financial burden for an otherwise futile exercise that is pushed onto teacher candidates—those least responsible for having created the problem in the first quality instance—but leaving potentially legitimate problems about teacher quality unaddressed.

Gatekeeping: Ensuring quality in education

The widely held assumption that student outcomes are inextricably linked with teaching quality (AITSL, 2011; Darling-Hammond, 2015; Organisation of Economic Co-operation and Development, 2005) has been a driving force for teacher education reforms (Wang et al., 2010). Recently, the selection and preparation of high-quality teacher candidates have been central to public policy discourses in countries such as the United States and Australia (AITSL, 2011; Hutt et al., 2018; Ledwell & Oyler, 2016). Consequently, we draw upon both these concepts in the current study—teacher quality and selection (in the form of gatekeeping)—to interrogate and further understand their relationship to education reform.

Theorising ‘quality’ within education is highly contentious (Rowe & Skourdoumbis, 2017; Sayed & Ahmed, 2015). Further, in relation to the teaching profession more specifically, the concepts ‘teacher quality’ and ‘teaching quality’ are often used interchangeably (Mockler, 2018a). Yet, the subtle difference between teacher and teaching quality is important. While the former refers to typically inherent traits and competencies associated with an individual, the latter emphasises the nature and characteristics of the strategies used to produce certain outcomes (Mockler, 2018b; Scholes et al., 2017). Conceived in this way, ensuring quality teachers is to ensure the ‘right’ people enter the profession in the first instance, while quality teaching focuses on what teachers do once they enter the profession and begin practising in schools.

Similarly, gatekeeping, particularly in the form of tests for both entry to and exit from teacher education programmes has been employed as a mechanism to address national and international pursuits to achieve teacher quality (Ledwell & Oyler, 2016; Malinen et al., 2012). Gatekeeping can be conceived as a policy solution (Shohamy, 2001), focused on selection criteria and procedures, that has been mobilised in a number of countries around the world (e.g., Caena, 2014; Heinz, 2013; Klassen & Kim, 2019). Shohamy (2001) argues that using gatekeeping tests to solve policy issues is appealing to policy makers because ‘they allow users to determine cutting scores in an arbitrary way and thus create quotas in a flexible manner’ (p. 86). She contends that tests are often used as a way for test users to seek permission to enter or exit and to demonstrate they have the knowledge required to make them legitimate members of the group being tested. With concerns regarding the seeming decline in education quality, and more specifically the quality of teachers, there has been a recent push for the need for more national and state policy initiatives that focus on selecting and preparing high-quality teachers through gatekeeping tests in both the US (Dover, 2018; National Council on Teacher Quality, 2019) and Australia (Australian Government Department of Education and Training [AGDET], 2015; AGDET, 2016; AITSL, 2019a, AITSL, 2011).

Ledwell and Oyler (2016) argue that, in the US, gatekeeping tests have historically been used in teacher education for two purposes: 1) to assess basic skills knowledge and/or pedagogical knowledge as an entry requirement, and 2) assess skills and competencies as a summative assessment for licensure purposes. Both gatekeeping purposes have traditionally been assessments that were ‘paper-and-pencil’ tests and under the purview of state-level legislation (Ledwell & Oyler, 2016, p. 121). However, this paper will focus primarily on gatekeeping tests, used internationally, that focus on basic skills testing and have been used primarily for selection into teacher education programmes, rather than teacher performance assessments that aim to assess classroom-readiness at the end of a teacher education programme (AITSL, 2019b; Ledwell & Oyler, 2016).

A discourse of inputs: Ensuring the ‘right’ people are selected into teacher education

The preoccupation with who enters the teaching profession as an approach to teacher education reform is shaped by a discourse of inputs (Barnes & Cross, 2018; Rowe & Skourdoumbis, 2017). The construction of the problem of teacher education (e.g., Cochran-Smith, 2004; Cochran-Smith et al., 2013; Ellis, 2010) is perceived as being rooted in the quality of candidates admitted into teacher education programmes (Barnes & Cross, 2018); that is, the failure of teacher education providers to select the right candidates with competencies deemed essential (e.g. literacy and numeracy skills) to being a quality teacher from the outset. While the use of federal or state-mandated basic skills tests for entry into a teacher education program is new to some countries, such as Australia, it has been common practice for a number of countries worldwide.

In the early 2000s, several countries established a regime of basic skills tests, assessing teacher candidates’ mastery over requisite literacy and/or numeracy skill, including the pre-professional skills test (Praxis I) in the US (e.g. Gitomer et al., 2011; Tatto, 2015), VAKAVA in Finland (e.g., Darling-Hammond, L, 2017; Malinen et al., 2012) and the Qualified Teacher Status numeracy and literacy tests in the United Kingdom (Department for Education, 2019; McNamara et al., 2002). Similarly, the Netherlands introduced a number of alternative teacher certification programmes, due to a shortage of teachers, which employed an entrance exam for ‘second-career teachers’ (Bouwer, 2007, p. 33). Bouwer (2007) argues that the intake assessments were used to legitimise selection and recruitment but that the predictive validity of these assessments were highly problematic.

A report by the European Commission in 2014 found that only a third of European countries have specified selection procedures and/or methods for their teacher education programmes (European Commission, 2014). In the case of Finland, VAKAVA was introduced in 2006 and developed by the eight faculties offering teacher education in Finland. The multiple-choice test requires teacher candidates to analyse academic articles and is part of a two-phase selection procedure that is used to select approximately 800–900 students from 7000 to 8000 applicants (Darling-Hammond, 2017; Malinen et al., 2012). This test was designed as not only a tool to select from a large number of applicants but to increase the chances that recent high school graduates, who had traditionally struggled to compete with candidates who had open university credits, a better chance of entry (Räihä, 2010). Unfortunately, the introduction of this policy has not been successful in increasing the admission of recent high school graduates, with only one fifth of those accepted being candidates who finished secondary school in that same year (Malinen et al., 2012). In 2015, the European Commission reported that Belgium (for Flemish-speakers only), the Netherlands, and the UK (England) also required their teacher candidates to take a literacy and/or numeracy exam as part of the selection process (European Commission/EACEA/Eurydice, 2015). However, the Qualified Teacher Status Professional Skills tests, which was once required for entry into a teacher education program in England, has recently been replaced by a new system to assess candidates’ literacy and numeracy skills within their teacher program, yet the details at the time of writing this paper were yet to be announced (Department for Education, 2019).

With national policy discourses highlighting the need to select and prepare high-quality teachers and the use of gatekeeping tests for selection into teacher, it is timely to interrogate the introduction of the LANTITE in Australia as a national solution to the teacher quality problem in Australia.

LANTITE: A (cheaper, one-size-fits-all) national solution

With public commentary reporting Australia’s national and international test scores either stagnated or in decline (Baroutsis & Lingard, 2017; Gorur & Wu, 2015; Robinson, 2018), several key policy announcements in the lead up to LANTITE’s 2016 introduction fuelled a failure narrative in which poor teachers had led to poor outcomes, heightening the nation’s appetite for reforms to teacher education. In 2011, the Australian Professional Standards for Teachers were released by the federal government under the auspices of the newly established Australian Institute for Teaching and School Leadership (AITSL), making it clear that quality teachers mattered for quality education. AITSL is a Commonwealth Government agency, funded by the Australian government, and consists of a board that provides national education oversight for the states and territories (AITSL, 2019a). In 2014, the federally established Teacher Education Ministerial Advisory Group (TEMAG) released a report with key recommendations that set into motion a series of initiatives for improving teacher education quality with an emphasis on inputs. For example, references were made for the need for more ‘transparent selection’ procedures (Teacher Education Ministerial Advisory Group, 2014, p. x), including mechanisms to better discriminate who can enter teacher education courses in the first instance, along with greater evidence of pre-service teachers’ classroom readiness as a condition for graduation.

AITSL was then commissioned by the Australian Government to lead the design and implementation of LANTITE, along with related reforms linked to the accreditation of initial teacher education programmes, fieldwork requirements and assessment strategies to determine classroom readiness (Australian Government Department of Education and Training [AGDET], 2015). As one specific instrument in this broader raft of reforms, LANTITE was designed to identify which teacher education applicants met a certain standard of literacy and numeracy to ensure only ‘the best’ would enter the profession (Teacher Education Ministerial Advisory Group, 2014, p. x). The ‘best,’ as defined by LANTITE, are those with ‘levels of personal literacy and numeracy … broadly equivalent to those of the top 30% of the population’ (AGDET, 2016, p. 6).

LANTITE in practice

Although LANTITE was developed federally and was designed to be a mechanism for selection, teacher education institutions were given the power to decide when their teacher candidates would be required to take the test. This is problematic given that students in some institutions are not required to complete the test until their final year of coursework, which allows the provider to generate income from students who might never be eligible to graduate (Dinham, 2013; Zyngier, 2016).

Of the 5,000 students who sat the pilot test in 2015, 92% of candidates passed the literacy component and 90% passed the numeracy component (Knott, 2016, para. 5). Once LANTITE commenced nationally in 2016, the Commonwealth Minister for Education and Training reported the pass rate as having increased to approximately 95% (Barry, 2017). Further, for the small percentage who fail, two additional attempts to re-sit the test are allowed—or more, if provided with a letter or support from their course provider (Australian Council for Education Research [ACER], 2019). Each test-taking attempt costs 185 USD Australian Council for Education Research [ACER] (2019) for both components (literacy/numeracy) to recover test development and administration fees and is usually paid for by the teacher education candidate (Australian Council for Education Research [ACER], 2019).

Thus, although LANTITE has been cost-neutral for the government to implement, the initiative has the potential to generate up to 3.7 USD million in revenue annually, with around 20,000 teacher education graduates sitting the test each year (Knott, 2016). With Australian universities struggling in the face of decreased public sector funding (Klopper & Power, 2014; May et al., 2013)—and teacher education programmes scorned as ‘cash cows’ because they provide easy-to-fill student places (Zyngier, 2016, p. 32)—the financial incentives afforded by a test such as LANTITE has raised concerns (Barnes & Cross, 2018). In addition to generating revenue, without any clear sense of how profits are being reinvested to support other measures that might also help to improve teacher education, such as course innovation, and echoing similar concerns about the commodification of US teacher performance assessment data (e.g., Petchauer & Baker-Doyle, 2016; Reagan et al., 2016), LANTITE has also created private sector opportunities, such as tutoring companies and publishers, to flourish with LANTITE seminars and support materials now being widely promoted to the graduate student market. Reflecting neoliberal trends in the reform of education more broadly—and the shift from education as a public good to private interest, and an emphasis on individual opportunity and competition within the free market (Harvey, 2007; Loh & Hu, 2014; Olssen & Peters, 2005; Peters, 2011; Reeves, 2018; Sleeter, 2008)—LANTITE directly contributes to the commodification of teacher education: a fiscally efficient solution to address the apparent problem of teacher quality, while simultaneously generating further economic growth through the expansion of new third-party interests that have not traditionally had stakes in teacher education (except now, for financial gain).

LANTITE has successfully provided at least the pretence of reform, having been mandated by all teacher education course providers at some point prior to candidates graduating and entering the profession. However, with 95% of applicants passing the test in each round, it is difficult to ascertain the extent to which this reform measure has been substantive.

By exploring quantitative and qualitative data sampled from one of Australia’s largest teacher education providers, the remainder of this paper considers this question of the extent to which LANTITE reflects meaningful reform in terms of both the numbers (test results) and perspectives of teacher education candidates themselves, and the impact of LANTITE on their development as teachers. We examine the extent to which LANTITE enables substantial education reform, to further understandings of the concepts of teacher quality and gatekeeping problematised earlier.

A mixed-methods approach

The 2016 introduction of LANTITE in Australia provided an opportunity to explore the impact of high-stakes entrance examinations to affect large-scale teacher education reform. This paper draws together findings from two parallel projects to investigate the effects; namely,

1. a large-scale (n = 2,013) non-experimental quantitative study identifying quantifiable indicators of ‘success’ within LANTITE results, and

2. a small-scale (n = 109) qualitative case study exploring the experiences and attitudes of final year pre-service teachers regarding recent policy changes, drawing specifically on the introduction of LANTITE.

While data from the large-scale quantitative study was collected in 2016/2017 and pass/fail rates were used to measure the impact of the reform in practice, the authors felt that the perspectives of pre-service teachers were needed to robustly interrogate the impact of the reform. Therefore, a survey and focus groups were conducted in 2018, exploring pre-service teachers’ attitudes and experiences with recent teacher education policy reforms. This study seeks to temper the rhetoric of education policy by numbers (Lingard et al., 2012) that focuses on statistics to justify policy decisions, by giving voice to the central actors in this policy initiative. The rationale behind this project was to interrogate the impact of a teacher education reform initiative, such as the LANTITE, on teacher education (e.g., who is included/excluded) and on teacher candidates to determine whether the reform was substantive.

For background, the larger quantitative study explored correlations between LANTITE outcomes and a range of variables (e.g., gender and course level). Similarly, the qualitative study explored the broader perspectives of the teacher candidates and their conceived notions of ‘quality teaching’ and ‘classroom readiness,’ but here we focus on their perspectives in relation to LANTITE.

The paper addresses the following questions:

To what extent are teacher candidates in an Initial Teacher Education program able to achieve a passing score on LANTITE?

What are the attitudes and experiences of fourth year teacher candidates from an Initial Teacher education program on LANTITE?

Ethics approval was given to obtain de-identified quantitative student data (n = 2,013) from an education faculty at a large metropolitan Australian university, which contained demographic details and academic information, including LANTITE scores. Student data were collected from students who sat the LANTITE from July 2016 (the introduction of LANTITE) to August 2017. The goal was to capture the impact of the reform, as depicted by pass/fail rates, to identify its impact as a gatekeeper into teacher education. In other words, the goal was to identify which students were failing and therefore classified as not being the best person for the teaching profession. The sample represented four course levels, with the majority of students (n = 861, 42.7%) enrolled at master’s level. This was followed by a dual-degree bachelor program (n = 484, 24.1%), a bachelor program with honours (n = 358, 17.8%), and a standard bachelor program (n = 260, 19.9%). The students comprised 77.9% females (n = 1,568) and 22.1% males (n = 445).

Descriptive statistics were used to analyse demographic attributes of the sample and assisted in understanding the scope and distribution of the responses on each indicator of success (e.g., course level and gender). This was important to identify if underlying statistical assumptions have been met.

For the qualitative data, ethics was received for surveys and focus groups with undergraduate students completing their final (fourth) year of an early years, primary, or secondary teacher education program. The survey and focus groups collected data about student teacher perceptions on a number of recent teacher education policies, of which LANTITE was one. Using an explanatory sequential design (Creswell & Plano Clark, 2017), the attitudinal scales captured beliefs across the 109 participants and were used to shape the semi-structured interview questions relating to LANTITE (see Appendix A). The survey had a response rate of 22% (n = 109, from approximately 500 final year undergraduate teacher candidates). As the survey was administered in the final semester of the candidates’ teaching degree, many potential participants were completing final assignments and applying for teaching positions. The 22% response rate, while not favourable and at the lower end of average response rates for online surveys (Nulty, 2008), was therefore expected.

Comprising 22 five-point Likert-scale items and 3 open-ended questions in total, the survey included five items focused specifically on the use of literacy and numeracy skills as determinates of teacher and/or teaching quality. For example, participants were asked to indicate the degree to which they agreed with statements such as ‘Strong literacy and numeracy skills are characteristics of a high-quality teacher’ and ‘Teachers need to have literacy and numeracy skills’ (see Appendix A). In the survey, the participants were given the opportunity to register their interest in contributing to a focus group discussion, which directed them to another page, where they provided their contact information, so as to keep their survey response anonymous. Four weeks post-survey, six focus groups were conducted with a total of 25 students. Although 50 of the 109 survey respondents indicated their willingness to participate in a focus group, only 25 responded to the post-survey email invitation to attend. Table 1 provides a snapshot of participants’ gender, domestic or international student status, Language Background Other than English (LBOTE) profile, and their intended teaching context (primary or secondary school).

Table 1. Focus group participant information

Participant	Gender	International student or domestic students	Language Background Other than English (LBOTE)	Primary or Secondary
1	Female	Domestic	Yes	Secondary
2	Female	Domestic	Yes	Primary
3	Female	Domestic	No	Secondary
4	Female	Domestic	Yes	Secondary
5	Female	Domestic	No	Primary
6	Female	Domestic	Yes	Secondary
7	Female	Domestic	No	Secondary
8	Female	Domestic	No	Secondary
9	Female	International	Yes	Secondary
10	Female	Domestic	No	Secondary
11	Female	Domestic	No	Secondary
12	Female	Domestic	No	Secondary
13	Female	Domestic	No	Secondary
14	Female	Domestic	No	Secondary
15	Female	Domestic	No	Secondary
16	Female	Domestic	No	Secondary
17	Female	Domestic	Yes	Secondary
18	Female	Domestic	No	Primary
19	Female	International	Yes	Secondary
20	Male	Domestic	No	Secondary
21	Male	Domestic	Yes	Secondary
22	Male	Domestic	Yes	Secondary
23	Male	Domestic	No	Secondary
24	Male	International	Yes	Secondary
25	Male	International	Yes	Secondary
Results	Male: 6 Female: 19	Domestic: 21 International: 4	LBOTE: 11	Primary: 3 Secondary: 22

Focus group transcripts were coded from which initial themes emerged (vertical analysis). Once these themes were established, their related codes were linked to key concepts from the literature on quality teaching and standardised testing (e.g., teacher education policies [impact on the profession] and testing principles [e.g., validity, fairness, authenticity]). A second analysis was then conducted to compare emerging themes and patterns across both the quantitative and qualitative data sets (horizontal analysis). The following section discusses the key themes from the analysis which focus primarily on 1) pass and fail rates on the test, 2) the test as a valid measure of quality and 3) LANTITE as a policy. The first theme focuses on the first research question, ‘To what extent are teacher candidates in an Initial Teacher Education program able to achieve a passing score on LANTITE?’, while the remaining two themes answer the second research question—’What are the attitudes and experiences of fourth year teacher candidates from an Initial Teacher education program on LANTITE?’

Policy by numbers

The following sections examine the proportion of students who passed and failed either the literacy or the numeracy component on their first attempt. Of the 2,013 student records that formed the data set for this study, not all students undertook both the literacy and numeracy components: 1,979 students sat the literacy test and 1,898 students sat the numeracy test.

Literacy

Of the 2,013 student records relating to LANTITE, at the time of data collection, 1,979 students sat the literacy test. Although 90.1% (n = 1,784) of students passed the literacy test on their first attempt, 9.9% (n = 195) failed. Of those 195 students who failed, 31 re-sat the test, with 17 passing on their second attempt. Of the 14 students who failed their second re-sit, 7 re-sat the test for a third time, with 4 eventually passing, leaving 3 who failed all three re-sits. When data was collected in August 2017, 164 of the original 195 students who failed had not re-attempted the test. The 3 students who failed after three attempts were all female: 2 were in a master’s program and 1 in a bachelor’s honour program. One of these students received approval to re-sit the test a fourth time, but the results were not available when this data was collected. Pass/fail rates for literacy based on attempt, are outlined in Table 2.

Table 2. Pass/fail rates for Literacy

Literacy test attempt	Pass	Fail
First (n = 1,979)	1,784 (90.1%)	195 (9.9%)
Second (n = 31)	17 (54.8%)	14 (45.2%)
Third (n = 7)	4 (57.1%)	3 (42.9%)

Numeracy

Of the 2,013 student records used in this study, at the time of data collection, 1,986 students sat the literacy test. While 95.6% (n = 1,898) of students passed the numeracy test on their first attempt, 4.4% (n = 88) who failed. Of those 88 who failed, 37 students sat the test a second time, with 21 passing. Of the 16 who failed the second attempt, 9 students sat the test for a third time, with 5 of those passing. Of the 2 students who sat the test a fourth time, 1 reached the necessary standard. At the time the data was collected, 50 of the 88 students who failed the test on their first attempt had not re-sat the test. The results for the numeracy test are summarised in Table 3.

Table 3. Pass/fail rates for Numeracy

Numeracy test attempt	Pass	Fail
First (n = 1,986)	1,898 (95.6%)	88 (4.4%)
Second (n = 37)	21 (56.7%)	16 (43.3%)
Third (n = 9)	5 (55.5%)	4 (44.5%)
Fourth (n = 2)	1 (50%)	1 (50%)

Overall, the findings suggest the number of students from this sample who passed the test prima facie was consistent with the national LANTITE pass rate of 90–95% (Barry, 2017; Knott, 2016). However, the findings also provide evidence that if students do fail, they still have at least a 50% chance of eventually passing the test on each subsequent re-sit. With up to four attempts allowed, LANTITE is not very effective at excluding students who seek entry into teacher education, as intended by the original TEMAG recommendation in terms of a gatekeeping mechanism. Therefore, as a gatekeeping mechanism aimed at reforming teacher quality, and selecting the ‘right’ people into teacher education, the data reveals LANTITE to have limited impact.

Important but may not be valid and fair—Views on testing principles

The survey of fourth year teacher candidates (n = 109) who had completed their initial teacher education at the same institution found that 70% of respondents thought teacher candidates should possess a high level of literacy and numeracy skills, and that assessing these skills is important. However, the 25 participants who participated in subsequent focus groups—all of whom had passed the test, although the number of attempts taken is unknown—revealed mixed feelings about the authenticity, validity and fairness of LANTITE as a means of assessing literacy and numeracy.

Two focus group participants (Participant 6 and 9) specifically commented on the authenticity of LANTITE, and their surprise that the test was contextualised into teaching scenarios. As Participant 6 commented:

I didn’t realise when I sat it that it would all be in context of what you actually do as a teacher. Like reading NAPLAN results off a graph. We can’t deny that those skills are important.

Although such comments spoke to LANTITE’s authenticity, other participants questioned the validity of the test, particularly for literacy, as they felt there was an overemphasis on vocabulary rather than ‘actual literacy skills.’ Two English major students (Participants 3 and 7) expected to do well on the literacy test, but found they performed better on the numeracy test because they had studied for it by pulling out their old Year 9 and Year 10 mathematics textbooks. Although they still did well on the literacy test, they felt it could be unfair for others who might not have had the same exposure to language and vocabulary as those who had done majors in areas such as English. Participant 3 argued:

If you’re not someone who is an avid reader, you’ve never seen that word before in your life, and it’s a disadvantage. It’s not really showing their literacy skills, it just means that they haven’t been exposed to that certain word and then they can’t answer the question because it’s asking for a synonym of it.

Similarly, Participant 23 in a different focus group argued that the focus on vocabulary and standard English:

speaks to old fashioned homogeneity. I am honestly put slightly in mind of- going really back to the white Australia policy of just, everyone speaks standard English to a high degree and that’s what they want all teachers to be. I think it very much speaks to the mindset of the people who designed it.

Notably, these comments were made by a domestic, white Australian student who felt the test did not cater for diversity in education. Four of the 25 participants in the focus groups were international students (and all passed the test), but the domestic students, seven of whom were multilingual speakers, also advocated strongly that the test could potentially make it more difficult for multilingual teacher candidates who, in their view, still made excellent teachers. Such observations echo similar concerns in the literature, on the persistent discrimination of teachers and graduate teachers from English second or additional language backgrounds, relative to English native speakers (e.g., Hélot, 2017; Tupas, 2015). The problem of ‘linguistic discrimination’ identified in Cho’s (2010) Canadian study of teacher candidates from non-native English-speaking backgrounds during their qualifying practicum, for example, was similarly evident in the experiences of some multilingual teacher candidates undertaking teaching fieldwork in Australian schools (Moloney & Giles, 2015). As Schmidt and Gagné (2015) conclude in their study of linguistic diversity within Canadian initial teacher education programmes, the number and range of post-program employment opportunities diminish unless graduates ‘fit the dominant teacher demographic, that is, white, middle class, female, Canadian-born, and monolingual English-speaking’ (p. 308).

In alignment with the quantitative data, the literacy pass and fail rates, while still high, were lower in comparison to numeracy pass rates, which may be indicative of the multilingual nature of the cohort. Unfortunately, the quantitative data collected did not include data to indicate whether students were domestic or international, or whether their first language was English. Therefore, the multilingual backgrounds of the focus group participants provide insight into the multilingual makeup of the larger population of teacher candidates who sat the test. Many of the teacher candidates who participated in the focus groups suggested the perceived overemphasis on ‘testing vocabulary’ may create an additional hurdle for multilingual teacher candidates. While these teacher candidates are still highly likely to pass the test, they may have to spend a lot of time studying for the test, and possibly paying to re-sit the test multiple times.

LANTITE … just another hurdle—Views on the impact of LANTITE as policy

While focus group participants acknowledged that LANTITE placed an additional impost on multilingual teacher candidates, they also argued that LANTITE, as a reform mechanism, was pointless. Two students, from different focus groups commented that they could not understand why there were university-specific admissions requirements (e.g., a benchmark score on their final Year 12 English and Mathematics exams), as well as LANTITE, which they saw as yet another hurdle. If students were already required to demonstrate a particular standard of literacy and numeracy though other measures, they questioned why they were required to demonstrate this again in an additional external measure:

It’s just another hurdle in like our path to try to graduate, really … But I don’t think it’s necessary anyway. I don’t think it’s accurate whatsoever (Participant 8).

This [LANTITE] is a waste of my time (Participant 6).

Others recognised that despite universities having general entry requirement (e.g., minimum scores are required in their English and Mathematics secondary school exams), there was a public perception that these were not stringent enough. However, rather than the need for an external test such as LANTITE, which added to their pressures as a student, the solution should be that universities are held more accountable for who they admit in terms of reconsidering their internal selection methods (e.g., requiring higher secondary school exam scores). For many of the focus group participants, LANTITE also seemed to signal the future of the teaching profession, as one increasingly shaped by a focus on standardisation, data and a lack of trust in teacher competence. For example, Participant 15 lamented:

Implementing the literacy and numeracy tests set the teaching profession back how many years, because we’ve had standardised tests in all our schooling lives … So, it’s kind of just embarrassing in a way … Because they basically don’t think you’re literate.

This particular student, who passed the test on the first attempt and entered university with very high secondary school exam marks, went on to explain the need for such a test implied, to others, those undertaking a teaching degree often have questionable skill levels in literacy and numeracy.

Four other graduates, across two different focus groups, argued that the test did nothing to substantively reform the kind of teacher preparation they received; rather, it simply focused on trying to prevent courses admitting students who lacked sufficient literacy and numeracy skills in the first place. That is, the test is seen as a quick fix to solve a problem that is less about teacher education than university entry requirements, which the following comment highlights:

It’s just the government being ….people being, like oh, teachers don’t know how to read and write. We’ll give them a test and that will solve everything. It’s quicker at the moment than making a standard for all the universities. Like going through that process of redoing and making sure that every university is offering a teacher related course that is meeting the criteria, is a big thing. So, I guess that’s why … the LANTITE [is] quicker in the interim.

Such comments suggest graduates recognise the importance of literacy and numeracy skills and LANTITE would be redundant if teacher education programmes were simply more accountable and transparent in how candidates are selected in the first instance. The test itself seemed to do little to reform or improve the actual nature or quality of the teacher education program they went on to receive.

What the statistics show and voices say: A limited reform measure?

With approximately 90–95% of teacher candidates passing LANTITE nationally (Barry, 2017; Riddle, 2015), the teacher education cohort examined in this paper was statistically average for the literacy test. Yet, the percentages of those recorded as failing in these headline results are not an accurate representation of the number of students who are then excluded from teacher education. The findings from this study suggest that if applicants opt to re-sit the test, there is a statistically higher than 50% chance they will eventual pass (and go on to enter the course, as originally intended). For example, of the 31 students who failed but re-sat the literacy test, all but three eventually passed. One more was permitted to sit the test a fourth time, but the result was not available when data was collected; however, it is statistically probable that they would have passed that re-sit. Wolkowitz (2011) found that scores on a national nursing school entry examination, administered in the US, were significantly higher on additional attempts, suggesting that scores increase with each re-sit and more students than not will ultimately go on to pass.

In the case of LANTITE, the potential for teacher candidates to increase their scores on each subsequent attempt results in the actual number of candidates being excluded from teacher education is very small, rending this initiative as having very limited impact as a reform measure. Further, coupled with recent graduates’ perceptions of the test as ‘just another hurdle,’ and a ‘waste of time,’ it seems warranted to question the role of LANTITE in enhancing the teaching profession. This aligns with Dover’s (2018) research on the US EdTPA, as he reports that candidates referred to the test as a ‘waste of time,’ and ‘redundant’ (p. 14) because they felt the test had limited educative potential and took valuable time away from their own professional growth. The test was an act of compliance not one that was educative.

Although the intention of LANTITE is to focus on teacher selection at the point of entry into teacher education, the reliance on an external mechanism such as LANTITE to ‘ensure quality’ provides limited incentive for teacher education providers to focus on the internal quality of their existing courses or own institutional entry requirements. Dover (2018) argues that ‘by outsourcing teacher evaluation, [Teacher Performance Assessments] disrupt the internal accountability of university-based teacher education, which relies on authentic, multifaceted, and longitudinal analysis of candidates by school- and university-based faculty’ (p. 2). Therefore, in the case of LANTITE, determining teacher and teaching quality is not only outsourced and disconnected for teacher education programmes, but as indicated by the high pass rates, also has very limited impact on reforming teacher education.

Yet, although the findings above suggest LANTITE has limited substantive impact on reshaping Australian teacher education, its rhetoric wields substantial power in positioning teacher education providers, teacher educators, and the teaching profession more generally, as incapable of raising their own standards without interventions and governance.

What might we learn? Teacher education policies globally

This study has interrogated statistical data and teacher candidate perspectives related to a recent flagship teacher education reform measure in Australia, LANTITE. The findings add to our conceptual understandings of teacher quality and gatekeeping—and their relationship to teacher education reform—as well as provide empirical and methodological insights.

As concepts, teacher quality and gatekeeping are inextricably linked, with teacher quality being the problem (in this case, with respect to the preparation of teachers through teacher education) and gatekeeping being the policy solution (though mechanisms such as LANTITE; Barnes & Cross, 2018). Understood in this way, the notion of teacher quality implies a discernible line between those who possess it and those who do not, warranting a gatekeeping test. A theoretical contribution of this paper has been the importance of validity in the relationship between teacher quality and gatekeeping. Within LANTITE, skills in literacy and numeracy are taken as proxies for teacher quality, yet our findings question whether such measures constitute valid indicators of quality. Rather, the data suggest that LANTITE is a hurdle that all students must complete (and pay for), but with little impact in terms of transformative outcomes or reform. Almost all students go on to complete their course of teacher education, despite the test. If gatekeeping measures are to be valid, it is important to identify what constitutes teacher quality. We contend from the findings of this study that teacher quality cannot be synonymous with literacy and numeracy skills alone.

In addition to furthering understanding of the relationship between teacher quality and gatekeeping, this study also highlights the need for empirical evidence that explores the actual impact of teacher education reforms. With an urgency in recent policy discourses that highlights the apparent need to ensure quality teachers and teaching in schools, it is vital to explore the extent to which these initiatives are achieving quality teaching and teacher reform rather simply being hurdles for the very people we want in our classrooms (see Goldhaber & Hansen, 2010; Graham, 2013; Petchauer & Baker-Doyle, 2016). Research on the use of gatekeeping basic skills tests in Finland, the Netherlands, and England has found, respectively, that reforms were not successful in achieving its primary aim to increase recent high school graduates (Malinen et al., 2012; Räihä, 2010), lacked predictive validity (Bouwer, 2007), and were eventually abandoned (Department for Education, 2019). There is a need to therefore continually interrogate, document, and challenge teacher education policies that may have been well-intended recommendations but are not effectively translated into practice. In the present study, although participants agreed that teachers should possess strong literacy and numeracy skills, the impact of the reform was limited in terms of discriminating ‘quality’ prospective students (with almost all applicants eventually admitted).

Reasons for this limited impact seem largely due to test design and management being left to an independent third-party, which allows multiple (yet costly) attempts to re-sit a test that can be quickly and efficiently administered, but little direct input or oversight from teacher education providers. This provides little incentive for those actually responsible for delivering teacher education to critically engage with their own programmes and introduce quality assurance measures to ensure candidates are equipped with a strong, practical and sustainable skill set upon graduation. As Cochran-Smith et al. (2018) argue, teachers and teacher educators need to position themselves as agents of reform, not the objects of reform, which we argue is the orientation of the LANTITE implementation. This involves changing the narratives that surround teacher education reform by having the profession itself, including teacher educators and teacher education providers, taking an active role in discussing teacher education in the media (Ulmer, 2016), and challenging how federal policies are framed as the only viable solution for the apparent teacher education problems these policies identify.

Finally, this study contributes to the literature on teacher education reform through its use of mixed methods to interrogate reforms both by numbers and through the voices of those most influenced by them. This paper advocates for greater use of mixed-method approaches to not only counter the increasing dominance of a policy by numbers approach to policy decisions in the era of big data (Lingard, 2010; West, 2012), but for a balance review of policy outcomes, both statistically and through the voices of those most impacted by these policies. In the case of this analysis, what numbers show and voices say about LANTITE’s impact on Australian teacher education reform, is there are emerging concerns about how high-stakes entrance examinations are being enacted, embraced and/or resisted in Australia and beyond, and their use as mechanisms to improve teacher quality.

Acknowledgments

We thank Associate Professor Paul Shield for his help and support with the statitical analysis.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Melissa Barnes

Melissa Barnes is a senior lecturer in the Faculty of Education at Monash University, working within the fields of teacher education, assessment, policy and TESOL. She teaches and leads research initiatives that focus on policy construction, interpretation and enactment, with a focus on how policies impact and shape teaching and learning. She has published in journals such as Critical Studies in Education, Higher Education and Discourse, among others. Melissa has been a classroom teacher in the US, Germany, Vietnam and Australia, collectively shaping her understanding and approach to teaching and learning.

Russell Cross

Russell Cross is Associate Professor in language and literacy education. He leads teaching and research initiatives with a focus on content and language integrated learning (CLIL), and teachers’ practice in languages education more broadly. With a focus on the sociocultural and political nature of teachers’ work and knowledge, Russell’s work has appeared in Modern Language Journal, Teachers and Teaching and Language & Education, among others. With Trevor Gale from the University of Glasgow and Carmen Mills from the University of Queensland, Russell is current Chief Investigator on the Australian Research Council’s national discovery project, Social justice dispositions informing teachers’ pedagogy in advantaged and disadvantaged secondary schools.

Appendix A: Data collection instruments

Semi-structured focus group questions (related to LANTITE, from larger survey)

• 1. To what extent do you feel that teachers should have strong literacy and numeracy skills?

• 2. What was your experience sitting the LANTITE?

• 3. How valid do you feel the LANTITE as a measure of literacy and numeracy skills? Can you provide examples?

• 4. To what extent is LANTITE a good selection mechanism? Why or why not?

• 5. What do you feel is the most effective way to ensure that teacher candidates have strong literacy and numeracy skills?

Likert scale items, rated from Strongly Agree to Strongly Disagree (related to LANTITE, from larger survey)

• 1. A quality teacher needs to be a high-achieving student.

• 2. Teachers need to have strong literacy and numeracy skills.

• 3. A standardised test is the best way to assess pre-service teachers’ literacy and numeracy skills.

• 4. Pre-service teachers should have strong literacy and numeracy skills before entering a teacher education program.

• 5. Foundational literacy and numeracy skills should be a content focus in teacher education programmes.

• 6. Strong literacy and numeracy skills are characteristics of a high-quality teacher

• 7. Pre-service teachers can develop into quality teachers over time.

• 8. The potential of a pre-service teacher to be a quality teacher can be determined before they enter a teacher education program.

• 9. A literacy and numeracy test is a good way to measure teacher quality.

Qualitative questions (focus groups)

• 1. What was your experience taking the LANTITE (e.g., it was difficult, confusing, etc.)?

• 2. To what extent, do you feel that the LANTITE is an important resource for ensuring high-quality teachers in Australian classrooms?