Swedish teacher conceptions of assessment: a focus on improving outcomes

ABSTRACT

Understanding teachers’ conceptions of assessment is a key objective in supporting assessment practices that lead to improved learning outcomes. Thus, inventories capable of identifying teachers’ assessment conceptions are important. The Teachers Conceptions of Assessment (TCoA) inventory was an early and influential measure of teacher assessment conceptions, but replication studies have shown that the model may be affected by policy and practice context. In the present study, a Swedish adaptation of the TCoA was administered twice, 18 months apart. A sample of 249 teachers were matched across the 2 time-points and their self-reported scores were analysed with confirmatory factor analysis and invariance testing. With good correspondence to the data, six of the nine factors in the TCoA were completely replicated and one factor was partially replicated. The model had sufficient similarity between time points to permit mean score comparisons, which were largely equivalent between times. The study indicates that the Swedish Teacher Conceptions of Assessment adaptation can be used reliably in Swedish primary and lower secondary schools as measure of teacher conceptions of the uses of assessment.

KEYWORDS:

• Sweden
• conceptions of assessment
• teachers
• invariance
• confirmatory factor analysis

Introduction

Given the importance of teacher beliefs as predictors of practice (Fives & Buehl, 2012), knowing how teachers conceive of assessment seems to be essential in understanding their role in implementing assessment practices that enhance learning outcomes. If teachers conceive of assessment purely in terms of summative evaluation of schools or teachers, it is unlikely that they will implement assessment practices with the purpose of improving learning. In contrast, teacher conceptions that assessment can be used to improve learning would be beneficial for implementing formative assessment practices. Such practices include monitoring student learning in order to adapt classroom practice in accordance with student learning needs (Black & Wiliam, 2009). Therefore, inventories capable of identifying teachers’ assessment conceptions are needed.

The Teachers Conceptions of Assessment (TCoA; G. T. L. Brown, 2003) inventory, developed in New Zealand, was an early and influential measure for exploring teacher thinking about the purposes of assessment. It has been evaluated in multiple educational jurisdictions and languages leading to the understanding that policy and practice contexts influence the responses (G. T. L. Brown et al., 2019). To date no study in Sweden has been conducted.

The aim of this paper is to contribute to that gap by examining in which ways the New Zealand model replicates in the Swedish context. We do so with a repeated measure invariance study of Swedish TCoA responses. Their responses produce partial replication of previously reported results from other low-stakes contexts, supporting the importance of ecological rationality in understanding teacher’s conceptions of assessment.

Formative assessment in classroom contexts

Systematic literature reviews and meta-analyses have shown that formative assessment can improve student achievement (e.g. Black & Wiliam, 1998; Hattie, 2009; Lee et al., 2020; Palm et al., 2017). Formative assessment that is aligned to curricular objectives and goals requires teachers to collect information about student progress, difficulties, and successes to plan instruction, design activities, ask questions, and provide feedback to learners (Bloom et al., 1981). Performance data can be interactions in a classroom (e.g. question and answer) but also includes more formal diagnostic testing (G. T. Brown & Hattie, 2012) or analysis of errors made in classroom or home practice (Bejar, 1984). From these kinds of performance data teachers can formatively make appropriate adjustments to their classroom instruction and to student learning activities so they are tailored to the students’ learning needs (Lai & Schildkamp, 2016).

Naturally, this approach to assessment requires resources (i.e. assessments that diagnose needs and time to plan responses), policies that prioritise using assessment formatively rather than solely for administrative or summative purposes, and teacher commitment to using assessment formatively for improvement. Further, considering the importance of the teacher’s active role in using performance data in this way, it is likely that teacher beliefs about assessment and feedback matter to the efficacy of these processes.

Teacher conceptions of assessment

Ajzen’s (1991) Theory of Planned Behaviour (TPB) identifies the importance of teachers’ beliefs and attitudes towards any phenomenon (in this case, assessment) as essential to understanding their intentions and actions. TPB also points to the importance of social norms and perceptions of behavioural control as predictors of intentions, behaviours, and outcomes. A substantial body of evidence highlights the importance of understanding teachers’ conceptions of assessment, as these are linked to their approaches to assessment and influence students’ learning and outcomes (Barnes et al., 2015; DeLuca et al., 2019; Fives & Buehl, 2012). Teachers’ conceptions of assessment shape how they assess students and the conclusions and interpretations they draw (Cheng et al., 2010; Hill & Eyers, 2016). Research also emphasises the significant impact of assessment conceptions on teachers’ decisions and practices (G. T. L. Brown, 2008; Opre, 2015; Vandeyar & Killen, 2007). This aligns well with Fives and Buehl (2012) model in which teacher beliefs act as filters, frames, and guides to cognitive resources that impact their actions. Of course, teacher beliefs about the proper role of assessment are not universal; they are context bound by the policy and practice framework in which they are employed (Bonner, 2016; G. T. L. Brown & Harris, 2009; Fulmer et al., 2015).

Research into teacher beliefs about assessment indicates that teachers have multiple conceptions of assessment with varying degrees of intensity (G. T. L. Brown, 2008). The focus of a belief (e.g. the type of assessment [formal vs. informal] and the object of evaluation [teacher/school vs. student]) matters to how strongly a conception of assessment is endorsed or rejected, and even to whether that conception is present in teachers’ conceptions of assessment. For example, in both New Zealand and Queensland, Australia, primary school teachers tended to not agree as much as secondary teachers did with the purpose of holding students accountable through assessment (G. T. L. Brown, 2011; Brown, Lake, et al., 2011). Arguably, this reflects the greater use of assessment among adolescents to prepare them for entry to further education or adulthood. Thus, because of the multiple purposes and uses of assessment, teachers have multiple and complex attitudes or conceptions in response to those uses within a jurisdiction.

It appears teachers are aware of formative and summative uses of assessment and their attitudes towards such uses vary depending on whether the student or the school is the focus of attention (Harris & Brown, 2009). The use of student assessment to evaluate schools and teachers especially creates tensions and negative responses from teachers (Bonner, 2016; Nichols & Harris, 2016). Teachers in high-stakes accountability systems, where public examinations of student achievement have considerable acceptance, seem to view assessment for making students accountable a positive aspect for improving both student learning and character (G. T. L. Brown et al., 2009; Brown, Hui, et al., 2011). The relative strength of these varying conceptions appears to be ecologically rational in that teachers in general endorse the policies and purposes that apply to their level of employment.

Hence, we should expect that Swedish teachers’ responses to the TCoA to reflect broad cultural values and specific educational policies about the nature, purpose, and practice of assessment.

The Swedish context

According to the collaborative European Values Study and World Values Survey spanning 2005 to 2022 (EVS/WVS, 2022), Sweden exhibits a distinct inclination towards secular-rational and individualistic values, emphasising equality, individual freedom, and well-being. This sentiment is reinforced by the six-dimensional Index of National Culture (INC) developed by Hofstede et al. (2010). Within the educational context, these values manifest in the Swedish system by granting teachers significant autonomy to interpret and implement the objectives outlined in the national curriculum. This extends to their ability to select appropriate teaching methods that align with the overarching goals of fostering student achievement (Helgøy & Homme, 2007).

The egalitarian ethos prevalent in Sweden extends to the students. Instances of discrimination among students, such as the creation of specialised classes or educational tracks for gifted or underperforming students, are infrequent (Hofstede et al., 2010; Hofverberg & Winberg, 2020). Practices like public evaluation and ranking of students based on academic achievement are generally considered unacceptable within the Swedish educational culture.

Swedish teachers in compulsory school (school years 1–9) commonly follow their students during one of three stages, each stage consisting of 3 years (Years 1–3, 4–6 or 7–9). Teachers in Years 1–6 are generalists, teach most subjects, and mainly teach one class in any given year. Teachers in Years 7–9 are subject specialists and only teach a few subjects. As a consequence, they most often teach students from different classes and school years in any given year. The Swedish curriculum adopts a goal-oriented approach, incorporating national standards for student achievement in Years 3, 6 and 9 (corresponding to ages 9, 12 and 15). While achievement grades are assigned in Years 6–9, the most significant implications lie in Year 9, influencing admission to upper secondary school. Grading is criterion-referenced, allowing multiple students to receive the same grade as long as they meet the established standards. Legally, the responsibility for providing adequate resources for education and conducting systematic evaluations lies with the municipality in which the school is situated, as stipulated by The Education Act (SFS 2010:800, 2010, p. 800, ch. 4 § 3). To support schools and teachers, the National Agency for Education (NAEd) offers national screening materials, assessment support, and standardised national tests. These tools serve diverse purposes, from informing decisions about support and guiding adaptations in teaching to providing aggregated estimates of student achievement at the school or system level for grading consistency and trend analysis (Skolverket [National Agency for Education], 2020).

National standardised tests (NSTs) are mandatory in Years 3, 6, and 9 and play a pivotal role in grading students in Years 6–9, although teachers are obligated to also consider other available information about students’ knowledge and skills in their evaluation process (SFS 2010:800, 2010, ch. 10, § 20a). Consequently, the NAEd advises teachers to employ diverse assessment situations for both formative and summative purposes (Skolverket [National Agency for Education], 2022), granting teachers across all school levels the autonomy to design and implement their own classroom assessments.

Despite limited research on Swedish teachers’ views on assessment and feedback, insights from Helgøy and Homme’s (2007) study, involving approximately 70 teachers and principals across seven schools in four major cities in Sweden and Norway, indicate that Swedish teachers, more so than their Norwegian counterparts, perceive national standardised tests as valuable tools for grading and enhancing teaching. In contrast to Norwegian teachers, Swedish educators do not view these tests as constraining their autonomy in interpreting national goals or organising their teaching methods to align with those goals.

Thus, we might expect a tendency among Swedish teachers to embrace a commitment to improvement-oriented conceptions of assessment, while also harbouring positive attitudes towards formal, standardised testing, acknowledging its possible uses for formative purposes. Furthermore, it is conceivable that teachers in closer proximity to the Year 9 decision point for entry into senior high school may demonstrate somewhat different belief priorities compared to their counterparts at the early stages of schooling.

Methods

This study used self-administered, self-reported survey inventories administered with a forced-choice ordinal agreement response scale at two time-points separated in time by 18 months to check model replicability over time (reliability). Analysis was done within the multiple indicators, multiple causes (MIMIC; Jöreskog & Goldberger, 1975) framework in which each survey item response is explained by a latent factor and a residual capturing the universe of unexplained variance. Further, each latent construct is manifested by multiple indicators. This study uses confirmatory factor analysis and invariance testing to establish the structure of responses and their stability over time. Furthermore, the study examines whether the strength of conceptions differed depending on teacher sex, possession of a teacher degree, length of teaching experience, or teaching stage. Teaching stage was chosen as a variable instead of schoolyear level since teachers in Years 1–6 will not teach the same schoolyear two consecutive years in a row, and teachers in Years 7–9 will teach students from different schoolyears the same year. Given the significant focus on the purpose and design of assessment within Swedish teacher education, it is reasonable to expect that certification status or possession of a teaching degree will influence teachers’ conceptions of assessment (TCoA). Similarly, teachers’ real-world experiences with assessment and its consequences are likely to shape their understanding and perceptions of assessment. Given the inconclusive previous results on the role of gender in teachers’ conceptions of assessment, albeit with a slight tendency towards studies finding no gender effects, we also investigated whether gender differences could be identified in the Swedish context.

Participants

A total of 461 teachers working between years 1 and 9 in a northern city in Sweden responded to the first wave of the survey. This is a 62% response rate from the municipality. A similar size group responded in the second wave of the survey held approximately 18 months later. After restricting the sample to those with complete data in both waves or those with <10% missing responses, 259 participants could be matched (Table 1). Before modelling began, multivariate outliers were identified with Mardia’s kurtosis method (Mardia, 1970). The cut score for the resulting Mahalanobis d² statistic was set at p = .001 for df equal to the number of variables in the model. Appendix A provides details of identification and sensitivity analysis for removal of 10 multivariate normality outliers. Removal of outliers at Time 1 made fit worse (|ΔAIC| = 15.69), indicating keeping the outliers is better. In contrast, removal of outliers created a superior fit at Time 2 (|ΔAIC| = 27.41). Since ΔAIC = Δ χ² (Sterba & Pek, 2012), a statistical significance test of these two values with 10 degrees of freedom change had p = .11 and .002, respectively. This indicated removing outliers had a not statistically significant effect on Time 1 but a statistically significant effect at Time 2. To ensure comparability between data sets, all 10 outliers were removed to make matched data sets (N = 249).

Table 1. Teacher demographic characteristics.

Demographic	Frequency n	%
Sex
Women	212	85
Men	34	14
Missing	2	0.4
Degree status
Held	205	82
Missing or no	44	18
Teaching experience
Less than 2 years	1	0.4
2–5 years	29	12
6–10 years	25	10
>10 years	168	68
Missing	26	10
School stage
Y1–3	83	33
Y4–6	76	31
Y7–9	76	31
Missing	14	6

Note. Matched for two administrations of the TCoA after outliers removed.

Consistent with school employment statistics, nearly with more than three factors, with.

of the sample were women and a similar proportion had a bachelor’s degree. Nearly ⅔ of the sample had more than 10 years teaching experience. A third taught Years 1–3, and nearly the same proportion taught in Years 4–6 and Years 7–9. Teachers indicated their specialist teaching subjects, with the option to indicate more than one subject. However, it is not possible to separate teachers into separate teaching subjects since in Y1–6 teachers are generalists teaching multiple subject areas. This means that, without studying exclusively teachers working above Year 6, it is not possible to identify possible subject or discipline-related differences.

Teachers conceptions of assessment (TCoA)

The Teachers Conceptions of Assessment (TCoA; G. T. L. Brown, 2003) inventory probes three dominant perspectives of assessment: (1) improvement of both teachers’ instruction and students’ learning on the grounds that assessment provides dependable diagnostic information; (2) demonstrating through assessment results that schools and students have fulfiled expectations; and (3) assessment is irrelevant or ignored because it is inaccurate or has bad effects (G. T. L. Brown, 2004). The New Zealand model comprises four correlated factors (i.e. Improvement, Irrelevance, School Accountability, and Student Accountability) and the first two factors had multiple subordinate factors (i.e. assessment is reliable, assessment is descriptive, assessment helps teachers, and assessment helps students within the improvement factor; assessment is bad, assessment has error, and assessment is ignored within the irrelevance factor). Each of the nine factors had three items.

The TCoA was developed in the low-stakes assessment context of New Zealand with primary school teachers and achieved replication with primary teachers in Queensland, Australia (G. T. L. Brown, 2006), and secondary teachers in New Zealand (G. T. L. Brown, 2011). Previous studies show that teachers endorse the improvement function, reject the irrelevance and assessment for school accountability conceptions, and have ambivalent views towards student accountability, with only secondary school teachers giving a somewhat positive endorsement of assessment for student accountability (G. T. L. Brown, 2017).

Nonetheless, the societal norms around high- vs. low-stakes uses of educational assessment impact teacher conceptions of assessment. Previous research comparing the TCoA survey results in Cyprus, Queensland, India, Hong Kong, China, Egypt, Spain, and Ecuador found that the New Zealand model was replicated only among primary teachers in Queensland (G. T. L. Brown et al., 2019), a context that shared the English language, an assessment for learning pedagogy, and low-stakes consequences for assessments. Hence, although the present survey uses primarily primary school teachers in a low-stakes assessment for learning context, it is still likely that linguistic and policy factors may necessitate modifications in the statistical model representing teachers’ conceptions of assessment to gain close correspondence with the data.

Survey design

The instruments were translated into Swedish by a translator, prioritising functional equivalence rather than literal equivalence. In contrast to back-translation which produces literal equivalence, functional equivalence prioritises similarity of meaning and well-constructed expression in the new language (International Test Commission, 2018). The translation was evaluated independently by three bilingual researchers who reported that the Swedish version had strong functional equivalence to the English and reflected appropriately assessment practices in Swedish terms.

A member checking (Tong et al., 2007) was conducted in two interview groups to find out how the teachers perceived certain items and understand the reasons why they answered the way they did. Ten teachers were recruited on a volunteer and convenience basis to participate. At separate occasions, two-group interviews (n = 4 and 6, respectively) were led by two of the authors. The interviews were focused on two items that indicated ambiguous teacher interpretation during analysis. In the interviews, the teachers were first given time to think individually about each issue and then shared their thinking in a joint discussion. The teachers had completed the survey, but their responses had been anonymous and so they were commenting on aggregate data results to which they had contributed. In these 1-hour discussions, we explored the respondents’ interpretations and views of the item content. The authors took field notes and conversations were audio-recorded and transcribed. The transcripts and field notes were analysed for themes by authors 2 and 5.

Items were presented in jumbled order. Participants responded using a positively packed, 6-point agreement scale. This type of scale has two negative options (Strongly Disagree and Moderately Disagree, scored 1 and 2 respectively) and four positive options (Slightly Agree, Moderately Agree, Mostly Agree and Strongly Agree, scored 3 to 6, respectively). This approach gives greater ability to discriminate the degree of positivity participants hold for positively valued statements and is appropriate when participants are likely to endorse statements (G. T. L. Brown & Shulruf, 2023). Hence, in circumstances when participants are expected to respond positively to a stimulus (e.g. teachers responding to a policy expectation), giving them more choices in the positive part of the response continuum produces good results.

Analysis

After deleting participants with >10% missing responses, and thereafter testing whether missing data were Missing Completely at Random (MCAR), the remaining missing values were imputed with the expectation maximisation algorithm (Dempster et al., 1977). This process creates a data set with no missing values for variance-covariance structural analysis. A statistically not significant result for Little’s (1988) Missing Completely at Random (MCAR) test or for the normed chi-square alternative (Wheaton et al., 1977) shows that the distribution of missing responses is completely random. At time 1, missing values ranged from none to 1.9%, with Little’s MCAR test not statistically significant at the normed chi-square level (χ² = 512.67, df = 455, p = .03; χ²/df = 1.13, p = .29). At time 2, missing values ranged from none to 2.3%, with Little’s MCAR test not statistically significant at the normed chi-square level (χ² = 728.86, df = 566, p < .001; χ²/df = 1.29, p = .26). Hence, the missing value imputation was accepted as a valid representation of teacher intended responses.

Just over half of the variables (30/54; 55%) violated normality for skewness (z > 3.29; Kim, 2013), while just six variables exceeded normality for kurtosis. All variables were corrected using the Box-Cox normalisation method implemented in Courtney and Chang (2018) normal R shiny app.¹ With this adjustment, normal theory covariance structure analysis was permissible (Bentler & Dudgeon, 1996).

While it is conventional to approach a new model with new data using exploratory techniques, in this case, there was a strong conceptual model that could legitimately be expected to have validity in this new context and sample. Consequently, we tested the original TCoA model for fit to the new data using confirmatory factor analysis, taking a true confirmatory approach. Our goal was to test if it were possible to recover the original model in order to permit comparison across samples, languages, and jurisdictions. In the event, a CFA model is admissible, but poorly fitting, modification indices can be consulted to identify items that violate simple structure (Revelle & Rocklin, 1979) or independence of residual assumptions (Barker & Shaw, 2015). Items with weak loadings (i.e. <.30; Bandalos & Finney, 2010) or with strong violation of model assumptions (i.e. loading on non-intended factors or item residuals) are good candidates for deletion.

Fit of CFA models is established by inspection of multiple fit indices (Hu & Bentler, 1999). Because the chi-square measure of discrepancy between a model and its underlying data is sensitive to sample size and model complexity, statistically non-significant values for the normed chi-square (i.e. χ²/df) provide evidence for correspondence between a model and its data (Wheaton et al., 1977). Further evidence for non-rejection of a model arises when the comparative fit index (CFI) is >.90 and the root mean square error of approximation (RMSEA) is <.08. However, both the CFI and RMSEA indices are sensitive to models with more than three factors, with the CFI tending to reject and the RMSEA tending to not reject under those conditions (Fan & Sivo, 2007). While the standardised root mean residual (SRMR) also varies with model conditions, that index can be relied upon even in large models with non-normal data. Although the sample SRMR will produce values greater than the population SRMR, suggesting that the model fits the data more poorly than it actually does (Maydeu-Olivares et al., 2018). Thus, greater reliance is put on the gamma hat >.90 and the standardised root mean residual (SRMR) <.08 because these are robust against sample size, model complexity, and model misspecification.

Scale reliability was estimated using the Coefficient H maximal reliability index, which is based on an optimally weighted composite using the standardised factor loadings (Hancock & Mueller, 2001). H >.80 provides strong evidence for the robustness of a scale.

To evaluate the impact of time on survey responses, invariance testing between two time-points was conducted. Nested invariance testing establishes if the item regression weights (i.e. metric equivalence) and intercept values (i.e. scalar equivalence) are equivalent across time. This method tests equivalence sequentially and stops if the additional constraint causes a change in the CFI index of more than .01 (G. T. L. Brown et al., 2017) or if a statistically significant result for the difference in chi-square, given the difference in degree of freedom, occurs. The equivalence of measurement weights is sufficient to meet requirements for creating and evaluating repeated scale scores over time (McArdle, 2007). However, if strict invariance is observed (i.e. equivalent weights at measurement and structural levels), we can conclude that the passing of 2 years did not change participant responses.

If at least metric invariance over time is achieved, factor means can be calculated by averaging the raw score for each item contributing to the factor. This unrefined method preserves the participant’s response scale metric producing easy interpretation of scores and is appropriate when simple structure (i.e. items belong to only one factor) is present (DiStefano et al., 2009). Differences in factor scores across time points were evaluated under the null hypothesis that there is no difference over time for each individual factor. Similarly, we evaluated individual hypotheses positing no differences between men and women, teachers at different career stages, those with or without a teaching degree, or those with varying levels of teaching experience. Since we did not test a joint hypothesis – namely, that there would be no differences for any of the factors across time points or demographic groups – no correction for multiple comparisons was required (Rubin, 2021).

CFA and invariance analyses were conducted in IBM SPSS Amos v29 (Amos Development Corp, 2022). Because 6-point ordinal scales function similarly to continuous variables (Finney & DiStefano, 2006), maximum likelihood estimation was used.

Results

Model development

Based on the Time 1 data, the original four-factor, hierarchical model described above (Panel A, Figure 1) was not admissible because the correlation matrix among the four main factors was not positive definite. Further, the Student Accountability factor had low prediction values loadings (i.e. <.30) for two items. Consequently, the whole factor was removed from the model. This produced an admissible but still somewhat poorly fitting model. Details of model development and items deleted are given in Appendix B.

Figure 1. Schematic model of New Zealand TCoA and Swedish TCoA results.

The Inaccuracy sub-factor in Irrelevance was removed because of strong violation of simple structure and two factors (i.e. Bad for Teaching and Student Improvement) had their residual values fixed to 0.005 to address negative error variance. This is permissible because the observed value was less than two times the standard error, meaning the true value was likely to be positive. Four of the Irrelevance items were removed for violating simple structure and independent residual assumptions. This left Irrelevance with two sub-factors (i.e. Assessment is Ignored and Assessment is Bad for Teaching).

In contrast, the hierarchical Improvement factor was fully replicated as four sub-factors each with three items. The School Accountability factor also was retained with its three items on assessment indicating school quality.

The Swedish TCoA model (Panel B, Figure 1) had 20 items in a correlated three-factor structure (i.e. Irrelevance, School Accountability, and Improvement), with four subordinate factors under Improvement and two subordinate factors for Irrelevance. Having established a highly similar model to the original model at Time 1, the Swedish TCoA model was tested in true confirmatory fashion with data from Time 2. This model had qualitatively acceptable to good fit indices at both time points (Time 1: χ²  = 319.23, df  = 163, χ²/df  = 1.96. p = .16; CFI = .89; gamma hat = .94; RMSEA = .062, 90%CI = .052–.072; SRMR = .063; Time 2: χ²  = 358.85, df  = 163, χ²/df  = 2.20, p = .14; CFI = .90; gamma hat = .93; RMSEA = .070, 90%CI = .060–.079; SRMR = .057) and this model was accepted as a good representation of Swedish teachers’ conceptions of assessment.

Table 2 provides factors, items in both Swedish and English, scale statistics, and loadings for both time-points. Just three of the seven factors had Coefficient H values >.80, suggesting less than robust scale characteristics. Overall, the fit indices support acceptance of this multi-dimensional self-report measure.

Table 2. Swedish TCoA factors, items, item loadings, and scale statistics.

Predictor	Loading
Dependent item/factor	2021	2023
Describe (2021: M = 4.02 (.80), H = .61; 2023: M = 3.93 (.86), H = .67; F(1,506) = 1.67, p = .20)
Dia1 Bedömning är ett sätt att fastställa hur mycket elever har lärt sig av undervisningen. [Assessment is a way to determine how much students have learned from teaching]	0.47	0.62
Dia2 Bedömning fastställer vad elever har lärt sig.[Assessment establishes what students have learned]	0.61	0.66
Dia3 Bedömningarna som görs i skolan mäter elevers förmåga till analytiskt och reflekterande tänkande. [Assessment measures students’ higher order thinking skills]	0.64	0.62
Improve Student Learning (2021: M = 4.10 (.96), H = .80; 2023: M = 4.33 (.85), H = .76; F(1,506) = 5.63, p = .02, η^2 = .011)
si1 Bedömningarna som görs i skolan ger återkoppling till elever om deras prestationer. [Assessment provides feedback to students about their performance]	0.63	0.61
si2 Bedömningarna som görs i skolan ger elever återkoppling på deras lärandebehov. [Assessment feedbacks to students their learning needs]	0.73	0.73
si3 Bedömningarna som görs i skolan hjälper elever att förbättra sitt lärande. [Assessment helps students improve their learning]	0.83	0.77
Improve Teaching (2021: M = 4.10 (.96), H = .68; 2023: M = 4.10 (1.03), H = .72; F(1,506) = 0.00, p = .99)
ti1 Bedömning är en integrerad del av undervisning. [Assessment is integrated with teaching practice]	0.61	0.61
ti2 Lärare modifierar pågående undervisning av elever baserat på bedömningsinformation. [Assessment information modifies ongoing teaching of students]	0.58	0.67
ti3 Bedömning möjliggör att olika elever kan få olika undervisning. [Assessment allows different students to get different instruction]	0.71	0.73
Valid (2021: M = 3.45 (.81), H = .74; 2023: M = 3.74 (.87), H = .78; F(1,506) = 14.85, p  < .001, η^2 = .029)
Rel1 Bedömningsresultat är tillförlitliga. [Assessment results are trustworthy]	0.61	0.71
Rel2 Bedömningsresultat blir desamma oavsett vem som bedömer. [Assessment results are consistent]	0.53	0.48
Rel3 Bedömningsresultat är pålitliga [Assessment results can be depended on]	0.81	0.83
School Quality (2021: M = 3.38 (.97), H = .82; 2023: M = 3.40 (1.05), H = .87; F(1,506) = 0.03, p = .86)
sq1 Bedömning ger information om hur bra det går för skolor. [Assessment provides information on how well schools are doing]	0.61	0.68
sq2 Bedömningsresultat är en rättvisande indikator på en skolas kvalitet. [Assessment is an accurate indicator of a school’s quality]	0.69	0.80
sq3 Bedömning är ett bra sätt att utvärdera en skola. [Assessment is a good way to evaluate a school]	0.87	0.90
Ignore (2021: M = 2.91 (.95), H = .61; 2023: M = 2.92 (1.03), H = .71; F(1,506) = 0.003, p = .96)
Ig1 Lärare genomför bedömningar men använder resultaten i låg utsträckning. [Teachers conduct assessments but make little use of the results]	0.56	0.59
Ig2 Bedömning är orättvist mot eleverna. [Assessment is unfair to students]	0.52	0.70
Ig3 Bedömningsresultat arkiveras och ignoreras. [Assessment results are filed & ignored]	0.65	0.69
Bad (2021: M = 3.10 (1.10) H = .47; 2023: M = 3.08 (1.16), H = .62; F(1,506) = 0.03, p = .87)
Ir4 Bedömning stör undervisningen. [Assessment interferes with teaching]	0.48	0.71
Ir5 Bedömningarna som görs i skolan har låg påverkan på undervisningen. [Assessment has little impact on teaching]	0.61	0.62
Improvement (2021: M = 3.93 (.70), H = 1.00; 2023: M = 4.02 (.74), H = 1.00; F(1,506) = 2.26, p = .13)
Improve Student Learning	1.00	1.00
Valid	0.71	0.79
Describe	0.88	0.98
Improve Teaching	0.86	0.86
Irrelevance (2021: M = 3.00 (.86), H = .98; 2023: M = 3.00 (.97), H = 1.00; F(1,506) = 0.005, p = .94)
Bad	0.99	1.00
Ignore	0.82	.86

N = 249, outliers removed; loadings are standardised beta weights; model is unconstrained.

Invariance

The sequential testing of constraints, based on the change in CFI (i.e. ΔCFI range: .000–.005), indicated full equivalence. The fit of this two-group, constrained model was good, especially for the more robust normed chi-square, gamma hat, and SRMR indices (χ²  = 922.98, df  = 393, χ²/df  = 2.35, p = .13; CFI = .85; gamma hat = .95; RMSEA = .052, 90%CI = .048–.057; SRMR = .071). Under the assumption of full equivalence, the scale inter-correlations (see Figure 1 for values) were identical. Improvement was positively but moderately correlated with School Quality and more robustly inverse with Irrelevance, suggesting that the teachers perceived that assessment was relevant for improvement and can be used to identify school quality. In contrast, the correlation between School Quality and Irrelevance was not different to zero, suggesting that the function of assessment results to demonstrate school quality elicited random responses as to assessment being irrelevant.

Scale scores

In general, mean scores (Table 2) were close to or slightly above moderate agreement for assessment helps teachers and students improve and assessment provides descriptive, diagnostic information. In contrast, both assessment provides reliable, valid information and assessment identifies school quality had means between slightly and moderately agree. Unsurprising, the means for assessment is irrelevant were just below slightly agree. Univariate ANOVA of mean scores by time showed a statistically significant increase for only two factors; that is, assessment helps students improve and assessment provides valid, reliable information. It is worth noting that both of these score differences for η² qualify as small effects (Cohen, 1988). All other factors had statistically not significant change in means across the two time-points. Hence, score differences over time were tiny or nil. Multivariate analysis of variance, using main effects and all two-way interactions, showed no statistically significant difference in scale means at either time point by teacher sex, possession of a bachelor’s degree, length of teaching experience, or teaching stage or any of the two-way interactions. Consequently, no further examination of possible univariate results was conducted.

Discussion

A Swedish adaptation of the New Zealand Teacher Conceptions of Assessment inventory (version IIIA) was administered twice, approximately 18 months apart, in one northern Sweden municipality. A sample of 249 teachers were matched after removal of outliers across the two time-points and their self-reported scores were analysed with confirmatory factor analysis, invariance testing, and means comparison. With good correspondence to the data, six of the nine factors in the TCoA-IIIA were completely replicated and the Assessment is Bad for Teaching factor was partially replicated (i.e. two of three items replicated). The model was fully equivalent between time points and statistically significant but small mean score differences were found between times for two factors. None of the teacher demographic characteristics had statistically significant relations to TCoA factor scores.

The student accountability factor identified in previous studies was completely absent in this study. Unlike the findings of a New Zealand interview study (Harris & Brown, 2009) in which teachers were very much opposed to the idea of holding students accountable, these items simply could not be explained by Swedish teachers’ conceptions of assessment. It is as if the conception of using assessment to evaluate and hold students accountable is simply non-existent. Within the Swedish context, the items for this construct seem to violate Swedish cultural norms and educational policies. For example, item sa1 (Assessment places students into categories) can be interpreted as measuring and sorting students themselves, which is not socially acceptable in Sweden’s egalitarian culture. Both items sa2 (Assessment is assigning a grade or level to student work) and sa3 (Assessment determines if students meet qualifications standards) target students’ competencies in relation to qualification standards. However, the national steering documents require that assessment of students’ competencies in relation to qualification standards should be done both formatively and summatively, a view also expressed in the member checking interviews. Further, item sa2 specifically concerns the assignment of grades, which are only given in Years 6 to 9. Given that the current sample had a majority of teachers working in earlier years, this may explain why the item did not work.

The School Quality factor (i.e. the conception that assessments can be used to judge the quality of schools) was fully replicated with modestly positive scores around slightly agree. The modestly positive correlation with Improvement is not obvious and grounds for explanations need to be explored in future studies with more comprehensive member checking.

The three items for the Inaccuracy factor (i.e. ir2 Assessment results should be treated cautiously because of measurement error; ir3 Teachers should take into account the error and imprecision in all assessment; and ir6 Assessment is an imprecise process) were completely removed in this study. These items refer to teachers taking into account the degree of error in any assessment or being cautious with potential misinformation from or within assessment. Information about the degree of uncertainty or imprecision in resulting scores is a characteristic most associated with standardised tests that can be used for formative purposes. Perhaps, when teachers use diverse (unstandardised) assessment methods, and also base their assessments on their own interaction with a class or an individual, judgements about error might not only be difficult to make but are also unlikely to happen.

One item linked to assessment having a negative or inappropriate influence upon teaching practices was removed from the Assessment is Bad for Teaching factor (i.e. ir1 Assessment forces teachers to teach in a way against their beliefs). Given the inverse relationship between the Improvement factor and the Irrelevance factor it is likely that, among these Swedish classroom teachers, assessment is embedded within teaching and does express the teacher’s curricular goals and pedagogical choices. Further, given the control Swedish teachers have over assessment practices, it is unlikely that they would consider assessment to have little impact on their teaching. The very essence of formative assessment is that assessment derives from teaching and informs further instruction. Nonetheless, the two remaining items for this factor were captured in this analysis showing that Swedish teachers were aware of the possibility that assessment could interfere with or have little impact on teaching. The mean score at both times was slightly agree suggesting there are instances in which assessment practices are seen as interfering with teaching.

Swedish teachers generally agreed that assessment helps teachers and students improve and provides diagnostic descriptions of student performance. There were small increases in endorsement of just two conceptions: that is, assessment supports student improvement and assessment is dependably reliable and valid. Consequently, there is much room for increased endorsement of these formative improvement-oriented conceptions. Nonetheless, the positive view of formative conceptions of the purpose and nature of assessment is a positive result, consistent with governmental priorities and cultural norms. This formative conception of assessment strongly mimics previously reported results from other low-stakes assessment contexts such as New Zealand and Queensland, Australia.

It may be that increased endorsement of the formative conception of assessment is constrained, not by teacher knowledge or beliefs, but rather the quality of assessment resources available to teachers. For example, New Zealand has deployed a computer supported diagnostic assessment system (the Assessment Tools for Teaching and Learning; G. T. L. Brown, 2013, 2019; G. T. Brown & Hattie, 2012) that teachers can use to identify student learning needs and monitor the effectiveness of their instructional practices. Those resources give classroom teachers the ability to effectively meet accountability expectations as well as support student learning. Such assessment protocols may help Swedish teachers further entrench the formative conceptions of assessment.

This study indicates that the Swedish adaptation of the TCoA can be used reliably in Swedish primary and lower secondary schools as a measure of their conceptions of the formative uses and purposes of assessment. Thus, it may be used in future studies to examine the need for, and effects of, professional development initiatives to support the development of teachers’ formative assessment conceptions. The model shows the structure of assessment beliefs. PD programs may administer the survey to participating teachers, so they are provided with data about their own conceptions. Time may then be allocated to discussions about their conceptions and relate them to research about effects of different kinds of assessment practices. Such PD components including information concerning research about effective assessment practices and teachers confronting their own conceptions with others may influence their conceptions towards endorsement of assessment for improvement (Dixon & Haigh, 2009). Such conceptions could affect their practices towards more formative assessment practices, which have been shown to positively affect student achievement.

Some limitations of the study can be identified. Rather than rely on a purely empirical, data-driven analysis (e.g. exploratory factor analysis), we used a model-driven theory that teachers have conceptions of assessment defined by four fundamental purposes of assessment. The model development protocols, in which items and scales were deleted, aimed to maximise the original New Zealand model. However, the 2021 model result has capitalised on existing data. Full replication with the 2023 data does suggest that there is reasonable stability in the observed results. Furthermore, the similarity of the Swedish TCoA results with the original New Zealand model does suggest some important similarities across very different societies and languages exist.

The measurements were performed in a single municipality in northern Sweden, which may reduce generalisability of the results. However, we posit that the findings here are likely representative of a broader cohort of Swedish teachers. While the school administration is local, national policy as manifested in, for example, national curriculum documents, national tests, guides for grading, and other national support material are the same nationwide in primary and lower secondary schools. However, assuring generalisability would require a national survey. The relatively modest coefficient H values (i.e. just three of 14 coefficients H > 0.80) indicate low stability. Further testing of the factor measurement models in an additional larger sample (ideally N > 500) of Swedish teachers is therefore warranted.

Since the data constitutes responses to a questionnaire, we made informed interpretations, corroborated by a small-scale member checking exercise, of the meaning of factors and path values. Subsequent more comprehensive qualitative studies exploring teachers’ interpretations of the questionnaire items by providing them with their own scores could provide further confidence in our explanations. Such work is essential in identifying whether there are additional and different conceptions of assessment in Sweden, than elsewhere. Such additional conceptions were found in Hong Kong (Hui, 2012) and it is likely there exist other ideas not yet identified in Sweden.

This study indicates that the Swedish Teacher Conceptions of Assessment adaptation can be used reliably in Swedish primary schools as a measure of teacher attitudes towards the uses of assessment in that context. The near replication of the New Zealand model may be understood in the light of the similarities of the Swedish and New Zealand contexts. Both have low-stakes assessment systems and emphasis on formative uses of assessment. However, the differences between the New Zealand and Swedish models found in this study underscore the need for validation (and, possibly, adjustments) of the inventory even when educational systems share many policy features.

Supplementary Material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/20004508.2024.2375111

Disclosure statement

The authors declare that there are no financial or conflict of interest issues in submitting this manuscript.

Additional information

Notes on contributors

Gavin T. L. Brown

Gavin T. L. Brown is a professor of education at the University of Auckland, focused on quantitative research methods and the psychology of assessment. He holds honorary positions in Umea, Thailand, and Hong Kong.

Catarina Andersson

Catarina Andersson is an associate professor of mathematics education. Her research focuses on how formative assessment can be implemented in schools and especially with students with disabilities.

Mikael Winberg

Mikael Winberg is a professor in mathematics and science education. His research focus is on how student epistemic beliefs and motivation affects their learning, and how these in turn are affected by the instructional design.

Björn Palmberg

Björn Palmberg is an associate professor in science and mathematics education. HIs research focuses on identifying and evaluating the best ways to teach mathematics.

Torulf Palm

Torulf Palm is a professor of science and mathematics education. His research focuses on formative assessment methods and impacts in mathematics instruction.

Notes

1. https://autopsych.shinyapps.io/normalr/.

Appendix A.

Multivariate Outlier Analysis Sweden TCoA

Mardia’s kurtosis test for multivariate outlier analysis was conducted in SPSS (v. 29.0.0). The full model had 27 manifest variables and 259 matched cases were available. Setting p = .001 for the 99.90% confidence interval, the critical value for Mahalanobis d² = 55.4760. This identified seven participants (2.7%) in Time 1 and five (1.9%) in Time 2 as multivariate outliers. Just two participants were outliers at both time points.

Table

Time 1		Time 2
ID	d^2	ID	d^2
85	57.50758
130	55.94020
		132	81.55013
135	58.13585
170	61.34894
356	80.61411	356	61.93047
389	66.87392
		405	56.65718
		461	76.88553
474	63.51504	474	61.59266

Best fit model (k = 20) comparison with and without outliers by time

Table

	Fit indices
Model	N	χ^2	Df	χ^2/df	CFI	$\hat{\gamma }$	RMSEA (90%CI)	SRMR	AIC
Time 1 All cases	259	303.54	163	1.862	.901	.95	.058 (.048–.068)	.061	397.54
Time 1 no outliers	249	319.23	163	1.958	.894	.94	.062 (.052–.072)	.063	413.23
Time 2 All cases	259	386.26	163	2.370	.886	.92	.073 (.064–.082)	.061	480.26
Time 2 no outliers	249	358.85	163	2.202	.900	.93	.070 (.060–.079)	.057	452.85

Removal of outliers at Time 1 made fit worse (|ΔAIC| = 15.69), indicating keeping the outliers is better. In contrast, removal of outliers created a superior fit at Time 2 (|ΔAIC| = 27.41). Since ΔAIC = Δ χ² (Sterba & Pek, 2012), a statistical significance test of these two values with 10 degrees of freedom change had p = .11 and .002, respectively. This indicated removing outliers had a not statistically significant effect on Time 1 but a statistically significant effect at Time 2. To ensure comparability between data sets, all 10 outliers were removed to make matched data sets (N = 249).

Sterba, S. K., & Pek, J. (2012). Individual influence on model selection. Psychological Methods, 17(4), 582–599. https://doi.org/10.1037/a0029253

Appendix B.

Model Development History showing model problems and solutions

Table

Model	Description	Problems
Lulea TCoA	Original NZ TCoA	SA factor low loadings (.00, .10, .31); correlations > 1.00 with Irrelevance and Improvement; covariance matrix not positive definite Ignore factor negative error variance
Lulea TCoA noSA	Remove SA factor	Fit not acceptable for CFI Inaccuracy factor residual MI > 10 to residuals of Factors Valid and Teaching
Lulea TCoA noSA ignfix noinacc	Inaccuracy factor removed, items loaded direct to irrelevance	Ir3 has low standardised loading (.16)
Lulea TCoA noSA ignfix noinacc noir3	Ir3 removed	Ir1 MI > 10 to SQ factor and sq3
Lulea TCoA noSA ignfix noinacc noir3,ir1	Ir1 removed	E439 (ir6) MI > 10 to 3 items & SQ factor
Lulea TCoA noSA ignfix noinacc noir3,1,6	Ir6 removed	Ir2 MI > 15 both ways to ti2
Lulea TCoA noSA ignfix noinacc noir3,1,6,2	Ir2 removed	E22 (SL factor) neg err var but 95%CI crosses zero
Lulea TCoA noSA sl,ignfix noinacc noir3,1,6,2	SL factor error var fixed to .005	Model had acceptable to good fit.

Note. N = 249, post normal R shiny box-cox transformation; post MVN outlier removal.

Deleted items

Table

Code	Item Text
sa1	Assessment places students into categories
sa2	Assessment is assigning a grade or level to student work
sa3	Assessment determines if students meet qualifications standards
ir1	Assessment forces teachers to teach in a way against their beliefs
ir2	Assessment results should be treated cautiously because of measurement error
ir3	Teachers should take into account the error and imprecision in all assessment
ir6	Assessment is an imprecise process