Formative assessment, growth mindset, and achievement: examining their relations in the East and the West

ABSTRACT

Both formative assessment and growth mindset scholars aim to understand how to enhance achievement. While research on formative assessment focuses on external teaching practices, work on growth mindset emphasises internal psychological processes. This study examined the interplay between three formative assessment strategies (i.e. sharing learning progressions, providing feedback, and instructional adjustments) and growth mindset in predicting reading achievement using the PISA2018 data. We focused specifically on samples from the West (the United States, the United Kingdom, Ireland, Canada, Australia, and New Zealand) and the East (Mainland China, Hong Kong SAR, Macau SAR, Chinese Taipei, Japan and Korea) which comprised of 109,204 15-year old students. The results showed that formative assessment strategies were positively, albeit weakly, related to a growth mindset in the East, but not in the West. In contrast, growth mindset was positively related to reading achievement only in the West, but not in the East. The impacts of different formative assessment strategies on reading achievement demonstrated cross-cultural variability, but the strongest positive predictor was instructional adjustments. These findings highlight the potential synergy between formative assessment and growth mindset in enhancing academic achievement as well as the importance of cultural contexts in understanding their roles in student learning.

KEYWORDS:

• Formative assessment
• growth mindset
• reading achievement
• PISA 2018
• sharing learning progressions
• feedback
• instructional adjustments

Introduction

Formative assessment has been widely recognised as a crucial facilitator of effective teaching and learning (Bennett, 2011; Black & Wiliam, 1998). Similarly, growth mindset has been found to be related to optimal learning-related outcomes, such as better academic achievement (Claro et al., 2016; Costa & Faria, 2018) and self-regulated learning (Burnette et al., 2012). Although both formative assessment and growth mindset emphasise identifying the gap between a student’s current achievement and the expected goal and then finding a way to close the gap (Black & Wiliam, 1998; Dweck, 2015), the research on these two areas have proceeded in parallel with limited cross-over of ideas.

Both formative assessment and growth mindset focus on enhancing performance. Formative assessment emphasises the external teaching process, while growth mindset underscores the importance of internal psychological processes. Theoretically, mindsets may function as interpretative lenses that filters what information students attend to and, therefore, could potentially mediate the effectiveness of formative assessment. As one of the hurdles of formative assessment is the failure to activate students’ responsibility and agency in the assessment process (Guo & Yan, 2018; Yan & Brown, 2021), investigating the role of growth mindset in the link between formative assessment and academic performance represents a meaningful direction to understand the mechanism through which formative assessment influences learning. Without taking students’ mindsets into account, it would be difficult to enhance their responsibility and agency in the formative assessment process.

Hence, this study aims to examine the interplay between formative assessment, growth mindset, and academic achievement in a reading test using data from PISA 2018. We also differentiate the three specific formative assessment strategies (i.e. sharing learning progressions, providing feedback, and instructional adjustments) so as to provide a nuanced picture about the relationships among the key variables.

Formative assessment

Formative assessment has been defined as activities that elicit evidence of student learning and can be used by teachers, students, or others to make decisions about future teaching or learning plans (Black & Wiliam, 1998). As indicated by the definition, formative assessment encompasses different practices in classrooms (Bennett, 2011). However, a lack of consensus on how formative assessment should be designed and implemented is a big challenge for researchers to interpret and compare the effects of formative assessment in different contexts (McMillan et al., 2013). As a response, Wiliam and Thompson (2008) proposed a framework in which various formative assessment practices are categorised into five key strategies and one ‘big idea’. The five formative strategies include 1) clarifying and sharing learning intentions and criteria for success; 2) engineering effective classroom discussions, questions, and learning tasks; 3) providing feedback that moves learners forward; 4) activating students as instructional resources for one another; and 5) activating students as the owners of their own learning (Wiliam & Thompson, 2008, p. 64). The ‘big idea’ refers to instructional adjustments aiming to cater for students’ needs that have been identified by formative assessment strategies.

Strategies 1 and 2 have different functions: the former points out what the students are expected to achieve, while the latter identifies students’ current performance levels. These two strategies, however, share a common feature – they both reflect and provide information about learning progressions that refer to the learning goal and the coordinated sequential teaching steps towards that goal (Catley et al., 2005; Duschl et al., 2011). Thus, Strategies 1 and 2 are operationalised as sharing learning progressions in this study. Strategy 3 (providing feedback) and the big idea (instructional adjustments) are supposed to indicate what needs to be done for students to achieve their learning goals. Strategies 4 and 5 highlight students’ own responsibilities in learning. As the PISA 2018 did not provide data on Strategies 4 and 5, this study focused on three formative assessment strategies: sharing learning progressions, providing feedback, and instructional adjustments.

The positive effect of formative assessment on academic achievement has been well documented. Black and Wiliam's (1998) meta-analysis reported effect sizes ranging from 0.4 to 0.7 favouring students receiving formative assessment. Other review studies (e.g. Bennett, 2011; Dunn & Mulvenon, 2009) also provided support for the efficacy of formative assessment in enhancing student academic achievement. In particular, formative assessment was found useful in enhancing students’ reading achievement. For instance, Förster and Souvignier (2014) found that learning progress assessment, similar to formative assessment, which provides teachers with diagnostic information on students’ learning progress had a positive impact on Grade 4 students’ reading achievement. Chen et al. (2019) also reported that formative assessment embedded into a reading annotation system significantly improved Grade 5 students’ reading comprehension performance through facilitating their self-regulated learning and reflection.

Growth mindset

The core idea of mindset theory is that students hold different ‘implicit theories’ or ‘mindsets’ about their intelligence (Dweck, 2017; Yeager & Dweck, 2012). For instance, some students view intelligence as something that they cannot change (a fixed mindset), whereas others view intelligence as something malleable that they can develop over time (a growth mindset). Mindsets are schematic knowledge structures that include beliefs about the stability of key attributes and organise the way students ascribe meaning to what happens to them in school. For example, the same event such as failing an exam can be interpreted in different ways. For a student with a growth mindset, failure may be seen as an opportunity to learn and strive harder next time. In contrast, for a student with a fixed mindset, failure may be interpreted as a verdict that one really is not good at a particular subject.

Because mindsets function as interpretative lenses, they have been found to predict important learning-related outcomes. Growth mindset has been associated with better academic achievement (Claro et al., 2016; Costa & Faria, 2018; Sisk et al., 2018), self-regulated learning (Burnette et al., 2012), mastery response after setbacks (Blackwell et al., 2007), and psychological well-being (King, 2012, 2017; Schleider et al., 2015), and enhanced sensitivity to task-relevant input leading to better learning (Schroder et al., 2014) among others. In contrast, having a fixed mindset is associated with lower levels of achievement, lower psychological well-being and giving up after failure (Smiley et al., 2016). With regard to the correlation between mindset and reading achievement, the findings in the literature are mixed. Petscher et al. (2017) found that both a global and language-specific growth mindset were associated with higher reading comprehension performance for elementary school students. In contrast, McCutchen et al. (2016) studied a sample of Grades 3–6 students and found that mindset was significantly related with maths achievement but not reading performance.

Despite the popularity of the growth mindset construct in both the popular press and scholarly circles, there are also challenges to its relevance. For example, a study conducted by Burgoyne et al. (2020) found that growth mindset effects are smaller than those commonly found for other motivational factors. One potential reason for the mixed or weak effects is the ongoing debate on the operationalisation of the growth mindset construct. In the literature, mindsets are construed both as a domain-general construct focusing on beliefs about intelligence (e.g. Blackwell et al., 2007) and a domain-specific construct focusing on specific subject areas such as maths or language (e.g. Degol et al., 2018; Lou & Noels, 2017). Indeed, Dweck (1996), the pioneer of mindset research, acknowledged that mindsets could be both domain-general and domain-specific. In PISA 2018, mindset was operationalised as a domain-general factor pertaining to students’ beliefs about the malleability of their general intelligence.

These challenges, notwithstanding, mindset proponents have made a strong case for the role of mindsets in learning and achievement. However, most of the existing mindset studies have been conducted within one cultural context and using convenience samples. We contribute to this ongoing discussion by drawing on a large-scale international dataset such as PISA that includes student participants from both the East and the West, thereby expanding the generalisability of past research.

The link between formative assessment and growth mindset

The research on formative assessment and growth mindset have proceeded in parallel with different foci: the former reflects a pedagogical approach while the latter mostly focuses on a psychological approach. Both formative assessment and growth mindset research emphasise telling the truth about a student’s current achievement and then finding a way to help him or her perform better (Black & Wiliam, 1998; Dweck, 2015). Hence, both are areas with great potential for improved practice to lead to improved learning outcomes (Bibbens, 2018) and there are possible synergies between them.

Theoretically, formative assessment practices directing students’ attention to learning process and strategies may nurture a belief that the learning ability is malleable, and the desirable outcomes could be achieved through hard work, effort, and the use of effective strategies. Through a focus on revision and progress towards better achievement, formative assessment has the potential to encourage a growth mindset among students (Sanchez et al., 2017). The learning gains brought by formative assessment might further strengthen such a belief that appreciates and acknowledges the value of hard work, effort, and effective strategies. Past studies argued that growth mindsets can be taught (Blackwell et al., 2007; Yeager et al., 2016). Most importantly, teachers’ practices influence their students’ mindsets (Haimovitz & Dweck, 2017). For example, Sun (2015) reported that teachers who adopted process-oriented teaching (such as frequent questioning, providing chances for explaining and revisions) promoted a growth mindset among students.

In addition, the way students react to formative feedback is likely to be guided by their mindsets (Sperling & Shapcott, 2012). A growth mindset is likely linked to better use of feedback, while a fixed mindset might result in maladaptive responses to feedback which, in turn, might have negative impact on learning outcomes (Mangels et al., 2006). However, empirical evidence focusing on the link between formative assessment and mindset, especially with large samples, is still limited.

Culture, formative assessment, and growth mindset

Another important factor often neglected in studies on formative assessment and growth mindset is the role of culture. Culture refers to the set of values, beliefs, and traditions that influence the behaviours of a social group (Triandis, 2002). Cultural context could influence both the formative assessment strategies teachers use as well as the implicit beliefs students have about their intelligence. Though the notion of culture is very complex, most education scholars attempt to simplify this complexity by juxtaposing Eastern versus Western cultures (Li, 2002, 2003, 2005). Eastern cultural contexts are usually synonymous to Confucian heritage cultures, while Western culture is synonymous to North America and English-speaking societies.

Culture might impact assessment beliefs and practices (Yan & Brown, 2021; Teasdale & Leung, 2000) which, in turn, not only influence teachers’ approaches to assessment (Davison, 2004), but also impact the ways in which students engage with assessment tasks (Carless, 2011) as well as their approaches to learning and studying (Struyven et al., 2005). Thus, the interpretation of the process and consequence of assessment needs to take the role of culture into account. Assessment principles that work well for one culture might not work in another culture if the principles are not well integrated with the local context. As Yan and Brown (2021) reported, the implementation of formative assessment, originated from the Western culture, is far less than satisfactory in Eastern classrooms where the use of public examinations for important decision-making prevails. A particular assessment practice could lead to varied consequences under different social norms. For instance, although Confucian heritage culture is believed to prioritise summative assessment at the expense of formative assessment (Wicking, 2020), preparation for high-stakes, summative assessments often lead to desirable self-regulated learning activities (Brown, 2021). What is regarded as rote learning in the West is actually a combination of memorisation and deeper levels of learning, rather than being simply equivalent to surface learning, in the East (Biggs, 1999; Kember & Wang, 2016).

Culture might also affect mindset and how it relates to key outcomes. Though growth mindsets have been found to be beneficial across different cultures (Grigorenko et al., 2001), there seems to be some cultural differences. For example, the concept of what intelligence is might vary across cultural contexts (Sternberg, 2004). Another cross-cultural difference is the way mindsets are associated with other variables in the nomological network. Studies have found that having a fixed mindset was found to be more detrimental for North American students compared to Chinese students (Burnette et al., 2012). Growth mindset was also associated with both learning/mastery and ego/performance goals among Chinese students (Chen & Wong, 2015). Though the relationship between growth mindset and mastery goal is cross-culturally universal (e.g. Burnette et al., 2012), the positive association between performance approach and growth mindset was largely absent in Western cultural contexts. For Chinese students, it might not be sufficient to merely master one’s academic tasks (i.e. mastery goal) but also to demonstrate academic success to one’s parents, family members, and teachers, which is more akin to performance goals.

Though there is relatively little research on how cultural forces impinge on formative assessment and growth mindsets, past empirical research has shown that culture might be an important yet neglected aspect of formative assessment and mindset. Motivational factors that are salient in one cultural context might not be as salient in other cultural contexts (King et al., 2018; King & McInerney, 2014; 2019; Li & Yamamoto, 2020). Bernardo et al. (2021) found that cultural beliefs affect the endorsement of a growth mindset and how mindsets are related to achievement outcomes. In their study, they found that in cultures wherein people believe that there are multiple routes to achieving a given outcome, the importance of growth mindset decreases. This is perhaps due to the understanding that intelligence and effort are not the only routes to academic success. Given that growth mindset research originated in North American Western contexts, there is a need to test the role of growth mindsets in other cultural contexts, which this study aims to address by explicitly focusing on Eastern and Western cultures.

This study

This study aims to bridge the fields of formative assessment and mindset to examine the relationship between formative assessment practices, student mindset, and academic achievement using the PISA 2018 data set. To provide a clear picture of the effect of formative assessment, we examined the role of three specific formative assessment strategies that are available in the PISA 2018 data, i.e. sharing learning progressions, providing feedback, and instructional adjustments. The specific questions guiding this study are:

1. What is the relationship between student reports of teachers’ formative assessment practices, students’ mindset and students’ reading achievement?

2. Does mindset mediate the relationship between formative assessment practices and reading achievement?

3. Is there a cultural difference regarding the relationship between student reports of teachers’ formative assessment practices, students’ mindset and students’ reading achievement?

Method

Data and participants

The study utilised the PISA 2018 data set, which consists of more than 600,000 15-year-old nationally representative students from across 79 countries/economies. We drew our data from Western English-speaking cultures (the United States, the United Kingdom, Ireland, Canada, Australia and New Zealand) and Eastern Confucian cultures (Mainland China, Hong Kong SAR, Macau SAR, Chinese Taipei, Japan and Korea) which has a total of 109,204 students. We excluded student participants with more than 5% missing values on the key variables of the study, limiting our sample to 41,632 students from the English-speaking countries/economies (51% female) and 41,208 students from Confucian countries/economies (49% female).

PISA uses a complex two-stage sampling method in recruiting student participants. This means that PISA samples various types of schools and subsequently samples the students within these schools. This process opens secondary data analysis to possible errors due to variance in (1) the probability of selecting schools and students within schools across participating countries/economies, (2) the adjustments done in students’ non-response across types of schools and characteristics, and (3) oversampling in selected types of school for national-level reporting (OECD, 2009b). The PISA dataset assigns 80 replicate weights based on Fay’s version of balanced repeated replication and a final weight for each student to account for these variances and errors. Furthermore, these weight replicates reflect the clustered characteristics of the data (Rutkowski et al., 2010). The current study utilised these replicate weights to ensure more robust statistical estimates (OECD, 2009b).

Measures

PISA is a triennial assessment programme that measures students’ achievement scores in three learning domains (i.e. reading, mathematics, and science), with one domain as the specific focus in every cycle. PISA 2018 focused on reading as its primary domain of assessment based on a literacy framework that operationalised reading as ‘understanding, using, evaluating, reflecting on and engaging with texts in order to achieve one’s goals, to develop one’s knowledge and potential and to participate in society’ (OECD, 2019a, p. 28).

Aside from standardised assessment, PISA 2018 had a survey component that included demographic information and other non-cognitive contextual measures associated with reading literacy. This study examined the questionnaire items related to the students’ growth mindset and perceived formative assessment. We also looked at the students’ demographic details and their achievement score in reading.

Independent variables

We selected items from the PISA student questionnaire that reflect teachers’ formative assessment activities and clustered them into three strategies: (1) sharing learning targets and progress, (2) providing feedback, and (3) instructional adjustments. Please refer to Table 1 for a summary of items, their classification, the scales used, and their internal consistency coefficients. We reversed coded sharing learning progressions to be consistent with the direction of other variables.

Table 1. Descriptive statistics and internal consistency

Variables		Over-all Sample		English Speaking		Confucian
	$\alpha$	M	SD	M	SD	M	SD	Scale	Items
Reading Achievement		512.71 (1.77)	103.70 (0.86)	505.18 (2.83)	106.74 (1.24)	523.96 (1.49)	97.88 (.93)
Growth Mindset		2.81 (.01)	.91 (0)	2.90 (.01)	.93 (.01)	2.68 (.01)	.86 (0)	1 – Strongly Agree; 4 – Strong Disagree	‘Your intelligence is something about you that you can’t change very much.’
Sharing Learning Progressions	.80	3.02 (.01)	.73 (0)	2.97 (.01)	.73 (.01)	3.13 (.01)	.72 (0)	1 – Every Lesson; 4 – Never or Hardly Ever	^ST102Q01TA The teacher sets clear goals for our learning. ^ST102Q02TA The teacher asks questions to check whether we have understood what was taught. ^ST102Q03TA At the beginning of a lesson, the teacher presents a short summary of the previous lesson. ^ST102Q04TA The teacher tells us what we have to learn.
Providing Feedback	.90	2.50 (.01)	.89 (0)	2.60 (.01)	.87 (.01)	2.33 (.01)	.89 (0)	1 – Never or almost never; 4 – Every lesson or almost every lesson	^ST104Q02NA The teacher gives me feedback on my strengths in this subject. ^ST104Q03NA The teacher tells me in which areas I can still improve. ^ST104Q04NA The teacher tells me how I can improve my performance.
Instructional Adjustments	.79	2.64 (.01)	079 (0)	2.62 (.01)	.78 (0)	2.67 (.01)	.80 (0)	1 – Never or almost never; 4 – Every lesson or almost every lesson	^ST212Q01HA The teacher adapts the lesson to my class’s needs and knowledge. ^ST212Q02HA The teacher provides individual help when a student has difficulties understanding a topic or task. ^ST212Q03HA The teacher changes the structure of the lesson on a topic that most students find difficult to understand.

Note: Based on samples from English-speaking and Confucian countries/economies (n = 109,204). Values in parentheses are standard errors.

Source: Programme for International Student Assessment (PISA) 2018.

Dependent variable

Our study focuses on reading achievement as our key dependent variable since PISA 2018’s main assessment domain is reading. We used the students’ reading proficiency scores in PISA to operationalise reading achievement. These scores were scaled to have a mean of 500 and a standard deviation of 100 using the Rasch model.

The reading proficiency scores, however, were not actual student test scores but were derived values. Given the range of items to be covered within a limited testing time, PISA was administered using a complex test booklet rotation scheme to minimise the cognitive burden on students (Kaplan & Su, 2018; Rutkowski et al., 2010). In this scheme, students only answered test questions from a block of test items representing the whole range of assessments. Then, the students’ proficiency scores were drawn from the population’s ability distribution derived from individual students’ performance on their assigned test questions (OECD, 2009b; Rutkowski et al., 2010). These individual proficiency estimates are referred to as plausible values indicating the range of possible scores that the students could reasonably have achieved in the test (OECD, 2009a, 2009b).

The PISA 2018 database provides 10 reading proficiency plausible values. Our study used these 10 plausible values in computing the estimates following the PISA data analysis manual’s (OECD, 2009b) procedure for estimation using plausible values.

Mediator

The mediator variable is growth mindset. We took this variable from the students’ response to PISA’s single-item question, which states: ‘Your intelligence is something about you that you can’t change very much’. The item was rated on a 4-point Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree). We reverse coded the values since the item indicated a fixed mindset such that higher scores reflected a stronger growth mindset.

Covariates

We included the following demographic details as covariates: socioeconomic status, gender, repetition of a grade level, and class size. PISA derived an index of economic, social, and cultural status (ESCS) to operationalise socioeconomic status. ESCS is an index derived from ‘the highest educational level of parents, the highest occupational status of parents and possessions in the home’ (OECD, 2009a, p. 84). The OECD average was pegged at zero with index values within a standard deviation of one. Consistent with its previous cycles, the PISA 2018 results showed that students with low ESCS have underperformed compared with students with high ESCS. Likewise, female students have a clear performance advantage over male students. Students who had repeated a grade level have also been found to have lower performance than those who did not. PISA also reported that after accounting for student and school ESCS, class size did affect, albeit minimal, student achievement. We controlled for these covariates to account for their potential confounding effects on this study.

Analytic strategy

Missing data

We excluded cases with no available data for the study’s variables of interest. Hence, the Canadian sample was not included in the study since it had no information on sharing of learning targets and progress. The rest of the samples have isolated high missing data; we assumed that these are due to PISA’s complex survey design. High missing data is expected in large scale assessments, but these can be deemed missing ‘by design’ and missing completely at random (Grund et al., 2021; Kaplan & Su, 2018). We employed a multiple imputation method using multivariate imputation via changed equations (MICE) package in R (R Core Team, 2019; Rubin, 1996; Van Buuren & Groothuis-Oudshorn, 2011). This approach is aligned with past studies that drew on the PISA dataset (e.g. Chiu et al., 2017, 2016).

Preliminary analysis

We generated the descriptive statistics of the key variables and covariates using the intsvy (Caro & Biecek, 2017), survey (Lumley, 2004), and svyPVpack (Reif & Peterbauer, 2015) packages in R. These statistical software took into account the 10 plausible values, weights and the 80 balanced repeated replication values that come with the PISA dataset in the analysis. We reported the statistical estimates and their standard errors consistent with the PISA data analysis manual (OECD, 2009b). Then, we proceeded with the bivariate correlation analysis and reported the Pearson’s r coefficient and the associated standard error due to sampling design.

We focused on effect size to determine the magnitude of association. We did not use Cohen’s (1988) general guideline for analysing effect sizes since this rule of thumb has been found to be outdated and overly optimistic (Gignac & Szodorai, 2016; Schäfer & Schwarz, 2019). Effect sizes in social sciences and psychology research tend to be smaller but no less meaningful (e.g. Bosco et al., 2015; Ferguson, 2009). For heuristic purposes, we followed Bosco et al.’s (2015) finding that a |r| coefficient between .1 to .25 indicate medium effect size for associations that involve attitudes and behaviours.

Primary analysis

We posited that formative assessment practices (i.e. sharing learning progressions, providing feedback, and instructional adjustments) would positively correlate with student achievement via growth mindset. To test this hypothesis, we created a structural equation model and fitted the model separately with the two sets of data–one for the English sample and another one for the Confucian sample – in R lavaan (R Core Team, 2019; Rosseel, 2012). The first set of data is from the English-speaking countries/economies: the United States, the United Kingdom, Ireland, Australia, and New Zealand. The second set of data is from the Confucian countries/economies: Mainland China, Hong Kong SAR, Macau SAR, Chinese Taipei, Japan, and Korea. We used ESCS, gender (being female), repeating a grade, and class size as covariates to account for their confounding effects. Figure 1 shows our hypothesised mediation model.

Figure 1. Conceptual framework of association between formative assessment practices and reading achievement as mediated by growth mindset

Analysis using plausible values and weights

Studies on the use of PISA dataset in secondary analysis have recommended that researchers compute individual estimates for each plausible value and aggregate them using Rubin’s method to derive the final estimate (Laukaityte & Wiberg, 2017; Rubin, 2004; Rutkowski et al., 2013). The PISA data analysis manual also emphasised the general rule is to use the 80 weight replicates in the dataset to consider the sampling variances for each estimate (OECD, 2009b, p. 129). This study followed the recommended procedures in computing statistical estimates.

Structural equation model analysis

We conducted a structural equation model analysis using lavaan package in R (Rosseel, 2012) and followed Anderson and Gerbing (1988) two-step approach in structural equation modelling for our statistical estimates. First, we conducted a confirmatory factor analysis using a robust maximum likelihood estimator. When the sample size is large, the chi-square statistic tends to be rejected. Hence, we focused on the other fit indices using the following conventional goodness of fit criteria: CLI and TLI $\ge$ 0.95 indicates a good fit while CLI/TLI $\ge$ 0.90 is acceptable; RMSEA/SRMR $\le$ 0.05 indicates a good fit while a value $\le$ 0.08 is acceptable (Hu & Bentler, 1995).

Next, we conducted the structural equation modelling using robust maximum likelihood to estimate the parameters. While the PISA data showed that students are nested within schools and countries/economies, we did not employ a multilevel structural equation as our parameters of interest are all within the student level. Instead, we applied the design-based method by fitting our structural equation model using the weights and replicates provided by the PISA dataset (Lumley, 2004; Stapleton et al., 2016). This procedure takes into account the nested nature of our data.

Individual structural equation models were created for each plausible value of reading achievement of our two samples. To account for PISA’s sampling design in our structural equation model estimates, we relied on Oberski’s (2014) lavaan.survey package. This package refits the structural equation model using the final weight and its 80 replicates in the PISA dataset. Then, the final parameter estimates were aggregated at the last stage (Little & Rubin, 2019; OECD, 2009b; Rubin, 2004). We computed the final standard error from the combined average sampling and imputation variance of the estimates (OECD, 2009b). Finally, we examined the significance of indirect effects of formative assessment practices on students’ reading achievement via growth mindset to confirm our hypothesis.

Results

Descriptive statistics and bivariate correlations

Table 1 shows the descriptive statistics of the independent variables, mediator, and dependent variable for the English-speaking and Confucian samples. Our correlation analysis found some common results among the two sets of samples. First, sharing learning progressions, providing feedback, and instructional adjustments are strongly correlated. Second, these formative assessment variables have weak associations with growth mindset. Third, sharing learning progressions and providing feedback are weakly correlated with reading achievement. Finally, adaptive teaching is moderately correlated to reading achievement in both samples. The key contrast between the two samples is that growth mindset has a strong correlation with reading achievement among English-speaking while they are only weakly correlated with achievement in the Confucian sample. Tables 2 and 3 below show the respective summary of variable correlations for the English-speaking and Confucian countries/economies. We also ran a correlation analysis per country/economy and included the results as an appendix due to space constraints (please refer to Appendix Table A).

Table 2. Bivariate correlations in the English-speaking sample (n = 44,679)

	Reading Achievement	Growth Mindset	Sharing Learning Progressions	Providing Feedback	Instructional Adjustments	ESCS	Class Size	Sex
Growth Mindset	0.24*** (0.01)
Sharing Learning Progressions	−0.07*** (0.01)	0.02*** (0.01)
Providing Feedback	0.05*** (0.01)	0.05*** (0.01)	0.40*** (0.01)
Instructional Adjustments	0.11*** (0.01)	0.06*** (0.01)	0.43*** (.01)	0.55*** (0.01)
ESCS	0.34*** (.02)	0.12*** (0.01)	−0.02*** (.01)	0.07*** (0.01)	0.10*** (0.01)
Class Size	0.03*** (0.02)	0.02** (0.01)	−0.03*** (.02)	−0.02*** (0.02)	−0.04*** (0.02)	−0.03*** (0.02)
Sex	0.11*** (0.01)	0.03*** (0.01)	−0.05*** (0.01)	−0.06*** (0.01)	0.00 (0.01)	−0.02*** (0.01)	0.00 (.01)
Repeated a Grade	−0.27*** (0.01)	−0.10*** (0.01)	0.05*** (.01)	−0.01** (0.01)	−0.03*** (0.01)	−0.18*** (0.01)	−0.01* (0.02)	−0.04*** (0.01)

Source: Programme for International Student Assessment (PISA) 2018.

Note: Standard error estimates are enclosed in parentheses. *** p < .001; ** p < .01; *p < .05

Table 3. Bivariate correlations in the Confucian sample (n = 41,872)

	Reading Achievement	Growth Mindset	Sharing Learning Progressions	Providing Feedback	Instructional Adjustments	ESCS	Class Size	Female
Growth Mindset	.03*** (.01)
Sharing of Learning Targets and Progress	.06*** (.01)	.08*** (.01)
Providing Feedback	.03*** (.01)	.07*** (.01)	.42*** (.01)
Instructional Adjustments	.10*** (.01)	.06*** (.01)	.45*** (.01)	.58*** (.01)
ESCS	.21*** (.01)	.01 (.01)	0 (.01)	.01* (.01)	.01* (.01)
Class Size	.22*** (.03)	.01** (.01)	.01** (.02)	−.02*** (.02)	−.02*** (.02)	−.15*** (.02)
Female	.09*** (.01)	−.04*** (.01)	−.02*** (.01)	−.11*** (.01)	0 (.01)	.02*** (.01)	0 (.01)
Repeated a Grade	−.14*** (.01)	.02* (.01)	−.01 (.01)	0 (.01)	−.02*** (.01)	−.16*** (.01)	.05*** (.02)	−.04*** (.01)

Source: Programme for International Student Assessment (PISA) 2018.

Note: Standard error estimates are enclosed in parentheses. *** p < .001; ** p < .01; *p < .05

Structural equation model

We posited a structural equation model wherein formative assessment practices (i.e. sharing learning progressions, providing feedback, and instructional adjustments) predicted student achievement via growth mindset. Confirmatory factor analysis results showed an excellent model-data fit under robust maximum likelihood estimator for both samples. Fit indices for the English-speaking sample were χ² = 329.45, df = 32, p < .001, scaling correction factor = 7.79; CFI = .99, TLI = 0.99, RMSEA = 0.04 with 90% Confidence Interval = [.036, .044] and SRMR = 0.02. The Confucian sample’s fit indices were χ² = 1318.63, df = 32, p < .001, scale correction factor = 3.15; CFI = .98, TLI = .97, RMSEA = 0.06 with 90% Confidence Interval = [.05, .06] and SRMR = 0.02.

After ascertaining the model-data goodness of fit, we added the structural component and proceeded with the path analysis. After refitting the structural equation to the survey design, the structural equation models consistently demonstrated good fit both for the English-speaking sample (χ² = 1027.32, df = 86, p < .001, scaling correction factor = 7.39; CFI = .97, TLI = .96, RMSEA = 0.04 with 90% Confidence Interval = [.04, .05], and SRMR = 0.03) and the Confucian sample (χ² = 1769.76, df = 86, p < .001, scaling correction factor = 3.31; CFI = .97, TLI = .96, RMSEA = 0.04 with 90% Confidence Interval = [.04, .05], and SRMR = 0.03). The model fit was consistent across the different SEM repetitions done for each plausible value across both English and Confucian samples so we no longer reported the values here.

Figures 2 and 3 show the structural equation models for the two samples showing the final standardised path estimates and their associated standard errors after controlling for the covariance.

Figure 2. Structural equation model of association between dimensions of formative assessment and reading achievement via growth mindset among English-speaking countries/economies. For clarity of presentation, error terms and estimates of covariances were not shown.***p < .001 *p < .05

Figure 3. Structural equation model of association between dimensions of formative assessment and reading achievement via growth mindset among Confucian countries/economies. For clarity of presentation, error terms and estimates of covariances were not shown. ***p < .001 *p < .05

Our structural equation model for the English-speaking sample showed a direct effect of growth mindset on reading achievement (ß = .20, SE = 1.39, p < .001). The relation between growth mindset and the three formative assessment strategies were not significant. Sharing learning progressions had direct negative and significant relation with reading achievement (ß = −.11, SE = 2.79, p < .001), while instructional adjustments had direct positive and significant relation with reading achievement (ß = .14, SE = 2.99, p < .05). The indirect effects of the three formative assessment strategies via growth mindset were not significant: sharing learning progressions (ß = .004, SE = .67, p = .355), providing feedback (ß = .002, SE = .59, p = .615), and instructional adjustments (ß = .01, SE = .71, p = .198).

The structural model for the Confucian sample did not show direct effect of growth mindset on achievement (ß = −.02, SE = .58, p > .05). Two formative assessment strategies showed significant and positive relation with growth mindset: sharing learning progressions (ß = .07, SE = .05, p < .001), and providing feedback (ß = .06, SE = .05, p < .001). Instructional adjustments had direct positive and significant relation with reading achievement (ß = .14, SE = 1.78, p < .05), while providing feedback had direct negative and significant relation with reading achievement (ß = −.09, SE = 1.43, p < .001). The formative assessment strategies of sharing learning progressions (ß = −.001, SE = .014, p = .063), providing feedback (ß = −.001, SE = .12, p = .086), and instructional adjustments (ß = .002, SE = .08, p = .312) did not have significant indirect effects on reading achievement via growth mindset.

Discussion

The purpose of this study was to investigate the relationship between formative assessment practices (i.e. sharing learning progressions, providing feedback, instructional adjustments), growth mindset, and academic achievement using the PISA 2018 data set. We also examined potential cultural differences regarding the relationship between Western English-speaking cultures (the United States, the United Kingdom, Ireland, Canada, Australia, and New Zealand) and Eastern Confucian cultures (Mainland China, Hong Kong SAR, Macau SAR, Chinese Taipei, Japan, and Korea).

The relationship between formative assessment and growth mindset

The correlational analysis revealed that all formative assessment strategies were positively and significantly related to growth mindset in both the East and the West. The SEM results showed that students who reported receiving more formative assessment practices were more likely to have a growth mindset only in the East but not in the West. In the East, two formative assessment strategies (sharing learning progressions and providing feedback) were positively, and weakly related to growth mindset. In the West, all the associations were weak and non-significant. These results partially support the claim that formative assessment helps develop a growth mindset among students, although the support is weak and constrained in the Eastern samples.

We speculate that the association between formative assessment strategies and growth mindset was underestimated due to the way PISA 2018 assessed these constructs. Growth mindset was assessed with only one item and it would have been better if multi-item measures that were more psychometrically robust were used. This, however, was not possible given the way PISA designed the questionnaire. Second, the growth mindset assessed in PISA was about general, but not domain-specific, growth mindset. However, formative assessment items specifically asked about teachers’ strategies in the language lessons. The different levels of specificity in the measurement of the two constructs might reduce the associations. Recent studies have argued the need to measure language-specific growth mindsets although research in this area is still very nascent as past studies have mostly focused on general mindsets (see King & Trinidad, 2020; Lou & Noels, 2016, 2017 for domain-specific examples). Our study is among the first to test the relationships between formative assessment and growth mindset using a large cross-sectional sample, and the findings must be interpreted with caution. To better test the correlation between these two constructs, future studies may consider using domain-specific instruments for growth mindset, using longer multi-item surveys that are more psychometrically valid, and using more rigorous data collection plans (e.g. longitudinal, or experimental designs).

The relationship between formative assessment and reading achievement, and the mediating role of growth mindset

Instructional adjustments directly predicted reading achievement in both the East and the West. The path coefficients (ß = .14 for both the Eastern and Western samples) reached the threshold of medium effect size according to Bosco et al.’s (2015) criterion and was larger than that of the other two strategies. Researchers have argued that formative assessment may not be as effective as expected if the practices (e.g. sharing learning progressions, providing feedback) are not accompanied with necessary instructional adjustments to meet students’ learning needs (Briggs et al., 2012; McMillan et al., 2013). This finding is in line with these arguments and highlights that instructional adjustments are essential to effective formative assessment (Wiliam, 2010; Wiliam & Leahy, 2007). Although the effect sizes reported in previous reviews of the effect of formative assessment (e.g. Bennett, 2011; Black & Wiliam, 1998; Dunn & Mulvenon, 2009) are usually larger than the effect size we found here, a direction comparison is inappropriate due to two reasons. First, as H. Li (2016) pointed out, the effect size of formative assessment reported in previous reviews are mainly about the differences between the experimental and the control groups in experimental studies, while the relationship revealed in the present study is derived from a cross-sectional sample that were nationally representative. The tight controls in experimental settings may lead to stronger effects but the effects we found might be more ecologically valid given the large sample size and representativeness of our sample. Second, as formative assessment is an umbrella term that covers various forms of strategies, the effect of a specific strategy (i.e. instructional adjustments in this study) is not directly comparable to the effect of formative assessment in general, or another different strategies in particular.

The effects of the other two formative assessment strategies on reading achievement demonstrated cross-cultural variability. The relation between sharing learning progressions and reading achievement was non-significant in the East, but significantly negative in the West. Conversely, providing feedback was negatively related to reading achievement in the East, but the relation was not significant in the West. The weak and non-significant relations (sharing learning progressions and reading achievement in the East; providing feedback and reading achievement in the West) reiterate that sharing learning progressions or providing feedback alone might not guarantee greater achievement (Shute, 2008; Van der Kleij et al., 2015). A general consensus is that formative assessment is useful only if students make sense of the formative feedback generated from the assessment process and actively use it (Carless & Boud, 2018; Hattie & Timperley, 2007). PISA 2018 has examined teachers’ formative assessment practices without gauging students’ engagement (e.g. interpret and/or use the feedback) in the formative assessment process. This missing information on an important link in the formative assessment loop calls for special caution in interpreting these results.

The negative relations (sharing learning progressions and reading achievement in the West; providing feedback and reading achievement in the East) are even more counterintuitive, but partially echoes the OECD report (2016) on the PISA 2015 data indicating that students who experienced more formative assessment had lower academic achievement, and the pattern differs across countries/economies. A possible explanation is that ‘the relationship runs not from teaching strategy to student success on the items, but in the opposite direction’ (OECD, 2016, p. 62), indicating that teachers are more likely to use formative-assessment strategies for those students with lower academic achievement. The results imply that teachers in the West may share more learning progressions with students with lower reading achievement, while teachers in the East tend to provide more feedback to students who are struggling with reading.

The results showed that growth mindset did not mediate the relationship between formative assessment strategies and reading achievement. Previous studies (e.g. Mangels et al., 2006; Sperling & Shapcott, 2012) argued that mindsets filter what information students attend to and alter the way they react to formative assessment and, therefore, potentially influence the effectiveness of formative assessment. Students with a growth mindset have a greater likelihood to conceive and use formative feedback in adaptive ways (Sperling & Shapcott, 2012). Our finding does not seem to support such a mediating role of growth mindset. To some extent, this result is not a surprise given that the growth mindset only weakly, or non-significantly, related to both formative assessment and reading achievement. However, this result should be interpreted with caution because it is possibly due to the way growth mindset was measured and the potential mismatch between the levels of specificity in the measurement of the constructs, as discussed earlier.

The relationship between growth mindset and reading achievement

There is also cultural difference regarding the direct relation between growth mindset and reading achievement. Growth mindset was positively related to reading achievement (ß = .20, p < .001) in the West, corroborating past studies in terms of positive impact of growth mindset on learning-related outcomes (e.g. Blackwell et al., 2007; DeBacker et al., 2018; Yeager et al., 2016). In the East, however, growth mindset was not significantly related to reading achievement. We posit three potential reasons, from the sociocultural perspective, to explain why a growth mindset is more powerful in predicting reading achievement in the West, but less so in the East. Firstly, the non-significant finding may be due to ceiling effects of hard work in the Eastern context where Confucian notions of effort and perseverance prevail (Hau & Salili, 1991). Cross-national studies have suggested that East Asians perceive failure to be due to lack of effort more than Westerners (Yan & Gaier, 1994). Westerners, on the other hand, have been found to value ability more (Ho, 2004). A growth mindset brings about learning gains by motivating a student to learning from failure and strive harder next time. When a student has been working very hard already, for example, students in Mainland China reported spending 57 hours per week studying (OECD, 2019b), a growth mindset may not increase hours of studying or learning performance any further perhaps due to ceiling effects (Yeager & Dweck, 2020). As Zhao (2020) surmised, ‘perhaps the Chinese students do not care or are not allowed to care if their intelligence is fixed or malleable because they have to study hard regardless’.

Secondly, a recent PISA report indicates that the gap in mindset between high and low achieving students was much smaller in the East compared to the OECD average suggesting that both high and low achieving students hold a more similar type of mindset (OECD, 2019b). This report is echoed by what we found in this study as the standard deviation of growth mindset in the East is smaller than that in the West (see Table 1). The smaller variance in the Eastern growth mindset data might also contribute to a non-significant correlation.

Clearly, culture might play a role in how growth mindsets are associated with achievement outcomes as demonstrated by our findings. The role of culture was also alluded to in previous studies (Bernardo et al., 2021; Costa & Faria, 2018). However, our understanding of these cultural intricacies remains incomplete. More research is needed to delve into the cultural universality and specificity of the growth mindset construct. Although work on mindsets and culture is still at its nascency, researchers can take cue from research on other motivational and non-cognitive factors which have been found to demonstrate both culturally universal and specific patterns (Lee, 2009; Lee & Chen, 2019; Liou, 2017).

Limitations and future directions

Similar to other studies applying the secondary analysis method, this study has limitations related to the structure of the PISA data. Firstly, PISA 2018 has only one item measuring mindset. Under most conditions, multi-item scales outperform single-item scales in terms of predictive validity (Diamantopoulos et al., 2012). A single-item often has more measurement error and, hence, less statistical power, than a multi-item index, so we have less confidence in its non-significant results although retain confidence in its significant results (Kennedy, 2008). Future empirical studies are suggested to use a multiple-item instrument to produce a better measure of mindset.

Secondly, this study relied on student-reported data. Whether student reports can authentically reflect teachers’ instructional practices is questionable (Beecham, 2009) and particular cautions are warranted to interpret the PISA student questionnaire data (Hopfenbeck, 2016; OECD, 2010). For example, Hopfenbeck and Maul (2011) found that a non-trivial portion of Norwegian students in the PISA 2006 might not have provided valid responses to the questionnaire, especially for students with poor academic performance. To provide a reliable depiction of teachers’ teaching practice, future investigations may consider more direct and objective data sources (e.g. observation data).

Third, the formative assessment practices covered in the PISA 2018 data are not comprehensive. Some important components of formative assessment, such as self- and peer-assessment, are missing. Compared with teacher-directed formative assessment, self- and peer-assessment require a higher level of student engagement wherein they need to continuously generate internal feedback, through an internal psychological process, to achieve a better performance (Yan & Brown, 2017; Yan et al., 2020; Nicol, 2020). Thus, that subset of formative assessment may have a closer relationship with a growth mindset underscoring that learning is incremental as opposed to just getting it or not (Sanchez et al., 2017). This could be an interesting field to explore the link between formative assessment and mindset.

Finally, since the data used in this study is cross-sectional, the application of SEM is insufficient to determine causality in non-experimental designs (Beatty & Brew, 2005; Kline, 2011). Future studies could test the causal relationship between formative assessment, growth mindset, and academic performance with experimental or longitudinal designs.

Conclusion

The findings of this study provide insights into the relationship between specific formative assessment strategies, growth mindset, and reading achievement. In general, formative assessment strategies were positively related to a growth mindset only in the East, but not in the West. Comparatively, instructional adjustments, among the three formative assessment strategies, had the strongest and positive impact on reading achievement. Growth mindset, in turn, positively predicted reading achievement in the West but not in the East. The mediating effect of growth mindset in the relationship between formative assessment strategies and reading achievement was not significant.

This study has two unique contributions to the literature. First, it bridges two inter-related strands of research, i.e. formative assessment and mindset. The interplay between these two strands, as demonstrated in this study, has the potential to strengthen the synergy between external pedagogical strategies and internal psychological processes. More research along this line could provide a more sophisticated understanding of the mechanism by which formative assessment and mindset impact student learning and how such impact could be maximised. Second, the cross-cultural variability in the relationship between specific formative assessment strategies, growth mindset, and academic achievement revealed in this study highlights the need to consider cultural contexts when interpreting the results of large-scale international assessment projects, like PISA, and designing formative assessment or mindset intervention programmes.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Zi Yan

Zi Yan is an Associate Professor in the Department of Curriculum and Instruction at The Education University of Hong Kong. His publications and research interests focus on two related areas, i.e., educational assessment in the school and higher education contexts with an emphasis on student self-assessment; and Rasch measurement, in particular its application in educational and psychological research. A recent book co-edited with Lan Yang is entitled Assessment as learning: Maximising opportunities for student learning and achievement.

Ronnel B. King

Ronnel B. King is an Associate Professor in the Centre for the Enhancement of Teaching and Learning at The University of Hong Kong. His research interests are on student motivation, well-being, and positive psychology/education.

Joseph Y. Haw

Joseph Y. Haw is a doctoral student specializing in curriculum and instruction at The Education University of Hong Kong. He is also a research fellow at the Ateneo de Zamboanga University, Philippines. He has been working in an integrated basic education school in the Philippines for seven years as School President prior to his admission to the doctoral program.  As an administrator, he focused on teacher and learner well-being, teacher formation, curriculum design, and organizational development. His current research focus is on teacher and student motivation.

Appendix

Table A. Descriptive statistics and bivariate correlations by countries/economies

CNT	n	Reading Achievement		Sharing Learning Progressions (SLP)		Providing Feedback (PFB)		Instructional Adjustments (IAD)		Growth Mindset		Formative Assessment Correlation with Reading Achievement			Formative Assessment’s Correlation with Growth Mindset			Growth Mindset’s correlation with Reading Achievement
		Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	SLP	PFB	IAD	SLP	PFB	IAD
A. English-speaking Countries/economies
Australia	14,273	502.63 (1.63)	108.66 (0.9)	2.89 (0.01)	0.76 (0.01)	2.61 (0.01)	0.84 (0)	2.7 (0.01)	0.78 (0)	2.87 (0.01)	0.87 (0)	−.06*** (.01)	.05*** (.01)	.11*** (0.01)	.02 (0.01)	.05*** (0.01)	.06*** (0.01)	.23*** (0.01)
United Kingdom	13,818	503.93 (2.58)	100.21 (1.27)	2.91 (0.01)	0.73 (0.01)	2.74 (0.02)	0.81 (0.01)	2.71 (0.01)	0.79 (0.01)	2.9 (0.01)	0.89 (0.01)	−.07*** (.02)	.05*** (.02)	.13*** (0.02)	.08*** (0.01)	.10*** (0.01)	.12*** (0.01)	.18*** (0.01)
Ireland	5577	518.08 (2.24)	90.7 (1.04)	2.73 (0.02)	0.74 (0.01)	2.56 (0.02)	0.82 (0.01)	2.52 (0.01)	0.75 (0.01)	2.97 (0.01)	0.83 (0.01)	−.10*** (.02)	.04** (0.02)	.09*** (0.02)	.09*** (0.01)	.08*** (0.01)	.11*** (0.01)	.17*** (0.02)
New Zealand	6173	505.73 (2.04)	106.3 (1.34)	2.86 (0.01)	0.73 (0.01)	2.72 (0.01)	0.81 (0.01)	2.72 (0.01)	0.77 (0.01)	2.84 (0.01)	0.88 (0.01)	−.13*** (.02)	.00 (0.02)	.12*** (0.01)	−.01 (0.01)	.04** (0.01)	.08*** (0.01)	.31*** (0.01)
United States	4838	505.35 (3.57)	107.89 (1.57)	2.99 (0.01)	0.72 (0.01)	2.57 (0.02)	0.88 (0.01)	2.59 (0.01)	0.78 (0.01)	2.9 (0.01)	0.94 (0.01)	−.06*** (0.02)	.05** (0.02)	.10*** (0.02)	.01 (0.02)	.04* (0.02)	.05*** (0.02)	.26*** (0.02)
B. Confucian Countries/economies
Hong Kong SAR	6037	524.28 (2.73)	99.48 (1.49)	2.85 (0.02)	0.76 (0.01)	2.39 (0.02)	0.76 (0.01)	2.55 (0.01)	0.75 (0.01)	2.39 (0.02)	0.85 (0.01)	.02 (0.02)	.02 (0.02)	.08*** (0.02)	.07*** (0.01)	.07*** (0.02)	.06*** (.01)	−0.03 (.02)
Japan	6109	503.86 (2.67)	97.12 (1.68)	3.03 (0.02)	0.71 (0.01)	2.05 (0.02)	0.81 (0.01)	2.48 (0.02)	0.75 (0.01)	2.76 (0.01)	0.84 (0.01)	.05*** (0.02)	−.03* (0.02)	.02 (0.01)	.07*** (.01)	.07*** (0.01)	.06*** (0.01)	.14*** (0.02)
Korea	6650	514.05 (2.94)	102 (1.65)	3.21 (0.01)	0.7 (0.01)	2.46 (0.02)	0.98 (0.01)	2.87 (0.02)	0.8 (0.01)	2.59 (0.01)	0.86 (0.01)	.10*** (0.02)	−.04*** (0.02)	.10*** (0.02)	.10*** (0.01)	.10*** (0.01)	.12*** (0.01)	.08*** (0.01)
Macau SAR	3775	525.12 (1.23)	92.11 (1.07)	2.91 (0.01)	0.68 (0.01)	2.16 (0.01)	0.71 (0.01)	2.36 (0.01)	0.7 (0.01)	2.48 (0.01)	0.88 (0.01)	.04* (0.02)	−.03* (0.02)	0.01 (0.02)	.10*** (0.02)	.09*** (0.02)	.09*** (0.02)	.07*** (0.01)
Mainland China	12,058	555.24 (2.75)	87.23 (1.72)	3.24 (0.02)	0.69 (0.01)	2.52 (0.02)	0.88 (0.01)	2.82 (0.02)	0.81 (0.01)	2.65 (0.01)	0.89 (0.01)	−.02 (0.02)	.01* (0.01)	.09*** (0.02)	.10*** (0.01)	.10*** (0.02)	.07*** (0.02)	−.11*** (0.02)
Chinese Taipei	7243	502.6 (2.84)	101.73 (1.46)	2.95 (0.01)	0.73 (0.01)	2.58 (0.01)	0.82 (0.01)	2.62 (0.01)	0.78 (0.01)	2.67 (0.01)	0.85 (0.01)	.01 (0.01)	.07*** (0.01)	.05*** (0.01)	.08*** (0.01)	.09*** (0.01)	.07*** (0.01)	.07*** (0.01)

Note: Standard Errors are enclosed in parentheses. *** p < .001; **p < .01; *p < .05; B-S-Z-J: Beijing-Shanghai-Zhejiang-Jiangsu