Normative and appearance performance-approach goal structures: Two-level factor structure and external linkages

Abstract

While personal performance-approach goals (PAp goals) have already been successfully bifurcated into normative and appearance PAp goals, the same distinction has not yet been applied to performance-approach goal structures (PAp goal structures). The present study therefore aimed to (a) test the factorial two-level structure of PAp goal structures to establish whether a distinction between normative and appearance components is empirically supported, and (b) explore relations to achievement and approach-oriented achievement goals (mastery-approach goals, normative and appearance personal PAp goals). This study relied on a sample of 1,004 secondary school students (53.38% females, 49 classes). Results from multilevel confirmatory factor analyses revealed that a model with two separate normative and appearance PAp goal structure factors on the individual student level, and one overall PAp goal structure factor on the classroom level fit the data best. On the individual student level, normative PAp goal structures positively predicted achievement, whereas appearance PAp goal structures negatively predicted achievement. Each PAp goal structure type showed the strongest relation to the matching personal PAp goal, but no relations to mastery-approach goals were found. On the classroom level, the high associations between the overall PAp goal structure and personal PAp goals raised concerns on their empirical distinctiveness.

Keywords:

• Achievement goal
• appearance
• classroom structure
• normative
• performance-approach goal structure

Achievement goal theory proposes that individuals differ in the goals they pursue in achievement situations and learning tasks (e.g., Elliot & McGregor, 2001; Pintrich, Conley, & Kempler, 2003; Senko, 2019). The theory generally distinguishes between two categories of approach-oriented motivational patterns: Students can adopt mastery-approach goals (MAp goals), and thus aim to develop competence, or performance approach goals (PAp goals) whereby the aim is to showcase competence or to perform better than others. Their avoidance-oriented counterparts describe the aim of avoiding to perform poorly (performance-avoidance goals, PAv goals) or of avoiding to do less well than one possibly could (mastery-avoidance goals, MAv goals) (e.g., Elliot, 2005; Elliot & Harackiewicz, 1996; Senko & Dawson, 2017). Achievement goal theory further holds that characteristics of the learning environment–the goal structures–evoke specific achievement goals in students (e.g., Ames, 1992; Urdan & Schonfelder, 2006). For instance, an environment in which students’ learning and individual progress are valued (mastery-approach goal structures, MAp goal strucures) elicits a personal MAp goal, whereas an emphasis on outperforming peers or competence demonstration within a classroom (performance-approach goal structures, PAp goal structures) feeds into students’ adoption of PAp goals (e.g., Urdan, 2010). So far, developments within the goal structure domain regarding goal structure types and structural differentiations have mimicked those in the personal achievement goal domain, for example concerning the separation of PAp and PAv parts, and the introduction of the MAv construct (see e.g., Elliot & McGregor, 2001; Karabenick, 2004; Middleton & Midgley, 1997 for personal goals, see e.g. Chim & Leung, 2016; Peng, Cherng, Lin, & Kuo, 2018 for goal structures). This makes sense conceptually–why should there be a different level of differentiation on achievement goal theory’s contextual and personal levels (e.g., Schwinger & Stiensmeier-Pelster, 2011)? Moreover, this pattern has been backed up empirically, with a recent meta-analysis (Bardach, Oczlon, Pietschnig, & Lüftenegger, 2019) confirming that each goal structures type is most strongly related to its matching personal achievement goal.

In recent refinements of achievement goal theory, however, the personal PAp goal has been further divided into two sub-types in order to overcome disagreements among researchers on its essence–competence demonstration or normative performance–and contradicting empirical findings: Normative PAp goals with an emphasis on performing better than others and appearance PAp goals with an emphasis on appearing talented (e.g., Hulleman, Schrager, Bodmann, & Harackiewicz, 2010; Senko, Hulleman, & Harackiewicz, 2011; Senko & Dawson, 2017). Research on goal structures has not kept pace with this development, although claims have been made in this direction (Bardach, Oczlon et al., 2019). Providing the first empirical examination of normative and appearance PAp goal structures, the present study therefore asks whether the normative-appearance distinction can be applied to the PAp goal structure and whether such a distinction can enhance our understanding of the classroom context, its constituents and relations to student functioning. Accordingly, we investigate the two-level factor structure of the PAp goal structure to test whether a bifurcation into normative and appearance facets is empirically supported, and explore effects on students’ motivational and achievement-related characteristics in terms of approach-based personal achievement goals and grades in mathematics classes.

Performance-approach goals and goal structures – towards a differentiated understanding?

Suggestions to divide the personal PAp goal trace back to researchers’ disagreement about core features of this achievement goal type (see e.g., Senko et al., 2011 for an overview). Initially, researchers considered the aim of PAp goals to be to demonstrate competence (e.g., Dweck, 1986), even though social comparisons have occasionally been acknowledged as additional components of PAp goals in that demonstrating competence might be achieved by performing better than peers (e.g., Duda & Nicholls, 1992; Nicholls, 1984). Some researchers have later re-defined outperforming peers as the critical feature of PAp goals (e.g., Elliot & Thrash, 2001; Senko & Harackiewicz, 2002) and questioned whether the PAp goal should include competence demonstration at all. Providing empirical support for a two-tier PAp goal, it has been shown that PAp goals can have different effects depending on their conceptualization either as an aim to outperform peers (normative PAp goals) or an aim to demonstrate competence (appearance PAp goals). For instance, Hulleman and colleagues (2010) demonstrated in their meta-analysis that normative PAp goals were positively and appearance PAp goals were negatively related to achievement (for a further meta-analysis focusing on multiple educational outcomes see Senko & Dawson, 2017; see also e.g., Edwards, 2014; Senko & Tropiano, 2016).

The discussion on what makes the ‘performance-approach’ has not yet been carried over to research on PAp goal structures. A chief reason for this might be that research on goal structures always lags a bit behind that on personal achievement goals. For instance, while the basic approach-avoidance distinction is widely accepted for the personal performance goal, it is not within the goal structures domain. In fact, there are still (mainly approach-focused) scales in use that purport measuring a ‘performance goal structure’ (e.g., Sideridis, Ageriadis, Irakleous, Siakali, & Georgiou, 2006, Skaalvik & Skaalvik, 2013), while others have claimed for studying PAp and PAv goal structures separately (e.g., Schwinger & Stiensmeier-Pelster, 2011). We assert that this broader debate might have made other, more fine-grained arguments located at the micro-level of the single goal structure type appear less relevant or urgent. However, in light of the convincing empirical evidence speaking in favor of PAp vs. PAv goal structures, we believe that research on goal structures should move on in that the time is ripe to address these further issues.

A further reason for the lack of interest to examine normative and appearance PAp goal structures could be that, among the different types of goal structures, the PAp goal structure and the general (in most instances mainly approach-based) performance goal structure probably suffer most from definitional inconsistencies and relatedly, messy measurement. As such, scales often do not only incorporate normative and appearance facets, but also a perplexing array of further features, such as differential teacher treatment, or teachers’ emotional responses (see e.g., the studies by Bong, 2005; Polychroni, Hatzichristou, & Sideridis, 2012; Sideridis, 2007). This confusion presents a major dilemma for theorizing on and measuring PAp goal structures and it might have made the normative vs. appearance issue less apparent. For example, even though emotional as well as social factors are undoubtedly important in classroom settings and can overlap with goal structures (e.g., Patrick, Kaplan, & Ryan, 2011), a broader understanding of goal structures might also hinder productive theory testing. It has long been criticized that “consistent operational definitions of goal structures are elusive” (Urdan, 2010, p. 93; see also Bardach, Oczlon et al., 2019) and mixing multiple components –among those social and emotional facets– under the umbrella term of goal structure will not resolve these ambiguities. Moreover, consider that in the school context, MAp, PAp and PAv goals refer to academic achievement goals (vs. e.g., social achievement goals, e.g., Ryan & Shim, 2006 or other social strivings). Conceptualizing (PAp) structures more narrowly as academic goal structures could thus sharpen its profile and enhance conceptual clarity.

On the other hand, inconsistencies also infiltrate scales that have undergone multiple revisions to un-muddy the measurement of goal structures (i.e., Midgley et al.'s (2000) pattern of adaptive learning survey, PALS, as the most widely used measure to map goal structures). Whereas the PALS ‘teacher’s goal-scale’ centers on normative comparisons (e.g., ‘My teacher tells us how we compare to other students’, ‘My teacher lets us know which students get the highest scores on a test’), the revised climate-focused ‘goal structure-scale’ puts competence demonstration (i.e., appearance) characteristics to the foreground (e.g., ‘In our class, getting good grades is the main goal.’, ‘In our class, getting right answers is very important’). To date, both scales are used to measure what is labeled a ‘PAp goal structure’ (e.g., Alivernini, Manganelli, & Lucidi, 2018; Bae & DeBusk-Lane, 2018; Chen et al., 2016; Cho, Lee, & Toste, 2018). However, the implications of the decision to either embrace a normative vs. appearance-focused PAp goal structure measure has not yet been addressed.

In conclusion, we believe that it is a timely moment to investigate whether the normative-appearance distinction that has successfully been applied to personal PAp goals can also be employed for PAp goal structures. Can it be distinguished whether a teacher underlines that the main purpose of engaging in academic work in the classroom is to demonstrate competence (appearance PAp goal structure) or to perform better than others (normative PAp goal structure) and do these two types of PAp goals produce different educational outcomes? Examining the empirical and theoretical viability and utility of such a distinction would align research on achievement goal theory’s personal and contextual part. If this distinction is supported, it might show researchers where to look next when conceptualizing and measuring PAp goal structures. If the distinction is not supported, it might indicate that we have reached a point at which goal structure’s theoretical development does not parallel that of personal achievement goals and that moving into other directions might be more fruitful.

On a more general level, testing different types of goal structures will sharpen and refine the understanding of goal structures. For example, a differentiation between normative vs. appearance PAp goal structures that stands up to the empirical test would prove that the field has been wrong to shy away from such endeavors and that painting the classroom context with a broad blush might have led to a loss of potentially useful information. For research practice, it can aid in ‘tidying up’ goal structures measures and urge researchers to carefully consider the content of the PAp goal structure measures (e.g., PALS teacher’s goal scale vs. climate scale as normative vs. appearance-focused scales) when choosing a measure for their study. Such a distinction could notably contribute to theory-building in the goal structure domain as well: It would add a new facet (normative vs. appearance) to existing theoretical frameworks of goal structures, hence challenging still widespread assumptions that contextual goal emphases are per se blurry and that broader categories are best suited to study goal structures. On the other hand, finding that an overall PAp goal structure rather than a two-tier PAp goal structure receives empirical support should lead us to re-think if we ask the right questions in research on (PAp) goal structures and whether directions other than a distinction between sub-components of PAp goal structures might be the way to go (e.g., enriching rather than dividing PAp goal structures, see e.g., Kaplan & Maehr, 2007).

External linkages of normative and appearance PAp goal structures – Relations to approach-based achievement goals and achievement

Foremost among the external linkages that would need to be addressed in research on normative and appearance PAp goal structures are achievement goals, especially their personal counterparts, the normative and appearance PAp goals. Carrying over achievement goal theory’s core assumption that each type of personal achievement goal should be best predicted by its contextual counterpart to research on the two PAp goal structure subtypes, normative vs. appearance PAp goal structures should be most strongly related to normative vs. appearance personal PAp goals, respectively. Normative and appearance messages are likely to co-occur to a certain degree in a PAp-focused environment; still, if teachers place relatively more emphasis on the one over the other, then students’ personal goal adoption should reflect this focus. Hence, even though (strong) cross-relations can be expected, the congruent relation to the ‘matching’ PAp goal type should be most pronounced. On the other hand, in accordance with the majority of prior research findings on the overall PAp goal structure (e.g., Bardach, Oczlon et al., 2019; Kim, 2015; Sideridis, 2007) and due to the shared valence component (i.e., the approach focus, Elliot & McGregor, 2001; Schwinger & Stiensmeier-Pelster, 2011) between PAp goal structures and personal MAp goals, both normative and appearance PAp goal structures can be expected to positively predict students’ adoption of personal MAp goals.

In addition, one might assert that normative and appearance PAp goal structures should yield unique effects on achievement, reflecting the effects of personal PAp goals (e.g., Hulleman et al., 2010). As such, normative PAp goal structures should be positively associated with achievement, and appearance PAp goal structures should be negatively associated with achievement. Aligned with theoretical considerations that the effects of goal structures on educational outcomes should best be understood as (partially) transmitted through personal achievement goals (e.g., Murayama & Elliot, 2009; Urdan, 2010), it might be sensible to include personal achievement goals (normative and appearance PAp goals) as mediators to account for personal goal effects.

Research goals

The present study aims to explore the empirical and theoretical usefulness of branching PAp goal structures into normative and appearance components. Given that students are nested within classes, and, specifically, when asked to rate their goal structures, students within the same class refer to the same teacher, a multilevel-modeling approach is warranted. Multilevel modeling accounts for the hierarchical structure of the data and enables researchers to simultaneously estimate all effects at the individual student level (L1) and the classroom level (L2) (e.g., Marsh et al., 2012; Morin, Marsh, Nagengast, & Scalas, 2014). Although goal structures as classroom climate constructs should predominantly be located at L2 reflecting classroom aggregates of students’ perceptions of normative and appearance PAp goal structures, we also estimated all goal structure effects at L1 (i.e., the effects of individual students’ perceptions of the normative and appearance PAp goal structure focus in class, e.g., Lüdtke, Robitzsch, Trautwein, & Kunter, 2009). We asked the following questions:

Research question 1: Which dimensional structure describes the PAp goal structure construct best? Although highly inter-correlated, we assume that the two-tier PAp goal structure model (i.e., two factors reflecting normative and appearance PAp goal structures) on both levels will provide the best fit to the data (see e.g., Lüftenegger, Bardach, Bergsmann, Schober, & Spiel, 2019, for personal goals).

Research question 2: Are normative and appearance PAp goal structures differentially related to approach-based achievement goals and achievement? We propose that both normative and appearance PAp goal structures will yield positive relations to all three approach-based achievement goals (for normative PAp goal structures: normative PAp goals > appearance PAp goals > MAp goals; for appearance PAp goal structures; appearance PAp goals > normative PAp goals > MAp goals). Drawing on research on personal PAp goals (e.g., Hulleman et al., 2010), we expect a negative relation between appearance PAp goal structures (and appearance PAp goals) and achievement and a positive relation between normative PAp goal structures (and normative PAp goals) and achievement.

Research question 3: Do normative and appearance personal PAp goals serve as mediators for the relations between normative and appearance PAp goal structures and achievement? We hypothesize that normative as well as appearance PAp goals will (partially) mediate the relation between normative and appearance PAp goal structures and achievement.

Method

Sample

This study was part of a larger research project that mainly focused on Austrian secondary school students’ (social) motivation (Bardach & Lüftenegger, 2018; see also Bardach, Graf et al., 2019). In Austria, secondary school starts at the age of 10, following four years of elementary education. At the age of 10, children move on to either an academic secondary school or a general secondary school. Students can remain in academic secondary schools for eight years and prepare for university. However, students can also leave the academic school after four years and enroll in a school that provides a more specialized career preparation. Students who enroll in general secondary schools stay for four years, after which they can move on to either an academic school or a school that provides specialized career preparation. Students from academic secondary schools in Austria (Gymnasium schools), i.e., the highest track of secondary schools, participated in this project.

The sample analyzed here contained the students in ‘lower’ secondary education; that is, students in grade 7 (50.10%) and 8 (49.90%). As the perceptions of goal structures have been found to differ depending on students’ educational level (e.g., Anderman & Midgley, 1997), and might also be influenced by age-related developmental characteristics, it was decided to focus solely on students from ‘lower’ secondary education to ensure a more homogenous sample.

A total of 1,004 students (mean age = 13.39 years, SD = 0.72 years, 53.38% females) from 49 mathematics classes responded to paper-and-pencil questionnaires delivered by trained research assistants during regular classroom hours in May and June 2018. All students participated voluntarily and with active parental consent. The consent rate was above 99%, meaning that less than 1% of students were not allowed to participate by their parents or themselves refused to participate. The students did not receive compensation for participating in the study.

Measures

This study was conducted in the context of mathematics, and, therefore, all items referred to mathematics and mathematics class. Moreover, we used students’ grades in mathematics as proxies of their achievement. A 6-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree) was employed as response format of the scales assessing goal structures and achievement goals.

Performance-approach goal structure

Students’ perceptions of PAp goal structures were measured with items adapted from Schwinger and Stiensmeier-Pelster (2011) and Spinath, Stiensmeier-Pelster, Schöne, and Dickhäuser (2002). Introduced by the phrase ‘In our math class, the point is …’, three items assessed the normative PAp goal structure (e.g., ‘… to get our work done better than the others’, α = .82), and three items assessed the appearance PAp goal structure (e.g., ‘… to demonstrate our classmates if we do well at a certain task’; α = .72)¹. All six items are reported in the Online Supplement A1.

Personal achievement goals

The measure of students’ personal achievement goals comprised scales assessing students’ personal MAp goal, and their normative and appearance PAp goals. These three scales were taken from the Achievement Goal Instrument for secondary education (AGI-SE; Lüftenegger et al., 2019). All items were introduced with the item stem ‘In math I mainly study …’, followed by three statements referring to MAp goals (e.g., ‘… so I can expand my knowledge’, α = .84), three statements referring to normative PAp goals (e.g., ‘…so I will be better than other students’, α = .88), and three statements referring to appearance PAp goals (e.g., ‘…so other people will think that I am good’, α = .87).

Achievement

We used students’ mathematics grades from their school report as indicators of achievement. In the Austrian school system, ‘1’ represents the best grade and ‘5’ is the lowest grade; hence, we conducted our analyses with grades recoded so that higher values reflected higher achievement.

Analyses

All analyses were performed with Mplus Version 8.2 (Muthén & Muthén, 1998-2017). Given the multi-level nature of our data, we specified the models tested in this study as two-level models. First, we investigated which factorial structure serves the PAp goal structure construct best and whether a distinction between the normative and appearance components was empirically supported (research question 1). Therefore, we set up a series of multi-level confirmatory factor analytic models (ML-CFAs) using the robust maximum likelihood estimator (MLR). We systematically varied the number of factors of the tested ML-CFAs (i.e., overall PAp goal structure vs normative PAp goal structure and appearance PAp goal structure) on L1 and L2: (a) Model 1: one overall PAp goal structure factor on both L1 and L2, (b) Model 2: one overall PAp goal structure factor on L1, two distinct factors (normative and appearance) on L2, (c) Model 3: two distinct factors (normative and appearance) on L1, one overall PAp goal structure factor on L2, (d) Model 4: two distinct factors (normative and appearance) on both L1 and L2. The best fitting model was determined via comparisons of the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and the Sample-Size Adjusted BIC (SABIC) with lower values indicating a better trade of between model-fit and model complexity. In addition, we conducted χ2 difference tests, which we adjusted for the respective scaling factor by applying a scaling correction as recommended for the MLR estimator (Satorra & Bentler, 2010).

Second, we examined relations between the PAp goal structure and students’ personal approach-based achievement goals as well as students’ achievement on both levels (research question 2) in multilevel models (MLM) using the MLR estimator, with the number of predictors at L1 and L2 depending on the best fitting model for the PAp goal structure (i.e., overall PAp goal structure factor vs. normative and appearance PAp goal structure factors, see above).We assessed the goodness of fit for the ML-CFAs and the MLM with the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) and considered typical cutoff scores taken to reflect excellent and adequate fit to the data: (a) CFI > .95 and .90, respectively; (b) RMSEA < .06 and .08, respectively (Hu & Bentler, 1999).

Third, we tested whether normative and appearance personal PAp goals (partially) mediated the relations between PAp goal structures (normative and appearance; overall) and achievement (research question 3). The ML mediation model was set up using a Bayesian estimator to test indirect effects because bootstrapping in conjunction with multilevel modeling is not available in Mplus 8.2. Thus, the Bayesian Markov Chain Monte Carlo (MCMC) estimation method was applied. Following recommendations by Hox, van de Schoot, and Matthijsse (2012), convergence was assessed using the Gelman-Rubin criterion with a stricter cutoff value of 0.01 rather than the default setting of 0.05. Eight chains were requested for the Gibbs sampler and a minimum number of 10,000 iterations were specified. Starting values were based on the maximum likelihood estimates of the model parameters. All trace plots were manually inspected to check for convergence. In all models, group-mean centering was applied to the variables at L1, whereas the variables at L2 were grand-mean centered.

Significance testing was performed at the .05 level. Our data set included 0.66% missing data at the item level. Full information maximum likelihood estimation (FIML; Enders, 2010) and Bayesian estimation were used to handle missing data.

Results

Factorial structure

ML-CFAs were carried out to examine the construct validity of the different PAp goal structure models. All models fit the data well (see Table 1). The model with one overall PAp goal structure factor at L2 and the two normative and appearance PAp goal structure factors at L1 had the lowest AIC, BIC, and SABIC. A ${\chi }^2$ difference test was applied to statistically compare this model with a model with two normative and appearance PAp goal structure factors at L1 and L2 which had the second lowest AIC, BIC, and SABIC. Results showed that having two PAp goal structure factors at L2 does not improve model fit, Δ${\chi }^2\left(1\right)=0.09,$ p = .769. Moreover, the latent correlation of normative and appearance PAp goal structure factors at L2 was .99, indicating one factor at L2. Hence, all further analyses were carried out relying on a measurement model of PAp goal structures with two distinct factors at L1 and one overall factor at L2. Table 2 shows descriptive statistics and latent correlations separately for L1 and L2 variables. Even though the two PAp goal structures at L1 were quite highly correlated (r = .62), we decided to treat them as separate factors due to the results of the model comparisons and because the aim of the paper was to explore potentially differentiated effects. Moreover, high inter-correlations between normative and appearance facets have also been observed on the personal goal level (see Lüftenegger et al., 2019). Table A1 in the Online Supplement provides information on cross-level invariance testing, and composite reliability coefficients for all scales. Please note that no cross-level invariance testing was performed for PAp goal structures, due to the different number of factors on L1 and L2.

Table 1. Factorial structure of performance-approach goal structures.

Model	χ^2	df	CFI	RMSEA	AIC	BIC	SABIC
One Factor at L1, One Factor at L2	201.49	23	.938	0.088	17726.94	17849.73	17770.33
Two Factors at L1, One Factor at L2	44.22	22	.992	0.032	17680.08	17807.79	17725.21
One Factor at L1, Two Factors at L2	223.01	22	.930	0.095	17729.51	17857.22	17774.64
Two Factors at L1, Two Factors at L2	35.54	21	.995	0.026	17683.26	17814.88	17729.13

Table 2. Descriptive statistics, bivariate correlations and intraclass correlation coefficients on the individual and classroom level.

Variable	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.
1. PAp goal structure normative (L1)
2. PAp goal structure appearance (L1)	.62
3. Personal goals - MAp goal (L1)	.02	.04
4. Personal goals - PAp goal normative (L1)	.54	.45	.26
5. Personal goals - PAp goal appearance (L1)	.52	.54	.20	.72
6. Achievement (L1)	.05	-.02	.10	.15	.05
7. Overall PAp goal structure (L2)
8. Personal goals – MAp goal (L2)							.22
9. Personal goals – PAp goal normative (L2)							.58	.28
10. Personal goals – PAp goal appearance (L2)							.67	.37	.72
11. Achievement (L2)							.31	.39	.12	.38
M	2.84	3.08	4.12	2.85	2.82	3.45	2.97	4.15	2.85	3.09	3.43
SD	1.08	1.05	1.23	1.15	1.26	1.05	0.26	0.52	0.27	0.28	0.36
ICC(1)	0.02	0.02	0.14	0.03	0.00	0.07
ICC(2)	0.28	0.33	0.77	0.38	0.08	0.59

Note. Statistically significant correlations at p < .05 are boldface.

Relations to achievement goals and achievement

At L1, normative PAp goal structures were significantly positively related to achievement (unstandardized $\widehat{\beta }$ = 0.09, p = .026), whereas appearance PAp goal structures were significantly negatively related to achievement (unstandardized $\widehat{\beta }$ = −0.07, p = .046). Moreover, at L1, normative PAp goal structures significantly positively predicted personal normative PAp goals (unstandardized $\widehat{\beta }$ = 0.50, p < .001) and personal appearance PAp goals (unstandardized $\widehat{\beta }$ = 0.37, p < .001). Normative PAp goal structures were not significantly related to personal MAp goals (unstandardized $\widehat{\beta }$ = −0.01, p = .437). A significant positive relation between appearance PAp goal structures and the two types of personal PAp goals was found (unstandardized $\widehat{\beta }$ = 0.23, p < .001 for normative PAp goals, unstandardized $\widehat{\beta }$ = 0.42, p < .001 for appearance PAp goals). The relation to MAp goals did not attain statistical significance (unstandardized $\widehat{\beta }$ = 0.05, p = .174). At L2, the overall PAp goal structure was not a significant positive predictor of achievement (unstandardized $\widehat{\beta }$ = .34, p = .087), but of personal normative PAp goals (unstandardized $\widehat{\beta }$ = 0.85, p < .001), appearance PAp goals (unstandardized $\widehat{\beta }$ = 0.66, p < .001), and MAp goals (unstandardized $\widehat{\beta }$ = .48, p = .031). Table 3 displays all standardized and unstandardized effects.

Table 3. Multilevel regression model results: PAp goal structures predicting personal goals and achievement.

Predictors	MAp goal		normative PAp goal		appearance PAp goal		Achievement
	Est. (SE)	Std. Est.	Est. (SE)	Std. Est.	Est. (SE)	Std. Est.	Est. (SE)	Std. Est.
Student Level (L1)
normative PAp goal structure	−0.01 (0.05)	−0.01	0.50 (0.04)	0.42	0.37 (0.04)	0.31	0.09 (0.05)	0.09
appearance PAp goal structure	0.05 (0.05)	0.04	0.23 (0.04)	0.18	0.42 (0.05)	0.34	−0.07 (0.04)	−0.07
Classroom Level (L2)
Overall PAp goal structure	0.48 (0.26)	0.27	0.85 (0.10)	0.93	0.66 (0.11)	0.90	0.34 (0.20)	0.33

Note. Est. = unstandardized estimate; Std. Est. = standardized estimate; Statistically significant results at p < .05 are boldface.

Mediating effects of personal normative and appearance PAp goals

We re-estimated the relations between normative as well as appearance PAp goal structures and achievement at L1, entering personal PAp goals as mediators. The ML mediation model, which was set up using a Bayes estimator, converged properly according to the trace plots. Appearance PAp goal structures were still significantly and negatively related to achievement (unstandardized $\widehat{\beta }$ = −0.09, p = .020), but the relation between normative PAp goal structures and achievement was not significant after re-estimation (unstandardized $\widehat{\beta }$ = 0.02, p = .362). Personal normative PAp goals significantly and positively predicted achievement (unstandardized $\widehat{\beta }$ = 0.20, p < .001) and personal appearance PAp goals significantly and negatively predicted achievement (unstandardized $\widehat{\beta }$ = −0.07, p = .034). Personal normative PAp goals fully mediated the relation between normative PAp goal structures and achievement (unstandardized $\widehat{\beta }$ = 0.10, p < .001) and partially mediated the relation between appearance PAp goal structures and achievement (unstandardized $\widehat{\beta }$ = 0.04, p < .001). The other indirect effects were not statistically significant (unstandardized $\widehat{\beta }$ = −0.03, p = .068, for appearance PAp goals mediating the relation between normative PAp goal structures and achievement, unstandardized $\widehat{\beta }$ = −0.03, p = .068 for appearance PAp goals mediating the relation between appearance PAp goal structures and achievement).

At L2, the overall PAp goal structure did not significantly predict achievement (unstandardized $\widehat{\beta }$ = 0.18, p = .502), but was significantly and positively related to personal normative PAp goals and personal appearance PAp goals (unstandardized $\widehat{\beta }$ = 0.83, p < .001, and unstandardized $\widehat{\beta }$ = 0.64, p < .001, respectively). Neither personal normative PAp goals, nor personal appearance PAp goals significantly predicted achievement (unstandardized $\widehat{\beta }$ = 0.44, p = .053, and unstandardized $\widehat{\beta }$ = −0.33, p = .111, respectively). None of the indirect effects was statistically significant (unstandardized $\widehat{\beta }$ = −0.20, p = .222, for personal appearance PAp goals as mediator, and unstandardized $\widehat{\beta }$ = 0.35, p = .106 for personal normative PAp goals as mediators). Table 4 shows all results from the ML mediation model.

Table 4. Multilevel mediation model results: goal structures mediating the relationship between personal goals and achievement.

Level	Predictor	Mediator	Outcome	Est. (SD)	95% CI	Std. Est.
Student Level (L1)
	normative PAp GS	normative PAp goal		0.50 (0.04)	[0.43, 0.58]	0.43
	normative PAp GS	appearance PAp goal		0.37 (0.04)	[0.29, 0.44]	0.31
	appearance PAp GS	normative PAp goal		0.23 (0.04)	[0.15, 0.30]	0.19
	appearance PAp GS	appearance PAp goal		0.42 (0.04)	[0.34, 0.50]	0.35
		normative PAp goal	Achievement	0.20 (0.04)	[0.12, 0.28]	0.24
		appearance PAp goal	Achievement	−0.07 (0.04)	[−0.15, 0.01]	−0.09
	normative PAp GS		Achievement	0.02 (0.04)	[−0.07, 0.10]	0.02
	appearance PAp GS		Achievement	−0.09 (0.04)	[−0.17, −0.00]	−0.09
	normative PAp GS	normative PAp goal	Achievement	0.10 (0.02)	[0.06, 0.14]
	normative PAp GS	appearance PAp goal	Achievement	−0.03 (0.02)	[−0.06, 0.00]
	appearance PAp GS	normative PAp goal	Achievement	0.04 (0.01)	[0.02, 0.07]
	appearance PAp GS	appearance PAp goal	Achievement	−0.03 (0.02)	[−0.07, 0.00]
Classroom Level (L2)
	overall PAp GS	normative PAp goal		0.83 (0.14)	[0.56, 1.10]	0.65
	overall PAp GS	appearance PAp goal		0.64 (0.14)	[0.37, 0.91]	0.56
		normative PAp goal	Achievement	0.44 (0.27)	[−0.10, 0.97]	0.44
		appearance PAp goal	Achievement	−0.33 (0.27)	[−0.85, 0.21]	−0.30
	overall PAp GS		Achievement	0.18 (0.28)	[−0.36, 0.73]	0.14
	overall PAp GS	normative PAp goal	Achievement	0.35 (0.23)	[−0.08, 0.85]
	overall PAp GS	appearance PAp goal	Achievement	−0.20 (0.18)	[−0.59, 0.13]

Note. GS = Goal Structure; Est. = Unstandardized Bayesian posterior median estimate; SD = Standard deviation of the posterior distribution. 95% CI = 95% Bayesian credibility interval; Std. Est. = Standardized Estimate; It should be noted that the confidence intervals are based on two-tailed tests, meaning that it is possible that results from one-tailed tests which are statistically significant include zero in the confidence intervals; Statistically significant results at p < .05 are boldface.

Discussion

PAp goal structures can, among other aspects, place emphasis on being better than others or on showing off one’s competence. Our work was designed to make these issues salient and to provide, to the best of our knowledge, the first empirical examination of normative and appearance PAp goal structures and their effects. With regard to factorial structure, it was found that the two types of PAp goal structures split structurally at L1, but not at L2. Methodologically, this illustrates the need to separately consider and measure phenomenon at each level of analysis (e.g., Dunn, Masyn, Jones, Subramanian, & Koenen, 2015). Conceptually, goal structures as a classroom climate construct (e.g., Morin et al., 2014; Marsh et al., 2012; see also e.g., Khajavy, Bardach, Hamedi, & Lüftenegger, 2018; Lam, Ruzek, Schenke, Conley, & Karabenick, 2015; Lau & Nie, 2008; Luo, Hogan, & Paris, 2011; Urdan, 2004) supposedly primarily ‘live’ at L2 and the one overall factor reveals that PAp goal structure characteristics become blurred and even indistinguishable at L2. At L1, the distinct measurement structure indicates that individual students’ perceptions can be differentiated according to whether students perceive that the classroom environment highlights normative vs. appearance messages. In conjunction with the low ICCs, our L1 findings coincide with one of the oldest assumptions in research on goal structures on the importance of the psychological environment, i.e., students’ unique interpretations of the context (e.g., Maehr & Midgley, 1991).

At L1, normative and appearance PAp goal structures furthermore exhibited different empirical profiles in our study. When posited as predictors of personal PAp goals in the ML regression model, each type of PAp goal structures showed the strongest relation to its matching personal PAp goal. These results were in accord with our hypotheses and support achievement goal theory’s basic assumption on the level of goal structure-subtypes. No differentiated pattern of findings was expected for the association between PAp goal structure types and MAp goals. The results demonstrated that neither normative nor appearance PAp goal structures statistically significantly predicted MAp goals at L1, which contrasts the wealth of empirical evidence on the association between PAp goal structures and personal MAp goals and our assumption of positive relations (e.g., Bardach, Oczlon et al., 2019; Kim, 2015; Sideridis, 2007). Furthermore, in the analyses in which personal goal effects were left out, the patterns of relations between PAp goal structure types and achievement followed that obtained for personal PAp goals (e.g., Hulleman et al., 2010): Normative PAp goal structures were positively related to achievement, whereas appearance PAp goal structures were negatively related to achievement. In concert with the effects on personal PAp goals, our results point toward the potential empirical and theoretical usefulness of a differentiated PAp goal structure at L1.

When personal PAp goals were entered as potential mediators in the ML mediation model, normative PAp goals fully accounted for the effect of normative PAp goal structures on achievement. These findings align well with those achievement goal theorists claiming that the context, i.e., the goal structure, should influence students’ personal goal adoption, which, in turn, affects educational outcomes (e.g., Urdan, 2010). Interestingly, normative PAp goals also mediated the effect of appearance PAp goal structures on achievement (direct negative effect, indirect positive effect). Moreover, in the mediation model, personal normative PAp goals were, as can be expected based on theory and prior empirical evidence (e.g., Hulleman et al., 2010), positively related to achievement, whereas appearance PAp goals proved to be negatively related to achievement at L1. At L2 in the mediation model, class average achievement was unaffected by overall PAp goal structures and the considered achievement goals. In addition, none of the mediating effects reached statistical significance. The overall PAp goal structure positively predicted both types of personal PAp goals in both the ML regression and the ML mediation model. However, the size of the relations to the personal appearance and normative PAp goal (standardized estimates > 0.9) in the ML regression casts serious doubts on whether we are measuring the context vs. the personal part of achievement goal theory at L2, or rather an overall PAp goal factor. On the other hand, we caution against overstating the L2 results, given that most of the variance of the constructs investigated here was within classes rather than between classes. This relates to the recently in research on goal structures raised question of whether the classroom is always the appropriate level of aggregation (e.g., Bardach, Lüftenegger, Yanagida, Spiel, & Schober, 2019; Bardach, Yanagida, Schober, & Lüftenegger, 2018).

Overall, our study highlights (a) the relevance of individual students’ perceptions–the psychological environment–in research on (PAp) goal structures and (b) suggests that achievement goal theory should embrace the possibility of different PAp goal structure facets. Our study has shown that, at L1, normative vs. appearance PAp goal structures do not only split but also seem to capture distinctive features of the classroom environment that are differentially related to achievement and achievement goals. Hence, we clearly advocate that scholars should continue research into this direction. We now need more studies juxtaposing normative and appearance PAp goal structures, potentially leading to extended theoretical models comprising a bifurcated PAp goal structure–given that our findings can be replicated with other samples and in other contexts. Irrespective of the outcomes of future studies, the crucial contribution to theory of the current work lies in the consideration of different types of PAp goal structures: To the best of our knowledge, systematic investigations of normative and appearance PAp goal structures were absent in the literature. Our study holds general methodological implications for research on goal structures too. Given the nuanced patterns of results, we suggest that researchers investigating goal structures at least indicate whether their PAp goal structure items include normative or appearance content or a mixture of both types. Concerning practical implications, should teachers be encouraged to foster normative PAp goal structures alongside MAp goal structures? We do not recommend this. Normative PAp goal structures might under certain circumstances facilitate achievement; nevertheless, the L1 effects found here were at best small and these slight advantages are furthermore probably outweighed by other features, such as positive relations to appearance PAp goals. As such, the current study explores new directions for achievement goal theory and research, but its findings can and should not be transferred to classroom practice, e.g., by advising teachers to emphasize normative PAp goal structure messages.

Study limitations and future directions for research

The cross-sectional design of our study is a limitation, as it impedes testing whether perceptions of goal structures truly precede students’ personal goal adoption (e.g., Lüftenegger, van de Schoot, Schober, Finsterwald, & Spiel, 2014), whether personal goals serve as a filter through which students perceive and make sense of the prevailing goal structures (e.g., Tapola & Niemivirta, 2008), or whether the contextual and personal aspects are reciprocally related (e.g., in that a matching environment further reinforces the dominant personal achievement goal). Moreover, even though we believe that, from a conceptual and empirical standpoint, achievement goals and achievement represent the most important outcomes to address in the first study on normative vs. appearance PAp goal structures, this focus automatically excludes other relevant external linkages, such as self-efficacy, implicit theories, self-regulated learning, test anxiety or socio-emotional variables (e.g., Bardach, Lüftenegger, Yanagida, Schober, & Spiel, 2019; Bostwick, Martin, Collie, & Durksen, 2019; Edward, 2014; Janke, Bardach, Oczlon, & Lüftenegger, 2019; Korn, Elliot, & Daumiller, 2019). Furthermore, while concerns about the measurement of (PAp) goal structures and their entanglement with personal goals are not new (e.g., Murayama & Elliot, 2009), we intend to revive them. In this vein, it has been suggested that the item referent might play a crucial role in that scales referring to the goal structure climate (as in the scales used in this study or in the PALS ‘goal structures’-scales), and not teachers’ practices (as with, e.g., the PALS ‘My teacher’s goal’-scales), produce stronger relations to achievement goals due to the higher conceptual overlap and often overlapping item content (Bardach, Oczlon et al., 2019; Karabenick et al., 2007). Hence, we do not know whether this overlap could have also influenced the relations between PAp goal structures and personal achievement goals found in our study. In a recent meta-analysis, however, scholars have also proposed that both scales might tap into slightly different goal structure realities: ‘Goal structure practices’ (measured with scales focusing on the teacher and his or her instructional strategies) precede and give raise to the broader ‘goal structure climate’ (measured with scales focusing on the classroom). The goal structure climate then translates into students’ adoption of personal achievement goals, and therefore exhibits stronger relations to achievement goals then the more distal goal structure practices (Bardach, Oczlon et al., 2019). As our study relied on scales with goal structures framed as classroom climate (‘In our mathematics class, …’), we highly recommend future research to study PAp goal structures using teacher-focused scales. Of course, theoretical progress in the field in terms of testing the suggested process model would best be achieved by using both version of scales in longitudinal studies. Finally, we have to keep in mind that the current study’s findings were obtained in an Austrian secondary education context in highest academic ability track schools in mathematics classes, meaning that they probably cannot be generalized to other contexts. For example, the subject of mathematics might present a specific context, e.g., in that mathematics tasks are inherently more structured and standardized than tasks in other subjects, such as writing an essay (e.g., Bardach et al., 2018). In addition, mathematics teachers might, on average, be more likely to employ specific instructional strategies to a lesser extent (e.g., autonomy support) and other strategies (e.g., controlling behavior) to a higher extent than teachers of other subjects and it has been suggested that students tend to report higher levels of anxiety in mathematics (e.g., Punaro & Reeve, 2012). Some of these or other factors specific to mathematics could then shape the effects of PAp goal structure types on outcomes in a way not observed in other school subjects, e.g., in that normative and appearance PAp goals blend together to a higher degree. Future research on PAp goal structures should therefore systematically investigate their structure and external linkages in a range of other subjects, educational contexts and countries.

Conclusions

To conclude, this study presents the first piece of evidence on normative and appearance PAp goal structures. Its findings provide support for the factorial differentiation and predictive validity of normative and appearance PAp goal structures at L1. This was not supported at L2, and furthermore, the overall PAp goal structure and the appearance and normative PAp goals were (empirically) almost indistinguishable here. Should we abandon the normative and appearance differentiation of PAp goal structures at this point? We do not think so. Rather, we hope that our work, and particularly the promising findings at L1 inspire researchers to further pursue examinations of two-tier PAp goal structures, while also attempting to explicate and tackle further measurement-related issues, such as the conflation of the L2 overall PAp goal structure with personal PAp goals.

Notes

1 It should be noted that there were initially eight PAp goal structure items, but two items were excluded due to theoretical and measurement-related considerations (e.g., one PAp goal structure item used exactly the same wording as a personal PAp goal item and was therefore excluded to avoid unnecessary construct overlap).