Exploring mechanisms of peer-effects in education: a frame-factor analysis of instruction

ABSTRACT

In the educational literature on peer effects, attention has been brought to the fact that the mechanisms creating peer effects are still to a large extent hidden in obscurity. The hypothesis in the study reported in this article was that the Frame Factor Theory (FFT) can be used to reveal such mechanisms. Using data from the Swedish TIMSS 2015 (N = 3761 students in 179 classes), a multilevel structural equation model (M-SEM) was specified in accordance with the FFT. As predicted, the SEM-model verified a strong relationship between class composition and a latent variable of limitations on instruction, a variable which in turn had a great impact on individual students’ results in TIMSS 2015. The study hereby seems to reveal a fundamental mechanism of peer effects and provides evidence for the FFT.

KEYWORDS:

• Pedagogy
• frame factor theory
• compositional effects
• contextual effects
• structural effects
• peer effects

Introduction

For more than 50 years, there has been extensive research conducted into peer effects in education, with many researchers pointing to the Coleman-report (1966) as a starting point in the US (Barr & Dreeben, 1977; Zimmerman, 2003). In educational research, the topic of interest has been labelled “compositional”, “contextual” or “structural” effects (Dreeben & Barr, 1988), while economists speak of “peer effects” (Arnott & Rowse, 1987; Epple & Romano, 2011; Yeung & Nguyen-Hoang, 2016). Yet, the issue at the heart of the research remains the same: How important are classmates for individual students’ results?

The combined literature on peer effects is extensive, but researchers have drawn attention to the fact that

there has been much less investigation of the ways in which they [peer effects] operate in schools or of their effects on school functioning or pupil progress. (Rutter & Maughan, 2002, p. 469)

It is possible to find studies revealing mechanisms behind peer effects. For instance, Lavy et al. (2011) report evidence that peer effects arise as a function of classroom environment. Their study showed that a larger share of boys was correlated with a poorer classroom environment (more noise, less order), which in turn had negative effects on individual results. Similar results were reported by Lazear (2001), Aizer (2009), Cheema and Kitsantas (2014) and Bäckström (2020). Thus, classroom discipline seems to be one of the mechanisms generating peer effects on students’ results.

Other studies have indicated that peer effects also might arise through the compositional effect that the class has upon the teacher’s instruction. Evidence of such effects was early reported by Bourke (1986). In his study, Bourke showed that the average student ability in class, mediated by class size, affected teaching practices. This, in turn, affected student achievement. Further evidence of such effects was reported by Blatchford et al. (2003), Garrison (2004) and Blatchford et al. (2007). Such a compositional effect has also been hypothesised by Thrupp (1999). Referred to as the “school composition effect”, the hypothesis is that compositional effects on achievement are mediated by effects on processes such as organisational, pedagogical (instruction) or psychosocial (classroom environment). The hypothesis was empirically tested and confirmed by Thrupp et al. (2002) and Dumay and Dupriez (2007). Similar results were reported by Opdenakker and Damme (2006).

In the study reported in this article, Lundgren's (1972) Frame Factor Theory was applied to the issue of peer effects, as an attempt to enhance the theoretical understanding and thus explore their mechanisms. Building on Carroll (1963), the theory predicts that class composition (mainly measured as prerequisite knowledge and aptitude) will steer and limit teachers’ instruction in class, in turn affecting student achievement. The theory was tested in a multilevel structural equation model using data from the Swedish TIMSS (Trends in International Mathematics and Science Study) 2015. The reported null-model showed that 27.4% of the variance in student achievement could be attributed to the between-class level. The full structural equation model explained 25.3% of this variance with the latent variable “Limitations on Instruction”, specified in accordance with the Frame Factor Theory, being the strongest predictor of achievement. Thus, the study seems to provide evidence of how compositional effects affect instruction, generating peer effects on individual outcomes.

The frame factor theory

As mentioned, several studies indicate that one important mechanism generating peer effects is the compositional effect the average class ability has upon teachers’ instruction. This indicates that the teachers’ practice in class to some extent is dependent on class composition. In contrast, the well-known theory of Mastery Learning (ML), formulated by Bloom (1974, 1984), advocates a view that the teacher’s instruction should be independent of class composition (Barr & Dreeben, 1977). Hence, the ML-recommendations of individualised, sequenced and cumulative learning with teachers tutoring rather than instructing students (Block & Burns, 1976; Bloom, 1984). The significant keywords above are “should be” independent; researchers have contested whether this is even possible in real life, since classroom management is in fact a collective activity (Arlin, 1984; Barr & Dreeben, 1977; Dahllöf, 1999), or as put by Nuthall:

In order to take responsibility for a class of 25–35 students who have different knowledge, skills, interests, and motivations, teachers have to develop ways of managing the class as a whole. It is impossible to focus on the individual learning of any one student for more than very brief periods. (Nuthall, 2005, p. 903)

A common ground for both ML and the Frame Factor Theory (FFT) is Carroll’s (1963) model of school learning. At the heart of the model is a notion of the relation between time needed (T_N) for students to learn a specific unit in the curriculum (CU), and the actual time at hand (T_A). If the time available is less than the time needed, then students will not learn: $(T_N)>(T_A)\to \mathrm{\neg }(CU)$

As mentioned above, Bloom’s ML-conclusion is that time available therefore must be individualised for each student. The FFT on the other hand is based on a view that classroom management and instruction is a collective practice (Dahllöf, 1967; Lundgren, 1972). Therefore, the theory is based on the hypothesis that the class-aggregated time needed (measured as prerequisite knowledge) will steer and limit (i.e. frame!) what teachers are able to do in terms of instruction. As Lundgren showed in his empirical study, this includes content, methods, timing and pacing of instruction (Lundgren, 1972). An original paradigm of the FFT is depicted in Figure 1.

Figure 1. Paradigm of the Frame Factor model (Lundgren, 1972, p. 43).

As shown in Figure 1, the FFT hypothesises that the teacher’s instruction in class will be steered and limited by a steering criterion group (SCG). According to the hypothesis, the SCG consists of the pupils between the 10th and 25th percentile of aptitude in the class. Teachers will – mostly subconsciously – use the advancement of this group to make decisions about content, methods, timing and pacing of the instruction, decisions which will affect all students in class. The SCG-hypothesis originates from Dahllöf (1967, 1970) and is supported empirically by Lundgren (1972).

As a result, research into peer-effects in general, and research employing the FFT in particular, relates to other research fields such as Opportunity to Learn (OTL), effects of classroom heterogeneity and ability grouping. By combining results from these fields, several contradicting results are revealed. For instance, Beckerman and Good (1981) reported evidence that both high- and low-aptitude students benefited from attending more favourable classes (measured as the share of high- and low-aptitude students in class). Similar results have been reported by Schindler Rangvid (2007), Ammermueller and Pischke (2009) and Yeung and Nguyen-Hoang (2016). Yet other research, into the effects of classroom heterogeneity, has reported that classroom heterogeneity does not have a negative effect on students’ outcome measured as students’ conceptual understanding (Decristana et al., 2017), neither has ability grouping any effects – neither positive nor negative – on students’ outcomes (Slavin, 1990).

Aim of the study

Given the different and somewhat contradicting results from previous research referred to above, there is a need of theoretical synthesis of different research fields which all relate to the issue of how important classmates are for individual students’ results. As referred above, there is reason to believe that some of this potential effect on student outcomes is mediated through the compositional effect on teachers’ instructional practices in class (Blatchford et al., 2007; Bourke, 1986; Dumay & Dupriez, 2007; Evertson et al., 1981; Opdenakker & Damme, 2006; Thrupp et al., 2002). In the study presented in this article, the FFT was applied to the issue of peer effects to initiate such a synthesis. Since peer effects are compositional effects, they occur on the between-class level. The hypothesis, therefore, was that between-class variance in student achievement could to some extent be explained by the steering and limiting effect the SCG had upon teachers’ instruction. This hypothesis was tested in the study. The research questions for the study reported were:

1. How does class composition affect teachers’ instruction?

2. If teachers’ instruction is affected by class composition, does this then affect students’ results?

Method

Data set and sample

The study was conducted using data from the Swedish TIMSS 2015 (Trends in International Mathematics and Science Study). The dataset contained mathematic test results and survey data for 4090 students in 8^th grade (on average 14 years old) and their 200 teachers in mathematics (Foy, 2017; Mullis & Hooper, 2016; Skolverket, 2016). Data on 8th grade were chosen for several reasons, including the fact that previous studies have shown that TIMSS test items have a stronger correlation to the Swedish 8th grade curricula than the 4th grade curricula (Skolverket, 2016).

Since the study was concerned with compositional effects in classroom instruction, there had to be more than a few students in the included classes. Only intact classes were included in the dataset. Descriptive statistical analysis revealed classes with missing data on class size, which therefore were excluded. Also, classes with less than 13 students proved to a large extent to be outliers in several variables included in the statistical modelling and of test results in mathematics (in the Swedish case, these classes are often for students with special needs or even located in smaller resource schools). These classes were therefore also excluded. The final dataset contained 3761 students in 179 classes, representing 92% of the students and 87% of the classes in the original sample. The gross and net sample is reported in Table 1 along with some descriptive statistics. Data were weighted using “house weights” on the between-class level (Rutkowski et al., 2010).

Table 1. Descriptive statistics for gross and net sample in the study.

	Gross sample	Net sample
Schools	150	142
Teachers	200	174
Classes	206	179
Average class size	23.3	23.6
Pupils	4090	3761
of which
share of boys	52%	52%
mother tongue*	3.64	3.65
parental education**	3.69	3.71
TIMSS mean score (BSMMAT01)	502.6	504.2

*Speaks test language at home, 1 = never, 4 = always.

** 1 = don’t know, 6 = university or higher.

Operationalisation and measures

According to the FFT, the average ability of the SCG will steer and limit the teacher’s instruction in class. Here, ability is defined and operationalised as prerequisite knowledge. One major problem in the study was that the data lacked measures of students’ actual prior knowledge in mathematics. The problem was solved by aggregating the mean test results of the SCG for each class, which is a somewhat similar approach to that which Kang (2007) used to study the importance of peers’ academic achievements in TIMSS for individual students’ own test results. It could be argued that this approach is insufficient, and that it could represent a circular argument where actual results are being used to explain actual results. But this was not the case in the analysis. First, the mean score of the SCG was only used as an independent variable, which the latent dependent variable “Limitations on Instruction” was regressed on. Individual test results were in turn regressed on that latent variable. Second, the mean score of the SCG only concerns some of the students, whilst this level in turn was used to explain the presence of limiting factors on instruction for the whole class. Third, the procedure was also based on previous research reporting strong correlations between prior knowledge and results (Nilsson & Tindrebäck, 2013; Seery, 2009), which implies that this proxy in fact would show a high correlation to actual prior knowledge if such measures could have been attained. One could argue that the third argument is insufficient, for example since Seery (2009) reports evidence based on high-stakes tests when TIMSS in fact is a low-stake test. Previous research has shown that this influences students’ motivation of taking low-stakes tests (Eklöf & Knekta, 2017), which in the Swedish case also seems to have some minor effects on students’ results (Skolverket, 2015). Other studies in the Swedish setting have reported strong correlations between students’ mathematical results in TIMSS, their final grades in mathematics (r = 0.74) and high-stakes national tests in mathematics (r = 0.73) (Wiberg, 2019). This suggests that correlations between test results and actual prior knowledge also should be strong.

In the TIMSS 2015 teacher survey, there are several variables that measure different limiting aspects on instruction. These include disruptive classroom behaviour, lack of prerequisite knowledge and lack of student motivation. The following variables were used to specify a latent variable of “Limitations on Instruction” (LIMIT_INSTR):

In your view, to what extent do the following limit how you teach this class?

• − Students lacking prerequisite knowledge or skills (BTBG15A),

• − Students suffering from a lack of sleep (15C),

• − Disruptive students (15D) and

• − Uninterested students (15E).

The respondents answered in a Likert scale ranging from “not at all” (1) to “a lot” (3). The Cronbach’s Alpha was 0.756.

The FFT also hypothesises that one of the mechanisms creating peer effects is that the limitations on instruction, originating from the level of the SCG, will steer the time spent on elementary and advanced curricula units and also the number of advanced units being covered in instruction (Lundgren, 1972). The higher the level of the SCG in class, the more time will be spent on more advanced and complex units.

In TIMSS 2015, there are some basic data on curricula coverage, but these survey items only concern whether a topic has been addressed in class or not, not how much time has been spent on the topic or in what way the topic was handled by the teacher. These variables were therefore assessed as not being theoretically adequate for the study at hand. In accordance with previous research (Hansson, 2010, 2012), two latent variables measuring teacher responsibility of learning were therefore specified using variables from the teacher survey. As Hansson has reported, the dimensions of teacher responsibility differ between classes due to their composition. Therefore, one latent variable was specified to measure the presence of “teacher responsibility for content” (T_INSTR), which was intended to capture responsibility for elementary parts of the curricula. The variables chosen for this latent variable were:

In teaching mathematics to this class, how often do you ask students to do the following?

• − Listen to me explain new mathematical content (BTBM18A),

• − Listen to me explain how to solve problems (18B) and

• − Memorise rules, procedures, and facts (18C).

The respondents answered in a Likert scale ranging from “never” (1) to “every lesson” (4). The Cronbach’s Alpha was 0.728.

Another latent variable was specified to measure the presence of more “Advanced Instruction” (T_ADVAN). The variables chosen for this latent variable were:

How often do you do the following when teaching this class?

• − Ask students to complete challenging exercises that require them to go beyond the instruction (BTBG14C),

• − Ask students to decide their own problem-solving procedures (BTBG14F) and

• − Ask students to work on problems for which there is no immediately obvious method of solution (BTBM18G).

The respondents answered in a Likert scale ranging from “never” (1) to “every lesson” (4). The Cronbach’s Alpha was 0.749.

The FFT hypothesises that teachers’ instruction to some extent is dependent on class composition. In that sense, the FFT contradicts Bloom’s ML-assumption of teachers’ instructional independence of class composition. But in this respect, the FFT also stands in contrast to some OTL-research. For instance, Klapp and Jönsson (2020) report results from a survey amongst Swedish students in 9th grade in which low-achieving students perceive that they primarily receive “simplified support” instead of “scaffolding support”, where the former consists of lowered standards and lowered expectations (thereby limiting the students’ opportunities to learn) and the latter consisting of “communicating expectations, helping students to structure their work, giving feedback, and building relationships”. Since such aspects – expectations, structuring work, explaining new concepts, giving feedback – have been shown to be positively related to students’ outcomes (Hattie, 2009; Rubie-Davies, 2010), it was deemed that a latent variable had to be specified to capture these dimensions so that the analysis could relate such dimensions to the dimension of limitations on instruction according to the FFT. By doing so, the study at hand could be able to contribute further to the understanding of such results as presented by Klapp and Jönsson (2020); why would this happen within the teachers’ instructional practices? How is students’ perception of these aspects related to the possible limiting effect class composition has upon teachers’ instruction? Empirical (and to some extent theoretical) explanations of this phenomena can be found in previous research (Blatchford et al., 2007; Bourke, 1986; Garrison, 2004), but by specifying such a latent variable, from the student surveys, some synthesis of results could be made possible.

Following results from the research of Rubie-Davies, discussed in Rubie-Davies (2010), one latent variable of such, “Instruction According to Students” (ST_INSTR), was specified using variables from the student survey. The variables chosen for this latent variable were:

How much do you agree with these statements about your mathematics lessons?

• − My teacher is easy to understand (BSBM18B),

• − My teacher has clear answers to my questions (18E),

• − My teacher is good at explaining mathematics (18F),

• − My teacher does a variety of things to help us learn (18H) and

• − My teacher tells me how to do better when I make a mistake (18I)

The respondents answered in a Likert scale ranging from “disagree” (1) to “agree a lot” (4). The Cronbach’s Alpha was 0.897. These variables were also aggregated to the between-class level so that the same latent variable could be specified on that level as well (SC_INSTR). The Cronbach’s Alpha at this level was 0.968.

In Table 2, descriptive statistics for all variables, including manifest control variables, are reported along with Cronbach Alpha values for each forthcoming latent variable (e.g. set of indicators).

Table 2. Descriptive statistics for all variables in the analysis.

		N	M	SD
Within-class	Instruction According to Students (ST_INSTR)
	S_INSTR_EASYUNDERST	3 711	3.05	0.86
	S_INSTR_CLEARANSW	3 706	3.04	0.87
	S_INSTR_EXPLGOOD	3 678	3.20	0.88
	S_INSTR_VARYHELP	3 702	3.21	0.84
	S_INSTR_TELLBETT	3 703	3.11	0.87
	Cronbach α = 0.897
Between-class	Advanced Instruction
	T_INSTR2_CHALLEXC	3 738	2.56	0.72
	T_INSTR2_PROBSOLVE	3 738	2.93	0.77
	T_INSTR2_NOSOLU	3 761	2.09	0.51
	Cronbach α = 0.749
	Limitations of Instruction
	T_LIMIT_LACKKNOW	3 624	2.08	0.59
	T_LIMIT_LACKSLEEP	3 609	1.71	0.59
	T_LIMIT_DISRUPT	3 623	1.72	0.61
	T_LIMIT_UNINTREST	3 624	1.90	0.58
	Cronbach α = 0.756
	Teacher Responsibility for Content
	T_INSTR1_LECTCONT	3 742	3.30	0.76
	T_INSTR1_LECTSOLVE	3 736	3.17	0.83
	T_INSTR1_MEMO	3 761	2.40	0.78
	Cronbach α = 0.728
	Instruction According to Students (SC_INSTR)
	SC_INSTR_EASYUNDERST	3 761	3.05	0.41
	SC_INSTR_CLEARANSW	3 761	3.03	0.39
	SC_INSTR_EXPLGOOD	3 761	3.20	0.46
	SC_INSTR_VARYHELP	3 761	3.21	0.36
	SC_INSTR_TELLBETT	3 761	3.11	0.34
	Cronbach α = 0.968
Dependent & control variables	BOY (Sex)	3 728	0.52	0.50
	MT (Mother Tongue)	3 725	3.65	0.68
	HEDREC (Home Educational Resources)	3 709	2.22	0.48
	PARED (Parents’ Level of Education)	3 568	3.71	2.19
	TIMSS-score (BSMMAT01)	3 761	504.2	70.10
	Level of SCG (Steering Criterion Group)	3 761	445.5	44.14

Validity and reliability

Since the study is based on secondary data (Johnston, 2014; Smith, 2008), there is a need for a thorough discussion of validity and reliability. First, how well does TIMSS 2015 correspond to the Swedish lower secondary curricula in mathematics? Analyses from the Swedish National Agency for Education [Skolverket] show that the outcomes that are measured in TIMSS do correspond to the curricula, especially for 8th grade (in comparison to 4th grade which is also included in TIMSS). Although, all parts of the curricula aren’t measured (Skolverket, 2016). Similar results are reported by Sollerman (2019).

Second, how well does TIMSS 2015 correspond to other measures of mathematic performance in Sweden? Sweden has had a long continuous decline in performance between 2000 and 2012 in both PISA and TIMSS. The same trends were reported in several other studies, suggesting a good external validity and reliability in TIMSS (Henrekson & Jävervall, 2016; Skolverket, 2014).

Third, are the surveys in TIMSS valid? In a study, Marsh et al. (2013) reported that differences in students’ motivation for school work could to some extent be explained by specific contextual factors for different countries and the usage of language in the TIMSS-survey. Despite this, they reported that correlations between motivation and test results were similar between countries in TIMSS. Similar results are reported for Sweden (Eklöf & Knekta, 2017) and in other international studies (Smith & López-Castro, 2017). Results of such give some support to the external validity and reliability of TIMSS, but it is important to point out that the validity of different questionnaire scales is not directly generalisable to the scales used in this study.

With this said, the trends of Swedish students’ motivation, as measured by TIMSS, are also verified in Swedish longitudinal surveys, further indicating external validity and reliability of the TIMSS-survey (Skolverket, 2019).

One concern with the validity of student surveys in TIMSS – as with all self-reported survey data – regards questions about students’ educational resources and their parents’ educational background. There are studies that have reported significant errors in measurement concerning these survey items (Engzell, 2019; Engzell & Jonsson, 2015), indicating a risk of underestimation of such effects. In the study reported in this article, correlation analysis was conducted to ensure that the measures of home educational resources that were used as independent control variables in the model at least showed expected correlations. The final model in Table 3 reports on these correlations. No other considerations were made.

Table 3. Results for all structural equation models (M1-4).

	M1			M2			M3			M4
	Std.est.	S.E.	P	Std.est.	S.E.	p	Std.est.	S.E.	p	Std.est.	S.E.	p
WITHIN-CLASS
Instruction According to Students (ST_INSTR)	0.18	0.02	0.00	0.18	0.02	0.00	0.17	0.03	0.00	0.17	0.03	0.00
Boy (S_SEX)	–	–	–	–	–	–	0.05	0.03	0.10	0.05	0.03	0.11
Mother tongue (S_MT)	–	–	–	–	–	–	0.07	0.02	0.01	0.07	0.03	0.00
Home educational resources (S_HMEDR)	–	–	–	–	–	–	0.23	0.02	0.00	0.23	0.02	0.00
Parents’ educational background (S_PARED)	–	–	–	–	–	–	0.12	0.02	0.00	0.12	0.02	0.00
R^2 (BSMATH01)	0.03	0.01	0.00	0.03	0.01	0.00	0.13	0.01	0.00	0.13	0.01	0.00
BETWEEN-CLASS
Teacher Responsibility for Content (T_INSTR)	−0.04	0.09	0.62	−0.04	0.08	0.68	−0.04	0.09	0.66	−0.04	0.09	0.67
Advanced Instruction (T_ADVAN)	0.10	0.12	0.37	0.08	0.10	0.41	0.10	0.10	0.34	0.10	0.12	0.38
Instruction According to Students (SC_INSTR)	−0.16	0.11	0.15	−0.12	0.09	0.19	−0.15	0.10	0.13	−0.17	0.10	0.10
Limitations of Instruction (LIMIT_INSTR)	−0.56	0.09	0.00	−0.55	0.08	0.00	−0.54	0.07	0.00	−0.43	0.12	0.00
Parents’ educational background (C_PARED)	–	–	–	–	–	–	–	–	–	0.28	0.08	0.00
R^2 (BSMATH01)	0.28	0.09	0.00	0.28	0.09	0.00	0.26	0.10	0.01	0.25	0.09	0.00
Mediation	No	–	–	Yes	–	–	Yes	–	–	Yes	–	–
Control variables	No	–	–	No	–	–	Within	–	–	All	–	–
ICC (BSMATH)	0.25	–	–	0.25	–	–	0.18	–	–	0.18	–	–
N (students)	3 761	–	–	3 761	–	–	3 530	–	–	3 530	–	–
N (classes)	179	–	–	179	–	–	179	–	–	179	–	–
χ^2 (mean) (st.dev.)	449.8 (5.11) (104)	–	–	457.8 (5.11) (107)	–	–	491.6 (4.36) (127)	–	–	514.3 (4.39) (141)	–	–
(df)
RMSEA (mean) (st.dev.)	0.03 (0.00)	–	–	0.03 (0.00)	–	–	0.03 (0.00)	–	–	0.03 (0.00)		–
CFI (mean) (st.dev.)	0.97 (0.00)	–	–	0.97 (0.00)	–	–	0.96 (0.00)	–	–	0.96 (0.00)		–
SRMR (within, mean) (st.dev.)	0.03 (0.00)	–	–	0.03 (0.00)	–	–	0.04 (0.00)	–	–	0.04 (0.00)		–
(between, mean) (st.dev.)	0.07 (0.00)	–	–	0.09 (0.00)	–	–	0.09 (0.00)	–	–	0.08 (0.00)		–

Regarding the reliability of TIMSS, and the variables used in the analysis, Cronbach’s’ Alpha measures have already been reported for each of the latent variables. They were assessed as being good enough for proceeding with the model since α = 0.7 < (Abedi & Baker, 1995; Ahmad et al., 2016; Bryman, 2008).

Data analysis

Peer effects are a result of pupils having peers. Thus, peer effects originate from the between-classroom variation. An analysis of peer effects therefore has to employ a multilevel approach to be able to handle both the selection problems in school systems (Manski, 1993; Sund, 2009) and the natural nested multilevel structure of school data (Heck et al., 2010; Hox, 2010).

In the study reported in this article, Multilevel Structural Equation Modeling (M-SEM) was employed, using MPLUS 8.2 with multilevel add-on. Data preparation was done using SAS Enterprise Guide 7.15.

Structural Equation Modeling (SEM) has grown rapidly within educational research (Gustafsson, 2009; Wang et al., 2017), mainly due to its flexible ability to combine the handling of complex relations between dependent and independent variables, mediation analysis and analysis of latent variables (Kline, 2015). SEM also provides useful tools for ensuring construct validity and reliability due to its inherent ways of testing internal consistency between multiple indicators (Abedi, 2002; Abedi & Baker, 1995; Marsh & Hau, 2007; Zhao & Gallant, 2012).

The aim of the study was to test the FFT’s ability to reveal and explain the mechanisms of peer effects. Variables in the analysis were therefore chosen in accordance with the theory. The SEM-framework – with its basis in Confirmatory Factor Analysis (CFA) – was therefore considered a suitable methodology (Kline, 2015).

Results

As a first step in multilevel analysis, an empty null-model is set up to investigate what variance in student results can be attributed to the within- and between-class levels. Such an analysis revealed an intraclass correlation coefficient (ICC) of 27.4%, a level which is similar to previous research into peer effects (Opdenakker & Damme, 2006; Rutter & Maughan, 2002). This means that about one fourth of the variance in individual students’ results could be attributed to which class the student attends.

The main variance of interest in the analysis, in reference to the aim of the study, was the variance on the between-class level. To clarify, the aim of the study was to explain this variance with a SEM-model derived from the FFT.

Measurement model

In SEM, the first step is to build a measurement model. The idea is to test whether the theorised model, with its latent variables, fits the data (Kline, 2015). Several measures are investigated to evaluate whether the model fits data or not. If it fits, then the dependent variable (in this case students’ test results) is introduced into a full structural equation model. If not, either a new model must be derived from the theory or the theory must be modified or rejected (Gustafsson, 2009; Kline, 2015).

In Figure 2, the measurement model is reported. Maximum likelihood estimation with robust standard errors (MLR) was used for estimation. All reported coefficients are standardised (STDYX in MPLUS). The coefficients are significant (p < 0.05) if nothing else is stated. The left-hand side of the model holds the within-class level. The right-hand side holds the between-class level.

Figure 2. Results for the measurement model.

The fit indices reveal that the model had a good fit to data. This conclusion though, was not supported by the χ²-analysis which reported a high (χ² = 371.9, df = 89) and significant (p = 0.000) value, indicating that the measurement model and actual data were significantly diverse from one another. This result was expected though, since the χ²-test is sensitive to large sample sizes (Gustafsson, 2009; Kline, 2015; Wang et al., 2017).

Apart from the χ²-test, all other indices revealed good or excellent model fit (Gustafsson, 2009; Hu & Bentler, 1999; Kline, 2015; Wang et al., 2017). The Root Mean Square Error of Approximation (RMSEA) was 0.029 (recommended < 0.05) and the Comparative Fit Index (CFI) was 0.972 (0.95 <), which suggested good model fit. The Standardized Root Mean Square Residual (SRMR) suggested a somewhat better fit on the within-class level (0.026) than on the between-class level (0.071) (recommended < 0.08), a result which was similar to those reported by Hansson (2010, 2012) for her model.

Starting on the within-class level, the analysis showed that the latent variable (ST_INSTR) explained most of the variance in the variables concerning the teacher’s clear answers and good explanations, with standardised factor loadings of 0.83 and 0.87 respectively. The least of the variance was explained in the feedback variable that the teacher tells students what they can do better, with a factor loading of 0.67.

The same latent variable, but on the between-class level (SC_INSTR), explained even more of the variance in the manifest variables. All factor loadings but one exceeded 0.90.

Continuing with the between-class level, the analysis showed that each latent variable explained less variance in some of the manifest variables. One example is T_INSTR which only explained 11.6% of the variance (.34*.34) in the manifest variable measuring how often students are asked to memorise rules, procedures and facts. In general, these lower factor loadings for a few of the variables might explain the somewhat lower SRMR-value for the between-class level than the within-class level.

Structural equation model

Since the fit indices reported a good model fit to data, the complete structural equation model was tested. The dependent variable with student test results (BSMMAT)¹ was introduced and a stepwise approach was employed. Four models were estimated (M1-4), with control variables and mediation tests added successively. The results and fit indices from the stepwise analysis are reported in Table 3.

In M1 the latent variables were used to estimate the dependent variable, without controls and tests for mediation. In M2 mediation tests for the latent variable “Limitations on Instruction” (LIMIT_INSTR), “Teacher Responsibility for Content” (T_INSTR) and “Advanced Instruction” (T_ADVAN) were added stepwise. Since the coefficients weren’t affected, the full mediating M2-model is reported in Table 3. As reported in Table 3, the mediation analysis had practically no impact on the estimated effect of “Limitations on Instruction” (LIMIT_INSTR) on the dependent variable (from −0.56 in M1 to −0.55 in M2). This is of great importance in reference to the research questions and the predictions done according to the FFT. This is discussed below.

In M3 controls were added on the within-class level and in M4 for the between-class level. As reported in Table 3, the final controls on the between class-level reduced the negative effect of “Limitations on Instruction” (LIMIT_INSTR) to some extent, from −0.54 in M3 to −0.43 in M4.

In Figure 3, the complete model is reported. The figure has been somewhat simplified to facilitate the interpretation. All indicators of the latent variables, as depicted in Figure 2, have been left out in Figure 3 (even though they were in the analysis).

Figure 3. Final structural equation model (M4). * = non-significant effects (p > 0.05).

As reported in Figure 3, the latent variable “Limitations on Instruction” (LIMIT_INSTR) had a great impact on students’ results (−0.43), a variable which in turn was affected by the level of the SCG (−0.44). These effects were all controlled for the mean level of parental education in class (C_PARED, 0.28). This was the main finding in the study. As hypothesised by the FFT, the steering and limiting effect class composition has upon teachers’ instruction seems to be a highly significant, and theoretically relevant, mechanism of peer effects in education. Similar results concerning school compositional effects were reported by Opdenakker and Damme (2006).

An interesting result in the analysis concerns the latent variable “Instruction According to Students” (ST_INSTR). On the within-class level, under control of student characteristics, the latent variable showed a positive effect (0.17) on students’ test results (the effect was equal to the combined effect of parents’ educational background (0.12) and students’ sex (0.05) and was similar to those reported in earlier studies (Hattie, 2009; Rubie-Davies, 2010). But on the between-class level, the same aggregated latent variable (SC_INSTR) showed a non-significant negative effect (−0.17) on students’ test results. This result seems to have implications regarding teachers’ management of classrooms, which will be discussed further on in the article.

Even though the measurement model indicated that the two latent variables of teachers’ instruction, (T_INSTR) and (T_ADVAN), did capture different dimensions of teacher responsibility in instruction² (Hansson, 2010), the structural equation model revealed that, firstly, the latent variable of limitations on instruction had no effect on the two mentioned variables and, secondly, that the two latent variables in turn had no effect on students’ test results.

As seen in Figure 3, the model was specified with mediation analysis (Baron & Kenny, 1986; Field, 2018; Hayes & Matthes, 2009) between LIMIT_INSTR and the two latent variables of teachers’ instruction, T_INSTR and T_ADVAN. The analysis indicated that none of the original effects of “Limitations on Instruction” (LIMIT_INSTR) were mediated through T_INSTR or T_ADVAN.

These results contradict the expected results according to the FFT. Two alternative interpretations are suggested. Either the predictions of the FFT are incorrect or the latent variables T_INSTR and T_ADVAN were not able to capture the dimensions of instruction implied by the FFT. The first interpretation would imply that the FFT is refuted, the second that the two latent variables have a poor construct validity. Regarding the problem with use of secondary data, that data has been gathered for other purposes than for the study at hand, I argue that the second interpretation is more plausible in reference to reported results and previous research concerning the FFT (Lundgren, 1972).

In total, the structural equation model explained 12.9% of the within class variance and 25.3% of the between-class variance in individual TIMSS-results. These results are comparable with several previous studies of school compositional effects (Dumay & Dupriez, 2007; Opdenakker & Damme, 2006).

Discussion and future research

The study reported in this article provides some evidence on the mechanisms of peer effects in education. Given the cross-sectional design of TIMSS, the possibilities of drawing causal conclusions are limited. But since the results are in line with previous findings on compositional effects in education (Beckerman & Good, 1981; Rutter & Maughan, 2002; Yeung & Nguyen-Hoang, 2016), and their relation to compositional effects on teachers’ instructional practices (Bourke, 1986; Dumay & Dupriez, 2007; Evertson et al., 1981; Garrison, 2004; Lundgren, 1972; Thrupp et al., 2002) the results seem to verify the relation between class composition, teachers’ instructional practices and students’ outcomes.

By applying Lundgren’s (1972) Frame Factor Theory (FFT) to the issue of peer effects, the study verified the prediction that some peer effects seem to arise due to the steering and limiting effect class composition has upon instruction. In reference to the research questions, the study revealed that class composition affects teachers’ instruction which in turn affects students’ results. The study also verified the Steering Criterion Group-hypothesis (SCG); the steering and limiting effect on instruction can be predicted by the mean level of the students in the 10th-25th percentile of aptitude in class. Hereby, the study also supports previous findings – but with another theoretical outlook – from research into school compositional effects (Dumay & Dupriez, 2007; Garrison, 2004; Opdenakker & Damme, 2006; Thrupp et al., 2002).

One FFT-prediction that the study could not verify was the modus operandi of the limitations on instruction. According to the FFT, the level of the SCG will affect the content, timing and pacing of instruction. Lundgren (1972) operationalised this phenomenon in curricula units covered, their level of complexity and time spent. With no adequate data of such in TIMSS 2015, two latent variables capturing different dimensions of teachers’ instruction were specified in reference to previous research (Hansson, 2010, 2012). But the structural equation model revealed no relations or effects as predicted by the FFT. Even though the measurement model showed good fit to data, it is impossible to rule out the possibility of poor construct validity for these two latent variables (T_INSTR and T_ADVAN) in reference to the FFT. This conclusion summons future research into peer effects and the FFT to further investigate whether such effects can be verified.

With this said, the study did verify previous research implying that disruptive classroom behaviour is one of several mechanisms generating peer effects (Aizer, 2009; Cheema & Kitsantas, 2014; Lavy et al., 2011; Lazear, 2001), since such variables were included in the latent variable (LIMIT_INSTR).

In reference to previous research into the importance of teachers’ classroom practices (Hattie, 2009; Rubie-Davies, 2010), the model also included a latent variable measuring “Instruction According to Students” (ST_INSTR). Interestingly, the analysis revealed a significant positive effect (0.17) on the within-class level, but a similar opposing – though non-significant – negative effect (−0.17) on the between-class level. One proposed interpretation is that this yet again indicates that teachers’ instruction in class is dependent on the composition of the class (Dreeben & Barr, 1988; Nuthall, 2005; Weinert et al., 1989). What teachers do in class affects individual students’ learning, but in managing a whole class, the teacher is somewhat bound by the instructional conditions set by the composition of the class and aggregated prerequisite knowledge. Such an interpretation is supported by the fact that the latent variable of “Limitations on Instruction” (LIMIT_INSTR) had a negative effect (−0.22) on the aggregated variable of “Instruction According to Students” (SC_INSTR) on the between-class level. These results therefore summon future research into peer-effects, opportunity to learn and compositional effects to further try to synthesise theoretical perspectives and empirical estimation to enhance our understanding of the mechanisms of peer-effects.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

Research funded by The National Union of Teachers in Sweden

Notes

1 In TIMSS IEA employs a method of “plausible values” (PV), i.e estimations of plausible values for each student. This is done so each student doesn’t have to answer each question in the test, thereby saving time. In TIMSS, five different PVs are reported (BSMMAT01-05) for each student. When analysing the results, MPLUS can be programmed to run the SEM-model on each PV and then report the mean estimated effect on the PVs. The fit indices are reported as means for each PV with a standard deviation between the estimates. This is reported in Table 3.

2 This was also confirmed with correlation analysis since R =  0.03 (p > 0.05).