Multilevel Design Parameters to Plan Cluster-Randomized Intervention Studies on Student Achievement in Elementary and Secondary School

Figures & data

Figure 1. Results from previous research on multilevel design parameters for student achievement in elementary and secondary school in the United States: (a) Between-classroom (${\rho }_{L2}$) and between-school differences (${\rho }_{L3}$), and explained variances by (b) pretest scores, (c) sociodemographic characteristics, and (d) pretest scores and sociodemographic characteristics at the student ($R_{L1}^2$), classroom ($R_{L2}^2$), and school level ($R_{L3}^2$). Note. Boxplots show distributions across all domains. The distributions in mathematics/science/reading are based on 341/12/370 values for elementary school (grades 1–6) and 266/93/223 values for secondary school (grades 7–12). The underlying data table can be obtained from the Open Science Framework (https://osf.io/2w8nt). In the upper panels of Figure 1a–d, design parameters obtained from two-level models (students at L1 within schools at L3) are shown as reported in the following studies: Bloom et al. (1999) reported ${\rho }_{L3}$ for elementary schools in 1 city. Bloom et al. (2007) reported ${\rho }_{L3},$ $R_{L1}^2,$ and $R_{L3}^2$ for pretests and sociodemographics for elementary and secondary schools in 5 districts. Brandon et al. (2013) reported upper bounds of the means of ${\rho }_{L3}$ across several years for elementary and secondary schools in 1 state. Hedberg et al. (2004) reported ${\rho }_{L3}$ and $R_{L3}^2$ for sociodemographics for elementary schools in 120 districts and for secondary schools on a nationwide basis (values are retrieved from Schochet, 2008). Hedges and Hedberg (2007a) reported ${\rho }_{L3},$ $R_{L1}^2,$ and $R_{L3}^2$ for pretests, sociodemographics, and their combination for elementary and secondary schools on a nationwide basis (across districts and states). Hedges and Hedberg (2013) reported ${\rho }_{L3},$ $R_{L1}^2,$ and $R_{L3}^2$ for pretests and sociodemographics for elementary and secondary schools in 11 states (with between-district variance pooled into between-school variance within states). Schochet (2008) reported ${\rho }_{L3}$ for elementary schools based on 3 studies conducted in 6 cities, 12 districts, and 7 states, respectively. Spybrook, Westine, et al. (2016) reported means of ${\rho }_{L3},$ $R_{L1}^2,$ and $R_{L3}^2$ across several years for pretests and sociodemographics for elementary and secondary schools in 3 states. Westine et al. (2013) reported means of ${\rho }_{L3},$ $R_{L1}^2,$ and $R_{L3}^2$ across 5 years for pretests, sociodemographics, and their combination for elementary and secondary schools in 1 state. In the lower panels of Figure 1a–d, design parameters obtained from three-level models (students at L1 within classrooms at L2 within schools at L3) are shown as reported in the following studies: Jacob et al. (2010) reported ${\rho }_{L2},$ ${\rho }_{L3},$ $R_{L1}^2,$ $R_{L2}^2,$ and $R_{L3}^2$ for pretests, sociodemographics and their combination for elementary schools in 6 districts. Xu and Nichols (2010) reported ${\rho }_{L2},$ ${\rho }_{L3},$ $R_{L1}^2,$ $R_{L2}^2,$ and $R_{L3}^2$ for pretests, sociodemographics, and their combination for elementary and secondary schools in 2 states. Zhu et al. (2012) reported ${\rho }_{L2},$ ${\rho }_{L3},$ $R_{L1}^2,$ $R_{L2}^2,$ and $R_{L3}^2$ for pretests for elementary and secondary schools on a nationwide basis.

Table 1. Results from previous large-scale studies on student achievement in Germany: Between-classroom (${\rho }_{L2}$) and between-school differences (${\rho }_{L3}$) by grade and domain.

					Verbal skills in German				English reading
Grade	Mathematics		Science		Reading		Listening
	${\rho }_{L2}$	${\rho }_{L3}$	${\rho }_{L2}$	${\rho }_{L3}$	${\rho }_{L2}$	${\rho }_{L3}$	${\rho }_{L2}$	${\rho }_{L3}$	${\rho }_{L2}$	${\rho }_{L3}$
Elementary school
1
2
3
4	0.02^a	0.25^b/0.27^c/0.15^a		0.26^d	0.04^e	0.22^f/0.24^g/0.17^e		0.27^h
Secondary school
5
6
7	0.03^i	0.45^i
8		0.51^j/0.49^k		0.41^l
9		0.56^m		0.54^m		0.58^m/0.48^n				0.55^n
10	0.03^i	0.47^o/0.62^i		0.44^p
11
12		0.52^q

Note. Design parameters in italic/normal print are based on national probability samples/representative samples of a certain state. ^aLehmann & Lenkeit (2008, Table 3.6). ^bHaag & Roppelt (2012, Figure 5.11). ^cMartin et al. (2013, p. 139). ^dMartin et al. (2013, p. 140). ^eLehmann & Lenkeit (2008, Table 3.3). ^fBöhme & Weirich (2012, Figure 5.3). ^gMartin et al. (2013, p. 138). ^hBöhme & Weirich (2012, Figure 5.4). ⁱBaumert et al. (2003, Figure 10.6) where values of represent ${\rho }_{L3}$ the sum of the variances between schools and between school types. ^jMartin et al. (2000, p. 77). ^kBaumert et al. (2000, p. 68). ^lMartin et al. (2000, p. 76). ^mBrunner et al. (2018, Table S2) with data from 15-year-old students of which about 65% attend grade 9; most remaining students attend grade 10. ⁿKnigge & Köller (2010, Table 10.1). ^oSenkbeil (2006, p. 298). ^pSenkbeil (2006, p. 299). ^qBaumert et al. (2000, p. 69).

Table 2. Number of students (L1), classrooms (L2), and schools (L3), and median cluster sizes by grade, large-scale study, and school track.

Grade	Study	Total					Academic track					Non-academic track
		N			Mdn		N			Mdn		N			Mdn
		L1	L2	L3	L1	L2	L1	L2	L3	L1	L2	L1	L2	L3	L1	L2
Elementary school
1	NEPS-SC2	6,731	1,020	374	6	2	–	–	–	–	–	–	–	–	–	–
2	NEPS-SC2	6,319	986	362	6	2	–	–	–	–	–	–	–	–	–	–
3	NEPS-SC2	5,554	888	354	6	2	–	–	–	–	–	–	–	–	–	–
4	NEPS-SC2	5,419	1,026	349	4	3	–	–	–	–	–	–	–	–	–	–
Secondary school
5	NEPS-SC3	5,380	452	225	12	2	2,340	155	76	15	2	3,040	297	149	10	2
6	NEPS-SC3	5,026	452	211	11	2	2,287	170	76	13	2	2,739	282	135	10	2
7	NEPS-SC3	6,279	614	266	10	2	2,980	254	105	11	2	3,299	360	161	8	2
9	NEPS-SC3	4,651	627	239	6	2	2,255	271	95	8	2	2,396	356	144	5	2
9	NEPS-SC4	14,640	975	518	15	2	5,098	292	146	18	2	9,542	683	372	14	2
9	DESI	10,543	427	219	25	2	4,308	163	82	27	2	6,235	264	137	24	2
9	PISA-I+	6,020	275	152	23	2	2,664	116	61	23	2	3,356	159	91	22	2
10	NEPS-SC4	10,031	824	402	12	2	3,770	298	118	12	2	6,261	526	284	12	2
10	PISA-I+	6,020	275	152	23	2	2,664	116	61	23	2	3,356	159	91	22	2
11	NEPS-SC4	4,566	n/a	175	26	n/a	4,087	n/a	143	29	n/a	479	n/a	32	14	n/a
12	NEPS-SC4	3,963	n/a	168	23	n/a	3,596	n/a	137	27	n/a	367	n/a	31	12	n/a

Note. Cells containing a dash indicate that tracking into different school types does not occur in elementary school. Cells containing n/a indicate that classroom-level information was not available as 11th and 12th grade students did not attend intact classrooms, but rather the grouping of students varied depending on the subject taught.

Figure 2. Multilevel design parameters for student achievement for the general student population without and with adjustment for mean-level differences between school types: (a) Between-classroom (${\rho }_{L2}$) and between-school differences (${\rho }_{L3}$), and explained variances by (b) pretest scores, (c) sociodemographic characteristics, and (d) pretest scores and sociodemographic characteristics at the student ($R_{L1}^2$), classroom ($R_{L2}^2$), and school level ($R_{L3}^2$). Note. Boxplots show distributions across all achievement domains. For grades 1–10, design parameters are based on three-level models (students at L1 within classrooms at L2 within schools at L3). For grades 11–12, design parameters are based on two-level models (students at L1 within schools at L3) as 11th and 12th grade students did not attend intact classrooms, but rather the grouping of students varied depending on the subject taught. This means that design parameters at L2 (i.e., ${\rho }_{L2}$ and $R_{L2}^2$) were estimated for grades 1–10 only. In Figure 2a, intraclass correlations $\rho$ were estimated in intercept-only models (model set 1). In Figure 2b, explained variances $R^2$ by pretests were estimated in pretest covariate(s) models (model set 2). In Figure 2c, explained variances $R^2$ by sociodemographics were estimated in sociodemographic covariates models (model set 3). In Figure 2d, explained variances $R^2$ by pretests and sociodemographics were estimated in pretest and sociodemographic covariates models (model set 4). To estimate design parameters that were adjusted for mean-level achievement differences between school types offered in German secondary education (L3 adjusted), dummy-coded indicator variables representing the various school types were added as additional covariates at L3. The complete collection of design parameters is available in Tables B1, B3, B5, B7, B9, B11, B13, and B15 in the Supplemental Online Material B on the Open Science Framework (https://osf.io/2w8nt).

Table 3. Normative distributions of multilevel design parameters for student achievement: (a) Between-classroom (${\rho }_{L2}$) and between-school differences (${\rho }_{L3}$), and explained variances by (b) pretest scores, (c) sociodemographic characteristics, and (d) pretest scores and sociodemographic characteristics at the student ($R_{L1}^2$), classroom ($R_{L2}^2$), and school level ($R_{L3}^2$).

Statistic	a. Model set 1		b. Model set 2			c. Model set 3			d. Model set 4
	Intercept-only model		Pretest covariate(s) model			Sociodemographic covariates model			Pretest and sociodemographic covariates model
	${\rho }_{L2}$	${\rho }_{L3}$	$R_{L1}^2$	$R_{L2}^2$	$R_{L3}^2$	$R_{L1}^2$	$R_{L2}^2$	$R_{L3}^2$	$R_{L1}^2$	$R_{L2}^2$	$R_{L3}^2$
Elementary school (grades 1–4)
Minimum	0.03	0.04	0.01	0.00	0.06	0.00	0.16	0.14	0.02	0.27	0.27
25th percentile	0.04	0.10	0.09	0.11	0.33	0.04	0.38	0.51	0.14	0.46	0.66
Median	0.05	0.11	0.24	0.27	0.39	0.10	0.61	0.63	0.30	0.67	0.77
75th percentile	0.06	0.15	0.36	0.37	0.59	0.10	0.66	0.69	0.38	0.82	0.82
Maximum	0.09	0.22	0.51	0.85	0.90	0.14	0.84	0.92	0.52	0.89	0.92
Secondary school (grades 5–12)
General student population
Minimum	0.01	0.09	0.08	0.09	0.00	0.00	0.16	0.16	0.08	0.31	0.61
25th percentile	0.03	0.27	0.14	0.45	0.87	0.01	0.35	0.80	0.17	0.69	0.95
Median	0.04	0.35	0.20	0.65	0.96	0.03	0.53	0.88	0.22	0.77	0.98
75th percentile	0.06	0.39	0.30	0.75	0.98	0.05	0.66	0.91	0.31	0.86	0.99
Maximum	0.13	0.59	0.56	0.95	1.00	0.10	0.89	0.97	0.57	0.97	1.00
General student population with adjustment for mean-level differences between school types
Minimum	0.02	0.03	0.08	0.06	0.01	0.00	0.15	0.24	0.08	0.27	0.51
25th percentile	0.04	0.08	0.14	0.41	0.72	0.02	0.34	0.54	0.17	0.70	0.84
Median	0.06	0.10	0.21	0.63	0.81	0.03	0.46	0.70	0.22	0.77	0.90
75th percentile	0.09	0.12	0.30	0.75	0.91	0.05	0.64	0.80	0.31	0.86	0.97
Maximum	0.18	0.22	0.56	0.94	0.99	0.10	0.90	0.96	0.57	0.97	1.00
Academic track
Minimum	0.01	0.01	0.07	0.05	0.01	0.00	0.07	0.27	0.08	0.19	0.64
25th percentile	0.04	0.04	0.11	0.51	0.52	0.02	0.44	0.46	0.15	0.72	0.82
Median	0.05	0.06	0.20	0.61	0.68	0.03	0.64	0.66	0.22	0.85	0.89
75th percentile	0.09	0.09	0.30	0.82	0.84	0.05	0.75	0.81	0.32	0.92	0.95
Maximum	0.21	0.23	0.54	0.94	0.97	0.10	0.92	0.94	0.55	0.98	0.98
Non-academic track
Minimum	0.01	0.07	0.07	0.07	0.02	0.00	0.11	0.14	0.07	0.52	0.45
25th percentile	0.04	0.16	0.16	0.41	0.81	0.02	0.42	0.66	0.18	0.74	0.91
Median	0.06	0.20	0.20	0.65	0.88	0.03	0.53	0.79	0.24	0.84	0.96
75th percentile	0.09	0.23	0.33	0.80	0.98	0.06	0.67	0.89	0.34	0.91	0.99
Maximum	0.15	0.36	0.59	0.94	1.00	0.21	0.90	0.99	0.61	0.97	1.00

Note. Statistics were calculated across achievement domains and are based on the estimates obtained from three-level models (students at L1 within classrooms at L2 within schools at L3) for grades 1–10 and two-level models (students at L1 within schools at L3) for grades 11–12 because 11th and 12th grade students did not attend intact classrooms, but rather the grouping of students varied depending on the subject taught. This means that statistics for estimates at L2 (i.e., ${\rho }_{L2}$ and $R_{L2}^2$) were calculated for grades 1–10 only. Statistics were calculated excluding meta-analytically pooled results of grade 9. The complete collection of normative distributions is available in Tables B2, B4, B6, B8, B10, B12, B14, and B16 in the Supplemental Online Material B on the Open Science Framework (https://osf.io/2w8nt).

Figure 3. How much bias may result in design parameters for student achievement for the general student population at the student (L1) and school level (L3) when the classroom level (L2) is ignored? Comparison of corresponding design parameters obtained from three-level models versus two-level models: (a) Between-school differences (${\rho }_{L3}$), and variances explained by (b) pretest scores, (c) sociodemographic characteristics, and (d) pretest scores and sociodemographic characteristics at the student ($R_{L1}^2$) and school level ($R_{L3}^2$). Note. The graph juxtaposes corresponding design parameters estimated by three-level models (x-coordinate; students at L1 within classrooms at L2 within schools at L3) with design parameters estimated by two-level models (y-coordinate; students at L1 within schools at L3). The black line marks congruence of three- and two-level design parameters. Larger labeled dots exceed a deviation of ± 0.20 between three- and two-level design parameters. For example, in Figure 3b, left grid (“Student Level (L1)”), the dot labeled with “a” (representing German vocabulary in grade 1) shows that $R_{L1}^2$ was 0.24 when specifying a three-level pretest covariate model, whereas $R_{L1}^2$ was 0.47 when specifying a two-level pretest covariate model. ^aVocabulary (NEPS-SC2, grade 1). ^bDeclarative metacognition (NEPS-SC2, grade 3). ^cBasic cognitive functions: Reasoning (NEPS-SC2, grade 2). ^dReading speed (DESI, grade 9, wave 2). ^eDeclarative metacognition (NEPS-SC2, grade 1). ^fDeclarative metacognition (NEPS-SC2, grade 3). ^gBasic cognitive functions: Perception speed (NEPS-SC3, grade 9). ^hReading speed (NEPS-SC2, grade 2). ⁱBasic cognitive functions: Perception speed (NEPS-SC3, grade 5). ^jDeclarative metacognition (NEPS-SC2, grade 3). ^kBasic cognitive functions: Reasoning (NEPS-SC2, grade 2). ^lBasic cognitive functions: Perception speed (NEPS-SC3, grade 9). ^mReading speed (NEPS-SC2, grade 2). ⁿBasic cognitive functions: Perception speed (NEPS-SC2, grade 2).

Figure 4. Flow chart to guide the choice of design parameters as a function of key characteristics of the target intervention. Note. Tables B1–B16 can be retrieved from Supplemental Online Material B. A comprehensive overview of the achievement measures analyzed in the present study is given in Table A5 in the Supplemental Online Material A. The Supplemental Online Materials are available on the Open Science Framework (https://osf.io/2w8nt).

Bloom, H. S., Bos, J. M., & Lee, S.-W. (1999). Using cluster random assignment to measure program impacts. Statistical implications for the evaluation of education programs. Evaluation Review, 23(4), 445–469. https://doi.org/10.1177/0193841X9902300405

 PubMed Web of Science ®Google Scholar

Bloom, H. S., Richburg-Hayes, L., & Black, A. R. (2007). Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29(1), 30–59. https://doi.org/10.3102/0162373707299550

 Web of Science ®Google Scholar

Brandon, P. R., Harrison, G. M., & Lawton, B. E. (2013). SAS code for calculating intraclass correlation coefficients and effect size benchmarks for site-randomized education experiments. American Journal of Evaluation, 34(1), 85–90. https://doi.org/10.1177/1098214012466453

 Web of Science ®Google Scholar

Hedberg, E. C., Santana, R., & Hedges, L. V. (2004). The variance structure of academic achievement in America. Annual meeting of the American Educational Research Association, San Diego, CA.

 Google Scholar

Schochet, P. Z. (2008). Statistical power for random assignment evaluations of education programs. Journal of Educational and Behavioral Statistics, 33(1), 62–87. https://doi.org/10.3102/1076998607302714

 Web of Science ®Google Scholar

Hedges, L. V., & Hedberg, E. C. (2007a). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87. https://doi.org/10.3102/0162373707299706

 Web of Science ®Google Scholar

Hedges, L. V., & Hedberg, E. C. (2013). Intraclass correlations and covariate outcome correlations for planning two- and three-level cluster-randomized experiments in education. Evaluation Review, 37(6), 445–489. https://doi.org/10.1177/0193841X14529126

 PubMed Web of Science ®Google Scholar

Spybrook, J., Westine, C. D., & Taylor, J. A. (2016). Design parameters for impact research in science education: A multistate analysis. AERA Open, 2(1), 1–15. https://doi.org/10.1177/2332858415625975

 PubMed Web of Science ®Google Scholar

Westine, C. D., Spybrook, J., & Taylor, J. A. (2013). An empirical investigation of variance design parameters for planning cluster-randomized trials of science achievement. Evaluation Review, 37(6), 490–519. https://doi.org/10.1177/0193841X14531584

 PubMed Web of Science ®Google Scholar

Jacob, R. T., Zhu, P., & Bloom, H. S. (2010). New empirical evidence for the design of group randomized trials in education. Journal of Research on Educational Effectiveness, 3(2), 157–198. https://doi.org/10.1080/19345741003592428

 Google Scholar

Xu, Z., & Nichols, A. (2010). New estimates of design parameters for clustered randomization studies. Findings from North Carolina and Florida. National Center for Analysis of Longitudinal Data in Education. http://www.urban.org/sites/default/files/alfresco/publication-pdfs/1001394-New-Estimates-of-Design-Parameters-for-Clustered-Randomization-Studies.pdf

 Google Scholar

Zhu, P., Jacob, R., Bloom, H., & Xu, Z. (2012). Designing and analyzing studies that randomize schools to estimate intervention effects on student academic outcomes without classroom-level information. Educational Evaluation and Policy Analysis, 34(1), 45–68. https://doi.org/10.3102/0162373711423786

 Web of Science ®Google Scholar

Haag, N., & Roppelt, A. (2012). Der Ländervergleich im Fach Mathematik [National Assessment Study in mathematics]. In P. Stanat, H. A. Pant, K. Böhme, & D. Richter (Eds.), Kompetenzen von Schülerinnen und Schülern am Ender der vierten Jahrgangsstufe in den Fächern Deutsch und Mathematik. Ergebnisse des IQB-Ländervergleichs 2011 (pp. 117–127). Waxmann.

 Google Scholar

Baumert, J., Trautwein, U., & Artelt, C. (2003). Schulumwelten—Institutionelle Bedingungen des Lehrens und Lernens [School contexts—institutional conditions for teaching and learning]. In Deutsches PISA-Konsortium (Ed.), PISA 2000. Ein differenzierter Blick auf die Länder der Bundesrepublik Deutschland (pp. 261–331). Leske + Budrich. https://doi.org/10.1007/978-3-322-97590-4_11

 Google Scholar

Martin, M. O., Foy, P., Mullis, I. V. S., & O’Dwyer, L. M. (2013). Effective schools in reading, mathematics, and science at the fourth grade. In M. O. Martin & I. V. S. Mullis (Eds.), TIMSS and PIRLS 2011: Relationships among reading, mathematics, and science achievement at the fourth grade—implications for early learning (pp. 109–179). TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timsspirls2011/downloads/TP11_Relationship_Report.pdf

 Google Scholar

Lehmann, R., & Lenkeit, J. (2008). ELEMENT. Erhebung zum Lese- und Mathematikverständnis. Entwicklungen in den Jahrgangsstufen 4 bis 6 in Berlin. Abschlussbericht über die Untersuchungen 2003, 2004 und 2005 an Berliner Grundschulen und grundständigen Gymnasien [ELEMENT: Study of reading and mathematics literacy. Development from grades 4 to 6 in Berlin. Final research report on the 2003, 2004, and 2005 assessments at primary schools and undergraduate academic tracks in Berlin]. Humboldt-Universität zu Berlin. https://www.researchgate.net/profile/Jenny_Lenkeit/publication/273380369_ELEMENT_Erhebung_zum_Lese-_und_Mathematik-verstandnis_-_Entwicklungen_in_den_Jahrgangsstufen_4_bis_6_in_Berlin_Abschlussbericht_uber_die_Untersuchungen_2003_2004_und_2005_an_Berliner_Grundschulen_und_/links/553f61600cf23e796bfb38c2.pdf?origin=publication_detail

 Google Scholar

Böhme, K., & Weirich, S. (2012). Der Ländervergleich im Fach Deutsch [National Assessment Study in German]. In P. Stanat, H. A. Pant, K. Böhme, & D. Richter (Eds.), Kompetenzen von Schülerinnen und Schülern am Ende der vierten Jahrgangsstufe in den Fächern Deutsch und Mathematik. Ergebnisse des IQB-Ländervergleichs 2011 (pp. 103–116). Waxmann.

 Google Scholar

Martin, M. O., Mullis, I. V. S., Gregory, K. D., Hoyle, C., & Shen, C. (2000). Effective schools in science and mathematics: IEA’s Third International Mathematics and Science Study. International Study Center, Boston College. https://timssandpirls.bc.edu/timss1995i/TIMSSPDF/T95_EffSchool.pdf

 Google Scholar

Baumert, J., Köller, O., Lehrke, M., & Brockmann, J. (2000). Anlage und Durchführung der Dritten Internationalen Mathematik- und Naturwissenschaftsstudie zur Sekundarstufe II (TIMSS/III)—Technische Grundlagen [Design and implementation of the Third Trends in International Mathematics and Science Study (TIMSS/III)—technical information]. In J. Baumert, W. Bos, & R. Lehmann (Eds.), TIMSS/III. Dritte Internationale Mathematik- und Naturwissenschaftsstudie. Mathematische und naturwissenschaftliche Bildung am Ende der Schullaufbahn: Vol. 1: Mathematische und naturwissenschaftliche Grundbildung am Ende der Pflichschulzeit (pp. 31–84). Leske + Budrich.

 Google Scholar

Brunner, M., Keller, U., Wenger, M., Fischbach, A., & Lüdtke, O. (2018). Between-school variation in students’ achievement, motivation, affect, and learning strategies: Results from 81 countries for planning group-randomized trials in education. Journal of Research on Educational Effectiveness, 11(3), 452–478. https://doi.org/10.1080/19345747.2017.1375584

 Web of Science ®Google Scholar

Knigge, M., & Köller, O. (2010). Effekte der sozialen Zusammensetzung der Schülerschaft [Impact of the social classroom composition of schools]. In O. Köller, M. Knigge, & B. Tesch (Eds.), Sprachliche Kompetenzen im Ländervergleich (pp. 227–244). Waxmann.

 Google Scholar

Senkbeil, M. (2006). Die Bedeutung schulischer Faktoren für die Kompetenzentwicklung in Mathematik und in den Naturwissenschaften [The relevance of school context factors for competence development in mathematics and science]. In PISA-Konsortium Deutschland (Ed.), PISA 2003. Untersuchungen zur Kompetenzentwicklung im Verlauf eines Schuljahres (pp. 277–308). Waxmann.

 Google Scholar

Data Availability Statement

A more detailed description of methods and results (Supplemental Online Material A), a comprehensive compilation of design parameters (Supplemental Online Material B), the R and Mplus scripts that underlie the statistical analyses of this article, and brief descriptions of where and how to access the data and material are available on the Open Science Framework at https://osf.io/2w8nt.

This article uses data from the National Educational Panel Study (NEPS): Starting Cohort 2–Kindergarten, https://doi.org/10.5157/NEPS:SC2:6.0.1, Starting Cohort 3–5th Grade, https://doi.org/10.5157/NEPS:SC3:7.0.1, and Starting Cohort 4–9th Grade, https://doi.org/10.5157/NEPS:SC4:9.1.1. NEPS is carried out by the Leibniz Institute for Educational Trajectories (LIfBi) in cooperation with a nationwide German network. Moreover, this article uses data from the Assessment of Student Achievements in German and English as a Foreign Language (DESI), https://doi.org/10.5159/IQB_DESI_v1, and from the Programme for International Student Assessment – International Plus 2003, 2004 (PISA-I-Plus 2003, 2004), https://doi.org/10.5159/IQB_PISA_I_Plus_v1.