Consequences of omitting the classroom in multilevel models of schooling: an illustration using opportunity to learn and reading achievement

Abstract

Statisticians have shown the theoretical extent of parameter distortion when the classroom level is ignored in multilevel analyses of schooling. This article illustrates the practical consequences of omitting the classroom for inferences drawn about the extent and mechanisms of schooling effects, using the relationship between reading achievement, opportunity to learn, and student composition. Findings indicate that omitting the classroom level inflates estimates of school-level variance, while at the same time underestimating the overall extent of variance related to schooling effects. Classrooms also moderate the effects of educational opportunities. Finally, compositional effects typically conceptualized at the school level may be best defined at the classroom level. Ignoring classroom nesting in the analysis thus not only underestimates the overall impact of schools but presents a distorted picture of the mechanisms through which the schooling environment influences student achievement. Special attention is paid to considering how modeling choices may inform education policy efforts.

Keywords:

• schooling effects
• multilevel modeling
• opportunity to learn

Notes

1. The scores used in this study correspond to the year 2000 national sample. A new norming sample was introduced in 2007 with the 10th edition of the test (SAT-10).

2. Additional analyses were carried out separately for third, fourth, and fifth grade. Patterns of partitioning within grade mirror those in the full sample and are not presented. However, the results provide assurance that classroom-level variance is not inflated by differences between grades.

3. Similar results were observed with writing OTL; results are presented only for reading OTL.

4. Expected scores for the mean OTL slope are estimated with basis on Equation (6) substituting the estimated slope parameter values reported in Table 4 and the sample mean values reported in Table 1. The figure shows two additional sets of values estimated using theoretical slopes one and two standard deviations above the mean reading OTL slope.

5. Table 7 presents parameter estimates only for the classroom- and school-level intercepts and slopes. Student-level fixed effects did not change appreciably from those presented in Table 4 and are omitted here.

6. By contrast, additional results not shown suggest that teacher reports of classroom OTL are not predictive of student performance over and above individual student reports (see Martínez & Goldschmidt, 2003).