Disciplinary climate, opportunity to learn, and mathematics achievement: an analysis using doubly latent multilevel structural equation modeling

References

• Aditomo, A., & Köhler, C. (2020). Do student ratings provide reliable and valid information about teaching quality at the school level? Evaluating measures of science teaching in PISA 2015. Educational Assessment, Evaluation and Accountability, 32(3), 275–310. https://doi.org/10.1007/s11092-020-09328-6
   Web of Science ®Google Scholar
• Arens, A. K., Morin, A. J. S., & Watermann, R. (2015). Relations between classroom disciplinary problems and student motivation: Achievement as a potential mediator? Learning and Instruction, 39, 184–193. https://doi.org/10.1016/j.learninstruc.2015.07.001
   Web of Science ®Google Scholar
• Asparouhov, T., & Muthén, B. (2010). Resampling methods in Mplus for complex survey data. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.310.1260&rep=rep1&type=pdf
   Google Scholar
• Barnard-Brak, L., Lan, W. Y., & Yang, Z. (2018). Differences in mathematics achievement according to opportunity to learn: A 4pL item response theory examination. Studies in Educational Evaluation, 56, 1–7. https://doi.org/10.1016/j.stueduc.2017.11.002
   Web of Science ®Google Scholar
• Berkowitz, R., Moore, H., Astor, R. A., & Benbenishty, R. (2017). A research synthesis of the associations between socioeconomic background, inequality, school climate, and academic achievement. Review of Educational Research, 87(2), 425–469. https://doi.org/10.3102/0034654316669821
   Web of Science ®Google Scholar
• Borgonovi, F., & Pokropek, A. (2019). Seeing is believing: Task-exposure specificity and the development of mathematics self-efficacy evaluations. Journal of Educational Psychology, 111(2), 268–283. https://doi.org/10.1037/edu0000280
   Web of Science ®Google Scholar
• Bronfenbrenner, U. (1979). Contexts of child rearing: Problems and prospects. American Psychologist, 34(10), 844–850. https://doi.org/10.1037/0003-066X.34.10.844
   Web of Science ®Google Scholar
• Carnoy, M., Khavenson, T., Loyalka, P., Schmidt, W. H., & Zakharov, A. (2016). Revisiting the relationship between international assessment outcomes and educational production: Evidence from a longitudinal PISA-TIMSS sample. American Educational Research Journal, 53(4), 1054–1085. https://doi.org/10.3102/0002831216653180
   Web of Science ®Google Scholar
• Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64(8), 723–733.
   Web of Science ®Google Scholar
• Cheema, J. R., & Kitsantas, A. (2014). Influences of disciplinary classroom climate on high school student self-efficacy and mathematics achievement: A look at gender and racial–ethnic differences. International Journal of Science and Mathematics Education, 12(5), 1261–1279. https://doi.org/10.1007/s10763-013-9454-4
   Web of Science ®Google Scholar
• Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural equation modeling: A multidisciplinary journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
   Web of Science ®Google Scholar
• Chen, F., & Cui, Y. (2020). Investigating the relation of perceived teacher unfairness to science achievement by hierarchical linear modeling in 52 countries and economies. Educational Psychology, 40(3), 273–295. https://doi.org/10.1080/01443410.2019.1652248
   Web of Science ®Google Scholar
• Chi, S., Liu, X., Wang, Z., & Won Han, S. (2018). Moderation of the effects of scientific inquiry activities on low SES students’ PISA 2015 science achievement by school teacher support and disciplinary climate in science classroom across gender. International Journal of Science Education, 40(11), 1284–1304. https://doi.org/10.1080/09500693.2018.1476742
   Web of Science ®Google Scholar
• Cogan, L. S., & Schmidt, W. H. (2015). The concept of opportunity to learn (OTL) in international comparisons of education. In K. Stacey & R. Turner (Eds.), Assessing mathematical literacy: The PISA experience (pp. 207–216). Springer. https://doi.org/10.1007/978-3-319-10121-7_10
   Google Scholar
• Cogan, L. S., Schmidt, W. H., & Guo, S. (2019). The role that mathematics plays in college-and career-readiness: Evidence from PISA. Journal of Curriculum Studies, 51(4), 530–553. https://doi.org/10.1080/00220272.2018.1533998
   Web of Science ®Google Scholar
• Dettmers, S., Trautwein, U., & Lüdtke, O. (2009). The relationship between homework time and achievement is not universal: Evidence from multilevel analyses in 40 countries. School Effectiveness and School Improvement, 20(4), 375–405. https://doi.org/10.1080/09243450902904601
   Web of Science ®Google Scholar
• Dumay, X., & Dupriez, V. (2007). Accounting for class effect using the TIMSS 2003 eighth-grade database: Net effect of group composition, net effect of class process, and joint effect. School Effectiveness and School Improvement, 18(4), 383–408. https://doi.org/10.1080/09243450601146371
   Web of Science ®Google Scholar
• Echazarra, A., Salinas, D., Méndez, I., Denis, V., & Rech, G. (2016). How teachers teach and students learn: Successful strategies for school (OECD Education Working Papers No. 130). OECD Publishing. https://doi.org/10.1787/5jm29kpt0xxx-en
   Google Scholar
• Enders, C. K. (2010). Applied missing data analysis. The Guilford Press.
   Google Scholar
• Floden, R. E. (2002). The measurement of opportunity to learn. In A. C. Porter & A. Gamoran (Eds.), Methodological advances in cross-national surveys of educational achievement (pp. 231–266). National Academy Press.
   Google Scholar
• Fox, R. (2001). Constructivism examined. Oxford Review of Education, 27(1), 23–35. https://doi.org/10.1080/03054980125310
   Web of Science ®Google Scholar
• Frenzel, A. C., Pekrun, R., & Goetz, T. (2007). Perceived learning environment and students’ emotional experiences: A multilevel analysis of mathematics classrooms. Learning and Instruction, 17(5), 478–493. https://doi.org/10.1016/j.learninstruc.2007.09.001
   Web of Science ®Google Scholar
• Gee, J. P. (2008). A sociocultural perspective on opportunity to learn. In P. A. Moss, D. C. Pullin, J. P. Gee, E. H. Haertel, & L. J. Young (Eds.), Assessment, equity, and opportunity to learn (pp. 76–108). Cambridge University Press.
   Google Scholar
• Guo, S., Li, L., & Zhang, D. (2018). A multilevel analysis of the effects of disciplinary climate strength on student reading performance. Asia Pacific Education Review, 19(1), 1–15. https://doi.org/10.1007/s12564-018-9516-y
   Web of Science ®Google Scholar
• Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.
   Google Scholar
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
   Web of Science ®Google Scholar
• Infantino, J., & Little, E. (2005). Students’ perceptions of classroom behaviour problems and the effectiveness of different disciplinary methods. Educational Psychology, 25(5), 491–508. https://doi.org/10.1080/01443410500046549
   Google Scholar
• Irvin, M., Byun, S., Smiley, W. S., & Hutchins, B. C. (2017). Relation of opportunity to learn advanced math to the educational attainment of rural youth. American Journal of Education, 123(3), 475–510. https://doi.org/10.1086/691231
   PubMed Web of Science ®Google Scholar
• Jia, Y., & Konold, T. (2021). Moving to the next level: Doubly latent multilevel mediation models with a school climate illustration. The Journal of Experimental Education, 89(2), 422–440. 1-19. https://doi.org/10.1080/00220973.2019.1675136
   Web of Science ®Google Scholar
• Khajavy, G. H., MacIntyre, P. D., & Barabadi, E. (2018). Role of the emotions and classroom environment in willingness to communicate: Applying doubly latent multilevel analysis in second language acquisition research. Studies in Second Language Acquisition, 40(3), 605–624. https://doi.org/10.1017/S0272263117000304
   Web of Science ®Google Scholar
• Lafontaine, D., Baye, A., Vieluf, S., & Monseur, C. (2015). Equity in opportunity-to-learn and achievement in reading: A secondary analysis of PISA 2009 data. Studies in Educational Evaluation, 47, 1–11. https://doi.org/10.1016/j.stueduc.2015.05.001
   Web of Science ®Google Scholar
• LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. https://doi.org/10.1177/1094428106296642
   Web of Science ®Google Scholar
• Lipowsky, F., Rakoczy, K., Pauli, C., Drollinger-Vetter, B., Klieme, E., & Reusser, K. (2009). Quality of geometry instruction and its short-term impact on students’ understanding of the Pythagorean Theorem. Learning and Instruction, 19(6), 527–537. https://doi.org/10.1016/j.learninstruc.2008.11.001
   Web of Science ®Google Scholar
• Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2 × 2 taxonomy of multilevel latent contextual models: Accuracy–bias trade-offs in full and partial error correction models. Psychological Methods, 16(4), 444–467. https://doi.org/10.1037/a0024376
   PubMed Web of Science ®Google Scholar
• Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13(3), 203–229. https://doi.org/10.1037/a0012869
   PubMed Web of Science ®Google Scholar
• Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology, 34(2), 120–131. https://doi.org/10.1016/j.cedpsych.2008.12.001
   Web of Science ®Google Scholar
• Mainhard, M. T., Brekelmans, M., & Wubbels, T. (2011). Coercive and supportive teacher behaviour: Within-and across-lesson associations with the classroom social climate. Learning and Instruction, 21(3), 345–354. https://doi.org/10.1016/j.learninstruc.2010.03.003
   Web of Science ®Google Scholar
• Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling: A Multidisciplinary Journal, 11(3), 320–341. https://doi.org/10.1207/s15328007sem1103_2
   Web of Science ®Google Scholar
• Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., & Köller, O. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47(2), 106–124. https://doi.org/10.1080/00461520.2012.670488
   Web of Science ®Google Scholar
• Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., & Nagengast, B. (2009). Doubly-latent models of school contextual effects: Integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behavioral Research, 44(6), 764–802. https://doi.org/10.1080/00273170903333665
   PubMed Web of Science ®Google Scholar
• Martínez, J. F. (2012). Consequences of omitting the classroom in multilevel models of schooling: An illustration using opportunity to learn and reading achievement. School Effectiveness and School Improvement, 23(3), 305–326. https://doi.org/10.1080/09243453.2012.678864
   Web of Science ®Google Scholar
• McDonnell, L. M. (1995). Opportunity to learn as a research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17(3), 305–322. https://doi.org/10.3102/01623737017003305
   Web of Science ®Google Scholar
• Morin, A. J. S., Marsh, H. W., Nagengast, B., & Scalas, L. F. (2014). Doubly latent multilevel analyses of classroom climate: An illustration. The Journal of Experimental Education, 82(2), 143–167. https://doi.org/10.1080/00220973.2013.769412
   Web of Science ®Google Scholar
• Mostafa, T., Echazarra, A., & Guillou, H. (2018). The science of teaching science: An exploration of science teaching practices in PISA 2015 (OECD Education Working Papers No. 188). OECD Publishing. https://doi.org/10.1787/f5bd9e57-en
   Google Scholar
• Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide.
   Google Scholar
• Nagengast, B., & Marsh, H. W. (2011). The negative effect of school-average ability on science self-concept in the UK, the UK countries and the world: The Big-Fish-Little-Pond-Effect for PISA 2006. Educational Psychology, 31(5), 629–656. https://doi.org/10.1080/01443410.2011.586416
   Web of Science ®Google Scholar
• Ning, B., Van Damme, J., Van Den Noortgate, W., Yang, X., & Gielen, S. (2015). The influence of classroom disciplinary climate of schools on reading achievement: A cross-country comparative study. School Effectiveness and School Improvement, 26(4), 586–611. https://doi.org/10.1080/09243453.2015.1025796
   Web of Science ®Google Scholar
• Organisation for Economic Co-operation and Development. (2009). PISA data analysis manual: SPSS (2nd ed.). https://doi.org/10.1787/9789264056275-en
   Google Scholar
• Organisation for Economic Co-operation and Development. (2013). PISA 2012 results: What makes schools successful (Volume IV): Resources, policies and practices. https://doi.org/10.1787/9789264201156-en
   Google Scholar
• Organisation for Economic Co-operation and Development. (2014). PISA 2012 technical report. https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf
   Google Scholar
• Preacher, K. J. (2015). Advances in mediation analysis: A survey and synthesis of new developments. Annual Review of Psychology, 66, 825–852. https://doi.org/10.1146/annurev-psych-010814-015258
   PubMed Web of Science ®Google Scholar
• Reynolds, K. J., Lee, E., Turner, I., Bromhead, D., & Subasic, E. (2017). How does school climate impact academic achievement? An examination of social identity processes. School Psychology International, 38(1), 78–97. https://doi.org/10.1177/0143034316682295
   Web of Science ®Google Scholar
• Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351–357. https://doi.org/10.2307/2087176
   Web of Science ®Google Scholar
• Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-scale assessment data: Issues in secondary analysis and reporting. Educational Researcher, 39(2), 142–151. https://doi.org/10.3102/0013189X10363170
   Web of Science ®Google Scholar
• Santibañez, L., & Fagioli, L. (2016). Nothing succeeds like success? Equity, student outcomes, and opportunity to learn in high-and middle-income countries. International Journal of Behavioral Development, 40(6), 517–525. https://doi.org/10.1177/0165025416642050
   Web of Science ®Google Scholar
• Scheerens, J. (Ed.). (2017). Opportunity to learn, curriculum alignment and test preparation: A research review. Springer. https://doi.org/10.1007/978-3-319-43110-9
   Google Scholar
• Scheerens, J., Witziers, B., & Steen, R. (2013). A meta-analysis of school effectiveness studies. Revista de Educacion, 361, 619–645. https://doi.org/10.4438/1988-592X-RE-2013-361-235
   Google Scholar
• Schlomer, G. L., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57(1), 1–10. https://doi.org/10.1037/a0018082
   PubMed Web of Science ®Google Scholar
• Schmidt, W. H., & Burroughs, N. A. (2013). Opening the black box: Prospects for using international large-scale assessments to explore classroom effects. Research in Comparative and International Education, 8(3), 236–247. https://doi.org/10.2304/rcie.2013.8.3.236
   Google Scholar
• Schmidt, W. H., Burroughs, N. A., Zoido, P., & Houang, R. T. (2015). The role of schooling in perpetuating educational inequality: An international perspective. Educational Researcher, 44(7), 371–386. https://doi.org/10.3102/0013189X15603982
   Web of Science ®Google Scholar
• Schmidt, W. H., McKnight, C. C., Houang, R. T., Wang, H., Wiley, D. E., Cogan, L. S., & Wolfe, R. G. (2001). Why schools matter: A cross-national comparison of curriculum and learning. Jossey-Bass.
   Google Scholar
• Schmidt, W. H., Zoido, P., & Cogan, L. (2014). Schooling matters: Opportunity to learn in PISA 2012 (OECD Education Working Papers No. 95). OECD Publishing. https://doi.org/10.1787/5k3v0hldmchl-en
   Google Scholar
• Sortkær, B., & Reimer, D. (2018). Classroom disciplinary climate of schools and gender – Evidence from the Nordic countries. School Effectiveness and School Improvement, 29(4), 511–528. https://doi.org/10.1080/09243453.2018.1460382
   Web of Science ®Google Scholar
• Stapleton, L. M. (2008). Variance estimation using replication methods in structural equation modeling with complex sample data. Structural Equation Modeling: A Multidisciplinary Journal, 15(2), 183–210. https://doi.org/10.1080/10705510801922316
   Web of Science ®Google Scholar
• Stoll, L., Bolam, R., McMahon, A., Wallace, M., & Thomas, S. (2006). Professional learning communities: A review of the literature. Journal of Educational Change, 7(4), 221–258. https://doi.org/10.1007/s10833-006-0001-8
   Google Scholar
• Teng, Y. (2020). The relationship between school climate and students’ mathematics achievement gaps in Shanghai China: Evidence from PISA 2012. Asia Pacific Journal of Education, 40(3), 356–372. https://doi.org/10.1080/02188791.2019.1682516
   Web of Science ®Google Scholar
• Urick, A., Wilson, A. S. P., Ford, T. G., Frick, W. C., & Wronowski, M. L. (2018). Testing a framework of math progress indicators for ESSA: How opportunity to learn and instructional leadership matter. Educational Administration Quarterly, 54(3), 396–438. https://doi.org/10.1177/0013161X18761343
   Web of Science ®Google Scholar
• van Hek, M., Kraaykamp, G., & Pelzer, B. (2018). Do schools affect girls’ and boys’ reading performance differently? A multilevel study on the gendered effects of school resources and school practices. School Effectiveness and School Improvement, 29(1), 1–21. https://doi.org/10.1080/09243453.2017.1382540
   Web of Science ®Google Scholar
• Wang, J. (1998). Opportunity to learn: The impacts and policy implications. Educational Evaluation and Policy Analysis, 20(3), 137–156. https://doi.org/10.3102/01623737020003137
   Web of Science ®Google Scholar
• Wang, M.-T. (2009). School climate support for behavioral and psychological adjustment: Testing the mediating effect of social competence. School Psychology Quarterly, 24(4), 240–251. https://doi.org/10.1037/a0017999
   Web of Science ®Google Scholar
• Wang, M.-T., & Degol, J. L. (2016). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review, 28(2), 315–352. https://doi.org/10.1007/s10648-015-9319-1
   Web of Science ®Google Scholar