Identifying current practices in modeling multi-year student achievement data: a systematic methodological review

References

• Barker, K.M., et al., 2020. Cross-classified multilevel models (CCMM) in health research: a systematic review of published empirical studies and recommendations for best practices. SSM - population health, 12 (December), 100661. doi:10.1016/j.ssmph.2020.100661.
   PubMedGoogle Scholar
• Bauer, D.J., et al., 2013. Analyzing repeated measures data on individuals nested within groups: accounting for dynamic group effects. Psychological methods, 18 (1), 1–14. doi:10.1037/a0030639.
   PubMed Web of Science ®Google Scholar
• *Belfi, B., et al., 2014. Inequality in language achievement growth? An investigation into the impact of pupil socio-ethnic background and school socio-ethnic composition. British educational research journal, 40 (5), 820–846. doi:10.1002/berj.3115.
   Web of Science ®Google Scholar
• *Belfi, B., Haelermans, C., and De Fraine, B., 2016. The long-term differential achievement effects of school socioeconomic composition in primary education: a propensity score matching approach. British journal of educational psychology, 86 (4), 501–525. doi:10.1111/bjep.12120.
   PubMed Web of Science ®Google Scholar
• Beretvas, S.N., 2008. Cross-classified random effects models. In: A. A. O’Connell, and D. B. McCoach, eds. Multilevel modeling of educational data. Quantitative methods in education and the behavioral sciences: issues, research, and teaching. Information Age Publishing.
   Google Scholar
• *Blanton, M., et al., 2019. Does early algebra matter? The effectiveness of an early algebra intervention in grades 3 to 5. American educational research journal, 56 (5), 1930–1972. doi:10.3102/0002831219832301.
   Web of Science ®Google Scholar
• Browne, W.J., Goldstein, H., and Rasbash, J., 2001. Multiple membership multiple classification (MMMC) models. Statistical modelling, 1, 103–124.
   Web of Science ®Google Scholar
• *Bulotsky-Shearer, R.J., and Fantuzzo, J.W., 2011. Preschool behavior problems in classroom learning situations and literacy outcomes in kindergarten and first grade. Early childhood research quarterly, 26 (1), 61–73. doi:10.1016/j.ecresq.2010.04.004.
   Web of Science ®Google Scholar
• Cafri, G., Hedeker, D., and Aarons, G.A., 2015. An introduction and integration of cross-classified, multiple membership, and dynamic group random-effects models. Psychological methods, 20 (4), 407.
   PubMed Web of Science ®Google Scholar
• Chung, H., et al., 2018. The impact of sample size in cross-classified multiple membership multilevel models. Journal of modern applied statistical methods, 17 (1), jmasm.eP2491. doi:10.22237/jmasm/1542209860.
   Web of Science ®Google Scholar
• Chung, H., and Beretvas, S.N., 2012. The impact of ignoring multiple membership data structures in multilevel models. British journal of mathematical and statistical psychology, 65 (2), 185–200. doi:10.1111/j.2044-8317.2011.02023.x.
   PubMed Web of Science ®Google Scholar
• *Csizmadia, A., and Ispa, J.M., 2014. Black-White biracial children's social development from kindergarten to fifth grade: links with racial identification, gender, and socioeconomic status. Social development, 23 (1), 157–177. doi:10.1111/sode.12037.
   Web of Science ®Google Scholar
• *Dağli, Ü.Y., and Jones, I., 2013. The longitudinal effects of kindergarten enrollment and relative age on children’s academic achievement. Teachers college record: The voice of scholarship in education, 115 (3), 1–40. doi:10.1177/016146811311500308.
   Web of Science ®Google Scholar
• *Desimone, L.M., Smith, T.M., and Phillips, K.J.R., 2013. Linking student achievement growth to professional development participation and changes in instruction: A longitudinal study of elementary students and teachers in title I schools. Teachers college record: The voice of scholarship in education, 115 (5), 1–46. doi:10.1177/016146811311500508.
   Web of Science ®Google Scholar
• *Droop, M., et al., 2016. Long-term effects of strategic Reading instruction in the intermediate elementary grades. Journal of research on educational effectiveness, 9 (1), 77–102. doi:10.1080/19345747.2015.1065528.
   Web of Science ®Google Scholar
• *Dumas, D., et al., 2019. Preschool mathematics intervention Can significantly improve student learning trajectories through elementary school. Aera open, 5 (4), 233285841987944–15. doi:10.1177/2332858419879446.
   Web of Science ®Google Scholar
• *Foorman, B.R., et al., 2010. Reading first in Florida: five years of improvement. Journal of literacy research, 42 (1), 71–93. doi:10.1080/10862960903583202.
   Web of Science ®Google Scholar
• *Friedel, J.M., et al., 2010. Changes in efficacy beliefs in mathematics across the transition to middle school: examining the effects of perceived teacher and parent goal emphases. Journal of educational psychology, 102 (1), 102–114. doi:10.1037/a0017590.
   Web of Science ®Google Scholar
• *Gadaire, D.M., Henrich, C.C., and Finn-Stevenson, M., 2017. Longitudinal effects of parent–child interactions on children’s social competence. Research on social work practice, 27 (7), 767–778. doi:10.1177/1049731516632592.
   Web of Science ®Google Scholar
• *Gallant, D.J., 2013. Using first-grade teacher ratings to predict third-grade English language arts and mathematics achievement on a high-stakes statewide assessment. International electronic journal of elementary education, 5 (2), 125–142.
   Google Scholar
• Goldstein, Harvey. 2011. Multilevel statistical models. 4th ed. Wiley.
   Google Scholar
• Goldstein, H., Burgess, S., and McConnell, B., 2007. Modelling the effect of pupil mobility on school differences in educational achievement. Journal of the royal statistical society series A: statistics in society, 170 (4), 941–954. doi:10.1111/j.1467-985X.2007.00491.x.
   Web of Science ®Google Scholar
• Grady, M.W., and Beretvas, S.N., 2010. Incorporating student mobility in achievement growth modeling: a cross-classified multiple membership growth curve model. Multivariate behavioral research, 45 (3), 393–419. doi:10.1080/00273171.2010.483390.
   PubMed Web of Science ®Google Scholar
• *Graves, S.L., and Ye, F.F., 2017. Are special education labels accurate for black children? racial differences in academic trajectories of youth diagnosed with specific learning and intellectual disabilities. Journal of black psychology, 43 (2), 192–213. doi:10.1177/0095798416636280.
   Web of Science ®Google Scholar
• *Grillich, L., et al., 2016. Effectiveness evaluation of a health promotion programme in primary schools: A cluster randomised controlled trial. Bmc public health, 16 (1), 679. doi:10.1186/s12889-016-3330-4.
   PubMedGoogle Scholar
• Grimm, Kevin J., Nilam Ram, and Ryne Estabrook. 2017. Growth modeling: structural equation and multilevel modeling approaches. Guilford Press.
   Google Scholar
• *Hallinger, P., and Heck, R.H., 2010. Leadership for learning: does collaborative leadership make a difference in school improvement? Educational management administration & leadership, 38 (6), 654–678. doi:10.1177/1741143210379060.
   Web of Science ®Google Scholar
• *Heck, R.H., and Hallinger, P., 2014. Modeling the longitudinal effects of school leadership on teaching and learning. Journal of educational administration, 52 (5), 653–681.
   Google Scholar
• *Heck, R.H., and Reid, T., 2020. School leadership and school organization: Investigating their effects on school improvement in Reading and math. Zeitschrift für Erziehungswissenschaft, 23 (5), 925–954.
   Web of Science ®Google Scholar
• *Hong, S., and You, S., 2012. Understanding Latino children’s heterogeneous academic growth trajectories: latent growth mixture modeling approach. The journal of educational research, 105 (4), 235–244. doi:10.1080/00220671.2011.584921.
   Web of Science ®Google Scholar
• *Huang, F.L., and Invernizzi, M.A., 2012. The association of kindergarten entry age with early literacy outcomes. The journal of educational research, 105 (6), 431–441. doi:10.1080/00220671.2012.658456.
   Web of Science ®Google Scholar
• *Huang, F.L., Moon, T.R., and Boren, R., 2014. Are the Reading rich getting richer? Testing for the presence of the Matthew effect. Reading & writing quarterly, 30 (2), 95–115. doi:10.1080/10573569.2013.789784.
   Web of Science ®Google Scholar
• *Hughes, J.N., Kwok, O.-M., and Im, M.H., 2013. Effect of retention in first grade on parents’ educational expectations and children’s academic outcomes. American educational research journal, 50 (6), 1336–1359. doi:10.3102/0002831213490784.
   Web of Science ®Google Scholar
• *Jenson, J.M., et al., 2010. Preventing childhood bullying: findings and lessons from the denver public schools trial. Research on social work practice, 20 (5), 509–517. doi:10.1177/1049731509359186.
   Web of Science ®Google Scholar
• *Johnson, U.Y., and Hull, D.M., 2014. Parent involvement and science achievement: a cross-classified multilevel latent growth curve analysis. The journal of educational research, 107 (5), 399–409. doi:10.1080/00220671.2013.807488.
   Web of Science ®Google Scholar
• *Judge, S., 2013. Longitudinal predictors of Reading achievement among at-risk children. Journal of children and poverty, 19 (1), 1–19. doi:10.1080/10796126.2013.765629.
   Google Scholar
• *Judge, S., and Watson, S.M., 2011. Longitudinal outcomes for mathematics achievement for students with learning disabilities. The journal of educational research, 104 (3), 147–157. doi:10.1080/00220671003636729.
   Web of Science ®Google Scholar
• *Keles, S., et al., 2018. The longitudinal association between internalizing symptoms and academic achievement among immigrant and non-immigrant children in Norway. Scandinavian journal of psychology, 59 (4), 392–406. doi:10.1111/sjop.12454.
   PubMed Web of Science ®Google Scholar
• Lai, M.H.C., and Kwok, O.-m., 2017. A model for time-varying cumulative classroom/teacher effects on students’ growth. Society for research in educational effectiveness conference, 8.
   Google Scholar
• *LeBoeuf, W.A., and Fantuzzo, J.W., 2018. Effects of intradistrict school mobility and high student turnover rates on early Reading achievement. Applied developmental science, 22 (1), 43–57. doi:10.1080/10888691.2016.1211481.
   Web of Science ®Google Scholar
• Leroux, A.J., and Beretvas, S.N., 2018. Estimation of a latent variable regression growth curve model for individuals cross-classified by clusters. Multivariate behavioral research, 53 (2), 231–246. doi:10.1080/00273171.2017.1418654.
   PubMed Web of Science ®Google Scholar
• *Li, J., and O’Connell, A.A., 2012. Obesity, high-calorie food intake, and academic achievement trends Among U.S. school children. The journal of educational research, 105 (6), 391–403. doi:10.1080/00220671.2011.646359.
   Web of Science ®Google Scholar
• Liberati, Alessandro, Douglas G. Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C. Gøtzsche, John P. A. Ioannidis, Mike Clarke, P. J. Devereaux, Jos Kleijnen, and David Moher. 2009. “The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.” Plos medicine 6 (7): doi:10.1371/journal.pmed.1000100.
   Web of Science ®Google Scholar
• *Liu, Y., Liu, H., and Hau, K.-T., 2016. Reading ability development from kindergarten to junior secondary: Latent transition analyses with growth mixture modeling. Frontiers in psychology, 7, 1659. doi:10.3389/fpsyg.2016.01659.
   PubMed Web of Science ®Google Scholar
• *Lloyd, J.E., Li, L., and Hertzman, C., 2010. Early experiences matter: lasting effect of concentrated disadvantage on children's language and cognitive outcomes. Health & Place, 16 (2), 371–380. doi:10.1016/j.healthplace.2009.11.009.
   PubMed Web of Science ®Google Scholar
• *Lochman, J.E., et al., 2012. Coping Power dissemination study: intervention and special education effects on academic outcomes. Behavioral disorders, 37 (3), 192–205. doi:10.1177/019874291203700306.
   Web of Science ®Google Scholar
• *London, R.A., and Castrechini, S., 2011. A longitudinal examination of the link between youth physical fitness and academic achievement. Journal of school health, 81 (7), 400–408. doi:10.1111/j.1746-1561.2011.00608.x.
   PubMed Web of Science ®Google Scholar
• *Lopes, V.P., et al., 2011. Motor coordination as predictor of physical activity in childhood. Scandinavian journal of medicine & science in sports, 21 (5), 663–669. doi:10.1111/j.1600-0838.2009.01027.x.
   PubMed Web of Science ®Google Scholar
• Luo, W., and Kwok, O.-m., 2012. The consequences of ignoring individuals' mobility in multilevel growth models. Journal of educational and behavioral statistics, 37 (1), 31–56. doi:10.3102/1076998610394366.
   Web of Science ®Google Scholar
• Mainhard, Tim, Sophie Oudman, Lisette Hornstra, Roel J. Bosker, and Thomas Goetz. 2018. “Student emotions in class: The relative importance of teachers and their interpersonal relations with students.” Learning and instruction 53: 109–119. doi:10.1016/j.learninstruc.2017.07.011.
   Web of Science ®Google Scholar
• McCoach, D., and Betsy, K., 2010. Using time-varying covariates in multilevel growth models. Frontiers in psychology, doi:10.3389/fpsyg.2010.00017.
   PubMed Web of Science ®Google Scholar
• *McCormick, M.P., et al., 2016. Do effects of social-emotional learning programs vary by level of parent participation? evidence from the randomized trial of INSIGHTS. Journal of research on educational effectiveness, 9 (3), 364–394. doi:10.1080/19345747.2015.1105892.
   Web of Science ®Google Scholar
• *McDermott, P.A., Rikoon, S.H., and Fantuzzo, J.W., 2014. Tracing children’s approaches to learning through Head Start, kindergarten, and first grade: Different pathways to different outcomes. Journal of educational psychology, 106 (1), 200–213. doi:10.1037/a0033547.
   Web of Science ®Google Scholar
• *McDermott, P.A., Rikoon, S.H., and Fantuzzo, J.W., 2016. Transition and protective agency of early childhood learning behaviors as portents of later school attendance and adjustment. Journal of school psychology, 54, 59–75. doi:10.1016/j.jsp.2015.10.003.
   PubMed Web of Science ®Google Scholar
• McNeish, D.M., and Stapleton, L.M., 2016. The effect of small sample size on two-level model estimates: a review and illustration. Educational psychology review, 28 (2), 295–314. doi:10.1007/s10648-014-9287-x.
   Web of Science ®Google Scholar
• Mehta, Paras. 2013. “N-Level structural equation modeling.” In Applied quantitative analysis in education and the social sciences. Eds. Petecher, Y., Schatschneider, C., & Compton, D. L. Routledge.
   Google Scholar
• Moerbeek, M., 2004. The consequence of ignoring a level of nesting in multilevel analysis. Multivariate behavioral research, 39 (1), 129–149. doi:10.1207/s15327906mbr3901_5.
   PubMed Web of Science ®Google Scholar
• *Moller, S., et al., 2013. Collective pedagogical teacher culture and mathematics achievement: differences by race, ethnicity, and socioeconomic status. Sociology of education, 86 (2), 174–194. doi:10.1177/0038040712472911.
   Web of Science ®Google Scholar
• *Morgan, P.L., et al., 2016. Science achievement gaps begin very early, persist, and are largely explained by modifiable factors. Educational researcher, 45 (1), 18–35. doi:10.3102/0013189X16633182.
   Web of Science ®Google Scholar
• *Morgan, P.L., Farkas, G., and Wu, Q., 2011. Kindergarten children’s growth trajectories in reading and mathematics: who falls increasingly behind? Journal of learning disabilities, 44 (5), 472–488. doi:10.1177/0022219411414010.
   PubMed Web of Science ®Google Scholar
• *Ogg, J., and Anthony, C.J., 2020. Process and context: Longitudinal effects of the interactions between parental involvement, parental warmth, and SES on academic achievement. Journal of school psychology, 78, 96–114. doi:10.1016/j.jsp.2019.11.004.
   PubMed Web of Science ®Google Scholar
• *Ortega, L., Malmberg, L.-E., and Sammons, P., 2018. Teacher effects on Chilean children’s achievement growth: a cross-classified multiple membership accelerated growth curve model. Educational evaluation and policy analysis, 40 (3), 473–501.
   Web of Science ®Google Scholar
• Palardy, G.J., 2010. The multilevel crossed random effects growth model for estimating teacher and school effects: issues and extensions. Educational and psychological measurement, 70 (3), 401–419. doi:10.1177/0013164409355693.
   Web of Science ®Google Scholar
• *Park, S., and Holloway, S.D., 2017. The effects of school-based parental involvement on academic achievement at the child and elementary school level: a longitudinal study. The journal of educational research, 110 (1), 1–16. doi:10.1080/00220671.2015.1016600.
   Web of Science ®Google Scholar
• *Pässler, K., and Hell, B., 2020. Stability and change in vocational interests from late childhood to early adolescence. Journal of vocational behavior, 121 (September), 103462. doi:10.1016/j.jvb.2020.103462.
   Google Scholar
• Petscher, Y., and Schatschneider, C., 2019. Using n -level structural equation models for causal modeling in fully nested, partially nested, and cross-classified randomized controlled trials. Educational and psychological measurement, 79 (6), 1075–1102. doi:10.1177/0013164419840071.
   PubMed Web of Science ®Google Scholar
• PRISMA-P Group, et al., 2015. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic reviews, 4 (1), 1.
   PubMedGoogle Scholar
• *Rambo-Hernandez, K.E., and McCoach, D.B., 2015. High-Achieving and average students’ Reading growth: contrasting school and summer trajectories. The journal of educational research, 108 (2), 112–129. doi:10.1080/00220671.2013.850398.
   Web of Science ®Google Scholar
• *Rambo-Hernandez, K.E., Peters, S.J., and Plucker, J.A., 2019. Quantifying and exploring elementary school excellence gaps across schools and time. Journal of advanced academics, 30 (4), 383–415. doi:10.1177/1932202X19864116.
   Web of Science ®Google Scholar
• Raudenbush, S.W., and Bryk, A.S., 2002. Hierarchical linear models: applications and data analysis methods. 2nd ed. Sage.
   Google Scholar
• *Ready, D.D., 2013. Associations between student achievement and student learning: implications for value-added school accountability models. Educational policy, 27 (1), 92–120. doi:10.1177/0895904811429289.
   Web of Science ®Google Scholar
• *Rugutt, J.K., and Nur-Awaleh, M.A., 2011. Mathematics learning patterns across two groups of African American students: a multilevel approach. International journal of learning, 21.
   Google Scholar
• *Salvas, M.-C., et al., 2011. Interplay between friends' aggression and friendship quality in the development of child aggression during the early school years. Social development, 20 (4), 645–663. doi:10.1111/j.1467-9507.2010.00592.x.
   Web of Science ®Google Scholar
• *Schulte, A.C., and Stevens, J.J., 2015. Once, sometimes, or always in special education: mathematics growth and achievement gaps. Exceptional children, 81 (3), 370–387. doi:10.1177/0014402914563695.
   Web of Science ®Google Scholar
• *Smyth, E., 2020. Shaping educational expectations: the perspectives of 13-year-olds and their parents. Educational review, 72 (2), 173–195. doi:10.1080/00131911.2018.1492518.
   Web of Science ®Google Scholar
• *Swanson, H.L., 2015. Growth in working memory and inhibition predicts literacy in English language learners: a cross-sectional and longitudinal study. Memory (Hove, England), 23 (5), 748–773. doi:10.1080/09658211.2014.927504.
   PubMed Web of Science ®Google Scholar
• *Swanson, H.L., Orosco, M.J., and Kudo, M.F., 2017. Do specific classroom Reading activities predict English language learners’ later Reading achievement? Reading & writing quarterly, 33 (3), 199–210. doi:10.1080/10573569.2016.1145563.
   Web of Science ®Google Scholar
• *Taylor, I.M., Spray, C.M., and Pearson, N., 2014. The influence of the physical education environment on children’s well-being and physical activity across the transition from primary to secondary school. Journal of sport and exercise psychology, 36 (6), 574–583. doi:10.1123/jsep.2014-0038.
   PubMed Web of Science ®Google Scholar
• *Tong, F., et al., 2017. Investigating the impact of professional development on teachers’ instructional time and English learners’ language development: a multilevel cross-classified approach. International journal of bilingual education and bilingualism, 20 (3), 292–313. doi:10.1080/13670050.2015.1051509.
   Web of Science ®Google Scholar
• Tranmer, M., Steel, D., and Browne, W.J., 2014. Multiple-Membership multiple-classification models for social network and group dependences. Journal of the royal statistical society series A: statistics in society, 177 (2), 439–455. doi:10.1111/rssa.12021.
   PubMed Web of Science ®Google Scholar
• Van den Noortgate, W., Opdenakker, M.-C., and Onghena, P., 2005. The effects of ignoring a level in multilevel analysis. School effectiveness and school improvement, 16 (3), 281–303. doi:10.1080/09243450500114850.
   Web of Science ®Google Scholar
• *Wei, X., Blackorby, J., and Schiller, E., 2011. Growth in Reading achievement of students with disabilities, ages 7 to 17. Exceptional children, 78 (1), 89–106. doi:10.1177/001440291107800106.
   Web of Science ®Google Scholar
• *Williamson, G.L., Fitzgerald, J., and Stenner, A.J., 2014. Student Reading growth illuminates the common core text-complexity standard. The elementary school journal, 115 (2), 230–254. doi:10.1086/678295.
   Web of Science ®Google Scholar
• *Williford, A., et al., 2016. The effect of bullying and victimization on cognitive empathy development during the transition to middle school. Child & youth care forum, 45, 525–541. doi:10.1007/s10566-015-9343-9.
   Web of Science ®Google Scholar
• *Zarra-Nezhad, M., et al., 2019. Supportive parenting buffers the effects of low peer acceptance on children’s internalizing problem behaviors. Child & youth care forum, 48, 865–887. doi:10.1007/s10566-019-09510-y.
   Web of Science ®Google Scholar