A Comparison of Methodologies for Scaling Longitudinal Social-Emotional Survey Responses

References

• Baker, F. B. (1992). Item response theory: Parameter estimation techniques. New York, NY: Marcel Dekker.
   Google Scholar
• Bandura, A. (1997). The anatomy of stages of change. American Journal of Health Promotion, 12(1), 8–10. doi:10.4278/0890-1171-12.1.8
   PubMed Web of Science ®Google Scholar
• Bauer, D. J., & Curran, P. J. (2016). The discrepancy between measurement and modeling in longitudinal data analysis. In J. R. Harring, L. M. Stapleton, & S. N. Beretvas (Eds.), Advances in multilevel modeling for educational research (pp. 3–38). Charlotte, NC: Information Age Publishing.
   Google Scholar
• Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6(4), 431–444. doi:10.1177/014662168200600405
   Web of Science ®Google Scholar
• Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021). Introduction to meta-analysis. New York, NY: John Wiley & Sons.
   Google Scholar
• Briggs, D. C., & Domingue, B. (2013). The gains from vertical scaling. Journal of Educational and Behavioral Statistics, 38(6), 551–576. doi:10.3102/1076998613508317
   Web of Science ®Google Scholar
• Briggs, D. C., & Weeks, J. P. (2009). The impact of vertical scaling decisions on growth interpretations. Educational Measurement: Issues & Practice, 28(4), 3–14. doi:10.1111/j.1745-3992.2009.00158.x
   Google Scholar
• Cai, L. (2022). Frequently asked questions. Vector Psychometric Group. Retrieved from https://vpgcentral.com/software/flexmirt/support/frequently-asked-questions/#MDML
   Google Scholar
• Caprara, G. V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G. M., Barbaranelli, C., & Bandura, A. (2008). Longitudinal analysis of the role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. Journal of Educational Psychology, 100(3), 525. doi:10.1037/0022-0663.100.3.525
   Web of Science ®Google Scholar
• Collaborative for Academic, Social, and Emotional Learning. (2005). Safe and sound: An educational leader’s guide to evidence-based social and emotional learning programs—illinois edition. Chicago, IL.
   Google Scholar
• Dadey, N., & Briggs, D. C. (2012). A meta-analysis of growth trends from vertically scaled assessments. Practical Assessment, Research & Evaluation, 17(14), 14–21.
   Google Scholar
• Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251. doi:10.3102/0013189X15584327
   PubMed Web of Science ®Google Scholar
• Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta‐analysis of school‐based universal interventions. Child Development, 82(1), 405–432. doi:10.1111/j.1467-8624.2010.01564.x
   PubMed Web of Science ®Google Scholar
• Dweck, C., Walton, G. M., & Cohen, G. L. (2014). Academic tenacity: Mindsets and skills that promote long-term learning. Seattle, WA: Gates Foundation.
   Google Scholar
• Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance–a critical literature review. Chicago, IL: Consortium on Chicago School Research.
   Google Scholar
• Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological & Personality Science, 8(4), 370–378. doi:10.1177/1948550617693063
   Web of Science ®Google Scholar
• Gehlbach, H., & Hough, H. J. (2018). Measuring social emotional learning through student surveys in the CORE districts: A pragmatic approach to validity and reliability. Stanford, CA: Policy Analysis for California Education, PACE.
   Google Scholar
• Gorter, R., Fox, J. -P., Riet, G. T., Heymans, M., & Twisk, J. (2020). Latent growth modeling of IRT versus CTT measured longitudinal latent variables. Statistical Methods in Medical Research, 29(4), 962–986. doi:10.1177/0962280219856375
   PubMed Web of Science ®Google Scholar
• Gorter, J. W., Stewart, D., Cohen, E., Hlyva, O., Morrison, A., Galuppi, B. … Punthakee, Z. (2015). Are two youth-focused interventions sufficient to empower youth with chronic health conditions in their transition to adult healthcare: A mixed-methods longitudinal prospective cohort study. British Medical Journal Open, 5(5), e007553. doi:10.1136/bmjopen-2014-007553
   PubMed Web of Science ®Google Scholar
• Hanson, B. A., & Béguin, A. A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26(5), 3–24. doi:10.1177/0146621602026001001
   Google Scholar
• Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482. doi:10.1086/504455
   Web of Science ®Google Scholar
• Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. doi:10.1080/10705519909540118
   Web of Science ®Google Scholar
• Kim, S., & Kolen, M. J. (2006). Robustness to format effects of IRT linking methods for mixed-format tests. Applied Measurement in Education, 19(4), 357–381. doi:10.1207/s15324818ame1904_7
   Web of Science ®Google Scholar
• Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practice. New York, NY: Springer.
   Google Scholar
• Kolen, M. J., & Brennan, R. L. (2014). Score Scales. In Kolen, M. J. & Brennan, R. L. (Eds.), Test equating, scaling, and linking (pp. 371–485). New York, NY: Springer.
   Google Scholar
• Kolen, M. J., & Tong, Y. (2010). Psychometric properties of IRT proficiency estimates. Educational Measurement: Issues & Practice, 29(3), 8–14. doi:10.1111/j.1745-3992.2010.00179.x
   Google Scholar
• Kowaleski-Jones, L., & Duncan, G. J. (1999). The structure of achievement and behavior across middle childhood. Child Development, 70(4), 930–943. doi:10.1111/1467-8624.00067
   Web of Science ®Google Scholar
• Kuhfeld, M., & Soland, J. (2022). Avoiding bias from sum scores in growth estimates: An examination of IRT-based approaches to scoring longitudinal survey responses. Psychological Methods 27 2 234–260 . Advance online publication. doi:10.1037/met0000367
   PubMed Web of Science ®Google Scholar
• Lindley, D. V., & Smith, A. F. M. (1972). Bayes estimates for the linear model. Journal of the Royal Statistical Society: Series B (Methodological), 34(2), 1–18. doi:10.1111/j.2517-6161.1972.tb00885.x
   Google Scholar
• Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
   Google Scholar
• McNeish, D., & Wolf, M. G. (2020). Thinking twice about sum scores. Behavior Research Methods, 52(6), 2287–2305. doi:10.3758/s13428-020-01398-0
   PubMed Web of Science ®Google Scholar
• Meyer, R. H., Wang, C., & Rice, A. B. (2018). Measuring students’ social-emotional learning among California’s CORE districts: An IRT modeling approach. Stanford, CA: Policy Analysis for California Education, PACE.
   Google Scholar
• Petersen, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8(2), 137–156. doi:10.3102/10769986008002137
   Google Scholar
• Soland, J. (2019). Modeling academic achievement and self-efficacy as joint developmental processes: Evidence for education, counseling, and policy. Journal of Applied Developmental Psychology, 65, 101076. doi:10.1016/j.appdev.2019.101076
   Web of Science ®Google Scholar
• Soland, J., Hamilton, L., & Stecher, B. (2013). Measuring 21st-century competencies: Guidance for educators. Santa Monica, CA: The RAND Corporation.
   Google Scholar
• Soland, J., & Kuhfeld, M. (2021). Identifying students who are off‐track academically at the start of secondary school: The role of social‐emotional learning trajectories. The British Journal of Educational Psychology. Advance Online Publication. doi:10.1111/bjep.12463
   PubMed Web of Science ®Google Scholar
• Soland, J., & Kuhfeld, M. (2022). A Multi-Rater Latent Growth Curve Model. Multivariate Behavioral Research, 57(5), 701–717.
   PubMed Web of Science ®Google Scholar
• Soland, J., Kuhfeld, M., & Edwards, K. (2022). How survey scoring decisions can influence your study’s results: A trip through the IRT looking glass. Psychological Methods. https://psycnet.apa.org/record/2022-79825-001
   Google Scholar
• Soland, J., Kuhfeld, M., Wolk, E., & Bi, S. (2019). Examining the state-trait composition of social-emotional learning constructs: Implications for practice, policy, and evaluation. Journal of Research on Educational Effectiveness, 12(3), 550–577. doi:10.1080/19345747.2019.1615158
   Web of Science ®Google Scholar
• Soland, J., & Sandilos, L. E. (2021). English language learners, self-efficacy, and the achievement gap: Understanding the relationship between academic and social-emotional growth. Journal of Education for Students Placed at Risk (JESPAR), 26(1), 20–44. doi:10.1080/10824669.2020.1787171
   Web of Science ®Google Scholar
• Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180. doi:10.1207/s15327906mbr2502_4
   PubMed Web of Science ®Google Scholar
• Tong, Y., & Kolen, M. J. (2007). Comparisons of methodologies and results in vertical scaling for educational achievement tests. Applied Measurement in Education, 20(2), 227–253. doi:10.1080/08957340701301207
   Web of Science ®Google Scholar
• Weeks, J. P. (2010). plink: An R package for linking mixed-format tests using IRT-based methods. Journal of Statistical Software, 35(12). doi:10.18637/jss.v035.i12
   PubMed Web of Science ®Google Scholar
• Wegner, D. M., & Giuliano, T. (1982). The forms of social awareness. In William Ickes, & Eric Knowles (Eds.), Personality, roles, and social behavior (pp. 165–198). New York, NY: Springer.
   Google Scholar
• West, M. R., Buckley, K., Krachman, S. B., & Bookman, N. (2018). Development and implementation of student social-emotional surveys in the CORE districts. Journal of Applied Developmental Psychology, 55, 119–129. doi:10.1016/j.appdev.2017.06.001
   Web of Science ®Google Scholar
• Yen, W. M. (1986). The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement, 23(4), 299–325. doi:10.1111/j.1745-3984.1986.tb00252.x
   Web of Science ®Google Scholar
• Yen, W. M., & Fitzpatrick, A. R. (2006). Item response theory. In R. Brennan (Ed.), Educational measurement (4th ed., pp. 111–153). Westport, CT: Praeger.
   Google Scholar