References
- Aleven, V., Beal, C. R., & Graesser, A. C. (2013). Introduction to the special issue on advanced learning technologies. Journal of Educational Psychology, 105, 929–931. https://doi.org/10.1037/a0034155
- American Institutes for Research. (2013). Are personalized learning environments the next wave of K-12 educatino reform? American Institutes for Research. www.air.org:
- Antonaci, A., Klemke, R., & Specht, M. (2019). The effects of gamification in online learning environments: A systematic literature review. Informatics, 6, 32. https://doi.org/10.3390/informatics6030032
- Asparouhov, T. (2005). Sampling weights in latent variable modeling. Structural Equation Modeling, 13, 411–434. https://doi.org/10.1207/s15328007sem1203_4
- Asparouhov, T., & Muthén, B. (2014). Auxiliary variables in mixture modeling: Three-step approaches using Mplus. Structural Equation Modeling, 21, 329–341. https://doi.org/10.1080/10705511.2014.915181
- Bakk, Z., Tekle, F. B., & Vermunt, J. K. (2013). Estimating the association between latent class membership and external variables using bias-adjusted three-step approaches. Sociological Methodology, 43, 272–311. https://doi.org/10.1177/0081175012470644
- Beal, C., Arroyo, I., Cohen, P., & Woolf, B. (2010). Evaluation of animal watch: An intelligent tutoring system for arithmetic and fractions. Journal of Interactive Online Learning, 9, 64–77. http://www.ncolr.org/jiol/issues/pdf/9.1.4.pdf
- Becker, C., & Gather, U. (1999). The masking breakdown point of multivariate outlier identification rules. Journal of the American Statistical Association, 94, 947–955.
- Belsley, D. A., Kuh, E., & Welsh, R. E. (1980). Regression diagnostics: Identifying influential data and sources of collinearity. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons.
- Bergner, Y., Kerr, D., & Pritchard, D. E. (2015). Methodological challenges in the analysis of MOOC data for exploring the relationship between discussion forum views and learning outcomes. International educational data mining society.
- Bergner, Y., & Von Davier, A. A. (2019). Process data in NAEP: Past, present, and future. Journal of Educational and Behavioral Statistics, 44, 706–732. https://doi.org/10.3102/1076998618784700
- Biernacki, C., Celeux, G., & Govaert, G. (2000). Assessing a mixture model for clustering with the integrated completed likelihood IEEE transactions on pattern analysis and machine intelligence. 22, 719–725.
- Bishop, C. D., Leite, W. L., & Snyder, P. (2018). Using propensity score weighting to reduce selection bias in large-scale data sets. Journal of Early Intervention, 40, 347–362. https://doi.org/10.1177/1053815118793430
- Bol, L., Campbell, K. D., Perez, T., & Yen, C. J. (2016). The effects of self-regulated learning training on community college students’ metacognition and achievement in developmental math courses. Community College Journal of Research and Practice, 40, 480–495. https://doi.org/10.1080/10668926.2015.1068718
- Bolck, A., Croon, M., & Hagenaars, J. (2004). Estimating latent structure models with categorical variables: One-step versus three-step estimators. Political Analysis, 12, 3–27. https://doi.org/10.1093/pan/mph001
- Bollen, K. A. (2012). Instrumental variables in sociology and the social sciences. Annual Review of Sociology, 38, 37–72. https://doi.org/10.1146/annurev-soc-081309-150141
- Bray, B. C., Dziak, J. J., Patrick, M. E., & Lanza, S. T. (2018). Inverse propensity score weighting with a latent class exposure: Estimating the causal effect of reported reasons for alcohol use on problem alcohol use 16 years later. Prev Sci. 20, 394–406. https://doi.org/10.1007/s11121-018-0883-8
- Bühlmann, P., & Hothorn, T. (2007). Boosting algorithms: Regularization, prediction and model fitting. Statistical Science, 22, 477–505. https://doi.org/10.1214/07-sts242
- Cochran, W. G. (1957). Analysis of covariance: Its nature and uses. Biometrics special issue on the Analysis of Covarince, 13, 261–281. https://www.jstor.org/stable/2527916
- Collier, Z. K., & Leite, W. L. (2017). A comparison of three-step approaches for auxiliary variables in latent class and latent profile analysis. Structural Equation Modeling, 24, 819–830. https://doi.org/10.1080/10705511.2017.1365304
- Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Wiley.
- Cook, R. D. (1977). Detection of influential observation in linear regression. Technometrics, 19, 15–18. https://doi.org/10.1080/00401706.1977.10489493
- Dean, N., & Raftery, A. E. (2010). Latent class analysis variable selection. Ann Inst Stat Math, 62, 11–35. 10.1007/s10463-009-0258-9
- Dhawan, S. (2020). Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology Systems, 49, 5–22. https://doi.org/10.1177/0047239520934018
- DiStefano, C., & Kamphaus, R. W. (2006). Investigating subtypes of child development: A comparison of cluster analysis and latent class cluster analysis in typology creation. Educational and Psychological Measurement, 66, 778–794. https://doi.org/10.1177/0013164405284033
- Florida Department of Education. (2016). Statewide assessment program information guide 2016-2017. Florida Department of Education.
- Fop, M., Smart, K. M., & Murphy, T. B. (2017). Variable selection for latent class analysis with application to low back pain diagnosis. The Annals of Applied Statistics, 11, 2080–2110. https://doi.org/10.1214/17-aoas1061
- Fraley, C., & Raftery, A. E. (2002). Model-based clustering, discriminant analysis, and density estimation. Journal of the American Statistical Association, 97, 611–631. https://doi.org/10.1198/016214502760047131
- Funk, M. J., Westreich, D., Wiesen, C., Sturmer, T., Brookhart, M. A., & Davidian, M. (2011). Doubly robust estimation of causal effects. American Journal of Epidemiology, 173, 761–767. https://doi.org/10.1093/aje/kwq439
- Guo, S., & Fraser, M. W. (2014). Propensity score analysis: Statistical methods and applications (2nd ed.). Sage Publications.
- Henry, K., & Muthén, B. O. (2010). Multilevel latent class analysis: An application of adolescent smoking typologies with individual and contextual predictors. Structural Equation Modeling, 17, 193–215. 10.1080/10705511003659342
- Hirano, K., & Imbens, G. W. (2004). The propensity score with continuous treatments. In A. Gelman & X.-L. Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete‐data perspectives: An essential journey with Donald Rubin’s statistical family (pp. 73–84). John Wiley & Sons.
- Hong, G. (2015). Causality in a social world: Moderation, mediation and spill-over. John Wiley & Sons.
- Hsiao, I. H., Huang, P. K., & Murphy, H. (2017, March). Uncovering reviewing and reflecting behaviors from paper-based formal assessment. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 319–328).
- Huggins-Manley, A. C., Beal, C. R., D’Mello, S. K., Leite, W. L., Cetin-Berber, D. D., Kim, D., & McNamara, D. S. (2019). A commentary on construct validity when using operational virtual learning environment data in effectiveness studies. Journal of Research on Educational Effectiveness, 12, 750–759. https://doi.org/10.1080/19345747.2019.1639869
- Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87, 706–710. https://doi.org/10.1093/biomet/87.3.706
- Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615–635. https://doi.org/10.1016/j.jeconom.2007.05.001
- Jing, Z., Kuang, H., Leite, W. L., & Huggins-Manley, A. C. (2019, November). Applying hybrid ant colony optimization algorithm in mixture item response theory model [Paper presentation]. Florida Educational Research Association (FERA) 64th annual meeting, St. Petersburg, FL.
- Jo, B., Muthén, B. O., Reise, S. P., & Duan, N. (2003). Longitudinal studies with intervention and noncompliance: Estimation of causal effects in growth mixture modeling. In S. P. Reise & N. Duan (Eds.), Multilevel modeling: Methodological advances, issues, and applications (pp. 112–139). Lawrence Erlbaum Associates Publishers.
- Kang, J., & Schafer, J. L. (2010). Estimating average treatment effects when the treatment is a latent class (Technical Report 10-05). PennState. http://stat.psu.edu/research/technical-report-files/2010-technical-reports/TR10-05.pdf:
- Lanza, S. T., Coffman, D. L., & Xu, S. (2013). causal inference in latent class analysis. Structural Equation Modeling, 20, 361–383. https://doi.org/10.1080/10705511.2013.797816
- Lanza, S. T., & Rhoades, B. L. (2013). Latent class analysis: An alternative perspective on subgroup analysis in prevention and treatment. Prevention Science, 14, 157–168. https://doi.org/10.1007/s11121-011-0201-1
- Lanza, S. T., Tan, X., & Bray, B. C. (2013). Latent class analysis with distal outcomes: A flexible model-based approach. Structural Equation Modeling, 20, 1–26. https://doi.org/10.1080/10705511.2013.742377
- Lastinger Center for Learning (2021, January 13). Algebra Nation. University of Florida. https://lastinger.center.ufl.edu/mathematics/algebra-nation/
- Leite, W. L. (2017). Practical propensity score methods using R. Sage Publishing.
- Leite, W. L., Aydin, B., & Gurel, S. (2019a). A comparison of propensity score weighting ethods for evaluating the effects of programs with multiple versions. The Journal of Experimental Education, 87, 75–88. https://doi.org/10.1080/00220973.2017.1409179
- Leite, W. L., Cetin-Berber, D. D., Huggins-Manley, A. C., Collier, Z. K., & Beal, C. R. (2019b). The relationship between Algebra Nation usage and high‐stakes test performance for struggling students. Journal of Computer Assisted Learning, 35, 569–581. https://doi.org/10.1111/jcal.12360
- Leite, W. L., Jimenez, F., Kaya, Y., Stapleton, L. M., MacInnes, J. W., & Sandbach, R. (2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50, 265–284. https://doi.org/10.1080/00273171.2014.991018
- Leite, W. L., Stapleton, L. M., & Bettini, E. F. (2018). Propensity score analysis of complex survey data with structural equation modeling: A tutorial with Mplus. Structural Equation Modeling, 26, 448–469. https://doi.org/10.1080/10705511.2018.1522591
- Lunceford, J. K., & Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: A comparative study. Statistics in Medicine, 23, 2937–2960. https://doi.org/10.1002/sim.1903
- Mao, H., Li, L., & Greene, T. (2019). Propensity score weighting analysis and treatment effect discovery. Statistical Methods in Medical Research, 28, 2439–2454. https://doi.org/10.1177/0962280218781171
- Marbac, M., & Sedki, M. (2017). Variable selection for model-based clustering using the integrated complete-data likelihood. Statistics and Computing, 27, 1049–1063. https://doi.org/10.1007/s11222-016-9670-1
- Marbac, M., & Sedki, M. (2020). VarSelLCM: Variable selection for model-based clustering of mixed-type data set with missing values. https://cran.r-project.org/package=VarSelLCM
- Masyn, K. E. (2013). Latent class analysis and finite mixture modeling. In T. Little (Ed.), Oxford handbook of quantitative methods (pp. 551–611). Oxford University Press.
- Mayer, A., Dietzfelbinger, L., Rosseel, Y., & Steyer, R. (2016). The effectliter approach for analyzing average and conditional effects. Multivariate Behavioral Research, 51, 374–391. https://doi.org/10.1080/00273171.2016.1151334
- McLachlan, G. J. (2011). Commentary on Steinley and Brusco (2011): Recommendations and cautions. Psychological Methods, 16, 80–81; discussion 89–92. 10.1037/a0021141
- McLachlan, G. J., & Peel, D. (2000). finite mixture models. John Wiley and Sons.
- Muthén & Muthén. (2017). Mplus (Version 8.1). Muthén & Muthén.
- Muthén, B., & Asparouhov, T. (2009). Multilevel regression mixture analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172, 639–657. https://doi.org/10.1111/j.1467-985X.2009.00589.x
- Pardo, A., Bartimote-Aufflick, K., Shum, S. B., Dawson, S., Gao, J., Gaševic, D., & Moskal, A. C. M. (2018). OnTask: Delivering data-informed, personalized learning support actions. Journal of Learning Analytics, 5, 235–249. https://doi.org/10.18608/jla.2018.53.15
- Park, Y., Yu, J. H., & Jo, I. H. (2016). Clustering blended learning courses by online behavior data: A case study in a Korean higher education institute. The Internet and Higher Education, 29, 1–11. https://doi.org/10.1016/j.iheduc.2015.11.001
- Prieto, L. P., Dlab, M. H., Gutiérrez, I., Abdulwahed, M., & Balid, W. (2011). Orchestrating technology enhanced learning: A literature review and a conceptual framework. International Journal of Technology Enhanced Learning, 3, 583. https://doi.org/10.1504/IJTEL.2011.045449
- R Development Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.R-project.org
- Ridgeway, G., McCaffrey, D., Morral, A., Burgette, L., & Griffin, B. A. (2013). Toolkit for weighting and analysis of nonequivalent groups: A tutorial for the twang package. Retrieved November 15, 2013, from http://cran.r-project.org/web/packages/twang/vignettes/twang.pdf
- Robins, J. M., Hernan, M. A., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology, 11, 550–560. https://doi.org/10.1097/00001648-200009000-00011
- Romero, C., Romero, J. R., & Ventura, S. (2014). A survey on pre-processing educational data. In A. Peña-Ayala (Ed.), Educational data mining (pp. 29–64). Springer.
- Roschelle, J., Feng, M., Murphy, R. F., & Mason, C. A. (2016). Online mathematics homework increases student achievement. AERA Open, 2, 1–12. https://doi.org/10.1177/2332858416673968
- Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. https://doi.org/10.1093/biomet/70.1.41
- Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516–524. https://doi.org/10.1080/01621459.1984.10478078
- Rubin, D. B. (1973). The use of matching and regression adjustment to remove bias in observational studies. Biometrics, 29, 185–203. https://doi.org/10.2307/2529685
- Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100, 322–331. 10.1198/016214504000001880
- Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine, 26, 20–36. https://doi.org/10.1002/sim.2739
- Saleem, A. N., Noori, N. M., & Ozdamli, F. (in press). Gamification applications in E-learning: A literature review. Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-020-09487-x
- Schuler, M. S., Leoutsakos, J. S., & Stuart, E. A. (2014). Addressing confounding when estimating the effects of latent classes on a distal outcome. Health Serv Outcomes Res Methodol, 14, 232–254. 10.1007/s10742-014-0122-0
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Company.
- Sheard, J. (2011). Basics of statistical analysis of interactions data from web-based learning environments. In C. Romero, S. Ventura, M. Pechenizkiy, & R. S. J. D. Baker (Eds.), Handbook of educational data mining (pp. 27–42). CRC Press.
- Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57, 1380–1400. https://doi.org/10.1177/0002764213498851
- Stapleton, L. M. (2002). The incorporation of sample weights into multilevel structural equation models. Structural Equation Modeling, 9, 475–503. https://doi.org/10.1207/S15328007SEM0904_2
- Stapleton, L. M. (2006). An assessment of practical solutions for structural equation modeling with complex sample data. Structural Equation Modeling, 13, 28–58. https://doi.org/10.1207/s15328007sem1301_2
- Steinley, D., & Brusco, M. J. (2011a). Evaluating mixture modeling for clustering: Recommendations and cautions. Psychological Methods, 16, 63–79. https://doi.org/10.1037/a0022673
- Steinley, D., & Brusco, M. J. (2011b). K-means clustering and mixture model clustering: Reply to McLachlan (2011) and Vermunt (2011). Psychological Methods, 16, 89–92. https://doi.org/10.1037/a0022679
- Sinclair, J., Jang, E. E., Azevedo, R., Lau, C., Taub, M. and Mudrick, N. V. (2018). “Changes in emotion and their relationship with learning gains in the context of MetaTutor,” in Intelligent Tutoring Systems. ITS 2018. Lecture Notes in Computer Science, Vol. 10858, eds R. Nkambou, R. Azevedo, and J. Vassileva (Cham: Springer), doi: https://doi.org/10.1007/978-3-319-91464-0_20.
- Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1–21. https://doi.org/10.1214/09-sts313
- The World Bank. (2020, February 13). How countries are using edtech (including online learning, radio, television, texting) to support access to remote learning during the COVID-19 pandemic. The World Bank. https://www.worldbank.org/en/topic/edutech/brief/how-countries-are-using-edtech-to-support-remote-learning-during-the-covid-19-pandemic
- Thoemmes, F. J., & Kim, E. S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46, 90–118. https://doi.org/10.1080/00273171.2011.540475
- U.S. Department of Agriculture. (2021a). Program Data Overview. Retrieved May 4, 2021, from https://www.fns.usda.gov/pd/overview.
- U.S. Department of Education Office of Educational Technology. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. U.S. Department of Education.
- U.S. Department of Education Office of Educational Technology. (2013). Expanding evidence approaches for learning in a digital world. Author.
- Van Horn, M. L., Feng, Y., Kim, M., Lamont, A., Feaster, D., & Jaki, T. (2016). Using multilevel regression mixture models to identify Level-1 heterogeneity in Level-2 effects. Structural Equation Modeling, 23, 259–269. https://doi.org/10.1080/10705511.2015.1035437
- Vermunt, J. K. (2010a). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18, 450–469. https://doi.org/10.1093/pan/mpq025
- Vermunt, J. K. (2010b). Mixture models for multilevel data sets. In J. Hox & J. K. Roberts (Eds.), Handbook of advancedmultilevel analysis (pp. 59–84). Routledge.
- Vermunt, J. K. (2011). K-means may perform as well as mixture model clustering but may also be much worse: Comment on Steinley and Brusco (2011). Psychological Methods, 16, 82–88; discussion 89–92. https://doi.org/10.1037/a0020144
- Xu, B., & Recker, M. (2012). Teaching analytics: A clustering and triangulation study of digital library user data. Educational Technology & Society, 15, 103–115. https://eric.ed.gov/?id=EJ992506
- Xu, H., Fang, G., Chen, Y., Liu, J., & Ying, Z. (2018). Latent class analysis of recurrent events in problem-solving items. Applied Psychological Measurement, 42, 478–498. https://doi.org/10.1177/0146621617748325