Is there order in the mess? A single paper meta-analysis approach to identification of predictors of success in learning analytics

References

• Agudo-Peregrina, Á, S. Iglesias-Pradas, M. Conde-González, and Á Hernández-García. 2014. “Can we Predict Success from Log Data in VLEs? Classification of Interactions for Learning Analytics and Their Relation with Performance in VLE-Supported F2F and Online Learning.” Computers in Human Behavior 31 (1): 542–50. doi:10.1016/j.chb.2013.05.031.
   Web of Science ®Google Scholar
• Alyahyan, E., and D. Düştegör. 2020. “Predicting Academic Success in Higher Education: Literature Review and Best Practices.” International Journal of Educational Technology in Higher Education 17: 3. doi:10.1186/s41239-020-0177-7.
   Web of Science ®Google Scholar
• Andres, J. M. L., R. Baker, D. Gasevic, G. Siemens, S. Crossley, and S. Joksimovic. 2018. “Studying MOOC Completion at Scale Using the MOOC Replication Framework.” In Proceedings of LAK’18, March 7–9. 2018, Sydney, NSW, Australia. doi:10.1145/3170358.3170369.
   Google Scholar
• Andres, J. M. L., R. S. Baker, D. Gašević, G. Siemens, and C. A. Spann. 2017. “Replicating 21 Findings on Student Success in Online Learning.” Technology Instruction Cognition and Learning 10 (4): 313–33.
   Google Scholar
• Bergdahl, N., J. Nouri, T. Karunaratne, M. Afzaal, and M. Saqr. 2020. “Learning Analytics for Blended Learning: A Systematic Review of Theory, Methodology, and Ethical Considerations.” International Journal of Learning Analytics and Artificial Intelligence for Education 2 (2): 46–79. doi:10.3991/ijai.v2i2.17887.
   Google Scholar
• Bernard, R. M., P. C. Abrami, E. Borokhovski, C. A. Wade, R. M. Tamim, M. A. Surkes, and E. C. Bethel. 2009. “A Meta-Analysis of Three Types of Interaction Treatments in Distance Education.” Review of Educational Research 79 (3): 1243–89. doi:10.3102/0034654309333844.
   Web of Science ®Google Scholar
• Brooks, C., and C. Thompson. 2017. “Predictive Modelling in Teaching and Learning.” In The Handbook of Learning Analytics, edited by C. Lang, G. Siemens, A. F. Wise, and D. Gašević, 61–8. Alberta: Society for Learning Analytics Research (SoLAR).
   Google Scholar
• Chen, G., V. Rolim, R. Ferreira Mello, and D. Gašević. 2020. “Let’s Shine Together! A Comparative Study Between Learning Analytics and Educational Data Mining.” In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 544–53. New York: ACM. doi:10.1145/3375462.3375500
   Google Scholar
• Conijn, R., C. Snijders, A. Kleingeld, and U. Matzat. 2017. “Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS.” IEEE Transactions on Learning Technologies 10 (1): 17–29. doi:10.1109/TLT.2016.2616312.
   Google Scholar
• Da, C., R. Hatala, and R. Brydges. 2011. “Technology-Enhanced Simulation for Health Professions Education: A Systematic Review and Meta-Analysis.” Jama 306 (9): 978–988. doi:10.1001/jama.2011.1234.
   PubMed Web of Science ®Google Scholar
• Dawson, S., D. Gašević, J. Siemens, and S. Joksimovic. 2014. “Current State and Future Trends: A Citation Network Analysis of the Learning Analytics Field.” In Proceedings of the fifth International Conference on Learning Analytics and Knowledge, ACM (2014), 231–240. doi:10.1145/2567574.2567585.
   Google Scholar
• Donnelly, R. 2010. “Harmonizing Technology with Interaction in Blended Problem-Based Learning.” Computers & Education 54 (2): 350–9. doi:10.1016/j.compedu.2009.08.012.
   Web of Science ®Google Scholar
• Fernández-Castilla, B., L. Jamshidi, and L. Declercq. 2020. “The Application of Meta-Analytic (Multi-Level) Models with Multiple Random Effects: A Systematic Review.” Behavior Research Methods 52: 2031–52. doi:10.3758/s13428-020-01373-9.
   PubMed Web of Science ®Google Scholar
• Finnegan, C., L. Morris, and K. Lee. 2008. “Differences by Course Discipline on Student Behavior, Persistence, and Achievement in Online Courses of Undergraduate General Education. Journal of College Student Retention, Research, Theory and Practice 10 (1): 39–54. doi:10.2190/CS.10.1.d.
   Google Scholar
• Gašević, D., S. Dawson, T. Rogers, and D. Gasevic. 2016. “Learning Analytics Should Not Promote One Size Fits All: The Effects of Instructional Conditions in Predicting Learning Success.” The Internet and Higher Education 28: 68–84. doi:10.1016/j.iheduc.2015.10.002.
   Web of Science ®Google Scholar
• Glass, G. 1976. “Primary, Secondary, and Meta-Analysis of Research.” Educational Researcher 5 (10): 3–8.
   Google Scholar
• Gurevitch, J., J. Koricheva, S. Nakagawa, and G. Stewart. 2018. “Meta-Analysis and the Science of Research Synthesis.” Nature 555 (7695): 175–82. doi:10.1038/nature25753.
   PubMed Web of Science ®Google Scholar
• Hagger, M. S., N. L. D. Chatzisarantis, H. Alberts, C. O. Anggono, C. Batailler, A. R. Birt, R. Brand, et al. 2016. “A Multilab Preregistered Replication of the Ego-Depletion Effect.” Perspectives on Psychological Science 11 (4): 546–73. doi:10.1177/1745691616652873.
   Web of Science ®Google Scholar
• Hardy, R. J., and S. G. Thompson. 1998. “Detecting and Describing Heterogeneity in Meta-Analysis.” Statistics in Medicine 17 (8): 841–56.
   PubMed Web of Science ®Google Scholar
• Harrer, M., P. Cuijpers, T. Furukawa, and D. Ebert. 2019. Doing Meta-Analysis in R: A Hands-on Guide. doi:10.5281/zenodo.2551803.https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in.
   Google Scholar
• Higgins, J. P. T., and S. G. Thompson. 2002. “Quantifying Heterogeneity in a Meta-Analysis.” Statistics in Medicine 21 (11): 1539–58.
   PubMed Web of Science ®Google Scholar
• Higgins, J. P., S. Thompson, J. Deeks, and D. Altman. 2003. “Measuring Inconsistency in Meta-Analyses.” BMJ 327 (7414): 557–60. doi:10.1136/bmj.327.7414.557.
   PubMed Web of Science ®Google Scholar
• Ilgen, J. S., J. Sherbino, and D. A. Cook. 2013. “Technology-Enhanced Simulation in Emergency Medicine: A Systematic Review and Meta-Analysis.” Academic Emergency Medicine 20 (2): 117–27. doi:10.1111/acem.12076.
   PubMed Web of Science ®Google Scholar
• Jayaprakash, S. M., E. W. Moody, E. J. M. Lauría, J. R. Regan, and J. D. Baron. 2014. “Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative.” Journal of Learning Analytics 1 (1): 6–47.
   Google Scholar
• Jeong, H., C. E. Hmelo-Silver, and K. Jo. 2019. “Ten Years of Computer-Supported Collaborative Learning: A Meta-Analysis of CSCL in STEM Education During 2005–2014.” Educational Research Review 28 (April): 100284. doi:10.1016/j.edurev.2019.100284.
   Web of Science ®Google Scholar
• Joksimović, S., D. Gašević, T. M. Loughin, V. Kovanović, and M. Hatala. 2015. “Learning at Distance: Effects of Interaction Traces on Academic Achievement.” Computers & Education 87: 204–17. doi:10.1016/j.compedu.2015.07.002.
   Web of Science ®Google Scholar
• Joksimović, S., O. Poquet, V. Kovanović, N. Dowell, C. Mills, D. Gašević, S. Dawson, A. C. Graesser, and C. Brooks. 2017. “How Do We Model Learning at Scale? A Systematic Review of Research on MOOCs.” Review of Educational Research, doi:10.3102/0034654317740335.
   Web of Science ®Google Scholar
• Jovanović, J., D. Gašević, S. Dawson, A. Pardo, and N. Mirriahi. 2017. “Learning Analytics to Unveil Learning Strategies in a Flipped Classroom.” The Internet and Higher Education 33 (4): 74–85. doi:10.1016/j.iheduc.2017.02.001.
   Web of Science ®Google Scholar
• Jovanovic, J., N. Mirriahi, D. Gašević, S. Dawson, and A. Pardo. 2019. “Predictive Power of Regularity of Pre-Class Activities in a Flipped Classroom.” Computers & Education 134: 156–68.
   Web of Science ®Google Scholar
• Jovanović, J., M. Saqr, S. Joksimović, and D. Gašević. 2021. “Students Matter the Most in Learning Analytics: The Effects of Internal and Instructional Conditions in Predicting Academic Success.” Computers & Education 172 (April): 104251. doi:10.1016/j.compedu.2021.104251.
   Web of Science ®Google Scholar
• Kizilcec, R. F., J. Reich, M. Yeomans, C. Dann, E. Brunskill, G. Lopez, S. Turkay, J. J. Williams, and D. Tingley. 2020. “Scaling Up Behavioral Science Interventions in Online Education.” Proceedings of the National Academy of Sciences. doi:10.1073/pnas.1921417117.
   PubMed Web of Science ®Google Scholar
• Klein, R. A., M. Vianello, F. Hasselman, B. G. Adams, R. B. Adams Jr, S. Alper, M. Aveyard, J. R. Axt, M. T. Babalola, and Š Bahník. 2018. “Many Labs 2: Investigating Variation in Replicability Across Samples and Settings.” Advances in Methods and Practices in Psychological Science 1 (4): 443–90.
   Google Scholar
• Kurex, S., and J. Jonkman. 2007. “A Comparison of Heterogeneity Variance Estimators in Combining Results of Studies.” Statistics in Medicine 26 (9): 1964–81.
   PubMed Web of Science ®Google Scholar
• Lou, Y., P. C. Abrami, S. Apollonia, and S. D’Apollonia. 2001. “Small Group and Individual Learning with Technology: A Meta-Analysis.” Review of Educational Research 71 (3): 449–521. doi:10.3102/00346543071003449.
   Web of Science ®Google Scholar
• Lust, G., J. Elen, and G. Clarebout. 2013. “Students’ Tool-Use Within a Web Enhanced Course: Explanatory Mechanisms of Students’ Tool-Use Pattern.” Computers in Human Behavior 29 (5): 2013–21. doi:10.1016/j.chb.2013.03.014.
   Web of Science ®Google Scholar
• Mangaroska, K., and M. Giannakos. 2019. “Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning.” IEEE Transactions on Learning Technologies 12 (4): 516–34. doi:10.1109/TLT.2018.2868673.
   Web of Science ®Google Scholar
• McShane, B. B., and U. Böckenholt. 2017. “Single-Paper Meta-Analysis: Benefits for Study Summary, Theory Testing, and Replicability.” Journal of Consumer Research 43 (6): 1048–63. doi:10.1093/jcr/ucw085.
   Web of Science ®Google Scholar
• ntHout, J., J. P. A. Ioannidis, M. M. Rovers, and J. J. Goeman. 2016. “Plea for Routinely Presenting Prediction Intervals in Meta-Analysis.” BMJ Open 6: 7. doi:10.1136/bmjopen-2015-010247.
   Web of Science ®Google Scholar
• Ochoa, X., S. Knight, and A. Wise. 2020. “Learning Analytics Impact: Critical Conversations on Relevance and Social Responsibility.” Journal of Learning Analytics 7 (3): 1–5. doi:10.18608/jla.2020.73.1.
   Google Scholar
• Open Science Collaboration. 2015. “Estimating the Reproducibility of Psychological Science.” Science 349 (6251): aac4716. doi:10.1126/science.aac4716.
   PubMed Web of Science ®Google Scholar
• Papamitsiou, Z., and A. A. Economides. 2016. “Learning Analytics for Smart Learning Environments: A Meta-Analysis of Empirical Research Results from 2009 to 2015.” In Learning, Design, and Technology, edited by Spector M., Lockee B., and Childress M. (eds.), 1–23. Cham: Springer.
   Google Scholar
• Pina, A. 2012. “An Overview of Learning Management Systems.” In Virtual Learning Environments: Concepts, Methodologies, Tools and Applications, 33–51. Louisville: Sullivan Univ. Syst. doi:10.4018/978-1-4666-0011-9.ch103
   Google Scholar
• Saltmarsh, J., E. Zlotkowski, and E. L. Hollander. 2011. “Indicators of Engagement.” Higher Education and Democracy: Essays on Service-Learning and Civic Engagement, 285–302. doi:10.1007/978-1-4615-0885-4_3.
   Google Scholar
• Saqr, M., U. Fors, and M. Tedre. 2017. “How Learning Analytics Can Early Predict Under-Achieving Students in a Blended Medical Education Course.” Medical Teacher 39 (7): 757–67.
   PubMed Web of Science ®Google Scholar
• Saqr, M., and S. López-Pernas. 2021a. “Modelling Diffusion in Computer-Supported Collaborative Learning: A Large Scale Learning Analytics Study.” International Journal of Computer-Supported Collaborative Learning 16 (4): 441–83.
   Web of Science ®Google Scholar
• Saqr, M., and S. López-Pernas. 2022. “The Curious Case of Centrality Measures: A Large-Scale Empirical Investigation.” Journal of Learning Analytics 9 (1): 13–31. doi:10.18608/jla.2022.7415.
   Google Scholar
• Saqr, M., O. Viberg, and H. Vartiainen. 2020. “Capturing the Participation and Social Dimensions of Computer-Supported Collaborative Learning Through Social Network Analysis: Which Method and Measures Matter?” International Journal of Computer Supported Collaborative 15: 227–48. doi:10.1007/s11412-020-09322-6.
   Web of Science ®Google Scholar
• Schneider, M., and F. Preckel. 2017. “Variables Associated with Achievement in Higher Education: A Systematic Review of Meta-Analyses.” Psychological Bulletin 143 (6): 565–600.
   PubMed Web of Science ®Google Scholar
• Schriger, D., D. Altman, J. Vetter, T. Heafner, and D. Moher. 2010. “Forest Plots in Reports of Systematic Reviews: A Cross-Sectional Study Reviewing Current Practice.” International Journal of Epidemiology 39 (2): 421–9. doi:10.1093/ije/dyp370.
   PubMed Web of Science ®Google Scholar
• Schwarzer, G., J. Carpenter, and G. Rücker. 2015. Meta-Analysis with R. Cham: Springer International Publishing. doi:10.1007/978-3-319-21416-0
   Google Scholar
• Shannon, C. E. 1948. “A Mathematical Theory of Communication.” Bell System Technical Journal 27 (3): 379–423. doi:10.1002/j.1538-7305.1948.tb01338.x.
   Google Scholar
• Siemens, G., S. Dawson, and G. Lynch. 2014. Improving the quality and productivity of the higher education sector — Policy and strategy for systems-level deployment of learning analytics. Office of Learning and Teaching, Australian Government, Canberra, Australia.
   Google Scholar
• Tempelaar, D. T., B. Rienties, and B. Giesbers. 2015. “In Search for the Most Informative Data for Feedback Generation: Learning Analytics in a Data-Rich Context.” Computers in Human Behavior 47: 157–67.
   Web of Science ®Google Scholar
• Thompson, S. G. 1994. “Systematic Review: Why Sources of Heterogeneity in Meta-Analysis Should be Investigated.” BMJ 309 (6965): 1351–5.
   PubMed Web of Science ®Google Scholar
• Viberg, O., M. Hatakka, O. Bälter, and A. Mavroudi. 2018. “The Current Landscape of Learning Analytics in Higher Education.” Computers in Human Behavior 89: 98–110.
   Web of Science ®Google Scholar
• Voyer, D., and S. D. Voyer. 2014. “Gender Differences in Scholastic Achievement: A Meta-Analysis.” Psychological Bulletin 140 (4): 1174–204.
   PubMed Web of Science ®Google Scholar
• Walker, E., A. Hernandez, and M. Kattan. 2008. “Meta-Analysis: Its Strengths and Limitations.” Cleveland Clinic Journal of Medicine 75 (6): 431–9. doi:10.3949/ccjm.75.6.431. PMID: 18595551.
   PubMed Web of Science ®Google Scholar
• Wecker, C., and F. Fischer. 2014. “Where is the Evidence? A Meta-Analysis on the Role of Argumentation for the Acquisition of Domain-Specific Knowledge in Computer-Supported Collaborative Learning.” Computers & Education 75: 218–28. doi:10.1016/j.compedu.2014.02.016.
   Web of Science ®Google Scholar
• Zacharis, N. Z. 2015. “A Multivariate Approach to Predicting Student Outcomes in Web-Enabled Blended Learning Courses.” The Internet and Higher Education 27: 44–53. doi:10.1016/j.iheduc.2015.05.002.
   Web of Science ®Google Scholar