A Commentary on Construct Validity When Using Operational Virtual Learning Environment Data in Effectiveness Studies

References

• American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: Author.
   Google Scholar
• Baker, R. S. (2016). Stupid tutoring systems, intelligent humans. International Journal of Artificial Intelligence in Education, 26(2), 600–614. doi:10.1007/s40593-016-0105-0
   Web of Science ®Google Scholar
• Baker, R. S., & Ocumpaugh, J. (2014). Cost-effective, actionable engagement detection at scale. Proceedings of the 7th International Conference on Educational Data Mining (pp. 345–346).
   Google Scholar
• Baker, R. S., Ocumpaugh, J. L., & Andres, J. M. A. L. (2018). BROMP quantitative field observations: A review. In R. Feldman’s (Ed.), Learning science: Theory, research, and practice. New York, NY: McGraw-Hill.
   Google Scholar
• Baker, R. S., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008). Why students engage in “gaming the system” behavior in interactive learning environments. Journal of Interactive Learning Research, 19(2), 185–224.
   Google Scholar
• Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review of future and visions. Journal of Educational Data Mining, 1(1), 3–17.
   Google Scholar
• Cope, B., & Kalantzis, M. (2016). Big Data comes to school: Implications for learning, assessment, and research. AERA Open, 2(2), 1–19. doi:10.1177/2332858416641907
   PubMed Web of Science ®Google Scholar
• D’Mello, S., Dieterle, E., & Duckworth, A. (2017). Advanced, analytic, automated (AAA) measurement of engagement during learning. Educational Psychologist, 52(2), 104–123. doi:10.1080/00461520.2017.1281747
   PubMed Web of Science ®Google Scholar
• D’Mello, S., Kappas, A., & Gratch, J. (2018). The affective computing approach to affect measurement. Emotions Review, 10(2), 174–183. doi:10.1177/1754073917696583
   Web of Science ®Google Scholar
• Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920. doi:10.1111/bjet.12230
   Web of Science ®Google Scholar
• DiCerbo, K. E., & Behrens, J. T. (2014). Impacts of the digital ocean on education. London, UK: Pearson.
   Google Scholar
• Feng, M., Roschelle, J., Heffernan, N., Fairman, J., & Murphy, R. (2014). Implementation of an intelligent tutoring system for online homework support in an efficacy trial. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Intelligent tutoring systems, ITS 2014. Lecture Notes in Computer Science (Vol. 8474, pp. 561–566). New York: Springer, Cham.
   Google Scholar
• Garcia-Alvareza, M. T., Novo-Cortib, I., & Varela-Candamioc, L. (2018). The effects of social networks on the assessment of virtual learning environments: A study for the social sciences degrees. Telematics and Informatics, 35(4), 1005–1017. doi:10.1016/j.tele.2017.09.013
   Web of Science ®Google Scholar
• Hutt, S., Grafsgaard, J., & D’Mello, S. K. (2019). Time to scale: Generalizable affect detection for tens of thousands of students across an entire schoolyear. Proceedings of the ACM CHI Conference on Human Factors in Computing Systems (CHI 2019). New York: ACM.
   Google Scholar
• Idemudia, E. C., & Negash, S. (2012). An empirical investigation of factors that influence anxiety and evaluation in the virtual learning environment. SAIS 2012 Proceedings, 20. Retrieved from https://aisel.aisnet.org/.
   Google Scholar
• Kai, S., Almeda, M. V., Baker, R. S., Heffernan, C., & Heffernan, N. (2018). Decision tree modeling of wheel-spinning and productive persistence in skill builders. Journal of Educational Data Mining, 10(1), 36–71. doi:10.5281/zenodo.3344810
   Google Scholar
• Klasnia-Milicevic, A., Ivanovic, M., & Budimac, Z. (2017). Data science in education: Big data and learning analytics. Computer Applications in Engineering Education, 25(6), 1066–1078. doi:10.1002/cae.21844
   Web of Science ®Google Scholar
• McNamara, D., Allen, L. K., McCarthy, S., & Balyan, R. (in press). NLP: Getting computers to understand discourse. In K. Millis, D. Long, J. Magliano, & K. Wiemer’s (Eds.), Deep learning: Multi-disciplinary approaches. New York, NY: Routledge.
   Google Scholar
• National Forum on Education Statistics. (2015). Forum guide to elementary/secondary virtual education data (NFES 2016-095). U.S. Department of Education. Washington, DC: National Center for Education Statistics.
   Google Scholar
• Ozuru, Y., Rowe, M., O'Reilly, T., & McNamara, D. S. (2008). Where's the difficulty in standardized reading tests: The passage or the question? Behavior Research Methods, 40(4), 1001–1015. doi:10.3758/BRM.40.4.1001
   PubMed Web of Science ®Google Scholar
• Park, J. Y., Joo, S. –H., Cornillie, F., van der Maas, H. L. J., & Van den Noortgate, W. (2019). An explanatory item response theory method for alleviating the cold-start problem in adaptive learning environments. Behavior Research Methods, 51(2), 895–909. doi:10.3758/s13428-018-1166-9
   PubMed Web of Science ®Google Scholar
• Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth.
   Google Scholar
• Shute, V. J., & Moore, G. R. (2017). Consistency and validity in game-based stealth assessment. In H. Jiao & R. W. Lissitz’s (Eds.), Technology enhanced innovative assessment: Development, modeling and scoring from an interdisciplinary perspective. Charlotte, NC: Information Age Publisher.
   Google Scholar
• Susnea, E. (2018). How big data analytics will reshape e-Learning. e-Learning and Software for Education, 4, 72–76.
   Google Scholar
• U.S. Department of Education Office of Educational Technology. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Washington, DC: Author.
   Google Scholar
• U.S. Department of Education Office of Educational Technology. (2013). Expanding evidence approaches for learning in a digital world. Washington, DC: Author.
   Google Scholar
• Weller, M. (2007). Virtual learning environments: Using, choosing and developing your VLE. New York: Routledge.
   Google Scholar