Toward a New Approach to the Evaluation of a Digital Curriculum Using Learning Analytics

Abstract

Understanding how an educational intervention is implemented is essential to evaluating its effectiveness. With the increased use of digital tools in classrooms, however, traditional methods of measuring implementation fall short. Fortunately, there is a way to learn about the interactions that users have with digital tools that are embedded into the technologies themselves: user data. The purpose of this article is to outline ways in which researchers can harness learning analytics and user data to gain a deeper understanding of the implementation of digital innovations and their impact on teaching and learning. We discuss four considerations for the integration of learning analytics and user data in implementation research, and we provide an example of integration from an evaluation of a digital science curriculum, STEMscopes.

Keywords:

• implementation research
• learning analytics
• digital learning

Supplemental Materials

A supplemental appendix for this article can be accessed on the publisher's website at http://dx/doi.org/10.1080/15391523.2015.999639

Additional information

Notes on contributors

Virginia Snodgrass Rangel

Virginia Snodgrass Rangel is an independent education researcher and owner of Third Coast Insight in Houston, Texas. Her research interests focus on program evaluation and implementation research, particularly in science education and educational technology and data use. Please address correspondence regarding this article to Virginia Snodgrass Rangel, 662 Lester Street, Houston, TX 77007, USA. E-mail: [email protected]

Elizabeth R. Bell

Elizabeth R. Bell is research manager at Accelerate Learning Inc. Her research interests focus on early science education and program evaluation.

Carlos Monroy

Carlos Monroy is research scientist in the Department of Computer Science at Rice University. His research interests focus on learning analytics, data mining, and information visualization.

J. Reid Whitaker

J. Reid Whitaker is currently the chief academic officer for Accelerate Learning Inc. His research interests include leadership, science education, and blended learning.