Predicting student understanding by modeling interactive exploration of evidence during an online science investigation

ABSTRACT

This study mined student interactions with visual representations as a means to automate assessment of learning in a complex, inquiry-based learning environment. Log trace data of 143 middle school students’ interactions with an interactive map in Research Quest (an inquiry-based, online learning environment) were analyzed. Students used the interactive map to make scientific observations for an evidence-based hypothesis. The examination of classification error using an artificial neural network, compared against the majority class for prediction, suggests that student performance on several metrics of critical thinking can be classified based on different patterns in interactions with visual representations. Two alternative methods are compared in this study for training and evaluating data-mined models of student performance. In accordance with the general consensus in the literature, the error estimates for models’ predictions were less variable using a student-level cross-validation. Implications of these findings for open-ended inquiry-based learning environments are discussed.

KEYWORDS:

• Inquiry-based learning
• educational data mining
• visual representation
• critical thinking
• open-ended learning environments

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was supported by funding from the University of Utah's Research Incentive Seed Grant Program. Visualizations and 3D models were made possible through funding from the Joseph and Evelyn Rosenblatt Charitable Fund and the IJ and Jeanné Wagner Foundation to the Natural History Museum of Utah.

Notes on contributors

Eric Poitras

Eric Poitras is Assistant Professor of Instructional Design and Educational Technology at the University of Utah. His research focuses on the application of educational data mining and learning analytic methods to improve the adaptive capabilities of instructional systems and technologies.

Kirsten R. Butcher

Kirsten R. Butcher is Associate Professor of Instructional Design and Educational Technology at the University of Utah where she also serves as the Director of the Center for the Advancement of Technology in Education. Her research examines the impact of interactive visuals and multimedia features in educational technologies on high-level cognitive processes and deep learning outcomes.

Matthew Orr

Matthew Orr is a graduate student in the Department of Educational Psychology at the University of Utah. His research focuses on the impact that tangible materials have on learning with digital and tangible 3D models and scaffolding learner interactions in digital learning environments.

Michelle A. Hudson

Michelle Hudson is a graduate student in the Department of Educational Psychology. Her work focuses on understanding collaborative learning conversations during technology use and determining the impact of various digital and concrete resources on students’ learning processes and outcomes in classroom and museum environments.

Madlyn Larson

Madlyn Larson is the Director of Education Initiatives at the Natural History Museum of Utah. Her work explores the ways in which informal institutions – particularly museums – can support K-12 teachers and students using authentic science experiences and technology-enabled instruction.