How learning analytics can early predict under-achieving students in a blended medical education course

Abstract

Aim: Learning analytics (LA) is an emerging discipline that aims at analyzing students’ online data in order to improve the learning process and optimize learning environments. It has yet un-explored potential in the field of medical education, which can be particularly helpful in the early prediction and identification of under-achieving students. The aim of this study was to identify quantitative markers collected from students’ online activities that may correlate with students’ final performance and to investigate the possibility of predicting the potential risk of a student failing or dropping out of a course.

Methods: This study included 133 students enrolled in a blended medical course where they were free to use the learning management system at their will. We extracted their online activity data using database queries and Moodle plugins. Data included logins, views, forums, time, formative assessment, and communications at different points of time. Five engagement indicators were also calculated which would reflect self-regulation and engagement. Students who scored below 5% over the passing mark were considered to be potentially at risk of under-achieving.

Results: At the end of the course, we were able to predict the final grade with 63.5% accuracy, and identify 53.9% of at-risk students. Using a binary logistic model improved prediction to 80.8%. Using data recorded until the mid-course, prediction accuracy was 42.3%. The most important predictors were factors reflecting engagement of the students and the consistency of using the online resources.

Conclusions: The analysis of students’ online activities in a blended medical education course by means of LA techniques can help early predict underachieving students, and can be used as an early warning sign for timely intervention.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Notes on contributors

Mohammed Saqr is assistant professor of medicine with interest in learning enhanced technology, learning analytics, and simulation of medicine, he is currently the head of e-learning unit, Qassim University, College of Medicine.

Matti Tedre is a professor at the University of Eastern Finland, a docent at Stockholm University, Sweden, and the author of Science of Computing: Shaping a Discipline (Taylor & Francis/CRC Press, 2014).

Uno Fors DDS, PhD, is professor of IT and learning as well as head of Department of Computer and Systems Sciences at Stockholm University, Sweden. Fors research focuses on Technology enhanced learning and especially on virtual cases for learning within the healthcare area. Fors has published more than 150 papers in scientific journals and conferences.

Glossary

Learning analytics: Measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.

Siemens G. Learning analytics: The Emergence of a Discipline.

American Behavioral Scientist. 2013;57:1380–1400.

Social network analysis: Social network analysis is the study of structure, it deals with the relational structure and patterns of relationships among social entities, which might be people, groups, or organizations.

Hawe P, Webster C, Shiell A. A glossary of terms for navigating the field of social network analysis. J Epidemiol Community Health [Internet]. 2004 Dec [cited 2017 Feb 1];58(12):971–5. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15547054

Outlier: An outlier is an observation which deviates so much from the other observations and might exert an exaggerated effect on the results (deviate more than 3 times the SD from the mean).

Hawkins D. 1980. Identification of Outliers: Chapman and Hall.

Betweenness centrality: Is a measure of the influence a node has over the spread of information through the network.

Hawe P, Webster C, Shiell A. A glossary of terms for navigating the field of social network analysis. J Epidemiol Community Health [Internet]. 2004 Dec [cited 2017 Feb 1];58(12):971–5. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15547054