Using “big data” to guide implementation of a web and mobile adaptive learning platform for medical students

Abstract

Background: Adaptive learning platforms (ALPs) can revolutionize medical education by making learning more efficient, but their potential has not been realized because students do not use them persistently.

Methods: We applied educational data mining methods to study United States medical students who used an ALP called Osmosis (www.osmosis.org) from 1 August 2014 to 31 July 2015. Multivariate logistic regressions modeled persistence on Osmosis as the dependent variable and Osmosis-collected variables as predictors.

Results: The 6787 students included in our analysis responded to a total of 887,193 items, with 2138 (31.5%) using Osmosis persistently. Number of items per student, mobile device use, subscription payment, and group membership were independently associated with persisting (p < 0.001 in all models). Persistent users rated quality more favorably (p < 0.01) but were not more confident in answer selections (p = 0.80). While persisters were more accurate than non-persisters (55% (SD 18%) vs 52% (SD 22%), p < 0.001), after adjusting for number of items, lower accuracy was associated with persistent use (OR 0.93 [95% CI 0.90–0.97], p < 0.01).

Conclusions: Our study of a large sample of U.S. medical students illustrates big data medical education research and provides guidance for improving implementation of ALPs and further investigation.

Acknowledgements

Authors thank Dr. Rebecca Eynon, University of Oxford supervisor for Mr. Menon’s Masters dissertation which forms part of this paper.

Disclosure statement

Work was conducted by Mr. Menon as part of a Master’s degree at Oxford University. Mr. Gaglani and Dr. Haynes are co-founders of Osmosis. Dr. Tackett receives support from Osmosis to assist with research projects.

All authors contributed to study design, data acquisition, analysis and interpretation, and writing the manuscript, and approve and are accountable for its final version.

Glossary

Adaptive learning platform: Interfaces that adapt key functional characteristics to learner needs and preferences.

Brusilovsky P, Peylo C. 2003. Adaptive and intelligent web-based educational systems. Int J Artif Intell Educ. 13:156–169.

Educational data mining: The process of extracting interesting, interpretable, useful and novel information from existing educational data.

Romero C, Ventura S. 2010. Educational data mining: a review of the state of the art. IEEE Trans Syst Man Cybern C Appl Rev. 40:601–618.

Notes on contributors

Mr. Menon is an MSc graduate of the Oxford Internet Institute and the Department of Education, University of Oxford.

Mr. Gaglani and Dr. Haynes are medical students at the Johns Hopkins University School of Medicine and co-founders of Osmosis.

Dr. Tackett, MD, MPH, is Instructor of Medicine at Johns Hopkins Bayview Medical Center.