Using learning analytics to evaluate a video-based lecture series

Abstract

Background: The video-based lecture (VBL), an important component of the flipped classroom (FC) and massive open online course (MOOC) approaches to medical education, has primarily been evaluated through direct learner feedback. Evaluation may be enhanced through learner analytics (LA) – analysis of quantitative audience usage data generated by video-sharing platforms.

Methods and results: We applied LA to an experimental series of ten VBLs on electroencephalography (EEG) interpretation, uploaded to YouTube in the model of a publicly accessible MOOC. Trends in view count; total percentage of video viewed and audience retention (AR) (percentage of viewers watching at a time point compared to the initial total) were examined. The pattern of average AR decline was characterized using regression analysis, revealing a uniform linear decline in viewership for each video, with no evidence of an optimal VBL length. Segments with transient increases in AR corresponded to those focused on core concepts, indicative of content requiring more detailed evaluation. We propose a model for applying LA at four levels: global, series, video, and feedback.

Discussion and conclusions: LA may be a useful tool in evaluating a VBL series. Our proposed model combines analytics data and learner self-report for comprehensive evaluation.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Glossary

Learning analytics: The examination and interpretation of meaningful patterns in data derived from learners in an educational context.

Siemens G. 2013. Learning analytics: The emergence of a discipline. Am Behav Sci. 57:1380–1400.

Notes on contributors

K. H. Vincent Lau, MD, is a neuromuscular medicine fellow at Massachusetts General Hospital and Brigham and Women’s Hospital.

Pue Farooque, DO, is an Assistant Professor of Neurology at Yale School of Medicine.

Gary Leydon, BS, is the Associate Director for Technology Services at the Teaching and Learning Center at Yale School of Medicine.

Michael L. Schwartz, PhD, is an Associate Professor of Neuroscience, Director of Medical Studies in Neurobiology and Associate Dean for Curriculum at Yale School of Medicine.

R. Mark Sadler, MD, is a Professor of Neurology at Dalhousie University.

Jeremy J. Moeller, MD, MSc, is an Assistant Professor of Neurology and Neurology Residency Program Director at Yale School of Medicine.