Development and piloting of an instructional video quality checklist (IVQC)

Abstract

Purpose

Medical education instructional videos are more popular and easier to create than ever before. Standard quality measures for this medium do not exist, leaving educators, learners, and content creators unable to assess these videos.

Materials and methods

Drawing from the literature on video quality and popularity, reusable learning objects, and multimedia and curriculum development principles, we developed a 26-item instructional video quality checklist (IVQC), to capture aspects of educational design (six items), source reliability (four items), multimedia principle adherence (10 items), and accessibility (six items). Two raters applied IVQC to 206 videos from five producers across topics from two organ systems (cardiology and pulmonology) encompassing four disciplines (anatomy, physiology, pathology, and pharmacology).

Results

Inter-rater reliability was strong. According to two-rater means, eight multimedia items were present in over 80% of videos. A minority of videos included learning objectives (46%), alternative language translations (41%), when the video was updated (40%), analogies (37%), or references (9%). Producer ratings varied significantly (p < .001) across 17 of 26 items. There were no significant differences according to the video topic.

Conclusions

IVQC detected differences in elements of instructional video quality. Future work can apply this instrument to a broader array of videos and in authentic educational settings.

Keywords:

• Instructional video
• video quality
• checklist
• medical curriculum
• IVQC

Glossary

Instructional Video: A collective term used for referring to videos which target cognitive learning goals including video lectures, tutorials, asynchronous instructional videos, and instructional videos for declarative knowledge.

Acknowledgments

The authors wish to thank Dr. David Green and the Educational Development Office’s Division of Innovations in Medical Education at the University of Miami Miller School of Medicine for support and assistance.

Disclosure statement

Dr. Tackett receives salary support from Osmosis for research and scholarship. The authors alone are responsible for the content and writing of this article.

Additional information

Funding

LS is funded by grants from the National Institute of Health (1R01HL140468) and the Miami Heart Research Institute.

Notes on contributors

Sean P. Schooley

Sean Schooley, MD, is now a Pediatrics Resident at Children’s Hospital of The King’s Daughters at Eastern Virginia Medical School.

Sean Tackett

Sean Tackett, MD, MPH, is Associate Professor of Medicine and International Medical Education Director for Division of General Internal Medicine, Johns Hopkins Bayview Medical Center, Baltimore, MD.

Lazaro R. Peraza

Lazaro Peraza, MD is now an Otolaryngology-Head & Neck Surgery Resident at the Mayo Clinic in Rochester, MN.

Lina A. Shehadeh

Lina Shehadeh, PhD, is Professor of Medicine and Director of Regenerative Medicine Scholarly Pathway for Medical Students at the University of Miami Miller School of Medicine, Miami, FL.