SOMOSAT: Utility of a web-based self-assessment tool in undergraduate medical education

Abstract

Background: Relatively few studies have rigorously assessed the effectiveness of computer-based self-assessment in medical education.

Aim: To assess whether an online self-assessment tool can be an effective adjunct to a traditional curriculum for second-year medical students.

Methods: The NYU School of Medicine Online Self-Assessment Tool (SOMOSAT) consists of >450 multiple-choice questions spanning disciplines of internal medicine, administered as separate modules focused on individual organ systems. Questions are coded on multiple dimensions, permitting second-year medical students to receive low-stakes, highly specific feedback regarding their knowledge and performance. Students can also review their answers to guide future study. We employed data collected during SOMOSAT operation to assess its utility and effectiveness.

Results: Overall, SOMOSAT accurately predicted student performance on future exams. SOMOSAT participants generally performed better than non-participants on subsequent graded course examinations (p < 0.05). Students using SOMOSAT subsequently experienced greater improvement in areas in which they initially performed poorly, compared with those in which they initially performed well. Students reported that SOMOSAT was most helpful in filling knowledge gaps, and providing opportunities to practice exam-style questions.

Conclusion: The ability of SOMOSAT to enhance learning and exam performance suggests that web-based self-assessment tools can be effective adjuncts to traditional educational methods.

Additional information

Notes on contributors

David E. Leaf

DAVID LEAF, MD, a recent graduate of NYU School of Medicine, is currently a resident in internal medicine at Columbia Medical Center. He served as lead writer and Student Director for the SOMOSAT project and hopes to pursue a career in academic nephrology.

Joseph Leo

JOSEPH LEO, BA, is a Database Analyst for the NYUSOM Office of Medical Education. He is responsible for the creation and ongoing maintenance of the NYU Exam Database which is used to store exam questions and images, generate exams, and track statistical information. Mr Leo helped to design and deploy the SOMOSAT system.

David E. Leaf

DAVID LEAF, MD, a recent graduate of NYU School of Medicine, is currently a resident in internal medicine at Columbia Medical Center. He served as lead writer and Student Director for the SOMOSAT project and hopes to pursue a career in academic nephrology.

Joseph Leo

JOSEPH LEO, BA, is a Database Analyst for the NYUSOM Office of Medical Education. He is responsible for the creation and ongoing maintenance of the NYU Exam Database which is used to store exam questions and images, generate exams, and track statistical information. Mr Leo helped to design and deploy the SOMOSAT system.

Phillip R. Smith

PHILLIP R. SMITH, PhD, MD, is an Associate Professor of Cell Biology who has been teaching medical students for over 25 years. He has had an ongoing interest in improving the computer-based delivery of teaching and self-evaluation materials and wrote the software used to deliver the quizzes described here.

Herman Yee

HERMAN YEE, PhD MD, is Associate Professor of Pathology. He has been actively involved in undergraduate medical education for the 12 years, including serving as the Pathology Course Director. In the evolution from traditional to an integrative curriculum, he was an early adopter of computerized multimedia, including digitization of histopathologic slides.

Arnold Stern

ARNOLD STERN, MD, PhD, is Professor of Pharmacology, and Assistant Dean for Extramural Education Programs and NYU School of Medicine. He is also the Course Director for Medical Pharmacology. Dr Stern was the Editor of the 8th and 9th editions of Pharmacology: Pre Test Self Assessment and Review.

Pamela B. Rosenthal

PAMELA ROSENTHAL, MD, is Assistant Professor of Medicine at NYU School of Medicine and is the current Course Director for Pathophysiology for the undergraduate medical program. She is also the Site Director for Rheumatology Education at Bellevue Hospital Center.

Eileen B. Cahill-Gallant

EILEEN CAHILL-GALLANT, MPA, is the Manager of the Office of Medical Education at New York University School of Medicine. She has served as coordinator for the second-year basic science curriculum, and is currently involved in student assessment, program evaluation and curriculum review.

Michael H. Pillinger

MICHAEL PILLINGER, MD, is Associate Professor of Medicine and Pharmacology, and served as Course Director for Pathophysiology at NYU. He is Director of Rheumatology Training, and Associate Director of the NYU Masters Program in Translational Investigation. He currently chairs the National Fellows’ In-training Examination for the American College of Rheumatology.