Abstract
Purpose
The purpose of this study was to enrich understanding about the perceived benefits and drawbacks of constructed response short-answer questions (CR-SAQs) in preclerkship assessment using Norcini’s criteria for good assessment as a framework.
Methods
This multi-institutional study surveyed students and faculty at three institutions. A survey using Likert scale and open-ended questions was developed to evaluate faculty and student perceptions of CR-SAQs using the criteria of good assessment to determine the benefits and drawbacks. Descriptive statistics and Chi-square analyses are presented, and open responses were analyzed using directed content analysis to describe benefits and drawbacks of CR-SAQs.
Results
A total of 260 students (19%) and 57 faculty (48%) completed the survey. Students and faculty report that the benefits of CR-SAQs are authenticity, deeper learning (educational effect), and receiving feedback (catalytic effect). Drawbacks included feasibility, construct validity, and scoring reproducibility. Students and faculty found CR-SAQs to be both acceptable (can show your reasoning, partial credit) and unacceptable (stressful, not USMLE format).
Conclusions
CR-SAQs are a method of aligning innovative curricula with assessment and could enrich the assessment toolkit for medical educators.
Acknowledgments
The authors would like to thank Krista Paxton and Ayomide Johnson for their instrumental contributions in the preparation of this manuscript.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Additional information
Funding
Notes on contributors
Judith M. Brenner
Judith M. Brenner, MD, is the Senior Associate Dean for Medical Education at the New York University Grossman Long Island School of Medicine, Mineola, NY.
Tracy B. Fulton
Tracy B. Fulton, PhD, is Professor of Biochemistry and Biophysics and a course director at University of California San Francisco.
Marieke Kruidering
Marieke Kruidering, PhD, is Professor in the Department of Cellular & Molecular Pharmacology and a course director at University of California San Francisco.
Jeffrey B. Bird
Jeffrey B. Bird, MA, is the Educational Research & Strategic Assessment Analyst and an assistant professor at the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY.
Joanne Willey
Joanne M. Willey, PhD, is the Leo A. Guthart Professor of Biomedical Sciences and Chair of the Department of Science Education at the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY.
Kelli Qua
Kelli Qua, PhD, is the Director of Evaluation and CQI Center for Medical Education at Case Western Reserve University School of Medicine.
Doreen M. Olvet
Doreen M. Olvet, PhD, is Assistant Professor and Medical Education Project Manager Department of Science Education, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY.