Open book tests: assessment of academic learning in clerkships

Abstract

An examination of an open-book testing approach in a family medicine clerkship seeks to determine whether this method more closely mirrors the discipline of family medicine, where practitioners refer daily to written resource materials in order to make clinical decisions without compromising the learning and assessment process. Student scores on the multiple-choice test were analysed by year, by quarter and by site using ANOVA. Students in the experimental site were interviewed to determine preparation style, use of text during test, as well as attitudes toward open-book testing. Analysis of variance showed that the interaction of site and year was significant at p = 0.03. The mean score of 88.2 for Maine students in 2002 was significantly different from the other three mean scores. The desired qualitative outcomes of the intervention were confirmed: reducing the anxiety of students, wider reading of the textbook, knowing the structure of the textbook as a learning resource, and deeper understanding of concepts and principles rather than time spent on memorization. While the difference in scores did reach statistical significance, it is important to note that the difference in mean score was only four points on a 100-point scale. Given the percentage of the total grade represented by the test score, it is unlikely that this difference represents any real difference in grade for students in Maine compared with Vermont. The students appeared to approach the textbook and therefore, perhaps, the body of knowledge as a whole with the orientation of a generalist. The MMC Clerkship Director recommended the implementation of the open-book approach to the Family Practice clerkship at all sites and the University of Vermont Medical School accepted the proposal. This recommendation supports advising students on the preparation for an open-book test and on tactics for the best use of the textbook during the test.