Discrepant performance on multiple‐choice and short answer assessments and the relation of performance to general scholastic aptitude

Abstract

We conducted correlational and performance discrepancy analyses on exam and achievement data taken from students in three psychology courses. Across courses, the same findings emerged. First, only a small fraction of students consistently performed more strongly on one type of assessment (e.g., multiple‐choice) than on another (e.g., short answer). Second, students’ multiple‐choice performance, above and beyond their short answer performance, accounted for variation in students’ standing on achievement measures unrelated to psychology (including high school class standing, American College Test score, and college grade point average). In contrast, students’ short answer performance, above and beyond their multiple‐choice performance, did not account for variation in students’ standing on those achievement measures. Our findings support the continued use of multiple‐choice items to assess student learning.

Acknowledgements

The authors thank Allen Keniston and two anonymous reviewers for comments on earlier versions of this manuscript.

Notes

1. In one of the three classes, Research Methods, many points were generated from activities besides the exams. Students completed, in pairs, a final research report of 15–20 pages in length. Students also individually completed nine statistical assignments, each of which included drafting a section of a research report in APA format. Finally, students individually completed eight article analyses. We did not use these variables as criteria in our analyses because there were no parallel assignments in the other courses. Across multiple semesters of Research Methods, however, correlations between individual exam scores and final research report scores have been moderate (rs .30–.37, ps < .05); correlations between individual exam scores and average statistical assignment scores have been high (rs .60–.70, ps < .001); and correlations between individual exams scores and average article analysis scores have been high (rs .59–.65, ps < .001). In other words, students’ exam performance is highly linked with their performance on other indices of their learning in the course. The lower correlations between exam performance and final report performance are likely due to unreliability produced by having final reports completed by students in pairs rather than individually.