New Multiple-Choice Measures of Historical Thinking: An Investigation of Cognitive Validity

Abstract

History education scholars have recognized the need for test validity research in recent years and have called for empirical studies that explore how to best measure historical thinking processes. The present study was designed to help answer this call and to provide a model that others can adapt to carry this line of research forward. It employed think-aloud protocols with 12 high school history students to explore whether selected multiple-choice items from a new test of historical thinking—the Historical Thinking Test (HTT)—elicit the intended aspects of historical thinking among high school history students and whether these new items elicit these constructs at higher rates than items from extant standardized history tests. Data revealed that HTT multiple-choice items did, to varying degrees, elicit aspects of historical thinking and that HTT items elicited the intended aspects of historical thinking at higher rates than selected multiple-choice items from extant standardized tests. Yet the HTT items also elicited construct-irrelevant reasoning, which weakens an argument for their use as measures of historical thinking. Implications for history testing and the efficacy of multiple-choice items as measures of historical thinking are discussed.

Keywords:

• assessment
• historical thinking
• multiple-choice

Notes

¹ Theoretical work on historical thinking in the United States has been heavily influenced by Bruner’s (1960/2009) seminal ideas about basing school curricula on the “structure of the disciplines” and has focused disproportionately on identifying on the processes and methods of the discipline. Although there is some common ground, notions of historical thinking in North America are distinct from those in Europe (cf. Seixas, 2017; Wineburg, 2001).

² The STAR Grade 11 History–Social Science test was discontinued in 2013 as the state focused on implementing English and mathematics tests aligned to the Common Core State Standards.