Using “Think-Alouds” to Understand Variations in Political Thinking

Abstract

One of the most important skills students can learn is how to use multiple, conflicting sources of information to formulate and defend positions on political issues. However, when we assign students to do this, all instructors see is the final product. Not knowing the process by which students do these assignments makes it difficult to help them perform the task better. I report on “think-alouds” done with upper-level political science majors and with introductory-level college students. Students use multiple articles about capital punishment to help them form/refine and defend their arguments about capital punishment. They do this while “thinking out loud” as they engage in the task. My results suggest that experts make more connections between sources and more effectively argue with the sources; knowing this will help me better scaffold this assignment to support novice students. I conclude by discussing possible future directions for work on the learning of political information skills.

Keywords:

• attitude formation
• capital punishment
• political thinking
• think-alouds

Earlier versions of this paper were presented at the 2008 American Political Science Association Teaching and Learning Conference (San Jose), the 2008 Midwest Political Science Association Conference (Chicago), and the 2008 Improving University Teaching Conference (Glasgow). I thank Rebecca Nowacek, Bob Bain, Matt Kaplan, and colleagues in my Project Group at the Carnegie Foundation for various conversations that have helped me develop the ideas in this paper, and Cherie Bryant for her research assistance. My sincere thanks to the students who agreed to participate in this research. Financial support for the project has been provided by the Faculty Research Support Fund, the College of Arts and Sciences and the Department of Political Science at Eastern Michigan University, and by the Spencer Foundation. None of the above should be implicated in any errors of fact or interpretation in this paper.

Notes

Small-N work is common in think-aloud studies, due to the intensive nature of the data collection and analysis of each of the cases. Like all empirical studies, these results call for replication and extension across a range of different settings; I discuss some future steps arising from this work at the conclusion of this paper.

It is important to note at the outset that this is not a study about the use of a pedagogical technique. While it certainly would be possible to use think-alouds as a classroom tool (as a different form of small-group work, for example), this study does not do that. Instead, it uses insights gained from the think-alouds to draw larger pictures of student learning in the political science classroom and then to draw implications from this for how we might refocus aspects of our teaching in order to achieve our learning outcomes.

On higher youth turnout during the 2008 primaries, see reports done by the CIRCLE (Center for Information & Research on Civic Learning and Engagement) Foundation, found at www.civicyouth.org. On higher youth turnout in the 2008 general election, see http://www.msnbc.msn.com/id/27562023/, accessed November 5, 2008. Early postelection reports estimated that about 50% of people aged 18–28 voted. This represents approximately 24 million young voters, an increase of about 2.4 million since 2004.

See Lupia (2006) for a particularly useful treatment of the concept of civic competence.

This assumes, of course, that there is such a thing as nonbiased information. Without debating this point, it seems clear that there are some sites that purport to neutrality and others that stretch the truth in order to drive home a point. The fact that reasonable people could not all agree on whether certain sites fit into the former or latter categories only magnifies the challenges.

Much of the critique of the Perry model arises from the fact that it was derived many years ago based on a study of male students at Harvard University. Extensions, such as Belenky et al. (1986), have improved the model and addressed some of its potential flaws. Still, it is striking the extent to which this and other extensions of the Perry approach support the basic premises and outlines of the model.

The fourth position that Perry describes, commitment to relativity, occurs when the student begins to develop his or her own point of view in a world of relativism. This leads to reexamining past attitudes and developing firm commitments regarding potentially new attitudes and beliefs. Ultimately, the learner recognizes that even these commitments constantly evolve, creating a lifelong commitment to examining these deeply held attitudes and beliefs. It is comparatively rare for college students to reach this stage (Moore 1994).

This initial division of students into being experts and novices is not presumed to be deterministic; it is quite possible that some so-called novices will exhibit behavior more characteristic of expertise than do some of the so-called experts.

The approach practiced by Diaz et al. is worth repeating within political science. It would, however, likely require a much larger project than the one detailed here.

Critically, it would not suffice to say that students will naturally gain these skills as their career in the discipline proceeds. It is true that many experts did figure out how to make those moves associated with successful performance on the task. However, these are top students, who ended up moving from an introductory class into advanced classes in the major and did well in these classes. For the many students, however, the introductory class will be their only political science course. For them, explicitly building this skill development into the first class is vital if we wish to help them understand and use the thought processes inherent in disciplinary work (and, perhaps, be motivated to take even more classes in the discipline).

The first six think-alouds were coded jointly by both myself and an undergraduate research assistant, in order to derive a common definition of what constituted mention of a new fact and a confirmatory fact (and to arrive at common definitions of other terms used in the data analysis). For the most part, such evidence was self-evident in the coding—statements were only accepted as fitting into either category if such placement was clear—and intercoder agreement was high. Given this, the final six think-alouds were coded by myself alone, with approximately four questionable items being discussed with the research assistant in order to validate the coding.

Full disclosure—I oppose capital punishment, largely on the grounds that it is difficult, if not impossible, to administer it equitably across race and class lines. I am very much persuaded by the opinion of Justice Blackmun in Callins v. Collins (1994), in which he argued that because of an inability to fairly and consistently apply death penalty statutes, he “no longer shall tinker with the machinery of death.” I am skeptical of the deterrent value of the death penalty but am willing to be persuaded.

In this case, the politically-engaged novice cited above acted more like a novice than an expert, finding nothing to fault in Pataki's reasoning. As a strong conservative, this may have been a case of where his ideological blinders trumped his ability to identify weaknesses in this particular version of the pro-death penalty argument.

My approach can be said to use a constructivist approach to learning, in which “students learn by constructing and reconstructing their understandings about the world and phenomena, via a process of actively attempting to make sense of one's experiences” (O'Brien 2008, p. 10). Think-alouds in which students work with others would be using the theory of social constructivism, which argues that learning is a social process that occurs through interaction and collaboration with others—learning as a process of social construction (Marshall 1996; O'Brien 2008).