Think-Aloud Interviews: A Tool for Exploring Student Statistical Reasoning

Abstract

Think-aloud interviews have been a valuable but underused tool in statistics education research. Think-alouds, in which students narrate their reasoning in real time while solving problems, differ in important ways from other types of cognitive interviews and related education research methods. Beyond the uses already found in the statistics literature—mostly validating the wording of statistical concept inventory questions and studying student misconceptions—we suggest other possible use cases for think-alouds and summarize best-practice guidelines for designing think-aloud interview studies. Using examples from our own experiences studying the local student body for our introductory statistics courses, we illustrate how research goals should inform study-design decisions and what kinds of insights think-alouds can provide. We hope that our overview of think-alouds encourages more statistics educators and researchers to begin using this method. Supplementary materials for this article are available online.

Keywords:

• Correlation and causation
• Misconceptions
• Sampling distributions
• Think-aloud interviews

Supplemental Materials

The recruiting script, interview protocol, and coding scheme are included as supplemental materials.

Acknowledgments

We are grateful to the editor, associate editors, and reviewers for their many helpful comments. Thanks to the Carnegie Mellon’s Eberly Center for Teaching Excellence and Educational Innovation for initial support developing this study; to David Gerritsen for initial advice on conducting think-aloud interviews; and to Gordon Weinberg for feedback and suggestions for questions, and for facilitating administration of the assessment to his courses. We also thank Sangwon Hyun, Ron Yurko, and Kevin Lin for contributing questions and assisting with think-aloud interviews. We are grateful to Christopher Peter Makris for extensive logistical support. Many thanks to our student participants, without whom this research would not have been possible.

Notes

1 Although we use the term “think aloud” to be consistent with the literature, communication need not be verbal. The key is to use a real-time communication method, so that participants are relying on short-term working memory in narrating while they solve the task, not reflecting on their solution afterwards. For instance, Roberts and Fels (2006) used a “gestural think aloud protocol” in a study with sign language users.