Conceptual Challenges in Coordinating Theoretical and Data-centered Estimates of Probability

Abstract

A core component of informal statistical inference is the recognition that judgments based on sample data are inherently uncertain. This implies that instruction aimed at developing informal inference needs to foster basic probabilistic reasoning. In this article, we analyze and critique the now-common practice of introducing students to both “theoretical” and “experimental” probability, typically with the hope that students will come to see the latter as converging on the former as the number of observations grows. On the surface of it, this approach would seem to fit well with objectives in teaching informal inference. However, our in-depth analysis of one eighth-grader's reasoning about experimental and theoretical probabilities points to various pitfalls in this approach. We offer tentative recommendations about how some of these issues might be addressed.

Acknowledgments

We thank the students and staff at Lynch Middle School, Holyoke, Massachusetts, and Amy Robinson for the bone drawings. This research was supported in part with a grant from the National Science Foundation, ESI 0454754. The views we express here do not necessarily represent those of the Foundation.

Notes

¹Erin is a pseudonym.

²In the excerpts that follow, I indicates the Instructor/Interviewer.

³In Jones et al. (2007), this page number was mistakenly given as 314.

⁴For one of many examples, see Billstein and Williamson's (1998) Book 2 of Middle grades MATH Thematics, pp. 28 and 33.

⁵See p. 1b of the Teacher's Guide of the grade 6 unit “How Likely Is It?” (Lappan, Fey, Fitzgerald, Friel, & Phillips, 1998).

⁶We do not discuss Erin's responses to the fourth and last task because our analysis of it suggested that she was confused about the graph we had given her to reason from, one that attempted to show the percentage of 2s from a die model settling down as the number of trials grew.

⁷One third is obviously (to us) on the high side of her results. She may have chosen that estimate under the assumption that we surely would have divided the spinner using some common unit fraction: “school” usually works like that.

⁸The words that are capitalized below were stressed in the interview.

⁹ Ireland and Watson (2009) reported similar findings with a student they interviewed who thought it might help to compare results across several replications and yet apparently did not see this as the same as drawing one large sample (p. 355).

¹⁰These frequencies were A = 50, B = 279, C = 244, D = 375, E = 52, and F = 0.

¹¹In practice, of course, it may be impractical to roll the die enough times to detect any difference between the true probability and the theoretical probability.

¹²The CMP unit Data About Us that Erin used does include more than one such activity, for example, flipping marshmallows and noting whether they land on an end or a side (see pp. 14–15 of the Teacher's Guide).

¹³Indeed, there is evidence from both the history of the development of statistical thinking and from classroom research that suggests that applying, for example, the idea of average as signal across different contexts is not a simple matter (Konold & Pollatsek, 2002; Lehrer & Schauble, 2007). Thus, taking ideas of informal inference developed in the context of probability and applying them to random sampling from populations of people (for example), will likely require carefully designed instruction.

¹⁴See the activity developed by Rider and Stohl Lee (2006).

¹⁵If you have collected data on ten pencils, you can blindly draw one from the collection, roll it a few times, and correctly identify which pencil it is. Each pencil has its own unique signal distribution.