The Role of Statistics Educators in the Quantitative Literacy Movement

Abstract

Discussions of quantitative literacy have become increasingly important, and statistics educators are well aware of the link between statistics education and quantitative literacy. Both the statistics education and quantitative literacy movements have emphasized the importance of students practicing skills in multiple contexts—a goal also consistent with a quantitative reasoning across-the-curriculum approach. In this paper, we consider two sources of information: 1) Our data from statistics courses and other quantitative-intensive courses at Lawrence University and 2) a review of the research literature on transfer of quantitative concepts across contexts. Through analysis of these sources, we further explore the link between statistics education and quantitative literacy, and argue for an across-the-curriculum approach to teaching quantitative reasoning. Moreover, we make specific suggestions to statistics educators on their role in the quantitative literacy movement.

• Across-the-curriculum approach
• Statistics education
• Transfer

1. Introduction: The Link Between Statistics and QL

As statistics educators, we are all familiar with the quantitative literacy (QL) movement. To make informed, intelligent decisions about critical issues such as health, politics, and the economy, college students must feel competent and confident in their quantitative skills. As Steen (2001, p. 2) and colleagues write in Mathematics and Democracy: The Case for Quantitative Literacy, “Quantitative literacy empowers people by giving them tools to think for themselves, to ask intelligent questions of experts, and to confront authority confidently.” The same can be said about statistics. Statistics is an applied branch of mathematics that provides tools for collecting and analyzing data to answer real questions. Statistics is naturally a key component of QL, and statistics educators can and should assist the QL cause. As Scheaffer (2003, p. 149) argues, “It would make practical as well as pedagogical sense to anchor the expansion of QL to the statistics teaching efforts of colleges and universities.”

Hence, statistical education should play a critical role in the QL curriculum. The understanding of chance, variability, sampling methods, data analysis, and decision-making are important statistical concepts as well as essential components of QL. Like the QL movement, the statistics education reform pedagogy encourages students to apply statistical and mathematical concepts to interesting, important problems in a variety of contexts. In this sense, statistics is a natural bridge between mathematics and QL; statistics teaches general quantitative principles, yet couches them in a variety of real-world contexts. Furthermore, statistics courses introduce students to sophisticated reasoning and open-ended questions (e.g., what design should be implemented? What analyses should be done?). Statistical training should be focused on conceptual mastery and skill development in a manner that promotes the transfer and application of concepts across contexts, including real-world contexts in which multiple-solution problems predominate. As Lovett (2001) demonstrates in her literature review and empirical work on statistics education, there is still considerable room for improvement when teaching transferable skills. With explicit attention to transfer, statistics courses can successfully foster quantitative literacy in students. Research on transfer demonstrates the importance of providing opportunities for students to practice QL skills and of explicitly showing conceptual connections across contexts. That is, an across-the-curriculum approach is essential to developing QL in students.

Leaders in the QL movement have also consistently recommended a quantitative-reasoning-across-the-curriculum (QRAC) approach to developing quantitative literacy (Cozzens 2003; Madison 2004; Tritelli 2004). A QRAC approach, reflecting the well-established writing-across-the-curriculum model, asks instructors in all disciplines to incorporate explanations of quantitative reasoning and quantitative problem solving in their teaching, and where appropriate, in their course requirements. A QRAC approach reinforces quantitative concepts and problem-solving processes, and allows students to practice and apply skills in new contexts. Furthermore, a QRAC approach often provides opportunities for students to practice skills in areas where they have greater familiarity or expertise, as in their major area. Research on learning (see, for example, Bransford, Brown, and Cocking 2001) and transfer (see, for example, Detterman and Sternberg 1993) has established that familiarity and expertise generally promote more sophisticated reasoning and problem solving. Teaching quantitative reasoning is a natural pedagogical fit in most science and social science courses, and is sometimes a more challenging fit in some humanities and fine arts courses. However, even in the sciences the QRAC approach has not yet become standard practice, and many faculty legitimately struggle with determining the best pedagogies for teaching QL and promoting transfer. In some cases, faculty feel unqualified or simply uninterested in teaching QL. Given that many fields use statistics, statistics educators can provide peer support and help faculty to find a common language to teach quantitative concepts across disciplines.

In this article, we argue that statistics education and statistics educators should play a pivotal role in the QL movement, particularly in helping colleges to incorporate QL across the curriculum. Our argument is based on two sources of information: 1) Our data from statistics courses and other quantitative-intensive courses at Lawrence University and 2) a review of the research literature on transfer of quantitative concepts across contexts. In our view, transfer of quantitative concepts to novel contexts is the greatest challenge to achieving QL. As Douglas Detterman (1993, p.13) concludes in his review of transfer studies, “The surprise is the extent of similarity it is possible to have between two problems without subjects realizing that the two situations are identical and require the same solution.” We believe statistics educators can play a critical role in promoting transferable skills in students and encouraging pedagogical changes and curricular development to support transfer. To this end, in the final section of the article we provide suggestions for the role of statistics educators in the QL movement.

2. Lawrence University Data

Lawrence University, a selective, private college of approximately 1400 undergraduate students, has general education requirements that include an across-the-curriculum competency requirement in quantitative reasoning. Instituted in 2001, the quantitative reasoning requirement is fairly new. As part of the implementation instructors submitted proposals for courses that would meet the requirement, which specifies among other things, explicit instruction in quantitative methods and quantitative reasoning. Once quantitative courses were approved and taught, the university assessed students' perceptions of these courses. We present the data from five terms of quantitative-intensive courses (49 different courses, 1400 students), with particular attention to comparisons between statistics courses (4 different courses, 235 students) and other quantitative courses. Data from these courses are used to explore whether students perceive quantitative courses as building QL skills that can be applied in new contexts, and whether particular types (e.g., statistics) or levels of study are more effective at developing transferable QL skills. We also administered to an algebra-based, introductory statistics class (n = 31), pre- and post-course attitudinal assessments that are based on the Dartmouth College Mathematics Across the Curriculum Survey (Korey 2000). These data allow us to explore how students' attitudes toward mathematics and statistics affect the development of QL skills.

2.1 Data on Quantitative-Intensive Courses

As shown in Appendix A, the student evaluation form for the quantitative-intensive courses focuses on student perceptions regarding opportunities to develop or improve QL, pedagogical techniques used to teach quantitative concepts, and whether the students had learned transferable skills. It should be noted that even though the dean requested that instructors of all quantitative-intensive courses administer the evaluation forms, compliance was less than perfect (e.g., for 2002–03 we have data from only 55% of the quantitative-intensive courses). Also, we summarize the student ratings by course level (Table 1) and discipline (Appendix B), though not by instructor. Consequently, disciplinary differences and variation within disciplines may also reflect student perceptions of particular instructors. With these cautions in mind, we present descriptive summaries of the data and interpretations below.

Table 1 summarizes the evaluation data according to the level of course: introductory (16 different courses, 787 students), lower (17 different courses, 414 students), or upper (16 different courses, 199 students). Introductory courses have no prerequisites, lower-level courses have at least one prerequisite and are at the 100 or 200 level (i.e. freshman and sophomore level), and upper-level courses are at the 300 or 400 level (i.e., junior and senior level). The quantitative requirement at Lawrence specifies that students take one quantitative-intensive course. Therefore, splitting the data in this way allows us to compare perceptions of students meeting the basic requirement to perceptions of students who elect to take additional quantitative courses. Given that many QRAC programs only require one or two quantitative courses, this comparison will inform our discussion of how statistics educators might facilitate the teaching of transferable skills in the context of such programs.

Looking at Table 1, most of the averages are above 7 (on a 10-point scale), indicating that students tended to evaluate the courses positively, although courses with prerequisites (both lower- and upper-level) were evaluated more positively than courses without prerequisites. Given our focus on QL, we highlight the items most relevant to the development of QL skills: opportunities to develop quantitative reasoning, feedback on quantitative work, learning concepts or skills that can be applied in other courses or that have practical applications, opportunities to explain reasoning, and amount and helpfulness of instruction on quantitative skills. To aid our interpretations, Appendix B summarizes the data by course level, as well as by department.

In terms of opportunities to develop quantitative reasoning and usefulness of feedback on quantitative work, ratings significantly improved as level of course increased. A pleasant exception to the student tendency to rate introductory courses lower was the finding that introductory statistics and introductory computer science were particularly strong in these areas (see Appendix B). Statistics courses with prerequisites (at both the lower- and upper-level) were also rated very highly in terms of opportunities to develop quantitative reasoning (lower-level courses: mean = 8.8, upper-level courses: mean = 9.1) and feedback (lower-level courses: mean = 8.8, upper-level courses: mean = 9.0).

Table 1. Course Evaluation Data, Summarized by Level of Course

Question	Level of Course	Sample Size	Mean	Std. Dev.
2. Usefulness of Feedback	Introductory Lower Upper	776 404 195	7.24 7.80 8.27	2.01 1.82 1.64
4. QR Opportunities	Introductory Lower Upper	769 404 193	7.17 7.79 8.34	2.10 1.90 1.73
5. Course Applications	Introductory Lower Upper	784 408 198	6.75 8.00 7.88	2.35 1.96 2.22
6. Practical Applications	Introductory Lower Upper	780 407 194	6.88 7.71 7.61	2.14 2.13 2.30
7. Explaining Reasoning	Introductory Lower Upper	782 408 197	7.07 8.31 8.75	2.34 1.86 1.65
8. QR Improvement	Introductory Lower Upper	784 408 196	6.35 7.31 7.66	2.19 1.86 1.89
9. QR Help Opportunities	Introductory Lower Upper	772 406 196	7.29 7.92 8.13	2.20 1.92 2.01
10. Clarity of Guidelines	Introductory Lower Upper	765 406 194	7.52 8.14 8.30	2.05 1.69 1.74
11a. Explicit Instruction	Introductory Lower Upper	764 408 193	7.08 7.76 7.82	2.12 1.90 1.98
11b. Helpfulness	Introductory Lower Upper	760 405 193	7.43 7.95 8.19	2.10 1.89 1.93
14-15. Difference in Skills	Introductory Lower Upper	739 390 172	1.37 1.02 0.82	1.48 1.08 1.41

NOTE: ANOVA analyses (by course level) for all questions showed significant results. In pairwise comparisons (using 0.05 as Tukey's family error rate) the introductory course mean was significantly different from both the lower-level and upper-level means for all questions. The lower-level and upper-level means were only significantly different on questions 2, 4, and 7.

In terms of applications to other courses or practical applications, the means in the introductory courses were significantly lower than the means in the courses with prerequisites. However, when the introductory data were broken down by department, some encouraging differences again emerged. In terms of applications to other courses, students rated statistics (mean = 7.5), computer science (mean = 7.5), and mathematics (mean = 7.4) significantly more highly than other introductory courses (t = 6.86, p‐value = 0.00). Similarly, statistics (mean = 7.9) and computer science (mean = 7.9) were rated significantly more highly than other introductory courses in terms of practical applications (t = 8.96, p‐value = 0.00). Interestingly, statistics (mean = 9.4) was rated significantly higher than mathematics (mean = 7.2) in terms of practical applications at both the lower (t = 8.09, p‐value = 0.00) and upper level (t = 5.59, p‐value = 0.00). As shown in Appendix B, this pattern also held for course applications. These findings are consistent with the argument that advanced math is increasingly abstract, whereas statistics emphasizes conceptual application, contributing to QL (Steen 2001).

Ratings of opportunities to explain reasoning also varied significantly with level of course. However, introductory statistics (mean = 7.1) did not stand out as particularly strong, whereas introductory physics (mean = 8.7) was rated strikingly higher than any other introductory course. For courses with prerequisites, statistics was rated highly in terms of explaining reasoning (lower-level courses: mean = 8.9, upper-level courses: mean = 9.0), as were most courses with prerequisites. The differences between introductory and higher-level courses in opportunities to explain reasoning may be accounted for by the smaller class sizes in upper-level courses. Nevertheless, if the goal is to improve QL, these results suggest that an area for possible improvement is to encourage students to monitor and explain their own reasoning at all course levels.

In terms of explicit instruction on quantitative skills and helpfulness of that instruction, introductory courses were rated significantly lower than courses with prerequisites. However, introductory statistics was rated significantly higher than other introductory courses in both the amount (t = 9.07, p‐value = 0.00) and helpfulness (t = 6.80, p‐value = 0.00) of quantitative instructions, and comparable to ratings of courses with prerequisites. For courses with prerequisites, statistics also stood out as particularly strong in these areas (see Appendix B).

Perhaps not surprisingly, when students were asked to rate the extent to which their overall quantitative skills improved due to a course, introductory courses were rated significantly lower than other courses. However, when asked to assess their initial level of quantitative skills and their skills at the end of the course (questions 14 and 15), the average improvement was significantly greater for students in introductory courses than in other courses. Introductory students seem to recognize that their beginning quantitative skills are at a lower level and that they have improved significantly after the introductory course, yet perhaps they also recognize that there is substantial room for further improvement in their QL skills.

In summary, the data from the course evaluations suggest that students perceive introductory courses less positively than courses with prerequisites. However, statistics stood out as a pleasant exception to this rule in most cases. That is, based on student reports, introductory statistics helped students to learn transferable skills. However, these data provide no basis to assume students will actually go on to use their skills, especially if they are fulfilling the minimal quantitative requirement. A QRAC approach, especially in colleges with minimal quantitative requirements, seems particularly important because it provides opportunities for students to practice learned quantitative skills in other contexts. Furthermore, students fulfilling only the minimal requirement may have less confidence in their quantitative skills, implying that instructors across the curriculum, particularly those in introductory quantitative courses, may need to focus not only on improving students' quantitative reasoning skills, but also on bolstering students' confidence and interest in QL. We will explore this point below with attitudinal data from a statistics course.

2.2 Introductory Statistics: Attitudinal Data

The attitudinal assessment was done in an elementary statistics course, which is taught using the principles of the statistics reform movement (e.g., use of real data; stressing of concepts over formulas; use of Moore and McCabe's practice-based, introductory text). The pre-assessment was given during the first week of class, and the post-assessment was given during the last week of the 10-week term. The attitudinal survey contained 51 items, 35 from the Dartmouth study that focuses on mathematics and 16 items reworded to focus on statistics (see Appendix C). Items are rated on a 5-point scale where a 5 corresponds to “strongly agree” and a 1 corresponds to “strongly disagree.” Based on Dartmouth's factor analysis (Korey 2000), we created composite scores reflecting the identified four factors: confidence in mathematical or statistical competency, perception of the practical utility of mathematical or statistical concepts, belief that mathematics or statistics contributes to personal growth, and level of interest in pursuing further study in mathematics or statistics. Table 2 presents the pre- and post-course comparisons on these four subscales.

Table 2. Subscale Results from the Dartmouth Attitudinal Survey (n = 31)

Subscale	Pre-Course Mean (Standard Deviation)	Post-Course Mean (Standard Deviation)	P‐value (Based on Paired t‐test)
Confidence	23.32 (5.00)	25.19 (4.45)	0.038
Practical Utility	42.61 (5.90)	44.61 (6.38)	0.050
Personal Growth	30.52 (5.31)	32.42 (4.19)	0.018
Interest	12.58 (3.44)	12.87 (3.21)	0.519

NOTE: The confidence subscale includes items 2, 3, 16 (reversed), 22, 27 (reversed), 40 (reversed), and 46 (reversed). The practical utility subscale includes items 10, 11, 15, 19 (reversed), 23 (reversed), 26, 30, 38, 39, 42 (reversed), 44, and 47. The personal growth subscale includes items 1, 12, 20, 24, 25, 28, 32, 43, 45, 49. The interest subscale includes items 4 (reversed), 9, 17, 21 (reversed), and 41.

Overall, students showed significantly improved confidence in their mathematical/statistical abilities. Specific item analyses showed that while 48% of students initially reported nervousness about learning statistics, only 16% reported nervousness on the post-test. In addition, the percentage of students who felt confident about being good at mathematics nearly doubled from pre- (15%) to post-test (29%). Clearly, though, there is still room for improvement.

Student perceptions of the practical utility of mathematics and statistics also significantly improved from pre- to post-test. We were delighted to find that, at post-course, 84% of the students thought statistics helped them to understand the world and 74% noticed familiar statistical concepts in other courses. After taking the introductory statistics course, 90% of students agreed or strongly agreed with the statement, “After I”ve forgotten all the formulas, I'll still be able to use ideas I“ve learned in statistics.” This understanding of the conceptual applicability of statistical thinking mirrors the attitudinal goals of the QL movement.

Similarly, students were significantly more likely to endorse items about mathematics and statistics contributing to their personal growth on the post-test. For example, at post-course, 74% of students agreed that statistics raises interesting new questions about the world.

Unfortunately, in terms of interest in pursuing further study, there was no significant change from pre- to post-test. We found only 23% of the students wanted to study more statistics. That is, even when students became convinced of the applicability of statistics in other areas and were more confident in their skills, they were reluctant to pursue further study. Garfield and Ahlgren (1994) found similar results in their evaluation of the nationwide Quantitative Literacy Project (66% of students felt statistics was useful, but only 35% wanted to learn more statistics). This reluctance points to a variety of challenges, including the incredible difficulty people have with transferring quantitative skills across contexts (see, for example, Detterman and Sternberg 1993). It should be noted that none of the items on the attitudinal scale measured students' confidence in actually using statistical skills in the future. Although we saw increased confidence in mathematical and statistical skills following the introductory course, students may feel that transferring and applying those skills in new contexts is a difficult task.

2.3 Summary

Overall both the attitude and course evaluation data are promising, yet they also highlight the need for some pedagogical improvements in order to develop QL in college students. What is perhaps most striking is the mismatch between students' awareness of conceptual applications and their reluctance to pursue those applications. That is, students recognize that concepts they have learned in areas like statistics have many practical applications, yet simultaneously report low interest in pursuing further coursework that requires statistical or mathematical reasoning. This finding suggests that students may not be confident in their ability to transfer learned skills, or may not be motivated to devote the effort necessary for quantitative problem solving. Also striking is the fact that most introductory courses did not fare as well as courses with prerequisites in terms of opportunities to develop quantitative reasoning and in teaching course and practical applications. The encouraging exception was statistics, suggesting that statistics educators may be in a good position to help strengthen other introductory courses and build a QRAC program. In order to help statistics educators prioritize pedagogical goals for a QRAC approach, we need to understand what promotes and what impedes successful transfer of skills across contexts.

3. Review of the Transfer Research Literature: Challenges for Educators

We define transfer as the ability to appropriately and accurately apply quantitative reasoning to new problems and in new contexts. This definition implies that teaching transfer has at least three challenges. The first is to help students master concepts and learn skills they can apply in future situations where quantitative problem solving is explicitly required. The second challenge is to motivate students to engage in analytical quantitative reasoning, rather than being satisfied with superficial processing of quantitative problems. The final challenge is to help students to develop metacognitive skills that allow them to identify real-world situations that logically demand quantitative reasoning and to avoid biases while engaging in the reasoning process. The research literature does not address all these practical challenges in teaching, but it certainly informs the discussion of how statistics educators might promote transfer and contribute to a QL program.

3.1 General Transfer Research

Despite a century's worth of research on transfer of learning, no general consensus exists about what constitutes transfer, the extent to which it occurs, and the mechanisms that account for transfer (Barnett and Ceci 2002). One of the more complicating issues is what constitutes transfer: must learners recognize the underlying principle on their own or can they be cued to apply a learned principle? Laboratory research often finds that people do not recognize analogous problem solving situations unless the analogy is explicitly pointed out to them (Detterman 1993). Some researchers question whether performance should be considered successful transfer if the student must be told to use a particular principle. Consequently, a challenge for educators is to ascertain what helps learners identify underlying principles so the analogy is apparent. If in a QRAC approach teachers explicitly point out underlying conceptual or formal similarities between quantitative problems, perhaps students will begin to recognize analogous problems on their own.

Furthermore, the ability to identify analogous underlying formalisms and concepts is greatly affected by learners' familiarity with an area. Familiarity affects whether people dig deep and identify underlying principles or get caught in the surface features of the problem. For example, Chi, Feltovich, and Glaser (1981) showed that when physics experts were asked to judge the similarity of physics problems, they did so on the basis of formulaic or structural similarity (e.g., problems that involve torque). In contrast, novices judged similarity based on the surface structure of the problems (e.g., mention of an inclined plane). Similarly, Reed and Evans (1987) found that college students did more poorly on acid-concentration mixture problems than on a more familiar temperature-of-water mixture task, even though the problems were isomorphic, requiring the same weighted-average principle for solution. More encouragingly, college students who were trained with the temperature prediction problems were able to apply the principle to solve the acid-concentration mixture problems, if given the simple hint that the principles were the same in both cases. Because of students' lower levels of familiarity and expertise within many areas, we hypothesize that promoting transfer will necessitate explicit instruction on recognizing the underlying formalisms and concepts in problems, using a common quantitative language across disciplines.

Other issues involved in understanding transfer include a) the degree of similarity between the context in which a principle is learned and the new context in which it should be applied, and b) the amount of time between initial learning and subsequent application. Barnett and Ceci (2002) offer a taxonomy for judging the extent to which far transfer (i.e., transfer to dissimilar contexts) occurs. They demonstrate in their review that very little of the lab research on transfer has used stringent transfer tests that require the kind of flexible, sophisticated reasoning and application of principles over a long period of time that educators desire and QL requires. Much educational research is also guilty of looking only at the transfer of isolated concepts on tests that occur very near the time of training (Lovett and Greenhouse 2000).

From an educational perspective, one productive line of research explores the cumulative effects of disciplinary training on QL. For example, Lehman and Nisbett (1990) examined the effects of disciplinary undergraduate training on conditional reasoning, and statistical and methodological reasoning. They found that social science training led to substantial improvements in statistical and methodological reasoning, whereas natural science and humanities training produced improvements in conditional reasoning and smaller effects on statistical reasoning. This research provides an encouraging start for a QRAC program and also points out the importance of identifying the QL concepts students should have in their repertoire and in what areas of the curriculum those concepts are being taught. However, this research does not address the specific pedagogies and experiences that lead to improvement. Consequently, the mechanisms necessary for fostering transfer of concepts across contexts still need to be ascertained.

Statistics education research has begun to describe such mechanisms by identifying pedagogical techniques that promote student learning (Garfield 1995; delMas, Garfield, and Chance 1999). Some important educational implications have emerged from this work. For example, delMas, et al. (1999) demonstrated that helping students to explicitly identify and confront their statistical misconceptions led to better statistical reasoning about sampling distributions when working with computer simulation demonstrations. Research in math education also supports this conclusion. Shaughnessy (1977) highlighted the importance of students articulating their misconceptions in learning finite mathematics through activity-based pedagogy. This research implies that teaching students to metacognitively monitor their reasoning process and thereby identify possible misconceptions is an integral part of promoting QL.

Students' tendency to categorize concepts they have learned as either domain-specific or as broadly applicable presents another challenge in promoting transfer. Bassok and Holyoak (1989) gave high school and college students training in solving isomorphic algebra (arithmetic progression word problems) problems or physics (motion in a straight-line with constant acceleration) problems. After training in one domain, they looked at transfer to the other domain. They found that algebra training, even if it involved training using word problems in a specific domain (e.g., money: salary increases or loans; motion, i.e., physics problems of bodies moving in a straight line with constant acceleration), led to successful application of the principles to physics problems. However, training in physics, with physics materials and units, did not lead to the solving of isomorphic algebra problems. They interpreted the findings as reflecting the students' beliefs that algebra is content free and can be applied across a variety of domains. Physics, on the other hand, is seen as more content bound—specified units are presented, leading students to conclude that physics equations are constrained to particular domains (e.g., motion concepts). Statistics educators can help address this problem of students too narrowly defining concepts. By leading across-the-curriculum discussions, statistics educators can help faculty to identify quantitative concepts taught in their disciplines and to create a shared language for describing these concepts.

A final obstacle in promoting transfer lies in understanding the extent to which the problem-solving context triggers reasoning biases. Reasoning biases occur for a variety of reasons not limited to but including invalid intuitive understanding based on practical experience, a tendency to favor evidence that confirms one's own hypotheses, and a tendency to under- or overestimate or ignore population base rates based on one's own limited experience. The extensive research on reasoning biases (e.g., Kahneman, Slovic, and Tversky 1982; Klaczynski and Gordon 1996) highlights the importance of determining not only what promotes transfer but also what may interfere with it. For example, the representativeness heuristic involves judging the probability of some event on the basis of the extent to which the event represents or resembles the expected and intuitively incorrect outcome, instead of considering the prior probability of outcomes. In their classic work, Tversky and Kahneman (1982a) asked people to judge the probability that someone was a librarian, farmer, salesman, pilot, or physician given the following description, “Steve is very shy and withdrawn, invariably helpful, but with little interest in people, or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.” People routinely estimated the probability that Steve is a librarian to be quite high, no doubt based on their stereotypes, and ignored the relevant base-rate information they were given (i.e., there were many more farmers than librarians) that should enter into any reasonable estimate. When not given the personality description, participants did use base-rate information correctly. Research on reasoning biases suggests that people are more likely to draw on statistical principles when making objective decisions or dealing with explicitly quantitative information than when making decisions about equally uncertain social situations or dealing with less easily quantified information. Statistics educators must help students to attend to base rates and quantitative information even when the linguistic demands in the problem encourage more subjective assessments.

Another interesting bias, the self-serving and confirmatory bias, refers to the tendency to treat evidence consistent with one's beliefs or goals more favorably than inconsistent evidence (see Haines and Moore 2003 for a review). Klaczynski and Gordon (1996) found that students judged goal-enhancing evidence to be more convincing than neutral or goal-threatening evidence, but interestingly, they were more likely to use statistical reasoning when presented with goal-threatening problems. That is, threatening evidence seemed to stimulate more sophisticated statistical reasoning. The good news is that transfer is more likely when people are engaged or feel their beliefs are challenged, and the bad news is that people require little empirical evidence if their beliefs are confirmed. Statistics educators can play a central role in encouraging students to routinely examine the quantitative evidence.

3.2 Statistics Transfer Research

Although research on reasoning biases gives somewhat discouraging impressions of people's understanding of probability (see, for example, Tversky and Kahneman 1982b; Konold 1989), recent transfer research on statistical training provides a reason for guarded optimism. Fong, Krantz, and Nisbett (1986) tested whether subjects understood the concept that the larger a random sample is, the better an estimate it is of the population (they refer to this concept as the “law of large numbers”). They found that brief formal training on the law of large numbers was effective in both increasing the use of statistical reasoning in everyday problems and increasing the quality of the statistical reasoning. This indicates that statistical training can lead to the acquisition of general rules that may be broadly applied. The researchers also assessed the law-of-large-numbers knowledge of students in an introductory statistics course (pre- and post-course), using the guise of a telephone opinion survey on sports. They found that training in the statistics course had a significant effect in enhancing the use of statistical reasoning on the survey questions. Such transfer of statistical knowledge to everyday problems is an important goal in the QL movement.

Kosonen and Winne (1995) extended the work of Fong and colleagues. They trained groups of undergraduates on concepts associated with sampling and the law of large numbers as well as on what kinds of reasoning errors to avoid (truism, egocentric bias based on their own experience, attribution to character's disposition and under-using objective data, and speculation—adding data to the problem or generating a hypothesis not founded on the problem's description). They found that training on avoiding biases helped students with the kinds of reasoning problems where they had the most difficulty (e.g., using base-rate information rather than dispositions when making judgments). The researchers speculated that the success of this brief training “can be attributed, in part, to statistical heuristics that were already but inadequately part of the students' repertoires for reasoning” (Kosonen and Winne 1995, p. 44). Hence, students may develop and use statistical heuristics, some correct and some incorrect, before entering a formal statistics class. A key finding from Bransford, et al. (2001, pp. 14–15) synthesis of the learning literature is the importance of addressing students' preconceptions: “If their initial understanding is not engaged, they may fail to grasp the new concepts and information that are taught, or they may learn them for purposes of a test but revert to their preconceptions outside the classroom.” Consequently, correcting and refining student intuition may, in fact, be more productive than simply teaching a concept from scratch. These research findings are encouraging because they demonstrate that students can be successfully trained to avoid reasoning biases.

Lovett (2001) did a series of studies on college students' reasoning about statistics using an approach that integrates cognitive theory as well as methods and findings from applied research on statistics education. Her team drew on cognitive theory to model how students learn statistical reasoning, and then tested the model empirically using both laboratory and statistics classroom-based designs. This approach allowed them to identify areas where statistics students showed good mastery (e.g., interpreting descriptive and inferential statistics), and diagnose specific areas of difficulty (e.g., choosing appropriate statistical approaches and drawing conclusions from statistical analyses). They used this information to design a computerized learning environment to help students overcome these difficulties. Students, even those with no prior statistical training, showed substantial improvement after a brief (45 minute) computerized interactive session, especially if they received specific feedback throughout their problem-solving attempts (as opposed to feedback at the end). Lovett's research demonstrates that detailed analysis of student learning can be used to successfully and specifically modify learning environments to provide practice and guidance in difficult areas, thereby helping to establish more comprehensive, transferable skills.

3.3 Summary and Implications

The significant body of research on transfer establishes that teaching students to apply skills in new contexts is very challenging. However, the statistics transfer research and the college curriculum research is more encouraging, although there is clearly room for improvement. The research also suggests that in order to achieve the type of transfer essential to QL, educators will need to reinforce concepts across as many contexts as possible and offer opportunities to practice quantitative problem solving as frequently as possible, as is done in a QRAC approach. Because of its focus on broadly applicable concepts and reasoning skills, statistical training can play a central role in a QL curriculum. If statistics educators determine how to pedagogically implement the suggestions derived from the transfer research, then the QL movement will be greatly strengthened.

4. Suggestions for Statistics Educators

In this paper, we have advocated an across-the-curriculum approach to developing QL. The transfer research strongly demonstrates that transfer and application of conceptual skills across contexts is very difficult for learners. In order for students to develop these sophisticated QL skills, teachers need to explicitly label, model, and reinforce them repeatedly in multiple contexts. For example, if students learn appropriate ways of evaluating sampling processes in their statistics course, which are then explicitly reinforced in subsequent history and psychology courses, perhaps they will be more likely to draw on these conceptual skills when faced with a sampling problem in, for example, a presidential election. Statistics educators can play a central role in establishing a QRAC approach to QL. As previously mentioned, our data indicate that students rate highly the practical applicability of statistics, even after a single introductory course. On the other hand, students had a difficult time identifying practical applications following instruction in other disciplines, such as geology or chemistry. Because statistics naturally contains broadly applicable concepts (and students see this), statistics teachers can create a model for teaching conceptual transfer across contexts. We list some suggestions below, in the hope of beginning a dialogue about the role of statistics training in developing QL curricula in higher education.

• Statistics educators can lead discussions to identify key QL concepts and create a shared vocabulary for labeling those concepts. Because statistics is used in so many fields and statisticians routinely consult with colleagues across the curriculum, statisticians are uniquely poised to articulate how similar statistical concepts are used in different contexts. Unless educators create a shared language and explicitly label concepts, the transfer research shows that students' lack of familiarity with a discipline is likely to impede sophisticated quantitative reasoning. Similarly, Lovett (2001) emphasizes the importance of creating a common language for statistics concepts across disciplines. Utts (2003) and Garfield (1995) have begun the task of identifying key statistical concepts to include in the QL repertoire.

• Statistics educators can teach students metacognitive skills for monitoring and verbalizing their reasoning processes. The statistics reform movement advocates active learning and the use of real data, yet discovery exercises alone do not always lead to a deep conceptual understanding that transfers successfully to new contexts. As our data demonstrate, introductory students often have fewer opportunities to articulate their reasoning, which may impede the development of deep conceptual understanding. Statistics teachers can help students derive the underlying concepts and principles, rather than focusing exclusively on the surface features of the problem. If this is done across the curriculum, students will have the opportunity to acquire the metacognitive skills and the language to articulate their own reasoning and conceptual understanding. Assessment data will be needed on actual QL skills to determine whether developing metacognitive skills actually promotes transfer.

• Statistics educators can teach students to do a bias check when solving quantitative problems. The research on reasoning biases and decision-making clearly establishes that everyone is prone to biases, like the representativeness heuristic and confirmation bias. Besides instruction on correct problem-solving approaches, students need tools to check for reasoning traps and faulty intuitions that could lead them astray.

• Statistics educators can help students to establish positive attitudes about quantitative problem solving and hence increase confidence about QL across disciplines. In our pre-post assessment of introductory statistics students' attitudes, two findings consistent with the math anxiety literature emerged and simultaneously provided encouragement that this anxiety can be conquered. First, in the pre-test, students rated themselves as not particularly good at mathematics. Their ratings significantly improved at the conclusion of the course. Student ratings of the extent to which statistics made them nervous also significantly improved following the introductory course. Although the mechanisms for the improvement are not clear, the data suggest that statistical education can help to build student confidence and improve attitudes about statistics.

• Statistics educators can develop pedagogies that include more opportunities with open-ended problems that have multiple solutions, allowing students to focus more on the reasoning process than on getting a particular right answer. If students are encouraged to compare and contrast multiple problems and multiple solution paths, they are more likely to acquire deeper, and consequently more transferable, conceptual understanding of the underlying principles (Lovett and Greenhouse 2000). Data-driven projects, like those now used in statistics courses, may be a useful way for students to work with open-ended problems

• Statistics educators can teach faculty how to assess the effectiveness of pedagogies used to teach quantitative skills. The recent research from the statistics education reform movement provides an excellent model for designing and interpreting classroom research (see, for example, delMas, et al. 1999). We recommend assessing students' understanding of statistical principles and willingness to engage in quantitative problem solving in new contexts, as well as their metacognitive ability to recognize when statistical reasoning is needed and to evaluate the validity of their own reasoning processes.

• Statistics educators can provide a support network for faculty interested in incorporating quantitative reasoning into their courses. Besides stimulating a general dialogue about QL goals across the curriculum, statistics teachers can offer individual consultation to help faculty develop particular pedagogies for teaching QL in their discipline. Ideas from the statistics reform movement (e.g., active learning, use of real data) can be extended and applied to other disciplines. These types of individual consultations create an opportunity for faculty who do not typically teach in a quantitative area to address any concerns they have and to articulate their own questions about quantitative concepts.

The process of developing an across-the-curriculum QL program actually models the type of quantitative reasoning we want students to acquire. As faculty identify key quantitative concepts and explain their applications to a variety of contexts, they model the kind of cognitive risk taking and problem solving that is central to QL. Such a dialogue allows faculty to make explicit their conceptual understandings and confront any misconceptions they may hold. Statistics educators can create a positive environment for this dialogue, which may increase teachers' confidence in their own quantitative skills and their ability to teach these skills to their students. In this way, statistics educators can help the QL movement gain momentum and achieve the kind of status and success that writing-across-the-curriculum programs have attained in higher education. In our increasingly technological world, QL is certainly as important to students as is good writing.

Appendix A —

Student Evaluation Form for Quantitative-Intensive Courses

Course Title & Number: ______________________ Term/Year: _________ 1. What types of quantitative work did you do in this class? Symbolic proofs? Never Sometimes Frequently Statistical analysis of data? Never Sometimes Frequently Interpretation of graphs? Never Sometimes Frequently Problem solving? Never Sometimes Frequently Computer programming? Never Sometimes Frequently Other _____________________________________________ 2. Usefulness of feedback on quantitative work 1 2 3 4 5 6 7 8 9 10 Not Useful Somewhat Very Useful 3. Did you receive feedback on quantitative work other than on quizzes or exams? No Yes On what? _____________________________ 4. Opportunities to develop quantitative reasoning (for example, but not limited to, analyzing evidence, detecting fallacies in reasoning, questioning assumptions and conclusions) 1 2 3 4 5 6 7 8 9 10 None Some Many 5. Did you learn concepts or quantitative skills that you will apply in other courses? 1 2 3 4 5 6 7 8 9 10 Not at all Some Many If so, please give examples: 6. Did you learn concepts or quantitative skills that have practical applications? 1 2 3 4 5 6 7 8 9 10 Not at all Some Many If so, please give examples: 7. How often were you asked to explain the reasoning behind your work? 1 2 3 4 5 6 7 8 9 10 Never Sometimes Very Often 8. Extent to which your overall quantitative skills improved due to this course 1 2 3 4 5 6 7 8 9 10 Not at all Somewhat Very Much 9. Opportunities for individual help on your quantitative skills 1 2 3 4 5 6 7 8 9 10 None Some Many 10. Clarity of guidelines and expectations for quantitative work 1 2 3 4 5 6 7 8 9 10 Not Clear Somewhat Very Clear 11. Amount of explicit instruction on quantitative skills 1 2 3 4 5 6 7 8 9 10 None Some Very Much 12. Helpfulness of the instruction on quantitative skills? 1 2 3 4 5 6 7 8 9 10 Not Helpful Somewhat Very Helpful 13. Please comment on how this course has helped you to develop your quantitative skills. What aspects of the course were particularly helpful? 14. Did you use the quantitative tutoring services of the Center for Teaching and Learning? Yes No If yes, how often? 15. To what extent did you find the assistance helpful? 1 2 3 4 5 6 7 8 9 10 Not at all Somewhat Very Helpful 16. Rate your quantitative skills at the beginning of the course. 1 2 3 4 5 6 7 8 9 10 Poor Average Excellent 17. Rate your quantitative skills at the end of the course. 1 2 3 4 5 6 7 8 9 10 Poor Average Excellent

Appendix B —

Course Evaluation Data, Summarized by Department and Level of Course

Table

		Mean (Standard Deviation, n)
Question	Department	Intro	Lower	Upper
2. Usefulness of Feedback	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	6.2 (2.1, 106) 7.2 (1.7, 115) 7.8 (1.8, 61) 6.8 (2.3, 120) 6.3 (2.1, 72) 7.4 (1.6, 108) 7.8 (1.6, 47) 8.2 (1.7, 141)	NA 7.8 (1.3, 66) 7.4 (3.1, 12) 7.7 (2.1, 27) NA 7.3 (2.1, 122) 7.7 (1.6, 98) 8.8 (1.3, 69)	NA NA 8.4 (1.4, 9) 8.1 (1.6, 72) 8.3 (1.4, 13) 8.2 (2.0, 59) 8.3 (1.3, 19) 9.0 (1.0, 23)
4. QR Opportunities	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	6.6 (2.1, 108) 7.2 (1.9, 114) 8.2 (2.4, 60) 6.7 (2.0, 121) 6.0 (2.0, 73) 6.8 (2.1, 103) 7.5 (1.8, 46) 8.2 (1.7, 138)	NA 7.7 (1.4, 66) 7.2 (2.4, 12) 7.4 (2.2, 26) NA 7.3 (2.1, 121) 7.9 (1.8, 99) 8.8 (1.4, 70)	NA NA 7.6 (1.9, 9) 7.9 (1.8, 71) 8.3 (1.0, 13) 8.6 (1.8, 58) 8.9 (1.7, 20) 9.1 (1.2, 22)
5. Course Applications	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	5.4 (2.3, 109) 7.1 (1.7, 115) 7.5 (2.2, 61) 6.7 (2.1, 122) 5.3 (2.6, 73) 7.4 (2.3, 109) 6.9 (2.2, 48) 7.5 (2.3, 142)	NA 7.7 (1.6, 66) 7.4 (2.5, 12) 7.4 (2.5, 27) NA 7.8 (2.1, 124) 8.0 (1.9, 99) 9.0 (1.5, 70)	NA NA 6.1 (2.8, 9) 7.7 (2.0, 73) 7.2 (2.4, 13) 7.7 (2.6, 59) 9.0 (1.1, 21) 8.9 (1.4, 23)
6. Practical Applications	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	5.5 (2.1, 109) 6.7 (1.8, 114) 7.9 (1.8, 59) 7.1 (2.0, 122) 6.1 (2.1, 73) 6.8 (2.4, 108) 7.1 (2.1, 48) 7.9 (1.8, 142)	NA 7.2 (2.0, 66) 8.3 (2.2, 11) 7.3 (2.3, 27) NA 7.2 (2.3, 123) 7.7 (2.1, 100) 9.1 (1.1, 70)	NA NA 7.4 (2.5, 9) 7.4 (2.2, 72) 7.9 (1.5, 13) 7.2 (2.6, 59) 7.2 (2.2, 18) 9.4 (1.0, 23)
7. Explaining Reasoning	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	7.0 (2.4, 110) 7.3 (2.1, 115) 5.9 (2.5, 60) 7.2 (2.3, 120) 6.0 (2.5, 73) 7.2 (2.2, 109) 8.7 (1.7, 48) 7.1 (2.3, 141)	NA 7.6 (2.0, 66) 6.8 (1.7, 12) 7.6 (2.1, 27) NA 8.1 (2.0, 124) 8.9 (1.6, 99) 8.9 (1.3, 70)	NA NA 6.9 (2.2, 9) 8.4 (1.7, 72) 8.6 (1.2, 13) 9.2 (1.4, 59) 9.3 (2.0, 21) 9.0 (0.9, 23)
8. QR Improvement	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	5.1 (2.4, 110) 6.7 (1.7, 113) 6.7 (2.1, 61) 6.2 (2.3, 123) 4.9 (2.3, 73) 7.0 (1.7, 109) 7.0 (2.0, 48) 7.1 (1.9, 141)	NA 7.1 (1.6, 66) 6.9 (1.8, 12) 6.5 (2.3, 27) NA 6.9 (1.9, 123) 7.4 (1.8, 100) 8.4 (1.5, 70)	NA NA 7.4 (2.3, 9) 7.5 (2.0, 72) 7.2 (1.7, 13) 7.8 (1.9, 58) 7.8 (2.0, 21) 8.0 (1.3, 23)
9. QR Help Opportunities	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	6.3 (2.4, 106) 7.5 (2.0, 114) 7.2 (2.0, 59) 6.5 (2.4, 121) 7.0 (2.3, 71) 7.5 (1.9, 108) 8.2 (2.0, 48) 8.2 (1.8, 139)	NA 8.2 (1.6, 65) 7.0 (1.5, 11) 7.1 (2.4, 27) NA 7.6 (2.3, 123) 7.6 (1.6, 100) 9.1 (1.1, 70)	NA NA 6.9 (2.2, 8) 7.4 (2.0, 73) 8.5 (1.3, 13) 8.7 (2.1, 59) 8.4 (1.7, 20) 9.0 (1.4, 22)
10. Clarity of Guidelines	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	6.9 (2.1, 109) 7.5 (1.8, 114) 7.8 (2.2, 60) 6.8 (2.2, 114) 6.7 (2.3, 72) 7.8 (1.7, 108) 8.0 (1.6, 47) 8.5 (1.7, 135)	NA 7.9 (1.3, 66) 7.5 (1.9, 11) 7.6 (2.3, 28) NA 7.9 (2.0, 122) 8.2 (1.5, 101) 9.0 (1.2, 68)	NA NA 7.3 (2.4, 9) 7.9 (1.6, 70) 8.5 (1.2, 13) 8.4 (2.1, 59) 8.8 (1.1, 20) 9.2 (1.0, 23)
11a. Explicit Instruction	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	6.0 (2.3, 109) 7.3 (1.8, 114) 7.1 (2.3, 60) 6.4 (2.2, 114) 6.3 (1.9, 72) 7.5 (2.0, 107) 7.5 (1.8, 47) 8.3 (1.6, 135)	NA 7.6 (1.6, 66) 6.8 (1.9, 12) 7.7 (2.1, 28) NA 7.8 (1.9, 123) 7.2 (2.1, 101) 9.0 (1.2, 68)	NA NA 7.2 (2.3, 9) 7.6 (1.8, 69) 7.8 (1.2, 13) 8.1 (2.2, 59) 7.1 (2.5, 20) 9.0 (1.1, 23)
11b. Helpfulness	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	6.6 (2.3, 106) 7.6 (1.8, 114) 7.4 (2.2, 60) 6.9 (2.2, 113) 6.5 (2.3, 71) 7.7 (1.9, 107) 7.9 (1.7, 47) 8.4 (1.8, 136)	NA 7.9 (1.6, 66) 7.0 (2.2, 12) 7.9 (2.0, 28) NA 7.4 (2.3, 122) 7.8 (1.6, 99) 9.2 (1.0, 68)	NA NA 7.6 (2.4, 9) 7.9 (1.8, 69) 8.0 (1.5, 13) 8.3 (2.3, 59) 8.4 (1.6, 20) 9.1 (1.1, 23)
14-15. Difference in Skills	Anthropology Chemistry Computer Science Economics Geology Mathematics Physics Statistics	1.0 (1.5, 105) 1.1 (1.4, 114) 1.1 (1.3, 57) 1.3 (1.4, 113) 0.7 (1.8, 69) 1.8 (1.2, 107) 1.5 (1.1, 43) 2.0 (1.5, 127)	NA 1.2 (1.1, 58) 0.8 (1.0, 12) 0.5 (1.1, 26) NA 1.1 (1.2, 123) 0.8 (0.9, 100) 1.2 (0.9, 61)	NA NA 1.2 (1.3, 5) 1.1 (1.0, 63) 1.0 (1.0, 12) 0.6 (1.6, 53) 0.4 (2.5, 18) 0.6 (0.7, 21)

Appendix C–

Dartmouth Attitudinal Survey Items (Korey 2000)

NOTE: Items 36–51 replace the term “mathematics” with “statistics” for selected items. All items are rated on a 5-point scale (1 — strongly disagree, 2 — disagree, 3 — neutral, 4 — agree, 5 — strongly agree).

1. To understand mathematics I sometimes think about my personal experiences.

2. I am good at mathematics.

3. If I work at it, I can do well in mathematics.

4. Most subjects interest me more than mathematics.

5. Mathematics is essentially an accumulation of facts, rules, and formulas to be memorized and used.

6. Good mathematics teachers show students the exact way to answer the questions they—ll be tested on.

7. Using a computer makes learning mathematics more complicated than it needs to be.

8. People who are good at mathematics can do mathematics quickly.

9. I enjoy learning new things in mathematics.

10. Mathematics helps me understand the world around me.

11. Mathematics has been an important tool to help me learn other subjects.

12. I have taken some mathematics courses in high school and college that were taught in a very interesting way.

13. For me, mathematics rarely involves exploration, investigation, or experimentation.

14. I like exploring problems using real data and computers.

15. Many situations in the world around me can be modeled mathematically.

16. I often feel like I'm missing something important in mathematics class.

17. I want to study more mathematics.

18. Working in groups helps me learn mathematics.

19. I rarely encounter situations that are mathematical in nature outside of school.

20. Doing mathematics helps me understand myself.

21. I try to avoid courses that involve mathematics.

22. When I get stuck on a mathematics problem, I can usually find my way out.

23. Becoming more proficient in mathematics prepares you for the next mathematics class, but that's about all.

24. Writing about mathematics makes it easier to learn.

25. In mathematics you can be creative and discover things for yourself.

26. After I've forgotten all the formulas, I'll still be able to use ideas I've learned in mathematics.

27. I'm never sure my answer is right until I'm given the solution.

28. Doing mathematics raises interesting new questions about the world.

29. Learning mathematics makes me nervous.

30. I often see familiar mathematical concepts in courses outside of mathematics.

31. Doing mathematics helps me think clearly and logically.

32. Mathematical thinking helps me make intelligent decisions about my life.

33. I don't really understand mathematics until I work it out for myself.

34. Expressing scientific concepts in mathematical equations just makes them more confusing.

35. I don't need a good understanding of mathematics to achieve my career goals.

36. Good statistics teachers show students the exact way to answer the questions they'll be tested on.

37. Using a computer makes learning statistics more complicated than it needs to be.

38. Statistics helps me understand the world around me.

39. Statistics has been an important tool to help me learn other subjects.

40. I often feel like I'm missing something important in statistics class.

41. I want to study more statistics.

42. I rarely encounter situations that are statistical in nature outside school.

43. In statistics you can be creative and discover things for yourself.

44. After I've forgotten all the formulas, I'll still be able to use instead I've learned in statistics.

45. Doing statistics raises interesting new questions about the world.

46. Learning statistics makes me nervous.

47. I often see familiar statistical concepts in courses outside of statistics.

48. Doing statistics helps me think clearly and logically.

49. Statistical thinking helps me make intelligent decisions about my life.

50. I don't really understand statistics until I work it out for myself.

51. I don't need a good understanding of statistics to achieve my career goals.