Puzzles in Statistical Reasoning

Figures & data

Table 1: SRA Correct reasoning scales and misconceptions scales; based on Garfield (2003)

Correct Reasoning Scales:
CC1:	Correctly interprets probabilities. Assesses the understanding and use of ideas of randomness, chance to make judgments about uncertain events.
CC2:	Understands how to select an appropriate average. Assesses the understanding what measures of center tell about a data set, and which are best to use under different conditions.
CC3:	Correctly computes probability, both understanding probabilities as ratios, and using combinatorial reasoning. Assesses the knowledge that in uncertain events not all outcomes are equally likely, and how to determine the likelihood of different events using an appropriate method.
CC4:	Understands independence.
CC5:	Understands sampling variability
CC6:	Distinguishes between correlation and causation. Assesses the knowledge that a strong correlation between two variables does not mean that one causes the other.
CC7:	Correctly interprets two-way tables. Assesses the knowledge how to judge and interpret a relationship between two variables, knowing how to examine and interpret a two way table.
CC8:	Understands the importance of large samples. Assesses the knowledge of how samples are related to a population and what may be inferred from a sample; knowing that a larger, well chosen sample will more accurately represent a population; being cautious when making inferences made on small samples.
Misconception scales:
MC1:	Misconceptions involving averages. This category includes the following pitfalls: averages are the most common number; failing to take outliers into consideration when computing the mean; comparing groups on their averages only; and confusing mean with median.
MC2:	Outcome orientation. Students use an intuitive model of probability that lead them to make yes or no decisions about single events rather than looking at the series of events; see Konold (1989).
MC3:	Good samples have to represent a high percentage of the population. Size of the sample and how it is chosen is not important, but it must represent a large part of the population to be a good sample.
MC4:	Law of small numbers. Small samples best resemble the populations from which they are sampled, so are to be preferred over larger samples.
MC5:	Representativeness misconception. In this misconception the likelihood of a sample is estimated on the basis how closely it resembles the population. Documented in Kahneman, Slovic, & Tversky (1982).
MC6:	Correlation implies causation.
MC7:	Equiprobability bias. Events of unequal chance tend to be viewed as equally likely; see Lecoutre (1992).
MC8:	Groups can only be compared if they have the same size.

Table 2: Components and scales of the Inventory of Learning Styles

Processing strategies	Regulation strategies	Learning orientations	Mental models of learning
Relating and structuring	Self-regulation of learning processes	Personally interested	Construction of knowledge
Critical processing	Self-regulation of learning content	Certificate directed	Intake of knowledge
Memorising and rehearsing	External regulation of learning processes	Self test directed	Use of knowledge
Analysing	External regulation of learning results	Vocation directed	Stimulating education
Concrete processing	Lack of regulation	Ambivalent	Co-operation

Table 3: Means of SRA Correct Reasoning scales and MisConceptions scales for Male and Female, Dutch and international students, and corresponding gender and nationality effects

Correct Reasoning	Females (N=779)	Males (N=1209)	p‐value	Effect size	Dutch (N=1080)	International (N=899)	p‐value	Effect size
CCtot: total Correct Reasoning	0.54	0.57	0.000	0.24	0.59	0.54	0.000	0.44
CC1: Correctly interprets probabilities	0.70	0.72	0.066	0.08	0.71	0.71	0.929	0.00
CC2: Understands how to select an appropriate average	0.68	0.74	0.000	0.24	0.77	0.68	0.000	0.36
CC3: Correctly computes probability	0.38	0.43	0.000	0.16	0.43	0.39	0.003	0.13
CC4: Understands independence	0.63	0.60	0.025	0.10	0.62	0.60	0.199	0.06
CC5: Understands sampling variability	0.21	0.29	0.000	0.27	0.30	0.23	0.000	0.25
CC6: Distiguishes betweeen correlation and causation	0.70	0.69	0.490	0.03	0.78	0.62	0.000	0.35
CC7: Correctly interprets two-way tables	0.71	0.78	0.000	0.20	0.81	0.71	0.000	0.27
CC8: Understands the importance of large samples	0.70	0.72	0.205	0.06	0.73	0.70	0.024	0.10
Misconceptions	Females (N=779)	Males (N=1209)	p‐value	Effect size	Dutch (N=1080)	International (N=899)	p‐value	Effect size
MCtot: total Misconceptions	0.33	0.30	0.000	0.27	0.29	0.33	0.000	0.34
MC1: Misconceptions involving averages	0.46	0.41	0.000	0.21	0.38	0.48	0.000	0.39
MC2: Outcome orientation	0.24	0.21	0.001	0.15	0.23	0.22	0.038	0.09
MC3: Good samples have to represent high % of population	0.17	0.14	0.004	0.13	0.15	0.15	0.566	0.03
MC4: Law of small numbers	0.33	0.25	0.000	0.29	0.24	0.31	0.000	0.24
MC5: Representativeness misconception	0.11	0.17	0.000	0.22	0.15	0.14	0.449	0.03
MC6: Correlation implies causation	0.24	0.26	0.201	0.06	0.19	0.30	0.000	0.33
MC7: Equiprobability bias	0.60	0.55	0.001	0.16	0.56	0.59	0.056	0.09
MC8: Groups can only be compared if they have the same size	0.31	0.24	0.002	0.14	0.27	0.27	0.761	0.01

Table 4: Correlations of SRA scales and course performance indicators and their two-sided p‐values: Homework bonus, scores in quizzes and final exam (N=680)

Performance indicator:	CCtot	p‐value	MCtot	p‐value
Homework bonus: Statistics period 1	−0.02	0.653	0.08	0.043
Homework bonus: Statistics period 2	−0.09	0.017	0.08	0.032
Homework bonus: Statistics period 3	−0.13	0.001	0.10	0.006
Homework bonus: Mathematics period 1	−0.12	0.001	0.14	0.000
Homework bonus: Mathematics period 2	−0.14	0.000	0.10	0.009
Homework bonus: Mathematics period 3	−0.06	0.093	0.03	0.388
Quiz score: Statistics period 1	0.01	0.792	0.00	0.922
Quiz score: Statistics period 2	−0.01	0.808	0.02	0.599
Final exam: Statistics period 1	0.24	0.000	−0.17	0.000
Final exam: Statistics period 2	0.06	0.131	−0.07	0.055
Final exam: Statistics period 3	0.07	0.072	−0.05	0.196
Final exam: Mathematics period 1	0.28	0.000	−0.18	0.000
Final exam: Mathematics period 2	0.18	0.000	−0.17	0.000
Final exam: Mathematics period 3	0.13	0.001	−0.17	0.000

Table 5: Average scores for SATS scales Affect, Cognitive Competence, Value and Difficulty, and the added scales Interest and Effort, and corresponding gender- and nationality effects, expressed by p‐values and effect sizes

SATS Scales:	Females (N=822)	Males (N=1290)	p‐value	Effect size	Dutch (N=987)	International (N=1060)	p‐value	Effect size
Affect	4.35	4.63	0.000	0.28	4.70	4.37	0.000	0.33
Cognitive Competence	4.79	5.06	0.000	0.33	4.93	4.99	0.123	0.07
Value	5.01	5.01	0.848	0.01	4.97	5.07	0.005	0.12
Difficulty	3.51	3.66	0.000	0.20	3.76	3.46	0.000	0.42
Added SATS Scales 2004:	Females (N=276)	Males (N=439)			Dutch (N=287)	International (N=428)
Interest	5.27	5.05	0.002	0.24	4.94	5.27	0.000	0.36
Effort	6.55	6.24	0.000	0.44	6.08	6.55	0.000	0.68

Table 6: Correlations between SRA and SATS scales and their two-sided p‐values (N=2031 for first four scales, N=687 for last two scales)

	CCtot	p‐value	MCtot	p‐value
Affect	0.12	0.000	−0.07	0.005
Cognitive Competence	0.12	0.000	−0.06	0.012
Value	0.10	0.000	−0.06	0.015
Difficulty	0.11	0.000	−0.10	0.000
Interest	0.02	0.533	0.05	0.174
Effort	−0.07	0.058	0.17	0.000

Table 7: Correlations between SRA and ILS scales and their two-sided p‐values (N=1767)

ILS scale	CCtot	p‐value	MCtot	p‐value
Relating and structuring	0.03	0.242	0.02	0.366
Critical processing	0.10	0.000	−0.10	0.000
Memorizing and rehearsing	−0.02	0.466	0.01	0.687
Analyzing	−0.06	0.011	0.04	0.057
Concrete processing	−0.01	0.808	−0.01	0.799
Self-regulation of learning processess	0.00	0.894	0.00	0.832
Self-regulation of learning content	0.00	0.981	−0.07	0.005
External regulation of learning processes	−0.03	0.162	0.07	0.003
External regulation of learning results	0.00	0.977	0.04	0.058
Lack of regulation	−0.01	0.806	−0.04	0.107
Personally interested	−0.05	0.057	0.01	0.697
Certificate directed	−0.02	0.483	0.05	0.037
Self test directed	−0.04	0.064	0.11	0.000
Vocation directed	−0.04	0.125	0.12	0.000
Ambivalent	−0.06	0.016	−0.02	0.355
Construction of knowledge	−0.08	0.000	0.09	0.000
Intake of knowledge	−0.09	0.000	0.15	0.000
Use of knowledge	−0.05	0.034	0.11	0.000
Stimulating education	−0.02	0.336	0.07	0.004
Co-operation	−0.08	0.001	0.06	0.007

Table 8: Correlations between selected ILS scales and SATS scale effort, and their two-sided p‐values (N=675)

ILS scale	SATS Effort	p‐value
Relating and structuring	0.22	0.000
Critical processing	0.12	0.002
Memorizing and rehearsing	0.19	0.000
Analyzing	0.29	0.000
Construction of knowledge	0.33	0.000
Intake of knowledge	0.22	0.000
Use of knowledge	0.31	0.000
Simulating education	0.17	0.000
Co-operation	0.19	0.000

Table 9: Gender effect (mean difference of males to females) in selected ILS scales, and two-sided p‐values in an independent samples t‐test (N=1215, 799 for males, females)

ILS scale	SATS Effort	p‐value
Relating and structuring	−0.008	0.810
Critical processing	0.100	0.001
Memorizing and rehearsing	0.013	0.659
Analyzing	−0.060	0.020
Construction of knowledge	−0.177	0.021
Intake of knowledge	−0.142	0.021
Use of knowledge	−0.142	0.024
Simulating education	−0.131	0.024
Co-operation	−0.169	0.025

Table 10. Standardized regression coefficients (), significance levels, and t‐values of regression models for SRA Correct Reasoning scores and MisConceptions scores of the complete models (Method Enter) and reduced models (SPSS Method Stepwise with entry significance level 5%, removal significance level 10%)

	CCtot: total Correct Reasoning score				MCtot: total MisConceptions score
	Method Enter		Method Stepwise		Method Enter		Method Stepwise
Explanatory variables		t		t		t		t
Constant		8.816		12.724		7.596		9.655
Nationality (dummmy for Dutch students)	0.205***	7.092	0.203***	7.815	−0.130***	−4.496	−0.120***	−4.462
Gender (dummmy for female students)	−0.081***	−3.051	−0.083***	−3.227	0.098***	3.667	0.099***	3.863
Relating and structuring	0.027	0.712			−0.034	−0.892
Critical processing	0.123***	3.836	0.125***	4.520	−0.092***	−2.869	−0.113***	−4.352
Memorizing and rehearsing	−0.024	−0.789			−0.024	−0.805
Analyzing	−0.060*	−1.857	−0.066*	−2.382	0.008	0.233
Concrete processing	−0.015	−0.418			−0.043	−1.211
Self-regulation of learning processes	0.001	0.037			0.023	0.660
Self-regulation of learning content	−0.013	−0.408			0.011	0.343
External regulation of learning processes	0.015	0.471			0.026	0.813
External regulation of learning results	−0.002	−0.068			0.006	0.169
Lack of regulation	0.040	1.362			−0.050*	−1.673
Personally interested	−0.054**	−1.987	−0.059**	−2.310	0.013	0.458
Certificate directed	0.025	0.832			−0.032	−1.073
Self test directed	−0.018	−0.561			0.069**	2.149
Vocation directed	−0.019	−0.538			0.062*	1.773	0.084***	3.037
Ambivalent	−0.037	−1.168			0.006	0.203
Construction of knowledge	0.002	0.056			−0.023	−0.642
Intake of knowledge	−0.017	−0.499			0.098***	2.886	0.096***	3.538
Use of knowledge	0.008	0.210			0.042	1.142
Stimulating education	0.036	1.140			−0.026	−0.810
Co-operation	−0.044	−1.437			0.011	0.374
Affect	−0.041	−1.076			0.062	1.625
Cognitive competence	0.099**	2.553	0.088***	3.223	−0.032	−0.831
Value	0.077***	2.676	0.067***	2.462	−0.075***	−2.623	−0.063**	−2.464
Difficulty	0.011	0.354			−0.070**	−2.199	−0.053**	−2.028
R-squared	0.091		0.085		0.088		0.078

* significant at 10%;

** significant at 5%;

*** significant at 1%; N=1466

Garfield, J. (2003), “Assessing Statistical Reasoning,” Statistics Education Research Journal [Online], 2 (1), 22–38. http://www.stat.auckland.ac.nz/~iase/serj/SERJ2(1).pdf

 Google Scholar

Konold, C. (1989), “Informal conceptions of probability,” Cognition and Instruction, 6, 59–98.

 Web of Science ®Google Scholar

Kahneman, D., Slovic, P. and Tversky, A. (1982), Judgment Under Uncertainty: Heuristics and Biases, Cambridge: Cambridge University Press.

 Google Scholar

Lecoutre, M.P. (1992), “Cognitive models and problem spaces in “purely random” situations,” Educational Studies in Mathematics, 23, 557–568.

 Google Scholar