Classroom Research: Assessment of Student Understanding of Sampling Distributions of Means and the Central Limit Theorem in Post-Calculus Probability and Statistics Classes

Figures & data

Table 1. Overall Scores: Pretest, Posttest, and Paired Differences (N = number of students)

Total Number of Questions = 27	Average Number of Correct Answers (Percent Correct)	Standard Deviation in Number (Percent)
Math 300 Pretest (N = 18)	9.9 (36.6%)	s = 3.20 (11.8%)
Math 300 Posttest (N = 18)	18.5 (68.5%)	s = 4.26 (15.8%)
Math 300 Paired Difference Post - Pre (N = 18)	8.6 (31.8%)	s = 5.45 (20.2%)
Math 400 Pretest (N = 7)	12.2 (45.0%)	s = 3.81 (14.1%)
Math 400 Posttest (N = 7)	18.9 (69.9%)	s = 2.97 (11.0%)
Math 400 Paired Difference Post - Pre (N = 7)	6.7 (24.9%)	s = 2.69 (20.0%)

Figure 1. Population Distribution (Top Left) and Possible Sampling Distributions (A-E) for the Irregular Population Distribution (please see the Assessment Tool Question 5 in Appendix B).

Table 2. Correct Answers by Category, Posttest

	Math 300 Post (N = 18)	Math 400 Post (N = 7)
Graphical Questions 4 and 5 Average Number (Percent) Correct with Standard Deviation. (Max Number Correct = 14)	9.6 (68.3%) s = 2.66 (9.9%)	7.6 (54.1%) s = 2.93 (10.9%)
Fact/Computational Average Number (Percent) Correct with Standard Deviation. (Max Number Correct = 13)	8.9 (68.9%) s = 2.75 (10.2%)	11.3 (86.8%) s = 0.76 (2.8%)
Average Total Number (Percent) Correct with Standard Deviation. (Max Number Correct = 27)	18.5 (68.5%) s = 4.26 (15.8%)	18.9 (69.9%) s = 2.97 (11.0%)

Figure 2. Correct Identification of Sampling Distributions for Math 300 and Math 400 Students.

Table 3. Math 300 Students Posttest Reasoning Pairs for the Irregular Distribution (Question 5)

Reasoning Category (delMas, et al. (1999a))	Number of Students (N = 18)	Percent of Students (N = 18)	Posttest Reasoning Pair Irregular Distribution (Question 5) (n = 4, n = 25)
Correct	5	27.8%	(C, E)
Good (Large-to-Small Normal)	2	11.1%	(B, E)
Large-to-Small Normal	7	38.9%	(A, E)
Large-to-Small Population	1	5.6%	(A, B)
Small-to-Large	2	11.1%	(E, D) or (E, C)
Other	1	5.6%	(C, D)

Table 4. Choice of Correct Sampling Distribution and Demonstration of Consistent Graphical Reasoning for Math 300 Students.

Determining the Sampling Distribution form the Irregular Population for Math 300 Students	Sample Size n = 4		Sample Size n = 25
	Pre-Test	Post-Test	Pre-Test	Post-Test
% Selecting the Correct Sampling Distribution (Number of Students from N = 18)	5.6% (1)	33.3% (6)	5.6% (1)	77.8% (14)
% with Consistent Graphical Reasoning (Number of Students from N = 18)	16.7% (3)	77.8% (14)	11.1% (2)	83.3% (15)

Table 5. Math 300 Posttest Comparison of Irregular Distribution Reasoning Pairs and Consistent Graphical Reasoning for n = 4

Posttest Reasoning Pair Irregular Population Distribution (Question 5(a)) (n = 4, n = 25)	Number (Percent) of Students (N = 18)	n = 4 Sampling Distribution Shaped More Like: (Question 5(b))		Variability of n = 4 Sampling Distribution compared to Population: (Question 5(c))		Number with Consistent Graphical Reasoning
		Answer	Number of Students	Answer	Number of Students
(C, E)	5 (27.8%)	Normal	5	Less	5		5
(B, E)	2 (11.1%)	Pop.	2	Less	2		2
(A, E)	7 (38.9%)	Pop.	7	Same	4		4
				Less*	3		0
(A, B)	1 (5.6%)	Normal*	1	More*	1		0
(E, D) or (E, C)	2 (11.1%)	Normal	2	Less	2		2
(C, D)	1 (5.6%)	Normal	1	Less	1	1
Totals	18	Normal Pop.	9 9	Less Same More	13 4 1	14

Table 6. Application of the Empirical Rule versus Theoretical Knowledge of the Standard Deviation of the Sampling Distribution.

Answer to Question 9d (std. dev. of Sampling Distribution is )	Answer to Question 2			TOTAL
	a, b, or e	c (uses pop. std. dev.)	d (uses standard error) correct answer
True correct answer	0	8	7	15
False	0	3	0	3
Total	0	11	7	18

Table

Sample Size (n)	Population Mean	Empirical Mean of Sample Means	Theoretical Mean of Sample Means (via the CLT)	Population Standard Deviation	Empirical Standard Deviation of Sample Means	Theoretical Standard Deviation of Sample Means (via the CLT)
1
5
25
50

Figure C.1. Examples of pairs of graphs for each type of reasoning for Question 5 (Figure 9 of delMas, et. al. (2002), used with permission).

Table C.1. Question 4 Answer Pairs with Category Classification

Answer Pair Classification (4a, 4e)	4e Answer (n = 16)
4a Answer (n = 4)	A (Population/Same)	B (Population/Smaller)	C (Normal/Smallest)	D (Approx Normal/Smaller)	E (Approx Pop/Same)
A (Population/Same)	Same	L-S Pop	L-S Normal	L-S Normal	S-L
B (Population/Smaller)	S-L	Same	Good	Good	S-L
C (Normal/Smallest)	S-L	S-L	Same	S-L	S-L
D (Approx Normal/Smaller)	S-L	S-L	Correct	Same	S-L
E (Approx Pop/Same)	L-S Pop	L-S Pop	L-S Normal	L-S Normal	Same

Table C.2. Question 5 Answer Pairs with Category Classification

Answer Pair Classification (5a, 5e)	5e Answer (n = 25)
5a Answer (n = 4)	A (Population/Same)	B (Population/Smallest)	C (Approx Normal/Smaller)	D (Population/Smaller)	E (Normal/Smallest)
A (Population/Same)	Same	L-S Pop	L-S Normal	L-S Pop	L-S Normal
B (Population/Smallest)	S-L	Same	S-L	S-L	L-S Normal
C (Approx Normal/Smaller)	S-L	L-S Pop	Same	Other	Correct
D (Population/Smaller)	S-L	L-S Pop	Good	Same	Good
E (Normal/Smallest)	Other	Other	S-L	S-L	Same

Figure C.2. Distribution of Math 300 Students (and Equally Likely Reasoning Pairs) into Reasoning Categories for the Skewed and Irregular Populations (Questions 4 and 5, respectively)

Table C.3. Consistent Graphical Reasoning Classification and Data for Math 300 (Pre) and Post Tests for Question 5a

Stated Variance of Sampling Distribution	Stated Shape of Sampling Distribution	5a - Sampling Distribution Chosen for Irregular Population n = 4 (Variance of Sampling Distribution Compared to Population)					Totals
		A Population (Same)	B Population (Same)	C Normal (Smaller) Correct Answer	D Population (Smaller)	E Normal (Smallest)
Less than Population	Normal	(1)		6*		2* (2)*	8 (3)
	Population	3	2* (1)*		*		5 (1)
Same as Population	Normal				(1)	(3)	0 (4)
	Population	4*	(1)				4 (1)
More than Population	Normal			(1)		(3)	0 (4)
	Population	1 (1)	(3)		(1)		1 (5)
Totals	8 (2)	2 (5)	6 (1)	0 (2)	2 (8)	18 (18)

Table C.4. Consistent Graphical Reasoning Classification and Data for Math 300 (Pre) and Post Tests for Question 5e

Stated Variance of Sampling Distribution	Stated Shape of Sampling Distribution	5e - Sampling Distribution Chosen for Irregular Population n = 25 (Variance of Sampling Distribution Compared to Population)					Totals
		A Population (Same)	B Population (Same)	C Normal (Smaller)	D Population (Smaller)	E Normal (Smallest)) Correct Answer
Less than Population	Normal		(1)	1*	2 (1)	14*	17 (2)
	Population	(1)	*		*		0 (1)
Same as Population	Normal	(2)				(1)	0 (3)
	Population	* (2)*					0 (2)
More than Population	Normal	(2)		(1)	(1)		0 (4)
	Population	(1)	1 (1)		(4)		1 (6)
Totals	0 (8)	1 (2)	1 (1)	2 (6)	14 (1)	18 (18)

Table C.5 Qualitative Survey Results for Math 300

Topic (Sample Size of 22 Students)	Post Avg.	Avg. Diff. (Post - Pre)	Paired Std. Dev.
My understanding of how to compute a sample mean versus how to compute the distribution (population) mean.	4.3	1.7	1.2
My understanding of how to compute a sample variance versus how to compute the distribution (population) variance.	4.0	1.6	1.6
My understanding of empirical probability distributions versus probability distributions.	3.9	1.8	1.3
My understanding of how to find probabilities associated with discrete random variables.	3.7	1.9	1.2
My understanding of how to find probabilities associated with continuous random variables.	3.8	2.0	1.2
My understanding of Bernoulli trials and their associated distributions.	3.4	1.6	1.6
My understanding of the Normal Probability Distribution.	4.1	2.1	1.3
My understanding of the difference between cumulative probability functions and probability mass/density functions.	3.0	1.5	1.6
My understanding of Mathematical Expectation and its properties.	3.5	2.1	1.2
My understanding of independent random variables.	3.9	1.8	1.1
My overall understanding of random variables.	3.6	1.4	1.2
My understanding of the Central Limit Theorem.	3.8	2.2	1.5
My understanding of Point Estimation.	3.1	1.3	1.6
My understanding of Confidence Intervals for Proportions.	3.2	1.4	1.4
My overall understanding of probability.	3.4	1.2	1.0
My overall understanding of statistics.	3.0	1.0	1.2

Table C.6 Identification of Correct Sampling Distribution for Question 4a — Skewed Distribution with Small Sample Size

Question 4a	Posttest		Totals
Pretest	Correct ID	Incorrect ID
Correct ID	0	0	0
Incorrect ID	3	15	15
Totals	3	15	18

Table C.7 Identification of Correct Sampling Distribution for Question 4e — Skewed Distribution with Large Sample Size

Question 4e	Posttest		Totals
Pretest	Correct ID	Incorrect ID
Correct ID	1	2	3
Incorrect ID	9	6	15
Totals	10	8	18

Table C.8 Identification of Correct Sampling Distribution for Question 5a—Irregular Distribution with Small Sample Size

Question 5a	Posttest		Totals
Pretest	Correct ID	Incorrect ID
Correct ID	0	1	1
Incorrect ID	6	11	17
Totals	6	12	18

Table C.9 Identification of Correct Sampling Distribution for Question 5e—Irregular Distribution with Large Sample Size

Question 5e	Posttest		Totals
Pretest	Correct ID	Incorrect ID
Correct ID	0	1	1
Incorrect ID	14	3	17
Totals	14	4	18

Table D-1: Major Mathematical Topics Covered

Topic
Sample Spaces, Outcomes, and Events
Relative Frequency Histograms
Basic Descriptive Statistics (Mean, Sample Mean, Variance, Sample Variance, etc.
The Probability Function and its Basic Properties
Methods of Enumeration (Combinatorics)
Conditional Probability and Independent Events
Bayes' Theorem
Basic Concepts for Discrete and Continuous Random Variables
Mathematical Expectation
Moment-Generating Functions
Bernoulli Trials and the Binomial Distribution
The Poisson Distribution
The Uniform Distribution (Discrete and Continuous)
The Exponential Distribution
The Normal Distribution
Distributions of Two Random Variables
Independent Random Variables
Distributions of Sums of Independent Random Variables
The Central Limit Theorem
Confidence Intervals for Means
Confidence Intervals for Proportions
Sample Size

delMas, R., Garfield, J., and Chance, B. (1999a), “A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students' Statistical Reasoning,” Journal of Statistics Education [Online], 7 (3). ww2.amstat.org/publications/jse/secure/v7n3/delmas.cfm

 Google Scholar

delMas, R., Garfield, J., and Chance, B. (2002), “Assessment as a Means of Instruction,” paper presented at the 2002 Joint Mathematics Meetings, [Online]. www.tc.umn.edu/~delma001/stat_tools/

 Google Scholar