The Statistical Reasoning Learning Environment: A Comparison of Students’ Statistical Reasoning Ability

Figures & data

Table 1 Possible matches grouped by years of experience.

Teacher ID	Total conformity	Years of experience	Completed coursework	Professional development	Percent free and reduced lunch
Possible teacher matches for zero years of teaching experience in statistics
1	0.537	0	4	13	45%
8	0.700	0	4	5	20%
9	0.743	0	3	10	75%
Possible teacher matches for 1 year of teaching experience in statistics
3	0.526	1	4	4	60%
6	0.605	1	6	5	65%
10	0.684	1	4	8	58%
Possible teacher matches for 2 years of teaching experience in statistics
2	0.525	2	5	6	65%
4	0.58	2	8	7	0%
14	0.759	2	2	6	60%
13	0.787	2	12	9	55%
Possible teacher matches for greater than 2 years of teaching experience in statistics
5	0.566	6	2	11	30%
7	0.587	4	2	3	50%
12	0.587	12	5	7	20%
11	0.785	7	5	15	24%

Fig. 1 Histograms of self-reported teaching practices for possible participants by subscale.

Fig. 2 Scatterplot of possible participants’ beliefs and practices subscales from the STI.

Fig. 3 Histogram of average across all SRLE constructs contained in survey.

Table 2 Comparison of matched teachers at all levels.

Higher conformity to SRLE teacher number	Lower conformity to SRLE teacher number	Difference in conformity levels	Years of experience teaching statistics	Completed coursework in statistics		Percent free and reduced lunch
				Higher	Lower	Higher	Lower
9	1	0.206	0	3	4	45%	75%
10	3	0.158	1	4	4	58%	60%
14	2	0.234	2	2	5	60%	65%
13	4	0.207	2	12	8	55%	0%
11	5	0.219	>2	5	2	24%	30%

Table 3 Student response rates.

	Low conformity classrooms					High conformity classrooms
Match	Teacher number	Matched tests	Total students	Response rate	Average class size	Teacher number	Matched tests	Total students	Response rate	Average class size
1	5	60	69	0.87	17.25	11	46	65	0.71	20.67
2	3	11	15	0.73	15	10	5	7	0.71	5
3	4	23	34	0.68	12.5	13	12	13	0.92	6.5
4	2	14	15	0.93	15	14	7	8	0.88	7
5	1	70	87	0.80	17.5	9	17	23	0.74	17
Total		178	220				87	116

Table 4 Teacher sample sizes of pretests, posttests, and matched tests.

	Low conformity					High conformity
Match	Teacher number	Pretest	Posttest	Matched tests	Number of classes	Teacher number	Pretest	Posttest	Matched tests	Number of classes
1	5	69	61	60	4	11	52	62	46	3
2	3	14	12	11	1	10	7	6	5	1
3	4	28	27	23	2	13	12	13	12	2
4	2	14	15	14	1	14	7	8	7	1
5	1	83	78	70	4	9	20	20	17	1
Total		208	193	178	12		98	109	87	8

Table 5 ANCOVA test for differences between conformity levels after controlling for pretests.

Source	Type III sum of squares	df	Mean square	F	Sig.	Partial eta-squared
Corrected model	1532.698${}^{\mathrm{a}}$	2	766.349	38.836	0.000	0.230
Intercept	1609.612	1	1609.612	81.570	0.000	0.239
Pretest	1527.173	1	1527.173	77.392	0.000	0.229
Conformity	18.797	1	18.797	0.953	0.330	0.004
Error	5130.580	260	19.733
Total	118237.000	263
Corrected total	6663.278	262

a The conformity factor tests for differences between high conformity and low conformity to SRLE principles grouping.

Table 6 ANCOVA test for differences between teachers after controlling for pretests.

Source	Type III sum of squares	df	Mean square	F	Sig.	Partial eta-squared
Corrected model	2351.098	10	235.110	13.740	0.000	0.353
Intercept	1678.883	1	1678.883	98.112	0.000	0.280
Pretest	1253.462	1	1253.462	73.251	0.000	0.225
Teacher	837.197	9	93.022	5.436	0.000	0.163
Error	4312.179	252	17.112
Total	118237.000	263
Corrected total	6663.278	262

a The teacher factor tests for differences among all teachers in the study.

Table 7 Adjusted class mean comparisons between matched teachers.

Match	High conformity teachers adjusted mean	Low conformity teachers adjusted mean	Difference	p-value for pairwise comparison
1	18.771	20.767	–1.996	0.017
2	24.680	17.205	7.475	0.003
3	25.149	22.208	2.941	0.088
5	18.916	20.335	–1.419	0.210
4	20.542	19.846	0.696	0.730

Fig. 4 Adjusted class mean visual comparison with national average.

Table 8 Comparison of percent of increase between matched teachers.

	High conformity				Low conformity
Match	Teacher number	Pretest percent correct	Posttest percent correct	Percent of increase	Teacher number	Pretest percent correct	Posttest percent correct	Percent of increase	Difference in average increase between low and high conformity
1	11	46.5%	49%	2.5%	5	46%	53%	7%	–4.5%
2	10	42.5%	61.5%	19%	3	36.8%	40.9%	4.1%	14.9%
3	13	40%	61%	21%	4	52.7%	63.3%	10.4%	10.6%

Table

	Strongly disagree	Disagree	Agree	Strongly agree
a) The content was presented mostly through the instructor’s lectures.	M	m	m	m
b) The instructor asked challenging questions that made me think.	m	m	m	m
c) The course frequently required students to work together.	m	m	m	m
d) The content was presented mostly through activities.	m	m	m	m
e) This course encouraged students to discover ideas on their own.	m	m	m	m
f) This course often used technology (e.g., web applets, statistical software) to help students understand concepts.	m	m	m	m

Table

	Seldom or not at all	A few times	Repeatedly
1. The need to base decisions on evidence (data)	m	m	m
2. Difficulties involved in getting good quality data	m	m	m
3. The study of variability is at the core of statistics	m	m	m
4. The need to select an appropriate model for making a statistical inference	m	m	m
5. The process of selecting an appropriate model for making a statistical inference	m	m	m

Table

	Not at all	To some extent	A major emphasis
6. Parametric methods (e.g., t-test, z-test)	m	m	m
7. Bayesian methods	m	m	m
8. Simulation/resampling (e.g., randomization, bootstrap methods)	m	m	m
9. Other (please describe):	m	m	m

Table

	Delivery of course content	Activities and assignments (e.g., homework, projects)	Assessments (e.g., quizzes, exams)
Statistical Analysis Packages (minitab, SPSS, IMP, StatCrunch, etc.)	q	q	q
Graphing calculators with built in statistical functions	q	q	q
Spreadsheet tools (e.g., Excel)	q	q	q
Web Applets	q	q	q
Conceptual Software (e.g., TinkerPlots, Fathom, etc.)	q	q	q
Other	q	q	q

Table

	Strongly disagree	Disagree	Undecided	Agree	Strongly agree
1. Rules of probability should be included in an introductory statistics course.	m	m	m	m	m
2. The topic of theoretical probability distributions (e.g., the binomial distribution) should be included in an introductory statistics course.	m	m	m	m	m
3. Students should learn how to read statistical tables of theoretical distributions (e.g., t-table, F-table).	m	m	m	m	m
4. Technology tools should be used to illustrate most abstract statistical concepts.	m	m	m	m	m
5. Students should learn the importance of using appropriate methods for collecting data.	m	m	m	m	m
6. Students should learn connections between the quality/nature of the data and inferences that are made.	m	m	m	m	m
7. Students should learn fewer topics in greater depth instead of learning more topics in less depth.	m	m	m	m	m
8. Lectures should be the primary way for students to learn statistical content.	m	m	m	m	m
9. Quizzes and exams should be used as the primary way to evaluate student learning.	m	m	m	m	m
10. Alternative assessments (e.g., projects, presentations,) should be used to provide important information about student learning.	m	m	m	m	m
11. All assessments should be regularly reviewed to see that they are aligned with important student learning goals.	m	m	m	m	m
12. Assessments should be used to provide formative feedback to students to improve their learning.	m	m	m	m	m
13. Students should be assessed on their ability to complete an open-ended statistical problem.	m	m	m	m	m
14. Students should be assessed on their statistical literacy (e.g., ability to read a graph, understand common statistical words, etc.).	m	m	m	m	m
15. Students should analyze data primarily using technology.	m	m	m	m	m
16. Statistics courses should be updated continually in light of developments such as new technology and common core curriculum requirements.	m	m	m	m	m
17. Statistics instructors should be actively engaged in the statistics education community.	m	m	m	m	m

Table

	0	1	2	3	4	5 or more
5. Live or prerecorded webinars (online seminars):	m	m	m	m	m	m
6. Workshops	m	m	m	m	m	m
7. Short courses/minicourses	m	m	m	m	m	m
8. Other:	m	m	m	m	m	m

Table

Name:

School:

E-mail:

Number of Students Enrolled at your school in grades 9–12:

Number of Students in your AP Statistics course:

Percent of Students with Free/Reduced Lunch: