Development of a Student Self-Reported Instrument to Assess Course Reform

Figures & data

TABLE 1 Descriptive Statistics for Courses Included in Analysis, Numbers Reported Are Self-Report Totals

Department	Course	Sections in Study	Total Enrollment^a	N of Participations	Control Class^b
Agronomy	Soil Science (AGRY 255)	2	304	83	No
Agronomy	Genetics (AGRY 320)	1	141	121	No
Chemistry	General Chemistry (CHM 115)	2	923	285	Yes
Communication	Principles of Persuasion (COM 318)	1	344	103	No
Math	Algebra and Trigonometry II (MA 154)	1	163	57	Yes
Political Science	Intro to Political Science (POL 101)	1	170	74	Yes
Psychology	Elementary Psychology (PSY 120)	16	712^c	161	No
		24	2,757	884 (32%)	3

^a Including all sections.

^b Included an experimental control class that was not reformed and will be used in future analysis to report on the efficacy of redesign on our campus.

^c 16 recitation sections of PSY 120 participated.

TABLE 2 Principle Components Analysis With Standardized Factor Loading of Classroom Experience Questionnaire Data

Component	Observation Data^a			Self-Reported Data^b
	1	2	3	1	2	3
Item 1	.620	.394	.385	−.116	.740
Item 2	.510	.702	.140	.729
Item 3	.676	−.386	.410	.852
Item 4	.721	−.325		.854	−.193
Item 5	.627	−.552	−.281	.768
Item 6	.848	.126		.811
Item 7	.751	.178		.817
Item 8	.603	−.483		.715		.169
Item 9	.851		−.199		.789
Item 10	.343	.182	.603	.695	−.178	−.251
Item 11	.742	.231	−.363	.587	.105	.231
Item 12		−.333	.413			.946
Item 13	.502	.195	−.559	.619	.210	.103
Component	Promax Rotation Component Correlation Matrix
	1	2	3	1	2	3
1	1.000			1.000
2	.453	1.000		.321	1.000
3	.372	.272	1.000	−.033	.047	1.000

Note. KMO = Kaiser-Meyer-Olkin.

^a n = 71; KMO and Bartlett Test Statistic = .684.

^b n = 649; KMO and Bartlett Test Statistic = .903.

KMO and Bartlett = p ≤ .001.

TABLE 3 Principle Axis Factor With Standardized Factor Loading of the Learning Community Construct

	Observation Data^a	Self-Reported Data^b
KMO and Bartlett Test statistic =	.804***	.833***
The Instructor:	Factor Loadings	Factor Loadings
Item 1: Encouraged students to learn from one another.	.772	.499
Item 2: Provided opportunities for students to challenge opinions expressed in class.	.618	.622
Item 3: Encouraged students to participate actively in class.	.893	.799
Item 4: Provided opportunities for students to ask questions.	.697	.747
Item 5: Allowed students to answer a question or solve a problem in more than one way.	.778	.705
Item 6: Maintained a climate of respect within the class for what others had to say.	.319	.592
Variance explained	49.48%	44.63%
Cronbach's α	.843	.820

Note. n = six items factored. KMO = Kaiser-Meyer-Olkin.

^a n = 71.

^b n = 649.

*** p ≤ .001.

TABLE 4 Principle Axis Factor With Standardized Factor Loading of the Constructivist Pedagogy (CP) Construct

	Observation Data^a	Self-Reported Data^b
The Instructor:	Factor Loadings	Factor Loadings
Item 7: Connected course content to students' experience and knowledge.	.392	.693
Item 8: Asked students to explain their ideas.	.610	.664
Item 9: Gave students adequate time to think about and/or discuss a new concept.	.985	.809
Item 10: Provided opportunities for students to process new information.	.859	.852
Variance explained =	55.48%	57.54%
Cronbach's α =	.798	.839

Note. n = four items factored. KMO = Kaiser-Meyer-Olkin.

^a n = 71; KMO and Bartlett Test statistic = .624.

^b n = 649; KMO and Bartlett Test statistic = .759.

KMO and Bartlett = p ≤ .001.

TABLE 5 Principle Axis Factor With Standardized Factor Loading of the Equity Construct

	Observation Data^a	Self-Reported Data^b
	Factor Loadings	Factor Loadings
Item 11: During the past week, who primarily guided the class discussion?	.738	.487
Item 12: Discussion in class generally followed which format (format options given)?	.674	.987
Item 13: During the past week, who primarily determined the topics covered during class?	.430	.405
Variance explained	39.46%	45.86%
Cronbach's α	.595	.630
Item 11% missing cases	62.0%	44.74%
Item 12% missing cases	59.2%	35.41%
Item 13% missing cases	2.8%	25.33%

Note. n = three items factored. KMO = Kaiser-Meyer-Olkin.

^a n = 71; KMO and Bartlett Test statistic = .616.

^b n = 649; KMO and Bartlett Test statistic = .556.

p ≤ .01; KMO and Bartlett = p ≤ .001.

TABLE 6 Concurrent Validity Statistics for Observation Data

		Value	Asymptotic SE	Approx. T
Ordinal by ordinal	Gamma	.597***	.047	12.714
	Spearman correlation	.792***	.046	10.466
Interval by interval	Pearson's R	.788***	.043	10.325
	N of valid cases	67

*** p ≤ .001.

TABLE 7 Concurrent Validity Statistics for Self-Reported Data

		Value	Asymptotic SE	Approx. T
Ordinal by ordinal	Gamma	.653***	.016	42.757
	Spearman correlation	.830***	.014	37.378
Interval by interval	Pearson's R	.826***	.013	36.792
	N of valid cases	632

*** p ≤ .001.