Assessment of teacher perceived skill in classroom assessment practices using IRT Models

Figures & data

Table 1. Factor loadings for exploratory factor analysis with promax oblique rotation of self-perceived skill in classroom assessment practices

	Crit. ref. test	Grading practices	Statistic appl.	Asses appl.	Essay items	Obj. items
1. Construct multiple choice items						.80
2. Construct essay items					.68
3. Write items for higher cognitive level				.60
4. Explain standardized exam scores			.72
5. Calculate central tendency			62
6. Conduct item analysis			.77
7. Revise items based on item analysis			.70
8. Assess individual class participation				.68
9. Assess problem-solving skills				.69
10. Use portfolio assessment				.53
11. Use assessment results for decisions				.73
12. Determine why students make mistakes				.73
13. Use assessment results to plan teaching				.76
14. Communicate classroom assess. results				.73
15. Include student improve when grading		.82
16. Including student effort when grading		.83
17. Assessing specific course objectives	.62
18. Develop systematic grading procedures		.56
19. Ensure test covers the material taught	.71
20. Use peer assessments		.62
21. Consistently grade essay question					.90
22. Use table of specification for test plan			.60
23. Develop rubrics (marking keys)		.53
24. Fairly assign grades to all students	.75
25. Use assesses results to evaluate class improvement				.56
26. Align test items with instructional obj	.72
27. Construct true or false items						.59
28. Provide feedback with comments when grading		.54
29. Calculate variability			.55

Notes: Extraction method: Principal axis factoring. Rotation method: Promax with Kaiser normalization, Ref. test = Reference testing, Appl. = Applications.

Table 2. Descriptive statistics and Cronbach’s alpha estimates for the self-perceived skill factors

	M	SD	α
Criterion referenced testing	3.65	.80	.88
Grading practices	2.85	.89	.82
Statistical applications	2.88	.89	.84
Assessment applications	3.23	.85	.88
Essay items	3.13	1.09	.75
Objective items	3.61	.99	.86

Table 3. Model fit indexes for partial credit model and graded response model

Model	AIC	BIC	Log-lik	p-value
PCM	51,100.85	51,631.82	−25,433.43	.001
GRM	50,539.88	51,197.91	−25,124	.01

Table 4. Model information partial credit model and graded response model

PCM	GRM	Model comparison
Total information = 102.99	Total information = 121.92	REPCM,GRM = 102.99
Information in (−10, 10) = 102.99 (100%)	Information in (−10, 10) = 121.92 (100%)	121.92
Based on all the items	Based on all the items	= .845

Table 5. Item threshold, discrimination values, and information for teachers perceived skills in classroom assessment items

Items	Extrmt1	Extrmt2	Extrmt3	Extrmt4	Discrim	Info
1. Construct multiple choice items	−3.319	−1.824	−.904	1.873	1.007	2.53
2. Construct essay items	−2.521	−1.363	−.372	1.959	1.057	2.55
3. Write items for higher cognitive level	−2.282	−1.097	−.116	1.740	1.510	4.41
4. Explain standardized exam scores to others	−1.556	−.438	.372	2.056	1.553	4.09
5. Calculate central tendency	−2.300	−.858	−.040	1.735	1.042	2.39
6. Conduct item analysis	−1.699	−.377	.604	2.259	1.498	4.09
7. Revise items based on item analysis	−1.89	−.861	.040	1.694	1.887	5.44
8. Assess individual class participation	−2.607	−1.289	−.394	1.406	1.690	4.88
9. Assess problem-solving skills	−1.781	−.724	.255	1.882	1.768	5.02
10. Using portfolio assessment	−.742	.250	1.110	2.557	1.340	3.16
11. Using assessment results for decision-making	−1.757	−.690	.311	1.852	1.869	5.43
12. Determining why students make mistakes	−2.040	−.782	.128	1.970	1.759	5.16
13. Using assessment results to plan teaching	−2.209	−1.054	−.119	1.514	2.050	6.27
14. Communicating classroom assessment results	−2.221	−1.161	−.332	1.413	2.030	6.04
15. Including student improvement when grading	−1.653	−.539	.379	2.026	1.646	4.52
16. Including student effort when grading	−1.488	−.341	.585	2.278	1.626	4.49
17. Assessing specific course objectives	−2.008	−1.084	−.227	1.514	1.626	4.31
18. Developing systematic grading procedures	−1.200	−.180	.772	2.271	1.527	3.95
19. Ensure the test covers the material taught	−3.253	−2.078	−1.142	.682	1.532	4.18
20. Use peer assessments	−1.587	−.523	.409	1.888	1.763	4.89
21. Fairly and consistently grading essay question	−1.613	−.535	.393	2.044	1.472	3.82
22. Use a table of specification for test planning	−1.124	.096	1.101	2.645	1.488	3.96
23. Develop rubrics (marking keys)	−1.946	−1.125	−.259	1.276	1.576	3.96
24. Fairly assign grades to all students	−3.506	−2.068	−1.058	.949	1.399	3.89
25. Use assesses results to evaluate class improvement	−2.594	−1.569	−.527	1.173	1.653	4.60
26. Align test items with instructional objectives	−2.741	−1.652	−.701	1.036	1.586	4.33
27. Construct True or false items	−2.882	−1.757	−.851	1.054	1.031	2.32
28. Provide written feedback with comments	−2.806	−1.578	−.590	1.062	1.560	4.29
29. Calculate variability (standard deviation)	−1.268	−.166	.696	2.376	1.311	3.21

Notes: Extrmt1: Threshold 1, Extrmt2: Threshold 2; Extrmt3: Threshold 3; Extrmt4: Threshold 4; Discrim: Item discrimination parameter. Info: Item information.

Figure 1. Characteristic and information curve for item 13

Figure 2. Characteristics and information curve for item 1