Measuring teaching skills in elementary education using the Rasch model

Figures & data

Table 1. Some characteristics of the teachers.

Percentage male teachers	9.2
Percentage of teachers working with 4–5-year-olds (Kindergarten)	19.4
Percentage of teachers working with 6–7-year-olds	37.3
Percentage of teachers working with 8–12-year-olds	43.3
Average amount of years of experience	9.57	SD 7.94
Average class size	21.46	SD 6.85

Table 2. Some characteristics of the schools.

Percentage of schools	Population	Sample
In the north of the Netherlands	17.7	5.4
In the middle of the Netherlands	60.8	69.9
In the south of the Netherlands	21.6	24.6

Table 3. Confirmatory factor analyses on 32 items and 1 factor.

Model fit for one-dimensionality Norm	CFI >.90	TLI >.90	RMSEA <.08
Result	.92	.92	.05

Figure 1. A scree plot of eigenvalues.

Table 4. Andersen’s log-likelihood ratio test for different teacher characteristics.

	A-χ^2	df	p value
Gender	43.85	29^1	.04
Gender leaving out Item 31	37.76	28	.10
Teaching experience (< 6 years versus ≥ 5 years of experience)	26.75	31	.69
Class size (< 21 students versus ≥ 20 students)	28.32	30^1	.55
Kindergarten vs. Group 3–4^2	88. 65	31	.00
Kindergarten vs. Group 3–4 leaving out Item 24	69.31	30	.00
Kindergarten vs. Group 3–4 leaving out Items 24, 32	55.79	29	.00
Kindergarten vs. Group 3–4 leaving out Items 24, 32, 25	44.02	28	.03
Kindergarten vs. Group 3–4 leaving out Items 24, 32, 25, 17	35.56	27	.09
Kindergarten vs. Group 5–8	87.88	31	.00
Kindergarten vs. Group 5–8 leaving out Item 32	75.32	30	.00
Kindergarten vs. Group 5–8 leaving out Items 32, 27	64.92	29	.00
Kindergarten vs. Group 5–8 leaving out Items 32, 27, 24	54.50	28	.00
Kindergarten vs. Group 5–8 leaving out Items 32, 27, 24, 12	49.58	27	.01
Kindergarten vs. Group 5–8 leaving out Items 32, 27, 24, 12, 17	45.03	26	.01
Kindergarten vs. Group 5–8 leaving out Items 32, 27, 24, 12, 17, 25	39.90	25	.03
Kindergarten vs. Group 5–8 leaving out Items 32, 27, 24, 12, 17, 25, 18	35.78	24	.06
Group 3–4 vs. Group 5–8	57.18	31	.00
Group 3–4 vs. Group 5–8 leaving out Item 16	44.03	30	.05
Group 3–4 vs. Group 5–8 leaving out Items 16, 27	39.56	29	.09

¹Item 1 and/or Item 2 had no variance in one of the groups.

²Group 3 starts when the students are about 7 years old; in Group 8, the students are about 12 years old.

Table 5. Confirmatory factor analyses on 32 items and 1 factor residual correlations set at 0.

Model fit for residual correlations set at 0 Norm	CFI > .90	TLI > .90	RMSEA <.08
Result	.93	.93	.05

Table 6. Andersen’s log-likelihood ratio test for teachers with low and high scores.

	Aicc χ2	df	p value
32 items	50.99	31	.00
Leaving out Item 24	39.49	30	.03
Leaving out Item 24 and Item 25	31.10	29	.15

A_icc χ2 = Andersen’s log-likelihood ratio test for testing equality of the item characteristic curves (icc).

Table 7. Slopes of the item characteristic curves.

	The teacher …	slope (a)	SE
1	Shows respect for learners in his/her behaviour and language	3.09	.23
2	Maintains a relaxed atmosphere	2.01	.27
3	Promotes learners’ self-confidence	2.68	.26
4	Fosters mutual respect	1.85	.28
5	Ensures the lesson proceeds in an orderly manner	1.53	.25
6	Monitors to ensure learners carry out activities in the appropriate manner	1.20	.40
7	Provides effective classroom management	1.96	.28
8	Uses the time for learning efficiently	2.35	.29
9	Presents and explains the subject material in a clear manner	2.64	.63
10	Gives feedback to learners	1.59	.26
11	Engages all learners in the lesson	1.62	.39
12	During the presentation stage, checks whether learners have understood the subject material	1.49	.36
13	Encourages learners to do their best	1.82	.23
14	Teaches in a well-structured manner	1.97	.39
15	Gives a clear explanation of how to use didactic aids and how to carry out assignments	1.52	.25
16	Offers activities and work forms that stimulate learners to take an active approach	1.54	.24
17	Stimulates the building of self-confidence in weaker learners	1.89	.20
18	Stimulates learners to think about solutions	1.90	.33
19	Asks questions which stimulate learners to reflect	1.63	.32
20	Let learners think aloud	1.65	.17
21	Gives interactive instructions	1.82	.26
22	Clearly specifies the lesson aims at the start of the lesson	.96	.22
23	Evaluates whether the lesson aims have been reached	1.01	.34
24	Offers weaker learners extra study and instruction time	1.15	.36
25	Adjusts instruction to relevant inter-learner differences	1.35	.28
26	Adjusts the processing of subject matter to relevant inter-learner differences	1.48	.34
27	Teaches learners how to simplify complex problems	1.76	.24
28	Stimulates the use of control activities	1.43	.27
29	Teaches learners to check solutions	1.54	.51
30	Stimulates the application of what has been learned	1.18	.26
31	Encourages learners to think critically	1.88	.20
32	Asks learners to reflect on practical strategies	1.91	.35

Table 8. Wright map for the ICALT31 scale (N = 400 teachers teaching 6–12-year-olds).

	Item parameters			Person parameters		Freq.
Domain	Item	Log δ	SE	Warm’s θ	SE	%
Beyond the basic standards of teaching 5.3%	The teacher …			−5.64	1.59	0
				−4.32	.96	0
	Shows respect for learners in his/her behaviour and language	−3.79 −3.79	.46
				−3.62	.78	0
				−3.12	.68	0
	Maintains a relaxed atmosphere	−2.76	.30
				−2.72	.62	0
				−2.37	.57	0
	Presents and explains the subject material in a clear manner	−2.21	.24
	Teaches in a well-structured manner	−2.05	.23
	Provides effective classroom management	−2.05	.23
	Promotes learners’ self-confidence	−2.00	.22
				−1.79	.52	1.3
	Ensures the lesson proceeds in an orderly manner	−1.61	.20
					.50	1.5
				−1.30	.49	.8
	Uses the time for learning efficiently	−1.08	.17			.5
				−1.07	.48	1.3
Basic teaching skills 11.3%				−.85	.47	1.8
	Fosters mutual respect	−.84	.16
				−.64	.46	2.3
	Gives a clear explanation of how to use didactic aids and how to carry out assignments	−.63	.15
	Engages all learners in the lesson	−.60	.15
	Encourages learners to do their best	−.51	.15
	Monitors to ensure learners carry out activities in the appropriate manner	−.43	.15	−.43	.46	1.8
	During the presentation stage, checks whether learners have understood the subject material	−.41	.15
				−.23	.45	3.0
	Gives feedback to learners	−.15	.14
				−.03	.45	2.5
Activating students 22.3%				.17	.45	4.3
	Offers activities and work forms that stimulate learners to take an active approach	.19	.13
	Gives interactive instructions	.26	.13
	Asks questions which stimulate learners to reflect	.26	.13
				.36	.45	3.0
	Let learners think aloud	.54	.13
				.56	.45	3.5
	Stimulates the building of self-confidence in weaker learners	.66	.13
				.76	.45	5.0
	Stimulates learners to think about solutions	.96	.12
				.97	.46	6.5
Differentiating teaching 21.0%				1.17	.46	5.3
	Offers weaker learners extra study and instruction time	1.07	.12
	Evaluates whether the lesson aims have been reached	1.30	.12
				1.39	.47	5.3
	Adjusts instruction to relevant inter-learner differences	1.33	.12
				1.61	.48	5.0
	Stimulates the application of what has been learned	1.76	.12
	Asks learners to reflect on practical strategies	1.77	.12
	Teaches learners how to simplify complex problems	1.80	.12
				1.84	.49	5.5
	Adjusts the processing of subject matter to relevant inter-learner differences	1.90	.12
Teaching learning strategies 25.5%				2.09	.51	8.5
	Encourages learners to think critically	2.17	.13
				2.35	.54	7.0
	Stimulates the use of control activities	2.46	.13
				2.65	.58	4.3
	Teaches learners to check solutions	2.68	.14
				2.99	.63	5.5
Almost perfect7.0%				3.42	.72	3.3
				4.01	.90	3.8
Perfect 7.8%				5.19	1.50	7.8

Table 9. Descriptives of the ICALT31 scale.

	n	µ	σ	Cohen’s δ	Significance
All	400	1.62	1.67
Gender
Male	41	1.70	1.77
Female	359	1.60	1.66	.06	n.s.
Years of experience (R with ICALT31 = .12; sign. = .02)
< 1 year	39	1.11	1.34
1–5 years	139	1.50	1.65	.25	n.s.
6–10 years	103	1.76	1.71	.16	n.s.
11–20 years	85	1.73	1.76	−.02	n.s.
> 20 years	34	1.95	1.72	.13	n.s.
Age group of students
6–7-year-olds	185	1.40	1.54
8–12-year-olds	215	1.80	1.76	.24	.02
Class size (R with ICALT31 = .16; sign. = .001)
< 11	34	1.04	1.54
11–15	40	1.44	1.80	.24
16–20	71	1.56	1.85	.07
21–25	125	1.53	1.52	.02
26–30	105	1.79	1.57	.17
> 30	25	2.58	1.91	.48	> 30 with < 11 .01

Table

Country	…	Date observation (dd-mm-yyyy)	…
School name	…	Class	…
Location	…	No of learners present	…
Level of education	0 = Kindergarten 1 = elementary education	Name observer	…
School denomination	0 = public 1 = private	Gender observer	F/M
Subject matter	…	Years of teaching experience observer	…
Name teacher	…	Occupation observer	0 = school teacher 1 = university teacher 2 = other, please specify…
Gender teacher Years of teaching experience teacher	M/F …

Observe the following behaviours and events

Rate¹ Please circle the appropriate answer:

1 = mostly weak; 2 = more often weak than strong; 3 = more often strong than weak; 4 = mostly strong

Observed² Please circle the appropriate answer:

0 = no, I have not observed this; 1 = yes, I have observed this

Table

Domain	Indicator: The teacher…		Rate^1	Examples of good practice: The teacher …	Observed^2
Safe and stimulating learning	1	…shows respect for learners in his/her	1 2 3 4	…lets learners finish their sentences	0 1
climate		behaviour and language		…listens to what learners have to say	0 1
				…does not make role stereotyping remarks	0 1
	2	…maintains a relaxed atmosphere	1 2 3 4	…addresses learners in a positive manner	0 1
				…uses and stimulates humour	0 1
				…accepts the fact that learners make mistakes	0 1
				…shows compassion and empathy for all learners present	0 1
	3	…promotes learners’ self-confidence	1 2 3 4	…gives positive feedback on questions and remarks from leaners	0 1
				…compliments learners on their work	0 1
				…acknowledges the contributions that learners make	0 1
	4	…fosters mutual respect	1 2 3 4	…stimulates learners to listen to each other	0 1
				…intervenes when learners make fun of someone	0 1
				…keeps (cultural) differences and idiosyncrasies in mind	0 1
				…stimulates solidarity between learners	0 1
				…encourages learners to experience activities as group events	0 1
Efficient organisation	5	…ensures the lesson proceeds in an orderly	1 2 3 4	Learners enter and settle in an orderly manner	0 1
		manner		…intervenes timely and appropriately in case of disorder	0 1
				…safeguards the agreed rules and codes of conduct	0 1
				…keeps all leaners involved in activities until the end of the lesson	0 1
				…makes sure that learners know what to do if they need help with their work and explains clearly when they can ask for help	0 1
				…makes sure learners know what to do when they have finished their work	0 1
	6	…monitors to ensure learners carry out activities in the appropriate manner	1 2 3 4	…checks whether learners have understood what they have to do	0 1
				…provides feedback on learners’ social functioning whilst carrying out a task	0 1
	7	…provides effective classroom management	1 2 3 4	…explains clearly which materials can be used	0 1
				The materials for the lesson are ready for use	0 1
				Materials are geared at the right level and developmental stage of the learners	0 1
	8	…uses the time for learning efficiently	1 2 3 4	… starts the lesson on time	0 1
				… does not waste time at the beginning, during, or at the end of the lesson	0 1
				…prevents any unnecessary breaks from occurring	0 1
				…does not keep learners waiting	0 1
	9	…engages all learners in the lesson	1 2 3 4	…creates learners assignments which stimulate active participation	0 1
				…asks questions which stimulate learners to reflect	0 1
				…makes sure that learners listen and/or continue working	0 1
				…allows for ‘thinking time’ after asking a question	0 1
				…also invites learners to participate who do not volunteer to do so	0 1
	10	…encourages learners to do their best	1 2 3 4	…praises learners who do their best	0 1
				…makes clear that all learners should do their best	0 1
				…expresses positive expectations about what learners are going to achieve	0 1
Clear and structured instructions	11	…presents and explains the subject material in	1 2 3 4	…activates prior knowledge of learners	0 1
		a clear manner		…gives staged instructions	0 1
				…poses questions which learners can understand	0 1
				…summarises the subject material from time to time	0 1
	12	…gives feedback to learners	1 2 3 4	…makes clear whether an answer is right or wrong	0 1
				…makes clear why an answer is right or wrong	0 1
				…gives feedback on the way in which learners have arrived at their answer	0 1
	13	…during the presentation stage, checks whether	1 2 3 4	…ask questions which stimulate learners to reflect	0 1
		learners have understood the subject material		…checks regularly whether learners understand what the lesson is about	0 1
	14	…teaches in a well-structured manner	1 2 3 4	The lesson is built up in terms of clear stages and transitions between stages	0 1
				The lesson builds up logically, going from the simple to the complex	0 1
				Activities and assignments are connected to the materials presented during the presentation stage	0 1
				The lesson offers a good variety of presentation, instruction, controlled practice, free practice, and so forth.	0 1
	15	…gives a clear explanation of how to use didactic	1 2 3 4	…makes sure that all learners know what to do	0 1
		aids and how to carry out assignments		…explains how lesson aims and assignments relate to each other	0 1
				…explains clearly which materials and sources can be used	0 1
Intensive and activating teaching	16	…offers activities and work forms that stimulate	1 2 3 4	…uses diverse forms of conversation and discussion	0 1
		learners to take an active approach		…offers controlled (pre-)practice	0 1
				…lets learners work in group	0 1
				…uses information and communication technology (ICT, e.g., digiboard, beamer)	0 1
				…employs a variety of instruction strategies	0 1
				…varies assignments	0 1
				…varies lesson material	0 1
				…uses materials and examples from daily life	0 1
				…asks a range of questions	0 1
	17	…stimulates the building of self-confidence in	1 2 3 4	…gives positive feedback on questions from weaker learners	0 1
		weaker leaners		…displays positive expectations about what weaker learners have to achieve	0 1
				…compliments weaker learners on their works	0 1
				…acknowledges the contributions made by weaker learners	0 1
	18	…stimulates learners to think about solutions	1 2 3 4	…shows learners the path they can take towards a solution	0 1
				…teaches strategies for problem-solving and referencing	0 1
				…teaches learners how to consult sources and reference works	0 1
				…offers learners checklists for problem solving	0 1
	19	…asks questions which stimulate learners to reflect	1 2 3 4	…waits long enough to give all learners the chance to answer a question	0 1
				…encourages learners to ask each other questions and explain things to each other	0 1
				…asks learners to explain the different steps of their strategy	0 1
				…checks regularly whether instructions have been understood	0 1
				…asks questions which stimulate reflection and learner feedback	0 1
				…checks regularly whether learners understand what the lesson is about	0 1
	20	…lets learners think aloud	1 2 3 4	…provides the opportunity for learners to think aloud about solutions	0 1
				…asks learners to verbalise solutions	0 1
	21	…gives interactive instructions	1 2 3 4	…promotes the interaction between learners	0 1
				…promotes the interaction between teacher and learners	0 1
	22	…clearly specifies the lesson aims at the start of	1 2 3 4	…informs learners at the start of the lesson about the lesson aim	0 1
		the lesson		…clarifies the aims of assignments and their learning purpose	0 1
Adjusting instructions and learner processing to inter-learner differences	23	…evaluates whether the lesson aims have been	1 2 3 4	…evaluates whether the lesson aims have been reached	0 1
		reached		…evaluates learners’ performance	0 1
	24	…offers weaker learners extra study and	1 2 3 4	…gives weaker learners extra study time	0 1
		instruction time		…gives weaker learners extra instruction time	0 1
				…gives weaker learners extra exercises/practices	0 1
				…gives weaker learners “pre- or post-instruction”	0 1
	25	…adjusts instructions to relevant inter-learner	1 2 3 4	…puts learners who need little instructions (already) to work	0 1
		differences		…gives additional instructions to small groups or individual learners	0 1
				…does not simply focus on the average learner	0 1
	26	…adjusts the processing of subject matter to relevant inter-learner differences	1 2 3 4	…distinguishes between learners in terms of the length and size of assignments	0 1
				…allows for flexibility in the time learners get to complete assignments	0 1
				…lets some learners use additional aids and means	0 1
Teaching learning strategies	27	…teaches learners how to simplify complex	1 2 3 4	…teaches learners how to simplify complex problem	0 1
		problems		…teaches learners how to break down complex problems into simpler ones	0 1
				…teaches learners to order complex problem	0 1
	28	…stimulates the use of control activities	1 2 3 4	…pays attention to prediction strategies for reading	0 1
				…lets learners relate solutions to the context of a problem	0 1
				…stimulates the application of alternative strategies	0 1
	29	…teaches learners to check solutions	1 2 3 4	…teaches learners how to estimate outcomes	0 1
				…teaches learners how to predict outcomes	0 1
				…teaches learners how to relate outcomes to the practical context	0 1
	30	…stimulates the application of what has been learned	1 2 3 4	… stimulates the conscious application of what has been learned in other (different) learning contexts	0 1
				…explains to learners how solutions can be applied in different situations	0 1
				…relates problems to previously solved problem	0 1
	31	…encourages learners to think critically	1 2 3 4	…asks learners to provide explanations for occurrences	0 1
				…asks learners for their opinion	0 1
				…asks learners to reflect on solutions or answers given	0 1
				…asks learners to provide examples of their own	0 1
	32	…asks learners to reflect on practical strategies	1 2 3 4	…asks learners to explain the different steps of the strategy applied	0 1
				…gives an explicit explanation of possible (problem-solving) strategies	0 1
				…asks learners to expand on the pros and cons of different strategies	0 1
Learner engagement	33	…are fully engaged in the lesson	1 2 3 4	…pay attention when instructions are given	0 1
				…participate actively in conversations and discussions	0 1
				…ask questions	0 1
	34	…show that they are interested	1 2 3 4	…listen actively when instructions are being given	0 1
				…show their interest by asking follow-up questions	0 1
	35	…take an active approach to learning	1 2 3 4	…ask follow-up questions	0 1
				…show that they take responsibility for their own learning process	0 1
				…work independently	0 1
				…take the initiative themselves	0 1
				…use their time efficiently	0 1

Version EE2018

Any reactions or comments? Please contact: [e-mail]

Table A1. Wright map for the ICALT24 scale (N = 500 teachers teaching 4–12-year-olds).

		Item parameters		Person parameters
		Log δ	SE	Warm’s θ	SE	%
Beyond the basic standards of teaching				−5.28	1.60	.0
				−3.96	0.97	.0
	Shows respect for learners in his/her behaviour and language	−3.61	.42
				−3.25	0.79	.0
				−2.73	0.69	.2
	Maintains a relaxed atmosphere	−2.70	.29
				−2.31	0.63	.2
				−1.95	0.59	.8
	Promotes learners’ self-confidence	−1.85	.21
	Provides effective classroom management	−1.77	.20
	Presents and explains the subject material in a clear manner	−1.73	.20
				−1.63	0.56	.4
	Teaches in a well-structured manner	−1.58	.19
				−1.34	0.54	1.4
	Ensures the lesson proceeds in an orderly manner	−1.32	.18
				−1.06	0.53	1.2
				−0.79	0.52	1.0
	Uses the time for learning efficiently	−.73	.15
Basic teaching skills	Fosters mutual respect	−.58	.15
				−0.53	0.51	3.6
	Engages all learners in the lesson	−.37	.14
	Encourages learners to do their best	−.31	.14
				−0.28	0.51	2.4
	Monitors to ensure learners carry out activities in the appropriate manner	−.10	.13
	Gives a clear explanation of how to use didactic aids and how to carry out assignments	−.19	.13	−0.02	0.51	4.2
				0.23	0.51	4.0
	Gives feedback to learners	.18	.13
Activating students	Offers activities and work forms that stimulate learners to take an active approach	.44	.12	0.49	0.52	8.2
	Asks questions which stimulate learners to reflect	.54	.12
	Gives interactive instructions	.58	.12
				0.76	0.53	4.6
	Let learners think aloud	.84	.12	1.04	0.54	6.2
				1.34	0.56	7.4
Differentiating teaching	Evaluates whether the lesson aims have been reached	1.63	.11	1.66	0.58	6.6
				1.99	0.60	7.4
	Stimulates the application of what has been learned	2.09	.11
	Adjusts the processing of subject matter to relevant inter-learner differences	2.33	.12
				2.37	0.64	9.4
teaching learning strategies	Encourages learners to think critically	2.50	.12
				2.79	0.69	8.8
	Stimulates the use of control activities	2.82	.12
	Teaches learners to check solutions	2.85	.12
Almost perfect				3.28	0.77	7.8
				3.94	0.94	5.8
Perfect				5.18	1.55	8.4

Table A2. Descriptives of the ICALT24 scale.

	Sample (N = 500)		Kindergarten (n = 96)		6–7-year-olds (n = 184)		8–12-year-olds (n = 215)
	µ	σ	µ	σ	µ	σ	µ	σ
Measurement 1	1.57	1.66	1.41	1.73	1.39	1.52	1.80	1.72
Cohen’s δ					−.01		.25
Significance					n.s.		.01

Note: Due to missing values, the addition of Kindergarten, 6–7-year-olds and 8–12-year-olds is lower than 500.