Does smartphone-assisted student feedback affect teachers’ teaching quality?

Figures & data

Figure 1. Theory of action.

Figure 2. Example of an item on a student phone screen.

Figure 3a and 3b. Examples of the feedback in the teacher web environment.

Table 1. Teaching aspects (items) related to the teaching dimensions and practices.

	Teaching aspects (items)	Teaching dimension	Related teaching practices
1	The teacher clearly indicated what I was going to learn.	Clear instructional approach.	Share learning objectives with students at the beginning of the lesson; have a clear beginning of the lesson.
2	The teacher explained the subject matter in such a way that I understood it well.	Clear instructional approach.	Introduce new material in small steps; give examples and model procedures; use clear language; ensure guided practice with students.
3	The teacher connected what I already knew with the lesson well.	Clear instructional approach.	Review on, and connect to, prior knowledge of students.
5	The teacher asked questions about the subject matter, which made me think.	Cognitive activation and deep learning.	Ask high-qualitative questions frequently to students in order to promote deep learning; model assignments; solve subject-matter-related problems together with students.
6	If I did not understand the subject matter, the teacher made sure I understood it.	Adaptive instruction.	Differentiate instruction; processing and/or assessment and adapt activities to the varying needs of students.
7	The teacher made sure we worked hard during the lesson.	Well-organised and structured classroom management.	Ensure an efficient classroom management and organisation; correct disruptive behaviour effectively; have clear classroom rules and routines.
8	The teacher made sure I was not afraid to say if I did not understand something.	Supportive and positive classroom climate.	Create a safe learning environment during the lesson.
9	The teacher created a good classroom climate.	Good student–teacher interaction.	Teachers are supporting and leading.
10	The pace of the lesson was good for me.	Adaptive instruction.	Adapt the pace of the lesson to the different learning needs of students.
11	If I gave an incorrect answer, the teacher explained why it was wrong.	Clear instructional approach.	Correct students’ incorrect answers; repeat instruction (in a different way, when necessary).
12	The teacher ensured my active participation in the lesson.	Clear instructional approach.	Make use of activating learning activities during the lesson.
13	The teacher gave enough time to work on the assignments in the lesson.	Well-organised and structured classroom management.	Provide extended practice and transfer by assignments and ensure independent practice of students.
14	At the end of the lesson, the teacher summarised what we had learned.	Clear instructional approach.	Have a clear ending of the lesson; review on the goal of the lesson with students and check whether the goal is achieved by most students.
15	I now can complete assignments about the subject matter on my own.	This item is meant to measure the effect of the lesson (did the lesson positively affect student learning?).
16	The teachers’ lessons will become even better if he/she…	This is an open question. Students can type their answer in the app to give a tip to the teacher about improvement of the lesson.

Figure 4. Measurement moments in the experimental group and the control group.

Table 2. Background characteristics of the 60 teachers (on M0).

		Experimental group			Control group
		N	(%)	M (SD)	N	(%)	M (SD)
Schools		23	(47.92)		25	(52.08)
Teachers		28	(46.77)		32	(53.33)
Teacher gender	Female	12	(42.90)		13	(40.60)
	Male	16	(57.10)		19	(59.40)
Teacher age				41.21 (2.05)			41.00 (1.93)
Years of teaching experience	General			11.23 (1.47)			14.00 (1.71)
	Math (SE)			10.84 (1.51)			11.09 (1.46)
Nationality	Dutch	31	(96.90)		28	(100.0)
	Other	1	(3.10)		0	(0.00)

Figure 5. The perceived areas by teachers for improvement (percentage of teachers) on the three measurement moments.

Figure 6. The improvement-oriented actions undertaken by teachers during the lessons at M1, M2 and M3 (percentages of teachers).

Figure 7. Percentage of teachers who reported they undertook the improvement-oriented actions outside the lesson on the three measurement moments (M1, M2 and M3).

Figure 8. Average teaching quality (a time period includes five days). Due to a holiday period in the Netherlands, no measurement was conducted in time periods 7, 8 and 9. Time period 18 was a holiday period as well, but because not all schools are off in the same week, two measurements were conducted in that period explaining the small decline in this period.

Table 3. Multilevel models.

	Null model	Model 1	Model 2	Model 3	Model 4
Fixed Effect	Model estimates (standard error)
Intercept (Y00)	2.953* (0.055)	2.934* (0.058)	2.819* (0.060)	2.930* (0.054)	2.819* (0.060)
DateCentred (Y10)		0.0003 (0.0003)	0.007* (0.001)	0.000* (0.000)	0.007* (0.001)
DateC_kwdr (Y01)			–0.000052* (0.000008)		–0.000* (0.000)
Random Part	Variance component (standard error)
Intercept (τ00)	0.076* (0.022)	0.076* (0.022)	0.076* (0.022)	0.064* (0.020)	0.076* (0.022)
Residual (σ^2)	0.026* (0.003)	0.026* (0.003)	0.020* (0.002)	0.025* (0.003)	0.020* (0.002)
Slope (τ11)				0.000^a (0.000)	0.000^a (0.000)
Covariance (τ01)				1.000^a (0.000)	–1.000^a (0.000)
Model summary
Deviance statistic	–66.409	–53.119	–68.991	–56.003	–68.991
# of parameters	3	4	5	6	7

*p < 0.01

^aThis covariance parameter is redundant. The statistic and confidence interval cannot be computed.

Figure 9. Predicted values of teachers’ teaching quality plotted against date.