Making sense of student data in teacher professional development

Figures & data

Figure 1. Graphical display of frames based on the concept of lesson segments.

Table 1. Survey of participants.

Participant	Age	Teaching experience (years)	Grade Level ^a	Grades
Bob	28	3	GSE	10–12
Kimberley	49	12	PUE	10–12
Paula	52	4	GSE	7–9
George	40	10	PUE	10–12
Vincent	49	11	PUE	10–12

Note. ^aPUE = Pre-university education, GSE = General secondary education

Table 2. Survey of the PDCA cycle.

PDCA cycle	Participants actions	Procedures of the PD program	Research data
PLAN	Set learning goals Design lesson Design SQ 1 Set expectations for learning	Instruct teachers how to design SQ 1	Lesson designs
DO	Teach lesson Gather student data using SQ 1 and SQ 2	Researchers design SQ 2 (for example, see Figure 1)
CHECK	Summarize the student data Reflect on the lesson using the students’ data	Offer frames Reflection questions in the online reflection format	Answers from reflection questions in the structured reflection format (‘Do you think that your specific lesson design has had an influence on the outcomes of SQ 1, and if so, how?’)
ACT	Propose intention to change	Offer frames Reflection questions in the online reflection format	Intentions from structured reflection format (‘Which change in your practice would positively influence students’ outcomes?’)

Note. SQ 1 = student questionnaire 1, which aims to collect data on student learning. SQ 2 = student questionnaire 2, which aims to collect data about students’ perception of the regulation of the learning processes.

Figure 2. An example of a student questionnaire (SQ 2) (taken from George’s lesson).

Table 3. Survey of lesson segments.

Lesson segment	Definition
Orientation	Introducing the subject, formulating goals, activating prior knowledge, and planning time and activities.
Test	Assessing to what extent the learning outcomes and/or processes match the pre-set goals.
Reflect	Looking back on results or processes, finding explanations for success or failure, finding improvements
Explanation	Explaining or presenting the content
Context with central questions	Introducing the context and attendant central questions or problems
Questions to Recall and/or Apply	Reproduction: assigning questions or tasks for which knowledge or skills learned earlier have to be literally repeated. Application: assigning questions or tasks in which knowledge acquired earlier has to be applied in new settings.
Answering questions	Answering the questions

Table 4. Illustration of how the analysis on the use of frames was done.

Frames^a	Interpretation	Use
Frame 1	‘The learning outcomes were high because I changed the sequence of my lesson; I started with a context’	‘I want to start with a context’ or ‘I want to present the concept before the answering phase’
Frame 2	‘Students had a role in the reflection phase; they had to present their answers to the rest of the class’	‘I want students to find information themselves, without my help’
Frame 3	‘Starting with a good, challenging context made students very active in answering the questions and resulted in high learning outcomes’	‘I want a good, authentic context that motivates students to find information’

Note. ^aFrame 1 = Lessons can be seen as specific sequences of lesson segments. Frame 2 = Regulation of the learning process can be done by either the teacher, or the students, or shared. Frame 3 = There are different types of contexts and these can have different functions.

Table 5. Survey of the numbers of frames used in this study to interpret (Check phase) or use (Act phase) student data.

Lessons	PDCA phase	No frame	Frame 1*	Frame 2	Frame 3
	ACT**		5	4
1	CHECK		4	3
	ACT		1	5	1
2	CHECK		2	3	1
	ACT	2		4
3	CHECK	2	1	1
	ACT	1	1	1	3
4	CHECK**	2	1	1	1

Note. *Frame 1 = Lessons can be seen as specific sequences of lesson segments. Frame 2 = Regulation of the learning process can be done by either the teacher, or the students, or shared. Frame 3 = There are different types of contexts and they can have different functions. **The PD program started by eliciting an intention (Act phase) and, therefore, the first Check phase is not included in this table. The PD program ended with the interpretation of students’ data from the fourth lesson (Check) and, therefore, the final formulation of an intention (Act phase) was not included in this table.

Table 6. Summary of the development directions after experiencing success or problems in the lessons.

Lesson	Success (S) or problem (P)	Keeps the changes, intentional change in new direction	Keeps the changes, intentional change in same direction	Keeps the changes, intentional change to repeat the success	Does not keep the changes, intentional change in new direction	Stops developing
Bob
1	S	X
2	S		X
3	P	X
4*	S
Kimberley
1	S		X
2	S		X
3	S		X
4	S
Paula
1	P		X
2	S		X
3	S	X
4	S
George
1	P		X
2	S			X
3	n/a	X
4	S
Vincent
1	S	X	X
2	P		X
3	S	X
4	S

Note. *The PD program ended after the interpretation (Check phase) of the fourth lesson. The formulation of intentions after the fourth lesson (Act) is therefore not included in this study.

Table A1. Paula’s development.

	PLAN			CHECK		ACT
Lessons in the PD program	Lesson design	Results SQ 1 compared with expectations	Results SQ 2 compared with the lesson design	Interpretation	Frame use	Intention	Frame use
Regular	Explain (teacher) → Questions to Recall and/or Apply (teacher) → Answering questions (shared)	n/a	n/a	n/a	n/a	I want pupils to be actively searching for information to answer the central question I want to start the lessons by using a context	Regulation Lesson segments
1st lesson	Context (teacher) → Answering questions (students) → Test (Shared)	80% of the students answered correctly, just as I expected	This was scored as I intended the lesson, but half the class felt as though I helped them a lot in answering the questions	‘Starting with a context had a positive influence on the learning outcomes’ ‘Designing a lesson in which students answer the questions rather independently had a positive influence on the learning outcomes’	Lesson segments Regulation	I want to let students answer the questions from the context completely by themselves, without my help	Regulation
2nd lesson	Context (teacher) → Answering questions (students) → Test (Shared)	50% completely correct, 50% for 2/3 correct. This was above my expectations	This was scored as I intended the lesson	“Starting with a relevant context and let them answer the questions independently involved them in the learning process and increased learning outcomes.	Lesson segments and Regulation	I want the students to design a context with attendant questions	Regulation
3rd lesson	Orientation (Teacher) → Context (students) → Answering questions (students)	Above my expectations	Most students answered that they were indeed responsible for the context and answering of the questions	By allowing them to come up with a context, the relevance of the topic increased. This had a positive influence on the learning outcomes.	Regulation	For the next topic, I would like a context with a real profession and I would like to introduce a little bit about the profession myself.	Context type
4th lesson	Context (teacher) →explain (teacher) →answering questions (students)→ test (shared)	As I expected	Mostly as designed, but students answered that they regulated the Test phase, where I intended it to be a shared regulation	By first presenting the doctor-context they have been paying attention to the explanation of the contraceptives. This had positive learning effect.	Regulation and Context type	n/a	n/a

Table A2. Survey of the frames used in the interpretation and use of students’ data in the Check and Act phases of the PDCA cycles.

	CHECK		ACT
Lessons	Interpretation	Frames	Intentions	Frames
Bob
	n/a	n/a	I want to start the lessons by using a context. I will create a context by elaborating on the application questions I normally assign at the end of the lesson	Lesson segments
Lesson 1	I think that starting with a context made students more active and serious this lesson. They also seemed to be more focused and involved.	Lesson segments	I want the students to work in groups when answering the questions, so that they can help each other.	Regulation
Lesson 2	I think that the learning outcomes were high because they worked together in answering the questions.	Regulation	I would like the students to answer the questions in groups and give them a role in the reflection phase.	Regulation
Lesson 3	There was not much of a working atmosphere due to the absence of many students who were on a study week. But because they had to exchange their answers and reflect as a class, they did their job.	n/a	I want to start with a relevant context, followed by relevant attendant questions so that students are challenged to study and learn.	Context type
Lesson 4	I think that the students were engaged in the topic of the lesson because of the relevant movie I showed and the funny attendant questions.	Context type	n/a	n/a
Vincent
	n/a	n/a	I want to start with a context and I would like students to think of one.	Lesson segments and Regulation
Lesson 1	I think that due to the context, the students were immediately focused on the topic. For some students, it seems hard that I’m not presenting everything before they start answering the questions.	Lesson segments and Regulation	I want students to answer the questions independently, without my help I want to limit the reflection to the problematic questions	Regulation n/a
Lesson 2	I think that the students have high learning outcomes because they had to find information themselves. In this way, the students started to think more deeply about the topic.	Regulation	I want students to reflect on their answers themselves.	Regulation
Lesson 3	The students seemed to get trapped in the context. I think that because of this, mainly the students’ discussion skills have been enlarged. Still, the results were good enough.	None	I want a tight connection between the context with attendant questions and the learning goals.	None
Lesson 4	The students clearly learned new concepts due to the tight questions I gave them in class. In addition, I was surprised that they discussed in class on the relationship between evolution and classification.	None	n/a	n/a
Kimberley
	n/a	n/a	I want to start with a context and first demonstrate how to proceed in such a context.	Lesson segments
			I want students to present the answers to each other.	Regulation
Lesson 1	Through using a context, I noticed that their thinking skills were addressed more than before. They started asking questions more deeply. I also noticed that the students made connections with other biological topics. All of this had a positive effect on the learning outcomes.	Lesson segments	I want to start again with a context and formulate attendant questions myself, but this time I won’t demonstrate first.	Regulation
Lesson 2	The use of a context and giving students more independence in answering the questions had a positive effect on the learning outcomes, even though it was hard for students to concentrate at that late hour of the day	Lesson segments	I want to start by using a context again. After that, students will have to think of their pre-existing knowledge, followed by a students’ task to find information in the textbook. I want to formulate good attendant questions with the context, so that students really have to find the details of the learning goals and learn enough.	Regulation n/a
Lesson 3	As a result of using a context and formulating good attendant questions, they learned what I wanted them to learn	Lesson segments	I want a relevant, interesting and detailed context and have students formulate the learning goals.	Context type and Regulation
Lesson 4	n/a	n/a	n/a	n/a
George
	n/a	n/a	I want to start by using a context that I normally present after my explanation. This will be followed by my explanation and then the students can find information in their textbooks.	Lesson segments and Regulation
Lesson 1	Starting the lesson with a context and my presentation really helped students to understand the complicated concept. I also think that students were more active in the lesson.	Regulation	I want to present the context by using an animation or short movie to get students interested and motivated. I want students to find information directly after the presentation of the context, without my explanation.	Context type Lesson segments and Regulation
Lesson 2	Starting with a relevant context helped the students to learn the concept	Context type	I want to try out the same lesson set-up for a less appealing topic to see if I can make that more interesting.	None
Lesson 3	n/a	n/a	I want to discover a process together with the students by using multiple contexts in a row.	Lesson segments
Lesson 4	I think that I have to try harder to motivate the pupils to work. I found the topic boring. It wasn’t ‘my day’ today.	n/a	n/a	n/a

Note. Frames: ‘Lesson segments’ = Lessons can be seen as specific sequences of lesson segments (Frame 1), ‘Regulation’ = Regulation of the learning process can be done by either the teacher, or the students, or shared (Frame 2) and “Context type = There are different types of contexts and these can have different functions (Frame 3). The interpretations and intentions are literal quotes from the participants.