Instructional Change after Participating in a Mathematics Professional Development Program: An Exploration of Impact

Abstract

This research study explored teachers’ self-reported uptake as well as observed instructional change after participating in a year-long professional development (PD) program focused on supporting the learning and teaching of transformations-based geometry. Analyses illuminate the degree and nature of the pedagogical shifts made by teachers who participated in this highly specified, videocase-based PD program. The treatment teachers’ lessons started out significantly lower on most aspects of their instructional quality relative to the control teachers’ lessons prior to the PD. However, the control teachers stayed the same or showed a decline in their instructional quality scores, whereas the treatment teachers made significant gains after the PD that brought them to the same level as the control teachers. The article concludes by considering how engagement with intentionally designed videocases during the PD may have contributed to the teachers' improvement in targeted dimensions of instructional quality.

Introduction

For many years mathematics educators, researchers, and policy makers have sought to understand the mechanisms through which mathematics teacher professional development (PD) improves student understanding and engagement (Hill et al., 2013; Kennedy, 2016; Koellner & Jacobs, 2015). Instructional change has been posited as the critical link between PD that promotes mathematics teachers’ knowledge and increased student learning (Desimone, 2009). In general, there is consensus that PD should foster improvements in teachers’ pedagogy, for example by encouraging teachers to provide increased access to rigorous mathematics content for all students (Kennedy, 2016).

A multitude of PD models have been designed to help teachers learn specific instructional strategies aligned to these recommended mathematical practices (Sztajn et al., 2017). Video-based PD is an especially popular and powerful tool in mathematics PD, allowing teachers to interpret and reflect on targeted instructional practices (Borko et al., 2011). Some video-based PD programs have indeed demonstrated an impact on classroom practice. For instance, Grant and Kline (2010) found that elementary school teachers’ reflection on a colleagues’ video supported their learning and subsequent changes in teaching. The teachers participating in this program made changes that included anticipating student thinking, teaching in more conceptual ways, changing their questioning patterns, and allowing students more time for inquiry without guidance. Similarly, Koellner and Jacobs (2015) investigated changes in classroom practice after middle school teachers participated in a two-year video-based PD model, the Problem-Solving Cycle (Borko et al., 2015). They reported incremental change in four areas including richness of the mathematics, student explanations, errors and imprecision, and student participation. The most notable changes were found in the ways that teachers engaged students in explanations and reasoning through mathematical tasks.

At the same time, the fact that a number of promising video-based PD programs did not have a significant effect on practice speaks to the difficulty of this effort. Jacob et al. (2017) investigated a commercially available mathematics PD that was intended to improve both teachers’ knowledge and their ability to elicit more student thinking and explanations. They found some evidence of increased teacher knowledge but no changes in instructional practice nor student achievement. Santagata et al. (2010) studied the impact of a large-scale video-based PD on teachers’ knowledge, practices and student learning. Their results did not indicate any significant changes in teacher knowledge or practice, which they partially attributed to the complexity of providing PD for teachers working in low performing schools.

Many teachers continue to struggle to reconceptualize their mathematics teaching and shifting to new instructional strategies has continued to prove difficult (Gallimore & Santagata, 2006). Kennedy (2016) explains that “any new idea offered by PD requires not merely adoption but also abandonment of the old way of thinking” (p.4). Changing practice requires not only shifts in the fundamental ways teachers think about teaching and learning but also their motivation to incorporate new instructional strategies. It is not yet clear why certain programs, whether video based or not, more effectively support instructional changes relative to others. Our research endeavors to add to this emerging literature base by describing a PD program that did have a positive influence on instruction and relating those instructional changes to the videocase design of the program.

Conceptual framework

The present study focuses on the classroom impact of the Learning and Teaching Geometry professional development program, which was based on a set of design features grounded in the literature on effective PD. The PD program follows a specified mathematics learning trajectory to support teachers’ understanding of transformations-based geometry. Teachers engage in challenging mathematical problems and view strategically selected and sequenced video clips that are intended to foster productive conversations about teaching and learning. Development of the PD program was conceptually grounded within several areas of the research literature, including the adaptive-specified nature of professional development programs, the power of video to support teachers’ learning, and the need for PD focused on geometric transformations. In this section, we briefly discuss these three target areas with a focus on how they related to the Learning and Teaching Geometry PD and to our study of the impact of participation on classroom practice.

Adaptive versus specified professional development programs

Professional development structures and features are important to consider when studying the impact of a mathematics PD program on teachers’ knowledge and instruction. Previously we posited that PD models fall on a continuum from adaptive to specified (Borko et al., 2011; Koellner & Jacobs, 2015). On one end of the continuum, there are adaptive models in which the learning goals and resources are derived from the local context and shared video is generally from the classrooms of one of the participating teachers. One-on-one instructional coaching (Desimone & Pak, 2017), video clubs (Sherin et al., 2009) and the Problem-Solving Cycle (Borko et al., 2015) are examples of well-researched adaptive mathematics PD models.

On the other end of the continuum, specified models of PD typically incorporate published materials that specify in advance teacher learning goals. In specified PD, the resources, tasks and video clips to be used in the PD are typically pre-selected to ensure a particular PD experience. One example of a well-known specified PD program is Cognitively Guided Instruction, which includes a textbook written by the research team and video clips intended to elicit inquiry and discussion focused on students’ thinking about arithmetic concepts (Carpenter et al., 1989). Similarly, the Learning and Teaching Linear Functions PD program incorporates math tasks, video cases, and other materials designed to deepen teachers’ understanding of ways to conceptualize and represent algebra content within their classroom practice (Seago et al., 2004; Seago et al. 2017). Understanding where a particular PD model is situated on the adaptability continuum enables not only differentiation between models, but also helps to clarify how and why a given model is structured in a particular manner.

As a specified model, the Learning and Teaching Geometry PD program incorporates a variety of resources in the service of supporting students’ understanding of transformations-based geometry. Resources include videoclips that highlight students’ geometric thinking and/or instructional moves and detailed facilitation supports that help maintain the intended mathematical and pedagogical storyline (Borko et al., 2014). These resources help to ensure that the PD will be implemented in alignment with the developers’ intentions and the explicitly stated learning goals, and that adaptations will be appropriate and productive. From the perspective of potential impact on instruction, the specified nature of the Learning and Teaching Geometry PD makes clear the targeted practices that teachers are directed to examine during the workshops and might later implement in their classrooms. For example, the videocases encourage teachers to notice and attend to focal math content, student sense-making, linking between representations, multiple solution methods, student explanations, questioning and reasoning.

The power of analyzing classroom video

Several recent and comprehensive reviews of the literature on video in PD point to the value of video as a learning tool that can promote improvements in instructional practice (Gaudin & Chaliès, 2015; Major & Watson, 2018). Incorporating video in a PD environment offers great potential for teachers to unpack the relationship between pedagogical decisions and practices, students’ mathematical work, and the mathematical content. Professional learning programs and courses that incorporate video representations of practice may foster the development of teachers’ noticing skills (Roller, 2016). As they attend to and make sense of instructional events viewed during PD workshops, teachers are also likely to consider the implications for their own practice (Santagata & Bray, 2016). In other words, what teachers notice appears directly relevant to how they elect to carry their learning into their classrooms (Sherin & van Es, 2009).

In specified PD programs, analyzing video of unfamiliar teachers’ practice offers a window into alternative teaching practices (Givvin et al., 2005; Zhang et al., 2011). Targeted viewing of another teacher’s classroom can prompt teachers to consider more wide-ranging instructional possibilities, while at the same time the experience can help teachers to see themselves in others and reflect on their own practice. Kleinknecht and Schneider (2013) found that teachers showed more emotional and motivational involvement when watching videos of other teachers’ lessons relative to watching videos of themselves.

In the present study, we anticipated that viewing video as part of the Learning and Teaching Geometry PD would have an influence on teachers’ desire and understanding to make instructional changes, as well as helping to determine the specific nature of those changes. In the PD program, video is embedded in an activity system that includes an intentional sequence of video along with carefully designed pre- and post-viewing experiences (van Es et al., 2019). As a result of teachers watching and discussing clips selected to showcase particular mathematical content and representations, instructional decisions focused on sense-making, and/or student explanations and reasoning, we expected that they would become not only more knowledgeable but also motivated to take up similar practices in their classrooms in order to support their students’ learning.

The need for professional learning in the area of geometric transformations

A critical feature of effective PD is that it addresses a problem of practice, meeting the professional needs of teachers (Scribner, 1999). Starting in middle school, the Common Core State Standards for Mathematics (CCSSM) contain a strong and consistent focus on geometric transformations—including their mathematical properties, how they can be sequenced, and their effect on two-dimensional figures in a coordinate plane. Furthermore, the standards state that congruence and similarity should be defined in terms of rotations, reflections, translations, and dilations. This increased emphasis on transformational geometry represents a dramatic shift from previous state standards (Teuscher et al., 2015; Tran et al., 2016) and much of the content is likely to be relatively new, especially to middle school teachers.

The Learning and Teaching Geometry PD program is closely aligned with both the content and practice standards in the CCSSM. The PD materials are intended to help teachers interpret and better understand how to implement a relatively large number of content and practice standards, such as making use of geometric structure, attending to precision, constructing arguments, and critiquing the reasoning of others (Seago et al., 2017). Of particular interest in the present study is not only the nature of teachers’ uptake from the PD as seen in their geometry lessons, but also the degree to which their mathematics lessons on any topic were impacted. In other words, to what extent does taking part in the Learning and Teaching Geometry PD influence teachers’ classroom practices broadly, across mathematical domains?

Context

Our paper is based on an efficacy study of a PD program. This randomized control study considered whether the Learning and Teaching Geometry PD program produced a beneficial impact on teachers’ mathematics knowledge, classroom teaching practices, and their students’ knowledge in the domain of geometry. In another report, we have written about the overall impact of the program (Jacobs et al., 2020); here we take an in-depth look at the impact on classroom teaching.

The PD program

The professional development program is targeted for teachers serving grades 6-12 and is designed to be implemented by a knowledgeable facilitator using a set of well-specified resources. The program supports 54 hours of face-to-face PD using a series of five modules, including a Foundation Module and four Extension Modules. The materials follow a specified sequence of events and offer access to specific and increasingly complex mathematical concepts that are presented within the dynamics of classroom pedagogy.

The modules use a total of 45 video clips, all of which are unedited segments selected from real classroom footage. The clips represent a range of grade levels, geographic locations, and student populations across the United States. By focusing on classroom video from across multiple and varied contexts, the materials provide insight into what an emerging understanding of similarity looks like as well as specific instructional strategies that can foster this understanding.

Within each module, the video clips were arranged to create a series of videocases, constructed using a “video in the middle” design. In this design, the viewing of video is sandwiched between pre- and post-viewing activities that are intended to support teachers’ engagement in and learning from the clips. The use of this design was based on the developers’ conjecture that productive conversations about the video footage required a highly intentional surrounding framework to ensure attention to the targeted mathematical content and practices. In addition, as part of most videocases, the teachers reflected on their own instructional practice and collaboratively discussed ideas for promoting student learning.

Example videocase

The Randy Videocase (see Figure 2) is one example that helps to illustrate the nature of the videocases and provide a glimpse into teachers’ experiences during the PD. This videocase has been the subject of several previous publications. Like all of the videocases used as part of the PD program, the Randy Videocase was designed to encourage teachers to deeply consider what a transformations-based approach to a specific geometric problem looks like, while at the same time attending to student thinking and pedagogical options and opportunities.

Teachers encounter the Randy Videocase during Session 3 of the Foundation Module, when they begin an in-depth exploration of dilation. As part of the pre-video activity, teachers work on the Rectangle Problem and share their ideas and solution methods. Solving this open-ended problem about which rectangles are mathematically similar allows the teachers not only to become familiar with the content, but to consider how their approach differs from others, as the Rectangle Problem readily lends itself to at least half a dozen different methods to determine similarity. There is time for individual problem solving as well as a whole group conversation, with a focus on sustained engagement in mathematical reasoning.

Next the teachers watch a short, 1.5-minute clip of an eighth-grade student named Randy solving the Rectangle Problem. Before watching the clip, the teachers look over a “lesson graph” that offers a brief illustration of the entire lesson and shows where this clip is situated. In the video Randy demonstrates to his classmates how he used tracing paper to determine whether the given rectangles were similar. Randy correctly explains that some of the rectangles share a common center of dilation and diagonal and are therefore dilation images. When prompted by his teacher, Randy provides a counterexample to show that the non-similar rectangles do not share a common diagonal line through their centers of dilation.

During the post-video activity, teachers are asked to use Randy’s dilation method for themselves to once again solve the Rectangle Problem. Then, as part of a whole group discussion, the teachers consider in a more precise way how Randy was thinking about the similarity and how they might teach this concept to their own students. The discussion is meant to hit on a variety of topics ranging from learning mathematics content to unpacking student thinking to considering pedagogical moves. As the participants play the role of both “learners” and “teachers,” they consider a variety of perspectives, which may ultimately lead to the thoughtful application of their learning in their own math classrooms.

Method and analysis

Sample

Participants in the Efficacy Study were 103 mathematics teachers serving grades 6-12 (47% middle school, 53% high school). The participants were recruited from several school districts in urban areas in two US states. These districts had established relationships with members of the research team and provided approval for interested teachers to participate. Participation was voluntary and we obtained informed consent from all participants. Teachers in both the treatment and control groups were compensated for their participation each year they attended PD workshops and/or contributed to data collection efforts.

Randomization was conducted at the school level, with 32 schools (49 teachers) assigned to the treatment group and 35 schools (54 teachers) assigned to the comparison group. The sample encompasses a diverse student population within and across two geographic regions of the United States. Table 1 provides information about the background characteristics of the participants. No significant differences were found on teachers’ backgrounds by treatment and control groups. For this study, observation ratings of pre- and post-PD lessons were obtained for a subsample of 88 teachers. There were no differences between teachers who were observed and those who were not observed and/or left the study on all but one of the measured background variables. Teachers who left the study or were not observed were less likely to have had an educational background in mathematics (p<.05).

Table 1. Demographic characteristics of teacher participants – percent of teachers (N = 103).

Demographic characteristic	Treatment (n = 49)	Control (n = 54)
Current grade level
Middle school	41%	52%
High school	59%	48%
Grade levels taught
Elementary	22%	21%
Middle school	61%	67%
High school	80%	80%
Years Teaching
0-5	44%	37%
6-10	23%	33%
11-12	25%	24%
>20	8%	6%
Teaching certification
Elementary education	20%	19%
Elementary math	2%	2%
Middle school math	43%	46%
Secondary math	80%	80%
Undergraduate or graduate degree in Math, Math Education	78%	80%

Note: No differences detected between treatment and control teachers’ backgrounds.

Professional development workshops

The PD workshops for treatment teachers began in Summer 2016 and continued throughout the 2016-17 academic year. Nine full days of professional development were offered, beginning with the five-day Foundation Module followed by four days of Extension Modules. Control teachers were offered the opportunity to participate in the same Learning and Teaching Geometry PD workshops during the 2017-18 school year, once pre- and post-PD data collection was completed. The same facilitator led all of the workshops after taking part in an extensive facilitator preparation process that included a multi-faceted assessment of fidelity. On average the treatment teachers attended seven of the nine full-day workshops.

Data collection and measures

Data on the impact of the PD on classroom practice included videos of mathematics lessons and reflections by the treatment teachers on what they took up and learned from their participation.

Videotaped classroom observations

Baseline video from all teachers was collected in Spring 2016 (prior to the summer workshops) and post-PD video was collected in Spring 2017. Teachers were filmed teaching any mathematical topic and class of their choosing; they were simply asked not to be filmed on a day when they were administering a test. Although we acknowledge the legitimate concern researchers have raised about the generalizability of findings based on individual lesson observations (Cohen & Goldhaber, 2016), due to logistical and financial constraints we were only able to film single lessons from teachers at each time point. In an attempt to mitigate this concern to some degree, we utilized a rigorous “observational system” for scoring the lessons (Hill et al., 2012) that included applying an observational protocol closely aligned with research-based best practices as well as the design and goals of the PD program, using knowledgeable and experienced coders, and establishing both initial and midpoint inter-rater agreement.

Teachers’ mathematics lessons were rated using the Math in Common teacher observation protocol (Perry et al., 2015). This protocol, developed as part of WestEd’s Math in Common study of K12 mathematics instruction, incorporates eight items that capture various elements of lesson quality. The Math in Common protocol includes three dimensions: (1) teacher work to support the richness of the mathematics, (2) student engagement in mathematical practices, and (3) actionable mathematics for students (see Table 2). The first two dimensions are rated on a 4-point scale (1= Not Present, 2 = Low, 3= Middle, 4 = High), while the mathematical content dimension is rated on a 3-point scale (1 = Novice, 2 = Apprentice, 3 = Expert).

Table 2. Math in common observation protocol.

Teacher work to support richness of mathematics	Student engagement in math practices	Actionable mathematics for students
Linking between representations Multiple solution methods Mathematical sense-making	Explanations Questioning and reasoning	Accuracy and coherence of math content Access to math content Student agency & ownership

Two members of the research team established initial inter-rater agreement of 92.5% on the Math in Common protocol overall, and between 80-100% on each item. To ensure that they were applying the items consistently throughout the coding process, the coders established mid-point agreement approximately half-way through the set of videotaped lessons. Midpoint inter-rater agreement was 91% on the protocol overall and between 80-100% on each item.

To investigate the impact of the PD on instruction, we focused on the three dimensions of the Math in Context protocol rather than single items. Because these dimensions measure underlying latent traits that are captured with more than a single item, we combined items based on procedures used by Hill et al. (2008). Using Cronbach’s alpha to compute the internal consistency for each dimension suggests that it is on the low end of the “reasonable" range for richness (three items, α=.67) and higher for mathematical practices (two items, α=.86) and actionable mathematics (three items, α =.83) (Taber, 2018). Despite having acceptable inter-rater agreement and internal consistency for the dimensions, we acknowledge there are unaccounted for factors (such as different implementation conditions, content areas, and rater pools) that could be sources of error and have an influence on our estimates of reliability (Hill et al., 2012). Therefore, we advise caution in interpreting these estimates as the true reliability of the instructional quality ratings is likely to be lower.

Teacher end of year reflection: uptake survey

Treatment teachers were given an uptake survey at the end of the PD (in Spring 2017) and again one year later (in Spring 2018). This survey was aimed at understanding what the teachers felt they learned from the workshops and what PD resources they were using or planning to use in their own classrooms. Teachers rated their uptake as either: generating new instructional materials based on what they learned from the PD, using many of the materials and/or pedagogical strategies in a substantive way with their students, using some of the materials and/or pedagogical strategies, or not using any of the materials and/or pedagogical strategies. Open-ended questions prompted teachers to describe in more detail the specific math content and pedagogical strategies from the workshops that impacted their teaching.

Quantitative analyses

Analyses based on the videotaped classroom observations focused on the three broad instructional quality dimensions measured by the Math in Common protocol and included pre-PD and post-PD data from the treatment and control groups. The final sample included observations from 88 teachers. Less than 3% of the sample lacked one (but not both) of the videotaped lessons. For these teachers, ratings of the missing observations were replaced by the average from the observed data for the group. Repeated measures analysis of variance were used to identify differences by participation in PD, time, and their interaction, followed by paired samples t-tests to identify change over time within groups and independent samples t-tests to identify change between groups within time. Spearman rank correlations measured the associations between the teachers’ responses to the uptake survey and changes in instructional quality.

Results

Changes in teachers’ instructional quality over time: Examining three broad dimensions

Clear patterns of change over time emerged on the three broad dimensions of instructional quality (teacher work to support richness of the mathematics, student engagement in mathematical practices, and actionable mathematics for students). Different trajectories for the treatment and control teachers’ videotaped lessons were identified for mathematical practices and actionable mathematics while both groups appeared to improve in richness. Results of the analyses within each dimension are described in the sections that follow (see Figure 1).

Figure 1. Changes in richness, engaging students in math practices, math content.

Figure 2. Overview of the Randy Videocase.

Teacher work to support richness of the mathematics

The richness dimension is intended to capture opportunities teachers provide for their students to engage in rich mathematical problem-solving. This dimension includes the following three codes: (1) linking between representations, (2) multiple solution methods or procedures, and (3) mathematical sense-making. The repeated measures analyses identified a significant time effect for this dimension (F = 4.893, p<.05). As shown in Figure 1, lessons from both the treatment and control groups appeared to improve from pre-PD to post-PD, but only the treatment group improved significantly (t = 2.722, p<.01, ES=.25). On average, the control teachers’ lessons received significantly higher ratings than the treatment lessons at pre-PD (t = 4.381, p<.05); however, no difference was detected on their post-PD scores. On the whole, after taking part in the Learning and Teaching Geometry PD, the treatment teachers showed measurable growth in providing their students with a richer mathematical classroom experience – including increased opportunities to make sense of mathematical content, more frequent connections between representations, and more often eliciting multiple solution strategies.

Student engagement in mathematical practices

The mathematical practices dimension is intended to capture evidence of students’ involvement in doing mathematics and includes two codes: (1) students provide explanations and (2) students engage in questioning and mathematical reasoning. Figure 1 illustrates the different patterns for the treatment and control groups based on a significant interaction effect for this dimension (F = 9.016, p<.01). Within group comparisons identified significant improvement from pre-PD to post-PD only in the treatment group (t = 3.236, p<.01) and this improvement was significantly larger than for the control group (t = 9.016, p<.01, ES=.61). On their baseline lessons, the control teachers were rated significantly higher than the treatment group (t = 8.156, p<.01) but again, by the conclusion of the PD workshops, no difference was detected between the groups. These results indicate that by the end of the PD the treatment teachers were eliciting student thinking more often and providing increased opportunities for their students to provide explanations and reason mathematically.

Actionable mathematics for students

The actionable mathematics dimension focuses on students’ equitable access to accurate and coherent mathematical content and includes three codes: (1) the accuracy and coherence of the mathematics, (2) access to mathematical content for all students, and (3) student agency, ownership, and identity of their mathematical contributions. A significant interaction effect was found for this dimension (F = 8.626, p<.01) indicating different trajectories in the treatment and control lessons (see Figure 1). The control group declined significantly from pre-PD to post-PD (t=-2.207, p<.05) while the treatment group improved, although not significantly. Similar to the other instructional quality dimensions, the control teachers’ lessons were at a significantly higher level than the treatment teachers when they began the study (t = 8.031, p<.01), but no difference remained at post-PD. These findings indicate that over time, the treatment teachers conducted lessons with increased coherence, access for students, and student agency that supported their learning of the mathematical content.

Teachers’ self-reported uptake from the PD workshops

Table 3 presents teachers’ responses to the uptake survey about the nature of their learning and use of the information and resources from the PD workshops. Almost all of the teachers (85%) who took part in the workshops responded that they used at least some of the materials in their classrooms at post-PD, and the same number (85%) said they did so one year later. At post-PD a large majority claimed that they were teaching “PD related content” in a manner that was different from before the PD (80%), that the workshops influenced their pedagogical strategies (73%), and that they planned to make additional changes in the future (73%). One year later, about half the group claimed that they were continuing to change their teaching in these regards. At post- PD most teachers (60%) could trace their new understanding of pedagogical practices back to a particular videoclip (or clips) they viewed in the workshops. Similarly, one year after the PD, most teachers (56%) reported there were aspects of the PD that influenced their classroom practice.

Table 3. Teachers’ self-reported uptake of the Learning and Teaching Geometry (LTG) PD across time – percent of treatment teachers.

Survey question	Response	Survey 2017 (post PD)		Survey 2018 (one year follow-up)
		n	%	n	%
Q1. How would you characterize your use of the LTG materials in your classroom?	Many/New	41	29%	34	32%
	Some		56%		53%
	None		15%		15%
Q2a. Have you done anything differently in terms of teaching LTG related content this school year (compared to last year)?	Yes	40	80%	34	53%
	No		20%		47%
Q2b. Do you plan on doing anything differently in the future in terms of LTG related content?	Yes	40	98%	32	41%
	No		2%		59%
Q3a. Have the workshops influenced the pedagogical strategies you used this school year (compared to last year)?	Yes	40	73%	31	58%
	No		27%		42%
Q3b. Do you plan to do anything differently in the future?	Yes	37	73%	29	45%
	No		27%		55%
Q4a. Are there any video clips from the LTG PD that stand out to you as influencing your understanding of the math content?	Yes	38	29%	32	28%
	No		71%		72%
Q4b. Are there any video clips from the LTG PD that stand out to you as influencing your understanding of pedagogical practices?	Yes	37	60%	31	35%
	No		40%		65%
Q5. Are there any other aspects of the LTG PD that you think influenced your classroom practice?	Yes	35	63%	32	56%
	No		37%		44%

¹Question 1 item responses were: New = Generate new materials for students; Many = Use many LTG PD materials and/or pedagogical strategies; Some = Use some LTG PD materials and/or pedagogical strategies; and None = Use none of LTG PD materials and/or pedagogical strategies.

Correlations between teachers’ survey responses and the quality of instruction dimensions highlight a relationship between their self-reported use of the workshop materials and their observed teaching quality. Specifically, teachers’ self-reported plans to implement ideas, strategies, and resources from the PD were significantly related to improvements in two instructional quality dimensions. Teachers’ plans to change their pedagogical strategies based on the post-PD survey (Q3b) was correlated with improvement in both richness of the mathematics in their lessons (r=.47, p<.01, n = 35) and engaging students in mathematical practices (r=.46, p<.01, n = 35). Taken together, these findings suggest that the increased instructional quality of the treatment teachers’ observed mathematics lessons can, at least to some degree, be traced to their participation in the PD and the transfer of what they learned from the workshops to their own classrooms.

Conclusion

Instructional improvements found across dimensions and lessons

This study aimed to understand the impacts of a specified, video-based PD program on teachers’ instructional practice. For reasons that remain unknown, the control group entered the study scoring higher on all of the broad dimensions of instructional quality captured by the Math in Common observation protocol. The control teachers were not significantly different from the treatment teachers on any of the background characteristics that data were collected on, therefore we are unable to speculate as to why their mathematics lessons scored higher at baseline. However, after participation in the PD, the treatment group made dramatic improvements in their teaching, enabling them to catch up to the control group.

The characteristics of high-quality instruction emphasized in the PD as well as the Math in Common protocol are aligned with theories of specific teaching practices that support student learning (Stein et al., 2017). The fact that 88% of the treatment teachers showed improvement on at least one instructional quality dimension suggests that the PD supported them to become better able to lead more effective mathematics lessons. In contrast, 66% of the control teachers improved on at least one dimension, a significantly (p<.05) smaller percentage relative to the treatment teachers.

It is important to highlight the fact that the treatment teachers’ gains in instructional quality were not limited to lessons featuring geometric content. The videotaped lessons captured a variety of mathematical topics, which the research team sorted into five categories based on the main content focus. At pre- and post-PD respectively the treatment teachers’ lessons covered geometry (57% and 48%), (pre)algebra (16% and 26%), (pre)calculus (5% and 10%), statistics (16% and 12%), and other content (6% and 4%). There were no significant differences on any of the instructional quality dimensions by topic, meaning that teachers’ improvements were consistent across content areas despite the fact that the mathematical focus of the PD was relatively narrow in scope. Also, the treatment teachers did not differ significantly from the control teachers on the percentages of lessons within the five content areas, implying that instructional quality differences between the two groups is not due to differences in the content focus of their lessons.

Conjectures about the mechanisms leading to instructional change

The field of mathematics education presently includes relatively few randomized control studies that report changes in instructional practice following an intervention. Although the data from this study do not enable us to identify with certainty the mechanisms that led to the treatment teachers’ changes in instructional practice, we can offer some conjectures as to why specific types of improvements may have occurred. By featuring a set of sequenced videocases that focus on mathematics content as well as particular pedagogical practices, the PD program was designed to prompt teachers to systematically reflect on targeted aspects of learning and teaching transformations-based geometry. Professional learning experiences that highlight a problem of practice within a video-based context, encourage self-reflection, and showcase new instructional options have been identified as especially effective for promoting teacher change (Santagata & Bray, 2016).

As discussed previously, the videocases within each PD workshop are intended to help teachers learn specific mathematics content as well as strategies for supporting students’ learning, in ways that relate to the content at hand as well as more generally across mathematical topics. The activities and conversations teachers engage in throughout each videocase closely correspond to the dimensions of instructional quality captured by the Math in Common observation protocol. Taken together, the three activities that comprise each videocase (pre-video discussions, video viewing, and post-video discussions) address not only the three broad dimensions but (most often) all of the specific items within each dimension. We illustrate this close correspondence between the videocases and the instructional quality dimensions by again considering the Randy Videocase described earlier (see Table 4). During the pre-video activity, teachers solving and discussing the Rectangle Problem corresponds to the richness codes “multiple solution methods” and “sense making,” as the teachers are encouraged to use different strategies and consider the degree to which each strategy is reasonable. Wearing a “student hat,” teachers use mathematical practices when they offer explanations and share their thinking or ask questions of one another. Lastly, the mathematics is made actionable when teachers are asked to use accurate mathematical language to describe and justify their ideas.

Table 4. Connections between observation protocol and learning and teaching geometry PD.

Learning and Teaching Geometry PD	Teacher work to support richness of mathematics	Student engagement in math practices	Actionable mathematics for students
Pre-Video Explore & discuss different ways to solve the rectangle similarity problem	Linking between representation Multiple solution methods Mathematical sense-making	Explanations Mathematical reasoning	Accuracy and coherence of math content Access to math content Student agency & ownership
Video View and discuss Randy’s dilation method, explanation & reasoning in solving the rectangle problem.	Linking between representation Multiple solution methods Mathematical sense-making	Explanations Mathematical reasoning	Accuracy and coherence of math content Access to math content Student agency & ownership
Post Video Examining & connecting representations of similarity	Linking between representations Multiple solution methods Mathematical sense-making	Explanations Mathematical reasoning	Accuracy and coherence of math content Access to math content Student agency & ownership Accuracy of mathematical content

Then, during the video, Randy engages in the mathematical practice of providing an explanation for his idea about why a procedure works, as well as the practice of drawing on mathematical reasoning by using a counterexample to show how a procedure generalizes. In both the videotaped lesson as well as the PD, Randy’s explanation and reasoning serves as an impetus for the rest of the class to further explore his use of tracing paper and dilation lines as a test for similarity across figures, helping teachers to access this relatively challenging content. In addition, the video clip offers a clear demonstration of student agency in determining how to solve the problem, as the teacher explicitly ascribes ownership of this particular method to Randy.

There are correspondences to both the richness of the mathematics and the actionable mathematics dimensions during the post-video activity. In terms of richness, the teachers are asked to consider how to help students make sense of similarity as involving an infinite set of similar figures, including how to represent this idea based on the given dilation images. In addition, the post-video activity prompts teachers to attend to the development of mathematical coherence by thinking about the distinction between similar rectangles and similar triangles, and how they can ensure access to these challenging math ideas for all learners.

This brief overview of the Randy Videocase is meant to provide a glimpse into the types of learning opportunities that teachers participating in the PD were afforded, as well as how such experiences might be viewed through the lens of the Math in Common protocol. Throughout each workshop, teachers engaged with videocases that showcased mathematics classrooms across the country and they took part in discussions that moved beyond a singular mathematical focus on transformations-based geometry. The conversations often challenged teachers to think broadly, both about mathematics and their approach to teaching mathematics, which might have inspired new ideas and motivated the types of changes in instruction that were documented from pre- to post-PD. As previously noted, there were no differences in the instructional quality of the treatment teachers’ lessons based on mathematical topics, indicating that their pedagogical improvements generalized across content areas.

Implications for educators and future research

The purposeful iteration of professional learning activities that include modeling, viewing, discussing, and reflecting on targeted instructional strategies during a year-long PD program may offer the needed ingredients as well as sufficient repetition for successful uptake. Other PD models have similarly found such components and their repetition to be effective, such as the Visual Access to Mathematics project (DePiper et al., 2021) where teachers engage in structured, online experiences that included planning lessons collaboratively, implementing the lessons, sharing student work, and reflecting on their practice. In the Problem Solving Cycle teachers also engage in multiple cycles of solving a rich math problem, planning and teaching a lesson involving that problem, and then using video to reflect on their own instruction as well as that of their colleagues (Jacobs et al., 2009; Koellner & Jacobs, 2015). Based on the positive outcomes of the Problem-Solving Cycle model, Visual Access to Mathematics, and the Learning and Teaching Geometry PD in terms of changing instructional practice, it seems plausible that this sort of sequence, when repeated often enough, contributes to a trajectory of teacher learning and improvement. In terms of the PD that is the focus of this paper, the use of videocases with specified activities, extensive visual resources and discussion questions may have been critical to organizing this type of productive, iterative sequence for the teachers to engage in. An important takeaway for educators may be the intentional use of this sequence in ongoing professional learning efforts, particularly if the primary goal is improving classroom practice.

The study reported here raises many questions regarding the impetus behind instructional change and why taking part in the PD led to improvements in classroom practice. More research is necessary to pinpoint the specific causal mechanisms associated with teacher change, including research on the nature of teachers’ participation during the PD, changes in their beliefs or vision, and their learning of the mathematics content. Numerous factors likely play a role in whether and how teachers elect to modify their practice (Gitlin & Margonis, 1995). Chazan et al. (2016) argued that the conceptualizations of teaching in mathematics education underestimate some of those factors, such as the role of societal, institutional, and mathematical contexts and the actual autonomy that individual teachers have in making choices about and implementing the recommended mathematical practices.

In addition, future research could explore whether teachers with varying levels of instructional quality at baseline show different patterns of uptake and growth after taking part in PD opportunities such as the one described in this paper. Due to our limited sample and the fact that the control teachers as a group entered the study with significantly higher ratings of instructional quality than the treatment teachers, we can not be certain that the teachers who participated in the Learning and Teaching Geometry PD would have exhibited such large gains in practice if they had started out at the same level as the treatment teachers. The quality of a teacher’s classroom instruction is notoriously challenging to quantify and trace over time (Schoenfeld, 2013), however more nuanced and smaller scale explorations of the relationship between teachers’ initial level of instructional quality and their professional learning outcomes would be extremely beneficial to the field (Hill, Beisiegel & Robin, 2013).

There is a strong demand for PD models that help teachers understand and utilize effective instructional strategies, especially those aligned to the Common Core Standards (Borko et al., 2011). This study demonstrates the potential for improvements in teachers’ classroom practices after participating in a specified, videocase-based PD and sets the stage for further inquiry into the factors associated with change in the intended direction. As reported elsewhere, there were some limited effects of teachers’ participation in the PD on student achievement), but it remains unclear whether and how shifts in classroom teaching influence students’ learning of specific content. Theories that specify the relationship between instruction and learning are in short supply (Hiebert & Grouws, 2007) and important future contributions to the field must continue to unpack the numerous and intertwined factors that are at play (c.f., Huffman et al., 2003; Kutaka et al., 2017; Stein et al., 2017).

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The Learning and Teaching Geometry Efficacy Study was supported by the National Science Foundation (NSF award #1503399). This study was approved by the University of Colorado Boulder’s Institutional Review Board, protocol #15-0524.