Exploring Barriers to Early Childhood Teachers’ Implementation of a Supplemental Academic Language Curriculum

ABSTRACT

Research Findings: This study used the Theoretical Domains Framework (TDF) to examine barriers to teachers’ implementation of a supplemental academic language curricula. Despite high satisfaction with this scripted curriculum, three past studies indicated heterogeneity in teachers’ fidelity of implementing the curriculum as well as difficulty going off script. Thus, our goal was to identify barriers to fidelity of implementation and map these onto possible behavior change techniques. Participants included 175 teachers from pre-kindergarten and kindergarten classrooms. We used mixed-method approaches that included surveys, focus groups, and interviews with teachers and their coaches. The most salient barriers to fidelitious implementation were: competing priorities for instructional time as well as limited teacher knowledge and skills. For some teachers, other barriers included difficulty changing habits, challenges with memory and attention processes, or lack of fit when the curriculum was used with populations beyond the designed scope. Practice or Policy: To understand why teachers may not consistently implement evidence-based curricula with fidelity, we explain steps other education researchers can use to apply the TDF and insights from the field of implementation sciences. We provide sample TDF survey questions and suggestions to help educators and researchers systematically revise the theory of change for curricular interventions.

Increasingly, experts argue that curricula are one potentially effective method for education reform (Chingos & Whitehurst, 2012; Whitehurst, 2009). However, when evidence-based curricula are provided, teachers do not always implement curricula in their classrooms as intended (Clements, 2007; Justice et al., 2009; Piasta et al., 2015). That is, there is either a gap between what curriculum developers intend and what classroom teachers actually execute with students (Mihalic et al., 2004) or there are individual or school factors that influence implementation (Domitrovich et al., 2015; Ransford et al., 2009). This is typically conceptualized as fidelity of implementation (FOI; Dane & Schneider, 1998; Dusenbury et al., 2003), meaning the extent to which an intervention is implemented in accordance with the original program design (O’Donnell, 2008). It is imperative that researchers study factors that influence FOI, as multiple studies have found better child outcomes when their teachers implemented evidence-based curricula with higher fidelity (Dane & Schneider, 1998; O’Donnell, 2008; Piasta et al., 2015; Wasik et al., 2006; Zucker et al., 2013). Likewise, a classroom curriculum may be effective in one set of circumstances, but not in others; thus, identifying and addressing implementation factors is important to maximizing benefits of classroom curricula (Weiss et al., 2014).

The current study examines a comprehensive list of possible barriers to implementation of a supplemental academic language curriculum, Developing Talkers (Zucker et al., 2019, 2013). In other words, this study views FOI as the outcome of interest and examines reasons why curricular activities were not implemented fully or factors that made implementation challenging. This and other curriculum supplements that address early language skills (e.g., Justice et al., 2010; Neuman et al., 2011; Pollard-Durodola et al., 2011) are important to study because early language skills are predictive of later reading and academic success (e.g., Dickinson, 2011; Duncan et al., 2007; Verhoeven et al., 2011). It is essential to go beyond asking whether teachers implemented language curriculum as intended. Educators and researchers need to attend to implementation drivers that interfere with (or support) use of evidence-based practices known to enhance children’s language development. The field of implementation sciences provides systematic methods to identify FOI barriers and facilitators as well as methods to map these behavioral determinants onto techniques to reduce barriers (Bauer et al., 2015).

Implementation Science Approaches

Implementation science methods study the uptake of evidence-based practices in ways that provide direction for educators and researchers to support classroom curriculum implementation. A prominent approach in this field is the Theoretical Domains Framework (TDF) that represents a collection of theories relevant to intervention implementation, with various theoretical perspectives and constructs grouped into 14 domains that explain possible determinants of behavior (Atkins et al., 2017; Michie et al., 2005). Although the TDF was developed by behavioral and community-health scientists, it is of great relevance to education. The TDF domains encompass three broad areas – Capacity, Motivation, and Opportunity. Table 1 defines the domains and constructs within the TDF, such as knowledge, skills, and social supports that may influence the extent to which individuals enact or change behaviors. The comprehensiveness and specificity of constructs can pinpoint reasons why people do/do not respond to behavioral interventions (see Atkins et al., 2017 for a full list of TDF versions and constructs). Table 2 outlines the steps in using the TDF and resources for studying determinants of behaviors. First, researchers and users of interventions begin by identifying specific target behaviors of an intervention that need to be improved. In education interventions, this may include reviewing the focal theory of change to specify which evidence-based teaching practices are the hypothesized mechanisms for increasing student outcomes. Second, researchers develop methods to gather data about all possible barriers to implementation; for example, using the TDF constructs to write survey and interview questions that allow end users to respond to a broad array of potential behavioral determinants of FOI. Third, researchers use the TDF domains to analyze multiple data sources for barriers. Fourth, researchers match the identified barriers to behavior change techniques (BCTs) that evidence suggests align with barriers educators face in delivering a curricular intervention.

Table 1. Theoretical domains framework domains & definitions (from Atkins et al., 2017)

Domain	Definition	Sample Constructs
Capacity
1. Knowledge	An awareness of the existence of something, including scientific rationale	Procedural knowledge Knowledge of task environment
2. Skills	An ability acquired through practice	Skill development Competence
3. Memory, attention, and decision processes	The ability to retain information, maintain focus, and make choices	Recall/Prospective memory Automaticity and attentional control Decision making/overload
4. Behavioral regulation	Managing or changing observed actions	Action planning Self-monitoring Breaking habit
Motivation
5. Beliefs about capabilities	Acceptance of reality about an individual proficiency/facility	Self-efficacy Perceived competence
6. Optimism	Confidence that desired goals will be attained	Optimism/pessimism Unrealistic optimism
7. Beliefs about consequences	Acceptance of reality about outcomes of a behavior	Anticipated outcome/regret Consequents
8. Professional role and identity	A set of behaviors displayed in a work setting	Professional identity Social identify/roles
9. Emotions	An individual’s experiential reaction to an event	Positive/negative affect Fear, Anxiety, Stress
10. Reinforcement	Dependent relationship between stimulus and response	Rewards/incentives Sanctions/punishment
11. Intentions	A conscious decision to perform a behavior	Stability of plans/intentions Stage of change model
12. Goals	End states an individual wants to achieve	Goals/target setting Priorities Action planning
Opportunity
13. Environmental context and resources	Circumstances of a person’s situation influencing behavior	Resources/materials Culture/climate
14. Social influences	Interpersonal processes that change thinking or behaviors	Social/group norms Modeling/support

Table 2. Steps for using the TDF for education research

Step 1: Identify Problem	Step 2: Collect Data across TDF Domains	Step 3: Identify Key Barriers to Address	Step 4: Identify Behavior Change Techniques
Review the intervention theory of change to identify specific behaviors to improve such as, FOI, teaching quality, or student learning.	Use surveys, interviews, and focus groups with questions that address all possible TDF domains. Collect classroom videos, lesson plans, or other observational and student data.	Analyze qualitative and quantitative data using approaches that classify responses and observations according to the TDF. Expand on the TDF definitions to create a codebook for themes in your data.	Match identified barriers to theoretically aligned and empirically tested approaches for changing behaviors in identified TDF domains.
See Atkins et al. (2017)	See Huijg et al. (2014)	See Atkins et al. (2017)	See Michie et al. (2008); Michie et al. (2013)

To date, researchers mostly use the TDF framework to study medical- and health-behavior change outcomes, such as considering barriers that prevent healthcare providers from implementing best practices in hospitals or medical settings. One study used the TDF framework to improve opioid prescribing after oncology surgery (J. S. Lee et al., 2018). Another study used the TDF to identify nurses’ barriers to using electronic medication management systems (Debono et al., 2017). A third examined physicians’ decisions to order unnecessary preoperative tests for low-risk patients (Patey et al., 2012). In these and other similar studies, one or more target behaviors are identified that warrant change, but uptake is inconsistent or challenging. In each study, researchers identified deterrents to behavior change by using the TDF domains as an analysis guide and then mapping identified barriers onto BCTs. For example, to address environmental barriers, surgeons needed on-screen prompts with normative data on the quantity of opioid to prescribe or nurses needed mobile computer workstations to use electronic medical systems. To address barriers related to social roles and beliefs about capabilities, physicians needed to clarify who was responsible for ordering routine tests and who had the authority to cancel unnecessary tests already ordered.

The TDF implementation science perspective also addresses “wicked problems” (Rittel & Webber, 1973); for example, if an adult wants to quit smoking, why can’t they stop? Or if an adult feels better when they diet or exercise, why don’t they stick with it? These implementation science perspectives are relevant to complex education problems as well. For example, if a well-designed curriculum enacts evidence-based practices, why don’t all teachers implement it consistently? Education researchers often argue that a cause of null findings is not that an intervention’s theory of change is entirely flawed but that change is only possible under ideal conditions (e.g., highly skilled teachers, schools with prerequisite “readiness” factors in place; Jacob et al., 2019). Thus, implementation science frameworks can uncover what makes curricular interventions difficult to implement. There is obvious value of implementation studies for educators and policymakers interested in scale-up and sustained use of evidence-based teaching practices (Weiss et al., 2014).

Teacher Fidelity of Implementing Curricula

The study of fidelity of intervention implementation is a multifaceted construct that traditionally includes: (a) adherence, (b) dosage or exposure, (c) participant responsiveness, (d) quality, and (e) program differentiation (Dane & Schneider, 1998; Dusenbury et al., 2003). Adherence and dosage represent structural fidelity features – linked to the structure of the curriculum – whereas responsiveness and quality represent process fidelity (Hill & Erickson, 2019). Our initial Developing Talkers studies reported adherence, dosage, and teacher responsiveness (Zucker, Cabell et al., 2021; Zucker et al., 2019; Zucker, Carlo et al., in press). Adherence refers to the level to which teachers deliver a curriculum following the guidelines. For example, this is the extent to which the teacher implement all steps of a lesson and where there is alignment between the suggested curriculum talking points and the observed teacher behaviors. Dosage refers to the amount of the intervention actually received. For instance, this can refer to the number of lessons implemented in a given week relative to the number suggested in the curriculum guidelines. Participant responsiveness includes the individual’s engagement and satisfaction with the curriculum and training supports. We focused on these variables because they are amenable to improvements through curriculum design and revision processes.

Various education researchers examining fidelity of curriculum implementation report wide variability in adherence and dosage. One study of a middle school drug prevention curriculum found that only 58% of the curriculum’s main points were implemented in the classroom; moreover, 63% of the adaptations teachers made to the written curriculum were judged ineffective (Dusenbury et al., 2005). In early childhood settings, Piasta et al. (2015) examined teacher FOI of a pre-kindergarten (pre-k) language and literacy curriculum and found substantial variability in adherence to components of each lesson, averaging 76–82% adherence. In terms of dosage, teachers only delivered 73% of the intended lessons, averaging 44 out of the 60 lessons (Piasta et al., 2015). In another study of a shared book reading curriculum, Justice, and colleagues found a range of 79–93% fidelity of implementation of specified targets, showing substantial variability in pre-k teachers’ adherence to a softly scripted curriculum (Justice et al., 2009). A pre-k study by Dickinson (2011) also showed wide variability, but on average teachers implemented only 38% of the core instructional targets of a language and literacy curricula. Importantly, this low FOI was one possible explanation for the lack of significant student impacts found in this study. These examples demonstrate that identifying underlying barriers in FOI is important if curricular interventions are to realize their potential as a mechanism for education reform (Chingos & Whitehurst, 2012).

Promise of Applying the TDF to Education Studies

The TDF framework is beginning to be applied to better understand FOI in education research. Within research on family engagement, Justice et al. (2015) applied the TDF to understand why families may not share books with their high-need, pre-k children. Families within schools were enrolled in a shared book reading program that explains the value of reading daily with your child and that children who are not read to at home experience a 1.4 million word gap relevant to peers (Logan et al., 2019). These researchers identified four primary deterrents to FOI, including time pressure, reading difficulties, skills, and beliefs about capabilities. More specifically, one salient barrier was caregivers’ own difficulties with reading, which was categorized within the TDF as an issue of self-confidence, within the domain of beliefs about capabilities. Therefore, Justice and colleagues (Justice et al., 2018) went on to experimentally test several potential BCTs and found that the most successful technique was providing parents with a small monetary incentive of 50 cents to complete each reading session.

Another education study used the TDF to examine the barriers to an obesity prevention program in Head Start by surveying directors about their perceptions and suggesting possible BCTs of: increasing staff training and assistance programs to improve perceived knowledge deficits; developing staff wellness programs to reduce negative self-perceptions about obesity; and providing healthy meals and snacks to all students and staff (Hughes et al., 2010). Some education researchers are applying the TDF early in the iterative design stage. For example, a study examined early childhood teachers’ barriers to use of positive behavior management strategies in classrooms in Jamaica to iteratively develop a low-cost, adaptable intervention to prevent school violence (Baker-Henningham, 2018). These education studies provide direction for studying barriers to FOI interventions because they recognize teachers are rational actors with specific reasons why they may or may not uptake new curricular practice and local conditions that hinder or facilitation implementation. The present study adds to this burgeoning area of research by using the TDF to identify barriers to early childhood teachers’ implementation of academic language curricula. Modifying survey items from Huijg et al. (2014), we illustrate in Table 3 how the comprehensive TDF domains can be applied to assess potential barriers to curriculum implementation. We retrospectively surveyed our study participants using a series of items like these that included Likert ratings and also asked them to rank their perceptions of the top barriers to implementation.

Table 3. Sample items across TDF domains to identify determinants of curriculum implementation

Domain	Sample Items to Adapt
Capacity
Knowledge	I know exactly what children are required to learn about [content area] within the grade I teach. I understand how to teach students [focal content area(s) in curriculum]. I know how to deliver [Curriculum Title] lessons according to the guidelines.
Skills	I have enough training in how to support students’ [content area] development. I have skills to teach students [focal skills in curriculum]. I had enough training and practice to deliver [Curriculum Title] lessons skillfully.
Memory, attention, and decision processes	Scaffolding children’s [content area] development is something I do automatically. Teaching [focal skills] is something I often forget. When planning lessons, it is hard to decide which [focal skills] are worth teaching. It is hard to remember to follow the prompts for [skills/strategies] in [Curriculum Title].
Behavioral regulation	There are so many requirements in the state guidelines about [content area] that it very hard to make plans that cover it all. I have a clear plan for when I will implement [Curriculum Title]. I have a clear plan with regard to delivering [Curriculum Title] even when there is little time.
Motivation
Beliefs about capabilities	There is little I can do to help my students who arrive at school with limited [focal skills]. For me, providing effective [content area] instruction is very difficult. I am confident I can deliver [Curriculum Title] following the guidelines even when there is little time. For me, delivering [Curriculum Title] is very difficult.
Optimism	In my work as a teacher, I’m always optimistic about students’ future [content area] achievement. I expect children can learn anything about [content area] that I teach well. When using [Curriculum Title], I am usually optimistic about how the lessons will go.
Beliefs about consequences	For me, teaching students [focal skill in curriculum] is NOT very enjoyable. For me, practicing [focal skills] with students is very useful. When I deliver [Curriculum Title] following the guidelines, it is very worthwhile for my students.
Professional role and identity	Supporting children’s [focal content] development is an important part of my work as a teacher. As a teacher, an important part of my job is to ensure students can [focal skill]. It is an important part of my job to deliver [Curriculum Title] lessons following the guidelines.
Emotions	When I work with students who have limited [focal skill], I feel uncomfortable. When I teach [content area], I feel stressed. When I teach [content area], I feel cheerful. When I deliver [Curriculum Title] lessons, it feels unpleasant.
Reinforcement	When I provide effective [content area] instruction, I get recognition from other teachers/administrators I work with. When I deliver improve my students’ [focal skills], I get financial or other incentives. When I deliver [Curriculum Title], I get valuable recognition or incentives.
Intentions	My lesson plans for next week will definitely include [focal skills]. I intend to deliver [Curriculum Title] following the guidelines next week. I will definitely deliver [Curriculum Title] in the next three months of school.
Goals	Other subjects are a much higher priority than [content area] when there is limited instructional time. Delivering [Curriculum Title] lessons is one of my highest priorities each week. In my agenda, other things are more urgent than delivering [Curriculum Title] lessons.
Opportunity
Environmental context and resources	Government and local authorities provide sufficient support to interventions for [focal content area]. In the school where I work, there are few resources available for [content area] instruction. My school leaders effectively supports me in delivering [Curriculum Title]. I can count on my school to have the supports I need to deliver effectively [Curriculum Title].
Social influences	Teachers with whom I work think that it is important to teach [content area]. I can count on support from [coach/others] when it is hard to improve children’s [focal] skills. I know the administrators at my school expect to see [focal instruction] daily. Other teachers that I respect deliver [Curriculum Title] following the guidelines.

Items are scored on a 7-point Likert scale from Strongly Agree to Strongly Disagree and several were adapted from Huijg et al. (2014).

Focal Curricular Intervention & Implementation Problems

A series of past-randomized-controlled trials examined the impact of a supplemental curricula, Developing Talkers, designed for use in pre-k and kindergarten classrooms (Zucker, Cabell et al., 2021; Zucker et al., 2019; Zucker, Carlo et al., in press). The curriculum targets academic language skills that include the ability to use sophisticated vocabulary and inferential language, representing formal language registers privileged in school and textbooks (Foorman et al., 2016). The curriculum includes three evidence-based teaching strategies based on substantial accumulated research. First, direct vocabulary instruction occurs before and during shared reading (for review see Marulis & Neuman, 2013). The goal is for students to use and understand sophisticated words of mature language users. Second, inferential level conversations are facilitated with open-ended questions; inferential talk requires reasoning and analysis around decontextualized topics that go beyond the immediate context to things that are not perceptually present (for review see Mol et al., 2009; Van Kleeck, 2008). Third, teachers learn to responsively extend and scaffold children’s responses to questions (for review see Roberts & Kaiser, 2011; Van de Pol et al., 2010). Scaffolding prompts in the curriculum provide the support that match children’s responses such that the conversation is simplified for incorrect responses (downward scaffolding) or the challenge is increased for correct responses (upward scaffolding). Teachers enact these three strategies in both whole-group shared reading as well as small-group extension activities. Small-group lessons are designed for review and more tailored support for a subset of three to five students who need more language support – that is, Tier 2 in multi-tiered systems of support frameworks (Greenwood et al., 2014).

The curriculum developers used an iterative design process involving researchers and practitioners seeking to ensure ease of implementation (cf. Clements, 2007; Penuel et al., 2011). The iterative process led to various design changes such as shortened instructional time to make lessons feasible to implement, placement of teacher talking points directly at the point of use in materials, and revision of curriculum-based measures to increase usability (Zucker et al., 2019, 2013). The curriculum provides softly scripted talking points throughout, meaning these are suggested talking points that a teacher can use his/her professional judgment to enact. Teachers report high satisfaction with the curriculum or even say, “I love it!” Yet despite this satisfaction, there were three complex problems we observed in past studies: a) small impacts on distal child outcomes, b) heterogeneity in FOI across studies, and c) teacher difficulties in going off-script.

Child Impacts

We previously reported causal evidence that teachers’ use of the Developing Talkers academic language curriculum improves child language outcomes. In each of three recent randomized trials of this intervention, we found the significant, medium to large effects on proximal, researcher-developed language measures (effect sizes [ES] range =.19 to 1.29, M = .71; Zucker, Cabell, in press; Zucker, Carlo, in press; Zucker et al., 2019). These impacts were expected because they measure students’ understanding of a selection of vocabulary words directly taught in the curriculum. But compared to other similar curriculum interventions (e.g., Piasta et al., 2015; Pollard-Durodola et al., 2011; Wasik et al., 2006) and benchmarks for practically meaningful effects (ES > .25-.30; What Works Clearinghouse, 2017, we found mostly non-significant and small effect sizes on distal, standardized language outcomes (ES range = −.11 to .39, M = .09). Although it is important for intervention studies to demonstrate gains on proximal measures, the ultimate goal is to demonstrate change on distal, standardized measures that capture the underlying language construct (Lonigan & Phillips, 2016). Therefore, understanding factors that hinder or facilitate implementation may enhance broad child outcomes in future applications of the curriculum.

Heterogeneity in FOI

Despite use of a continuous improvement approach to enhance each version of the curriculum across studies, we observed variability in adherence and dosage over time. Table 4 summarizes FOI levels across three studies that included (a) classroom observations of adherence to implementing the Developing Talkers lessons according to the guidelines, (b) teacher logs of the lessons they had time to deliver, and (c) teacher end of program satisfaction ratings. Details of the methods for collecting these data are summarized in Online Appendix A1. In the first study, teacher’s overall adherence to lesson steps was 73.15%, but this improved in the latter two studies, nearing 90% (i.e., Study 2, 90.53%; Study 3, 89.47%). These improvements may suggest revisions to training and materials made each version more feasible for classroom use. However, we do not see the same pattern of FOI improvement when considering dosage. Dosage declined substantially from the second study’s high level of 96.78% to third study with only 73.17% of lessons delivered. Notably, there was considerable heterogeneity in the number of lessons implemented in the third study, as evidenced by the large standard deviation (M = 122.20, SD = 34.96 – see Table 4). Thus, we see substantial variability across studies with structural fidelity ranging from moderate to high fidelity levels (i.e., >80% is “high”; <50% is low; Hill & Erickson, 2019). Teachers reported satisfaction with the curriculum across all three studies on a 4-point scale where four represents higher satisfaction. (see Table 4, Range = 3.32 to 3.78). Implementation studies such as the present work may elucidate the conditions that support consistently high fidelity.

Table 4. Fidelity of implementation & satisfaction scores across three studies

	Study 1				Study 2				Study 3
Curriculum Component	Min – Max points	M	SD	%	Min – Max points	M	SD	%	Min – Max points	M	SD	%
Adherence
Vocabulary	0–2	1.52	0.67	72.50	0–8	7.48	0.20	93.50	0–5	4.79	0.65	95.76
Inferential Questions	0–5	3.63	1.20	72.69	0–11	9.56	0.31	86.91	0–7	5.50	2.02	78.57
Scaffolding	0–5	3.69	0.64	73.71	0–3	2.28	0.41	76.00	0–2	1.23	0.86	61.46
Extensions	0–5	3.84	0.95	76.81	0–5	4.46	0.29	89.20	0–5	4.42	1.13	88.33
Total	0–17	11.76	3.53	73.15	0–27	24.42	0.94	90.53	0–19	17.00	2.38	89.47
Dosage	–	–	–	–	0–82	79.36	2.13	96.78	0–167	122.20	34.96	73.17
Teacher Satisfaction	1–4	3.32	0.98	–	1–4	3.62	0.59	–	1–4	3.78	0.44	–

Struggles Going off Script

When looking at teachers’ adherence to individual lesson components, FOI data in Table 4 show that teachers struggled to implement the scaffolding approach (62.46% to 76.00% adherence) more than other focal evidence-based practice. This was likely due to how scaffolding is less amenable to scripting because it requires teacher skill and practice to respond appropriately to each child’s verbalization. In comparison, fully scripted features of the curriculum such as vocabulary instruction and inferential questions were consistently easier for teachers to implement as designed (Range = 72.50% to 95.76%). Teachers reported high satisfaction with the Developing Talkers upward and downward scaffolds. For example, one teacher explained, “I learned a different strategy and I learned different ways of getting more one-to-one response from them … which was awesome.” Nonetheless, this implementation study sought to understand teachers’ and coaches’ insights into barriers to implementation of complex practices such as scaffolding.

The Present Study

The primary aim of this study was to understand barriers to FOI of the Developing Talkers curriculum so that identified, malleable factors can be addressed in future work. As stated, implementation studies are important to sustainability, replication, and future scaling of the intervention. Therefore, we used the well-established TDF implementation science method for identifying all potential determinants of implementation behaviors. Our primary research question was: What are barriers to teachers’ implementation of an academic language curriculum supplement? Our hypotheses were that limitations in TDF domains of knowledge and skill would be the major barriers; we did not expect barriers in domains such as optimism, memory, or attention/decision making. Although our prior iterative design studies suggested we needed to make these supplemental lessons shorter (Zucker et al., 2013), we did not expect challenges implementing these shortened versions that featured relatively brief 15- to 20-min whole-group and 8- to 10-min small-group lessons.

A second aim of this study was to use identified FOI barriers to revise our theory of change. This would allow us to conceive of potential adaptive interventions to maximize impacts of the curriculum. Our initial theory of change is shown in the top panel of Figure 1. We retrospectively applied the TDF to our research because it provided a comprehensive tool for us to examine barriers to FOI and consider ways to improve implementation. We expected to revise the theory of change to include additional methods to build teacher capacity around identified barriers to FOI.

Figure 1. Theory of change before and after this implementation study

We used a mixed-methods approach to understand perspectives of early childhood teachers, as well as their curriculum trainers/coaches. We sought coach input because teachers may not have felt comfortable articulating problems to curriculum developers or may not have been aware of some of their misunderstandings about the curriculum. Our data sources included a quantitative survey of coaches and qualitative analysis of teachers’ open-ended survey responses, focus groups, and interviews.

Method

To analyze teacher implementation barriers, we used data from participants across three projects examining the development and impact of the Developing Talkers curricular suite. Study 1 was a multi-year development study and included two subsamples of teachers who used the core academic language supplemental curriculum in English instructional settings in pre-k and kindergarten classrooms (Zucker et al., 2019). Study 2 was a randomized-controlled trial in bilingual-instruction pre-k classrooms (50% Spanish, 50% English) that utilized a Spanish version of the curriculum used in small-group settings; note this used the same procedures as the English version but was not a direct translation given differences in linguistic demands across the two languages (Zucker, Carlo et al., in press). Study 3 was a randomized-controlled trial using a derivative of the English curriculum as well as additional literacy and family engagement supports (Zucker, Cabell et al., 2021). All studies were conducted in a Southern State between 2015 and 2018. The studies collectively took place in both rural and urban locales. Across all participating sites, the vast majority of students were characterized as economically disadvantaged (93% to 100%) and about 77% were dual language learners.

Participants

Drawn from the three larger studies, participants included in the present analyses were 175 lead teachers (98% female) who implemented the supplemental curriculum in either pre-k (n = 159) or kindergarten (n = 15) treatment classrooms. As detailed in Online Supplemental Table A2, the research occurred in a variety of classroom settings (49 public, 45 Head Start, 70 private childcares that accept welfare-to-work subsidies). The pre-k settings served students eligible for needs-based pre-k programs. Teachers were mostly Hispanic or African American. All kindergarten teachers (drawn from Study 1) held either a Bachelor’s degree or higher and a state teaching license. Most pre-k teachers held an Associate’s degree or higher and about two-thirds held a state certification for teaching pre-k. Study 3 pre-k, Head Start teachers had less education with only 41% holding a state teaching license, notably less than teachers in the other studies. Teachers represented varying levels of experience, with a range of 1 to over 20 years in the classroom. Twenty-nine coaches also participated in this study. All coaches were trained by the curriculum developers to support teachers’ FOI. Most coaches held master’s degrees in education or related fields.

Procedures

All teachers implemented a version of the Developing Talkers curriculum for 13 to 23 weeks in their classrooms. Books featured in the curriculum were approximately half narrative and half informational genres, with accompanying lessons. Each book was read three times within a week. The curriculum included lessons for use at different levels within Multi-tiered System of Support (MTSS) frameworks. That is, the whole-group lessons represented the Tier 1 or universal level of the curriculum supports. In addition, targeted Tier 2 small-group instruction occurred three or four times per week for a subgroup of about four students who needed more support because they were deemed “Tier 2 eligible” by their district’s screening criteria. However, Study 2 took a different approach that implemented all lesson components exclusively in small-group settings to focus on Tier 2 eligible students.

Each book included six target vocabulary words introduced before reading with child-friendly definitions and the aid of picture vocabulary cards. During reading, vocabulary elaborations revisited the meaning of focal sophisticated, academic vocabulary (using criteria from Beck et al., 2008). Each book also included several inferential level comprehension questions, printed on stickers and placed at the point of use throughout the cover and pages of the text. These questions became more challenging across the three readings of each book and targeted skills such as making inferences, drawing logical conclusions, or making connections to the text. To support extended conversations, scaffolding prompts were provided for each guiding question. The scaffolding model encouraged extended conversations. To scaffold upward, teachers to ask more challenging, follow-up questions if children answered a guiding question correctly; however, for children who gave incorrect responses, the downward scaffolding prompts simplified the question to support the child in a producing a successful verbalization. Finally, the small-group extension activities reviewed focal vocabulary and included opportunities to use these words such as categorizing picture cards (e.g., examples/non-examples), acting out vocabulary with gestures and verbalizations, or asking questions vocabulary-related pictures.

Teachers attended a professional training designed to explain the rationale and how to use the curriculum; trainings ranged from one to three days. In addition, two to six sessions of coaching supports were provided by qualified and trained research staff, including both remote video coaching (Study 1) and in-person coaching (Studies 2 and 3). In all studies, teachers also received training and curriculum supports to manage the whole- and small-group lessons. For example, classroom management tools such as “voice level” posters taught students expectations for working independently in centers while teachers worked in small groups. Or “equity sticks” were used in whole-group lessons as a way to draw student names and ensure all students had an opportunity to answer comprehension questions.

Data Collection & Measures

We combined quantitative data sources to describe patterns across the three studies. This includes end of program surveys, focus groups, and interviews as well as a retrospective TDF survey that solicited coach perspectives on major barriers affecting teacher FOI of the curricula. Table A2 of the online supplemental materials details available data sources.

Teacher End of Program Surveys (Studies 1, 2, and 3)

Intervention teachers in all three studies provided feedback on their experience and opinions of the curriculum materials at the conclusion of the intervention. Teachers responded to survey items relating to the feasibility, usability, and effectiveness of curriculum components. Items varied somewhat across studies, but always included open-ended questions “What worked well … ” and “What did NOT work well … ” for aspects of the curriculum supplement.

Coach End of Program Surveys (Studies 1, 2, and 3)

At the conclusion of each intervention period, an online feedback survey was sent to coaches; a total of 48 coaches responded (68.57% response rate). Open-ended questions varied somewhat across studies, but all included: (a) What is working well and not working well for teachers you supported in using the curriculum supplement? (b) What feedback or recommendations do you have to improve the program? and (c) What topics/areas of the program did you need to clarify [for teachers]?

Coach Retrospective Survey (Studies 1, 2, and 3)

For the present study, we conducted a retrospective online survey with coaches using questions designed to assess potential barriers within each construct in the TDF (Atkins et al., 2017; Michie et al., 2005). This survey was sent to coaches of all three studies. Given the different timing of studies, this was completed 1.5 months after Studies 2 and 3, but approximately 12 months after Study 1. However, all coaches responding from Study 1 had current exposure to the curriculum because they continued to coach non-study teachers on the use of the program within an ongoing statewide professional development (PD) program. Nineteen of 27 unique coaches responded to the survey (70% response rate). From a set of 28 items, respondents were asked to mark (Yes/No) any barriers they thought were relevant to their teachers’ implementation of the curricula. Following Huijg et al. (2014), questions assessed all domains of the TDF such as: skills (“Teachers do not have the skills to facilitate conversations in curriculum lessons that address higher-level, inferential topics.”), time resources (“Delivering other programs/lessons is always more urgent than delivering this curriculum’s lessons.”), or memory/attention (“Teachers find it difficult to remember to pay attention to prompts in the curriculum that they should do something, such as ask a question or explain a word.”). Coaches also went on to rank their top barriers and describe their perceptions with responses to open-ended questions: (a) Please write about any barriers you observed to using the curricula; and (b) Please describe barriers to transferring practices to support academic language skills to other classroom contexts beyond the curriculum.

Focus Groups (Studies 1 and 3)

We conducted a total of three 60-min focus group: two groups with Study 1 coaches and one with Study 3 teachers. The timing of Study 1 focus groups was after intervention, whereas Study 3’s focus groups occurred during iterative development (before the experimental stage). All focus groups were moderated by the first author with support from research staff; the purpose was to collect feedback to inform changes in the next iteration of curriculum materials. One staff member was responsible for recording and transcribing participants’ responses.

There were two Study 1 focus groups that used a video-conference format with five coaches per group. Sample focus group questions that yielded information on barriers to implementation were: (a) How much time could teachers devote per week to activities in the curriculum supplement? (b) Which activities within the curriculum fit most easily into typical classroom routines? What activities were more challenging to layer over the core curriculum? (c) How easy or difficult was teachers to implement small-group activities?

The Study 3 focus group was held face-to-face with four non-experimental study pre-k teachers who field-tested portions of the curriculum before the randomized-controlled trial. Research staff explained that teachers’ feedback would inform curriculum revisions before the experimental phase. Sample questions relevant to implementation barriers included: (a) What is working well? What is not working well within language activities in whole- and small-group settings? (b) How much time could you devote per week to each of the curriculum activities? (c) What are some strengths and weaknesses of these methods for introducing vocabulary words?

Immediate Post-intervention Teacher Interviews (Study 3)

Finally, we conducted phone interviews with seven teachers following Study 3. We invited a subsample of teachers who represented both relatively higher and lower FOI in order to learn about possible barriers for different types of teachers. We used a semi-structured interview format that asked: (1) Can you tell me about some of the things that made it difficult to use the curriculum with the whole class? What was difficult for small group?, (2) What were some of your barriers that made it hard to use the curriculum as part of your everyday routines?, and (3) How did you decide ways to change the curriculum materials or suggested schedule to fit your needs? Can you tell me how you came up with the above strategy? (4) Can you tell me what changes you made to the curriculum to fit the needs of your students? Interviews ranged from 5 to 21 minutes in length, and were recorded and transcribed for further analysis.

Data Analysis

We first analyzed quantitative data in the retrospective TDF surveys to understand the chief domains that were perceived as barriers. These data describe the proportion of respondents who ranked each TDF domain as a barrier.

To analyze the qualitative data we used NVivo 12 software and followed Miles and Huberman (1994) approaches for data reduction, data display, and conclusion drawing. We first combined qualitative data from all sources into one dataset. To reduce data, we first identified any responses as positive/strengths versus negatives/barriers; the barriers data were separated from the larger dataset for further analysis. We developed an initial codebook using the TDF domains and constructs as an initial guide (Atkins et al., 2017); as shown in Online Supplemental Table A3 we updated the codebook with study-specific descriptive phrases found in the data (e.g., Social Influences domain includes “grade level team” or “administrator feedback”). These codes were not mutually exclusive and could co-occur. We identified 11 domains in our data that fit within the original 14 TDF domains. Although we used mostly deductive approaches, inductive approaches were considered in a second pass through the data to allow themes to emerge; these included one additional domain (Classroom Management) that overlapped across TDF constructs. All data were double-coded by two to three research staff, including the first two authors, averaging .92 agreement when any disagreements arose between the three coders in establishing the codebook. We used a consensus approach to resolve any disagreements by reviewing any data chunks with disagreement.

Finally, we moved to a mixed-method conclusion drawing stage. The authors reviewed the alignment between the highest rankings of quantitative barriers and the most prominent themes in the qualitative data. We reviewed all qualitative data within the top five quantitative barriers as well as one additional domain that was not prominent in the rankings but was salient in the qualitative data. We then discussed patterns in the data to summarize results.

Results

In Table 5, we detail the proportion of coaches who identified each domain as a barrier to implementation in the quantitative survey. The number of times each TDF domain was coded within the qualitative data is also shown in Table 5. These collective quantitative and qualitative barriers data indicate that the top barriers were: (a) environmental context and resources, (b) teacher skills, and (c) teacher knowledge. There were some other domains that appeared to be barriers in certain conditions, with the more frequent barriers including: (d) behavior regulation, (e) memory processes, and (f) beliefs about consequences. The quantitative and qualitative data both indicated that some areas were not salient barriers such as optimism, intentions, or negative emotions associated with the curriculum. Likewise, social/professional role was not a barrier, as teachers believed it was their job to facilitate their students’ oral language.

Table 5. Coaches’ retrospective TDF survey & all qualitative data counts on barriers to implementation

Domain	Quantitative Proportion Identified as Barrier^c	Qualitative Response Count^d	Illustrative Comments for Salient Barriers
Top Barriers to Implementation
Opportunity
Environmental Context and time constraints	71.88%	60	“The challenge was that we had two different programs. And I have to implement both programs, and it took the time away from a lot of things that I couldn’t get a chance to do.” – Teacher
Capacity
Skills in language facilitation and classroom management ^a	71.88%	61	“They think the script is exactly what they are supposed to say so if I child gives a correct answer, they don’t follow up with a higher-level question, they just go on to the next question on the script.” – Coach
Knowledge of effective pedagogical approaches	53.13%	43	“The teachers don’t have the background knowledge to really understand the strategies. They can do whatever the stickers [scripts] prompt them to do but they can’t apply it to other contexts because they don’t understand it.” – Coach
Potential Barriers in Some Contexts
Behavioral Regulation and changing habits^b	59.38%	15	“Even after a thorough training, teachers go back to school and get back in their normal routines.” – Coach
Memory and Attention to lesson prompts^b	64.58%	14	After training, “The kit sits for some time and they eventually forget how to use [it]. Then later when they are instructed to use the kit, they have difficulty remembering what it looks like to deliver a lesson.” – Coach
Motivation
Beliefs about Consequences for select students	25.00%	41	“English isn’t some students’ primary language. In the small group, too … my students were mostly Arabic … So when you implement it [with limited English speakers], you’re asking questions about it, they have no clue what I’m talking about.” – Teacher
Not Salient Barriers
Beliefs about Capabilities to impact learning	56.25%	5
Social Influences of other educators	53.13%	8
Goals and desire to improve teaching	29.17%	13
Emotions or stress associated with curriculum	31.25%	4
Professional Role as language facilitator	31.25%	1
Optimism about curriculum lessons	0%	0
Intentions to deliver curriculum	0%	0

^aAn emergent qualitative theme, that overlaps with various TDF domains focuses on interaction between student’s behavior regulation and teacher’s skill in influencing these outcomes.

^bThese domains completely overlapped in qualitative responses; therefore we combined these in our results and discussion.

^cThese quantitative data are from the coaches’ retrospective TDF survey.

^dThese qualitative data sources included all teacher and coach focus group, open-ended survey items, and interviews.

What are Top Barriers to Teachers’ Implementation?

Contrary to expectations, the dominant barrier was environmental context and priorities for instructional time. As expected, teacher knowledge and skills were a primary barrier.

Environmental Context Barriers: Competing Priorities

This domain was the most salient barrier identified, with 60 coded qualitative responses and 71.88% of coaches ranking this as a barrier. Although teachers were eager to implement the program, many had difficulty adding a 20- to 30-min supplemental curriculum to existing core curriculum and district requirements. Most coaches felt school leaders did not provide adequate classroom time or teacher preparation time for teachers to effectively deliver the supplemental program. A coach from Study 2 noted adding the supplement was challenging because, “Teachers are overwhelmed with many district initiatives that are being closely monitored by district and administration.”

Many teachers struggled with merging the new supplemental academic language curricula with the themes or topics in their core curriculum. A coach from Study 1 explained: “The main area that was concerning for the teachers was that they were not going to be covering the same unit of study with the supplement as with the [core] curriculum,” which was recently adopted because it was on the State Education Agency approved curricula list. Likewise, a Study 3 teacher said, “It’s hard with two curricula.” In short, even a relatively brief supplemental curriculum was not always feasible to layer into a myriad of other competing priorities.

Microelements of many classroom schedules also made it difficult to implement the supplemental curriculum. One teacher from Study 3 said, “The only problem I had was just my scheduling, just time to do the small groups because I was actually trying to … incorporate it, and not having the kids sit too long. It was time. Not time within the lessons, but MY time … with being on that … strict schedule.” This teacher then referred to fixed elements in her Head Start schedule such as outdoor play, lunch, and dental hygiene. When asked how teachers shortened the lessons to fit into available time, one teacher from Study 3 explained that, rather than adapting, she simply did not implement lessons at all. A coach from Study 1 also explained some teachers “felt that four days of small groups … was too much” and suggested a reduced number of small groups could be used each week if time was limited.

Skill Barriers: Eliciting Conversations while Managing the Classroom

The Skill domain was the second most coded in our data, with 53 qualitative responses and 71.88% of coaches ranking it as a salient barrier. A prominent theme coaches described were that teachers lacked skills to facilitate multiple-turn conversations with young children, including the ability to scaffold children’s communication attempts. As stated, possible upward challenge prompts and downward simplification prompts were suggested scaffolds in the curriculum supplement; many teachers struggled to use these prompts as designed. For example, teachers often used a downward scaffold for a correct response instead of the appropriate upward scaffold to provide challenge for the student, such that the scaffold was not responsive to the child’s level of understanding. Coaches from Study 1 said, “Some teachers did not know when to scaffold up or down, even though they said and seemed to understand it during the [training] session.” Most coaches described reviewing and practicing the scaffolding model during coaching sessions. Yet, some coaches felt that teachers relied too heavily on the scripted scaffolding prompts or seemed unable to go off-script to respond appropriately to a child’s verbalization in conversations beyond the curriculum.

Another theme in the qualitative data was that many teachers lacked classroom management skills required to deliver these lessons. Specifically, teachers often lacked the skill to manage small groups while other children were supposed to work independently at centers. A teacher from Study 3 explained that during small groups: “You’ll have certain kids that want to come over there to the table and do the small-group activity when they’re supposed to be doing another activity.” Several coaches stated that teachers needed broad training on effective classroom management approaches to implement a curriculum with both whole- and small-group instruction.

Knowledge Barriers: Language Development and Procedural Knowledge

The Knowledge domain was coded 43 times in the qualitative data and identified as a barrier by 53.13% of coaches in the TDF survey. The first barrier was teachers’ limited knowledge about how to be a responsive conversation partner to support children’s language development. For example, coaches from Study 1 said, “Many teachers lack their own vocabulary and reading comprehension skills” or struggle to pronounce vocabulary words. Indeed, several coaches noticed teachers who spoke English as a second language and/or had only a high school degree who did not have sufficient vocabulary knowledge to explain academic words in child-friendly ways, explaining, “They can only do this if they have the script.” Other coaches identified teachers’ lack of knowledge about “understanding multi-tier support systems,” or “the difference between informational and narrative style books” and “background knowledge to really understand the strategies” such as open-ended inferential questions and subsequent scaffolds.

A second barrier was lack of procedural knowledge about how to use the curriculum supplement. Particularly in the earlier stages of intervention, coaches reported spending large amounts of time on how to properly prepare for lessons or navigate lesson materials. In post-training feedback surveys, coaches from Study 1 noted, “Teachers needed basic clarification on the set up [of materials],” and “the differences between ‘essentials’ and ‘extensions’ for whole-group lessons.” Another coach from Study 1 mentioned teachers needed support, “getting familiar with looking at the unit at-a-glance and how it worked as far as them understanding what they needed for that lesson.” However, coaches agreed, when procedures were explained again in coaching sessions, the teachers were able to understand the curriculum procedures.

What are Potential Barriers to FOI in Some Contexts?

Three additional domains were identified as potential barriers, but were much less salient than the top three. These data indicate that for some teachers or contexts other barriers may arise.

Memory and Behavior Barriers: Changing Habits and Remembering

About 60% of coaches surveyed felt two TDF domains of Behavior Regulation and Memory, Attention, Decision Processes were a barrier to implementation. These domains were represented in 15 and 14 qualitative responses, respectively. These domains also heavily overlapped in the qualitative data; therefore, we consider these results concurrently. We did not hypothesize that memory/attention would be a potential barrier given a scripted curriculum, but this was problematic for some teachers. The first behavior regulation barrier related to the lack of automaticity in using a new curriculum or difficulty changing existing teaching habits. One teacher from Study 3 described the challenge of facilitating multiple-turn conversations as hard to “get into the repetition of it. That was something that was difficult … because it wasn’t something that I was used to doing.” Another Study 3 teacher explained that using a softly scripted curriculum required a change of habits and mindset, explaining her self-talk as, “Okay, I’m going to ask them this question from this book.” But she did not consistently use the scripted questions because, as she said, “Sometimes, it was easier for me just to do it [my] way than to [think], okay, let me see, what does it tell me to ask today?” Many teachers explained that with the new curriculum you had to “get into the groove” or that it “just becomes a little more natural” with time. A Study 3 teacher described similar ideas that, “I had to get used to drawing the [equity/name] sticks versus just saying ‘Oh okay, well, he raised his hand, so what do you think?’” Several coaches observed that teachers did not have time to prepare for lessons, which decreased their ability to follow all curriculum steps.

The second aspect of these barriers was cognitive overload related to difficulties remembering curriculum components or paying attention to important cues. Coaches noted that several teachers struggled to pay attention to cues in the curriculum that prompted instruction, such as remembering to teach a word or ask a comprehension question. Moreover, many teachers struggled to listen and respond contingently to children’s verbalizations to keep the conversation going as they scaffolded children’s responses. Several teachers explained, “In the beginning, of course, I was still trying to learn and remember ‘Okay, what am I supposed to do?’” Coaches described a cognitive overload for some teachers such that “the program is valuable but overwhelming.” A coach from Study 1 explained, “During class [trainings] teachers do state that they understand how to implement, but during their actual teaching time they do not follow all of the steps.” When prompted, coaches described these as problems in remembering the curriculum steps they had previously seemed to understand during training exercises and role plays; such problems worsened when teachers reported not using lessons immediately after training. This theme of feeling overwhelmed was often linked to other required curriculum materials or schedule constraints such that a Study 3 teacher described this as “a lot of back-and-forth.”

Beliefs about Consequences for Select Students

Although only 25.00% percent of coaches identified the domain of Beliefs about Consequences as a barrier in the TDF survey, those items broadly addressed how engaging or worthwhile the curriculum was for students. In the qualitative data, this domain emerged with 41 codes explaining barriers for subgroups of students. The first concern was that the initial version of the curriculum took too long to deliver. One coach from Study 1 commented, “Teachers think there are too many parts to the curriculum supplement and it is too much time for students to be sitting and therefore causes students to be distracted.” Thus, the steps in lessons and materials were reduced in Studies 2 and 3 to better match young children’s attention spans, particularly in the pre-k setting. Another concern was around student engagement with some lesson features. Several teachers expressed “concern about reading the same book three times in a week” because they felt children became bored hearing the same text repeatedly. When rereading lengthy books, a teacher noted, “They would get bored … and hyperactive and they wouldn’t pay attention to me.” In addition, one teacher said, “I have concerns about kids remembering this many words. Maybe only 1 word [per book].” Thus, the training materials were revised to explain the value of rereading texts and the need to explicitly teach a broad range of vocabulary to ensure children’s academic success (see Zucker et al., 2019).

Some activities were not a good fit for students with unique needs such as English learners or special needs. One teacher in Study 3 said, “I had very few English speakers. So for the Spanish ones … the words were like, wow … they couldn’t even pronounce the words.” Note this study used the English version of the curriculum. We cannot speculate if the Spanish version of the curriculum would have been a better fit for these students; however, teachers who used the Spanish version in Study 2 did not articulate any concerns about the sophisticated, academic vocabulary when it was presented in student’s home language. Another Study 3 teacher explained how the small-group lessons were too advanced for her students: “The other barrier I would have to say was that because you picked the ones [students] with the lowest scores, there was a language barrier. One was on the [Autism] spectrum. The others were already delayed in speech, so that was a barrier.” The curriculum was not designed for children with such special needs; thus, these comments suggested the developers needed to clarify for whom the curriculum was/was not designed.

Discussion

This study synthesized information from three past studies to identify barriers to teacher implementation of a supplemental academic language curriculum in preschool and kindergarten classrooms. Teachers’ and coaches’ perspectives were analyzed using the Theoretical Domains Framework (TDF; see Atkins et al., 2017) to understand all possible barriers to FOI. The TDF afforded us comprehensive coverage of all possible constructs that might explain why teachers do not consistently implement this curriculum with high FOI. Despite adequate teacher satisfaction, we identified five major barriers to FOI, including: (a) time constraints and competing priorities within school environments; (b) limited teacher skill in facilitating academic discourse; (c) limited teacher procedural and developmental knowledge; (d) difficulties with teacher behavioral change, memory, or attention processes; and (e) lack of fit when using the curriculum with certain subgroups of students. Some of these constructs, such as memory and environmental fit, were not expected barriers that we could have detected with our initial data collection approach that focused more narrowly on traditional FOI variables. We discuss these barriers in light of the extant literature on early childhood curriculum implementation. Then, we consider theoretically informed behavior change techniques (BCTs) that could reduce barriers and revise our theory of change to enhance implementation processes.

Barriers to Implementation of Curricula

Many of the barriers to implementation identified in the present study are in keeping with prior work examining teachers’ use of curricula. Timing and resources within the school environmental context presented as the first barrier for implementation, particularly finding time to “fit in” the curriculum among competing instructional priorities. We did not expect this would be a prominent barrier since the supplement only required 25 to 30 minutes about three times per week; this was within the scheduling expectations our earlier piloting work suggested was acceptable. Yet, teachers in these studies felt inundated with various mandates and priorities from schools and districts. Increasingly since the outset of Common Core State Standards in 2010, curricula are viewed as a potential lever to reform education through materials aligned with rigorous standards (Chingos & Whitehurst, 2012). Yet there are many challenges for districts and states in adopting new curricula because, even if an evidence-based curriculum is provided, it is hard to ensure teachers will have the time to permit high FOI (Polikoff, 2015). Moreover, teachers in the present studies did not always have clear guidance from school and district leaders about which curriculum materials were the “backbone” or mandate and which were optional resources (Polikoff, 2018). Higher levels of administrative support is related to quality of supplemental curriculum implementation (Ransford et al., 2009). These and other aspects of the larger school system (beyond the classroom) have lead some researchers to consider readiness ratings for deciding if a school system is ready to implement changes (Jacob et al., 2019; Mihalic et al., 2004).

As expected, limitations in teacher skill and knowledge emerged as important barriers in our data. Prior work shows that teachers’ disciplinary knowledge relates to the opportunities they provide students for language development (e.g., Schachter et al., 2016). It is essential to ensure teachers have the skills and knowledge to enact curriculum well-given studies that demonstrate these areas are significantly related to children’s language growth (Cabell et al., 2015; Cash et al., 2015; Connor et al., 2009). For example, teachers’ skill in scaffolding language and exposing children to advanced language models are hypothesized to be key mechanisms through which conversations improve children’s language ability (Nicholas et al., 2001). Although descriptive studies show substantial variability in teachers’ ability to engage in quality conversations with children (Cabell et al., 2015), academic language curricula should support teachers to act as facilitators of classroom conversations that span multiple turns on a given topic. Yet, like other language curriculum studies (Piasta et al., 2015), teachers in our samples had trouble going “off-script” to use scaffolding strategies to extend conversations and adjust talk to fit individual student needs (i.e., making a question easier or more difficult). Some teachers’ limited skill in managing their classroom also presented barriers to implementation. Indeed, the extent to which shared book reading curricula are effective for children’s language learning may depend on the quality of classroom management (Cabell et al., 2019). Finally, teachers’ own vocabulary and reading skill appeared to be a barrier for a small number of participants (e.g., teachers who were not fluent in English).

A smaller number of teachers experienced difficulties changing behaviors and remembering how to enact the new Developing Talkers curriculum; these barriers were unexpected because the intent of such scripted approaches is to reduce demands on teachers. Some teachers can experience attention problems or stress when they enact new curricula (e.g., Napoli, 2004). However, memory/attention and behavior change problems were not universal issues for teachers in our samples. In other implementations studies similar teacher reported stressors – ranging from specific resistance to new teaching approaches to general emotional exhaustion, burnout, and low self-efficacy – were associated with greater difficulty implementing curriculum supplements (Domitrovich et al., 2015; Ransford et al., 2009). Likewise, individual teacher perceptions of their ability to attend to and recall curriculum procedures may influence implementation delivery (Pollard & Courage, 2017). These individual-level factors may also explain why there we observed substantial variability in structural fidelity measures (dosage, adherence to scaffolding strategy).

For the final barrier, a small number of teachers’ negative beliefs about consequences of using the curriculum suggested the need for further improvements to the materials. Thus, revisions were made to include: (a) better explanation of the developmental skill progressions upon which the lessons were built (cf. Clements, 2007); (b) for whom the curriculum is/is not designed, including foundational language skills that should be supported when targeting academic level language (Van Kleeck, 2008); and (c) development of a bilingual Spanish/English version (rather than only English or Spanish). Arguably more research is needed to validate progressions for teaching of academic vocabulary words, as current best practices rely on data collected more than four decades ago and professional judgment (Biemiller, 2010). Alternatively, other solutions to this barrier could be to not use a supplemental academic language curriculum (or this particular curriculum), but to enhance the core curriculum to ensure rigorous classroom discourse in the primary unit of study.

Comparisons across Studies

Although this series of three studies represented an ongoing continuous improvement process to refine one supplemental curriculum, it is worth noting some differences in findings across studies. Competing priorities were a consistent theme in studies 1, 2, and 3, but for different reasons. In Study 1, lesson duration was too long, making instructional time the key constraint. Thus, an outcome of the first study was to shorten lesson length (from 35 minutes to 25–30 minutes). In Study 2, the competing priority was that the district had launched several recent initiatives that were all closely monitored for FOI. In Study 3, the issue was that these Head Start teachers had not only several curricula but also multiple scheduled requirements such as oral health and handwashing. Another consistent theme across studies was that teacher skill and knowledge to enact scaffolding was the most challenging element only reaching moderate implementation levels from 61% to 76% across studies.

A finding that was unique to Study 3 was the lack of fit of the academic language focus for students from diverse language backgrounds. This study occurred in Head Start settings that featured students from “superdiverse” linguistic backgrounds (Baker & Páez, 2018), meaning that 66% of students were dual language learners and spoke a variety of home languages (e.g., Urdu, Creole; see Zucker et al., 2021). Although Study 2 also featured dual language learners, these students all spoke Spanish as their native language; therefore, the academic language intervention was delivered in this first language to ensure a strong, broad linguistic background for processing these cognitively challenging concepts. Implementation was slightly higher in Study 2 for all aspects of FOI, perhaps suggesting the importance aligning academic language support with students’ native language.

Behavior Change Techniques that Could Reduce Barriers

After identifying barriers to FOI of the Developing Talkers curriculum, we completed the final step recommended by the TDF of identifying Behavior Change Techniques [BCTs] to reduce barriers. As shown in Table 2, this is essential as step to systematically map identified behavioral determinants onto evidence-based techniques for changing these behaviors (Michie et al., 2008, 2013). In meta-analytic syntheses, Michie et al. identified 137 effective techniques that bring about behavior changes and suggested a theoretically and empirically driven procedure for selecting relevant BCTs. For example, if lack of skills were the barrier, it might be appropriate to select rehearsal, homework, or modeling of skills; however, if motivation to perform the skill was the barrier, then persuasive communication or other techniques are better aligned BCTs. For the present study, up to 21 BCTs have general evidence of effectiveness for addressing the five domains identified as determinants of behavior. Table 6 summarizes all relevant techniques and barriers they address. We discuss below a set of BCTs (noted in italics) that can be layered together in curriculum implementation such as: time management, implementation planning, prompts/cues, or graded tasks.

Table 6. Linking identified determinants of behavior with behavior change techniques

		Techniques Judged to be Effective in Changing Identified Domains^a
BCTs	Definition	1. Environment	2. Skills	3. Knowledge	4. Memory, Regulation	5. Beliefs Consequences
Capacity
Monitoring	Trusted others record data on behavioral performance		●		○	○
Self-monitoring	Record behavioral performances (e.g., log, diary)		●		●	●
Rewards	Contingent valued consequence		●		○	○
Graded tasks	Perform easy tasks first; gradually increase difficulty		●		○
Skill practice	Practice making decisions or solving problems linked to target behavior		●		○
Rehearsal	Perform behavior repeatedly		●		○
Homework	Set tasks to complete on own		●
Perform behavior in different settings	Repeat behavior in the same setting and different settings		●		○
Planning implementation	Make a plan to perform behaviors; scheduling		○		●
Stress management	Undertake behaviors to reduce stress or impact of stressors		○		○
Problem solving	Identify and plan ways of overcoming barriers		○			○
Goal setting	Set behavioral goal		●	○
Information	Provide information about antecedents or consequences of the behavior			●		●
Opportunity
Environmental changes	Change objects or the environmental context to facilitate the target behavior	●			○
Prompts, triggers, cues	Stimulus that elicits behavior (e.g., text message, e-mail)	○			●
Time management	Action planning around perceived time shortage	○			○
Motivation
Feedback	Receive feedback on target behavior				○	●
Personalized message	Tailor techniques to individual context, traits, resources				○	○
Personalized experiments	Test hypotheses by collecting and interpreting data				○	○
Persuasive communication	Credible sources present arguments in favor of behavior			○		●

^aThese techniques and judgments are derived from Michie et al. (2008), Michie et al. (2013)). ● = Agreed on effective use; ○ = Uncertain on effectiveness, but potentially useful.

Organizational and Environmental Supports

To address the issue of competing priorities in the environmental context, teachers need time management support from school building leaders that allow preparation time. This may include implementation planning discussions to revise grade level schedules and layer in a new supplemental curriculum. Teachers who experience high levels of school leader support may perceive the supplemental curriculum as a “permanent innovation” (Ransford et al., 2009, p. 525). Indeed, in Study 2 high levels of school leader support have led to sustained implementation of this supplemental program for many years after the grant ended, in part, because school leaders built Developing Talkers time blocks into daily language arts schedules for pre-k and kindergarten teachers.

Other helpful environmental supports observed were the use of prompts or cues on a visual schedule or with classroom materials that triggered desired behaviors. For example, Study 2 and 3 teachers displayed sentence strips of each day’s “guiding question” and picture vocabulary cards before shared reading to focus students on these topics during shared book reading. A few teachers displayed digital questions and vocabulary cards before reading; technology supports can facilitate use of supplemental curricula as teachers click through each step in the lesson cycle (Myrtil et al., 2018). In all three studies, teachers who implemented with high fidelity often enlisted students with curricular tasks such as a student helper to randomly draw “equity sticks” so children took turns answering comprehension questions. Other teachers used physical reminders like a piggy bank the curriculum calls the “word bank,” which a student helper would put a coin into each time a child used a focal academic word (Zucker, Cabell et al., in press).

Educative Curricula

To enhance teachers’ attention and decision processes, an educative curriculum that uses graded tasks to gradually increase teacher responsibility for planning components of academic language instruction may be more beneficial than the softly scripted format. Although the scripted approach we used was intended to ease burdens, it may have the unintended effect of reducing teacher capacity to build skills and knowledge of focal evidence-based practices. Modern curricula are often highly scripted to: (a) limit training requirements, (b) ensure FOI of research-tested programs, and (c) narrow the range of individual teacher factors influencing instruction (e.g., novice vs. expert teachers). In contrast, educative curricula are intended to promote both teacher learning and student learning simultaneously by explaining underlying principles and by making teachers an agent in the instructional design and enactment (Davis & Krajcik, 2005). Although educative curricula may include some scripting to provide specificity and support (Remillard, 2000; Remillard & Reinke, 2012), a distinction is that teachers have more autonomy in using scripts as well as more lesson planning homework. These educative design features promote deliberate skill practice, which diverges from scripted approaches that may elicit “mindless performance” (Chi, 2011). Educative curricula could also promote problem solving. For example, a reported barrier was that the supplemental curriculum did not match the core curriculum themes. Because language and vocabulary develop in relatively malleable sequence (Beck et al., 2008), teachers could flexibly reorder units to match their core program. (Note this may not be possible for literacy/math curriculum supplements that follow specific developmental sequences [(e.g., Clements, 2007]).

We are currently studying a Developing Talkers Adaptive version that includes these educative features (Zucker & Logan, 2019). This version does not require teachers to create lessons from scratch, but instead uses a gradual release of responsibility for teachers to prepare an increasing number of lesson components, distributed over time. It is possible that this less scripted approach could improve teachers’ generalized use of evidence-based practices known to increase children’s academic language. This may also enhance teachers’ motivation to use these practices, as research from other disciplines suggests these sort of self-made products affords users creativity and increases valuation of their work (e.g., Demiroglu & James, 2012; Kelly, 2008), particularly when it is comparable to expert creations (Norton et al., 2012). Emerging research on educative curriculum supplements suggests they can increase teacher effectiveness (Bleses et al., 2017; Neuman et al., 2015; Remillard, 2000), particularly when teachers have prior experience with the curriculum (Quinn & Kim, 2017).

Training and Consultation

To address knowledge barriers and beliefs about consequences, teachers may benefit from more information on evidence-based practices. Although the present teachers attended one to three training days, standalone workshops are a generally less effective form of PD (Sheridan et al., 2009). More distributed practice – in which informational material is encountered across multiple sessions – better supports learning (e.g., Dunlosky et al., 2013). In addition, consultation with curriculum experts or language/literacy coaches could improve implementation. Such technical assistance and coaching sessions support persuasive communication because messages are tailored to their individual context or teacher mindsets (Noar et al., 2007). Individual teacher traits are possible determinants implementation. For example, if teachers’ pedagogical beliefs align with the curriculum teachers may demonstrate higher self-efficacy that promotes FOI (Cobanoglu & Capa-Aydin, 2015).

Additional research is needed to understand individual teacher factors – such as attitudes, beliefs, and readiness for change – that are associated with target behavior changes. Examining these potential moderators can advance the science of teacher PD and could signal for whom particular messaging, curriculum, or PD approaches are likely to be effective. Individualized sessions with a trusted coach or colleague can also support monitoring of implementation (Landry et al., 2009; Pianta, Mashburn et al., 2008). Studies 2 and 3, as well as studies by other teams (e.g., Justice et al., 2009; Piasta et al., 2015), included methods for teachers to track their implementation of curricula. This type of self-monitoring that includes recording behavioral performance – such as keeping a log, diary, or tracking in a mobile app – can enhance attention to target behavior changes.

Constant Learning Communities

An increasingly popular PD approach uses professional learning communities (PLCs) to support instructional improvements (e.g., Parkinson et al., 2015). PLCs generally function as a collaborative community of educators exchanging teaching experiences and improving teachers’ pedagogy by targeting teachers’ skills, motivation, and self-efficacy (J. C. K. Lee et al., 2011; Vangrieken et al., 2017). These communities typically support behavioral rehearsal and feedback (Early et al., 2017). Learners are often more motivated in activities that involve cooperative goals, like those in PLCs, rather than individualistic goals (Johnson et al., 1981).

Teacher knowledge and skills are likely to improve with some form of collaborative feedback from peers (or from trained coaches), as both methods encourage deliberate practice and reflection (Denton & Hasbrouck, 2009; Salas et al., 2012). Likewise both forms of PD use goal setting; setting achievable goals and tracking progress toward these goals is as a key component within programs that increase teacher knowledge and skills (e.g., Landry et al., 2009). For example, Pianta and colleagues have repeatedly tested coaching approaches that set goals based on teachers’ scores on the Classroom Assessment Scoring System (CLASS), an observational measure of classroom quality (Pianta, La Paro et al., 2008). Other promising approaches set goals around specific instructional behaviors and settings (Crawford et al., 2013).

Revised Theory of Change

We integrated these BCTs into our revised theory of change, to explain how enhanced implementation processes could maximize future impacts of the Developing Talkers curriculum. As shown in the bottom panel of Figure 1, to effectively implement a supplemental curriculum designed to enhance academic discourse, teachers need opportunities within their schedule to plan for and implement the program. Likewise, the identified barriers predominantly fall into domains that suggest capacity building is a key driver of behavior change. For teachers to routinely implement these supplemental lessons and generalize the three evidence-based practices (direct vocabulary instruction; open-ended, inferential level questions; responsively scaffolding children’s discourse), they require sustained PD opportunities for skill rehearsal, practice, and reflection. Some teachers may require more personalized coaching or other supports to adapt the supplemental curriculum to fit their particular student needs or their individual pedagogical competencies. Although the TDF includes several constructs that address motivation, these data did not suggest this was a barrier to implementation; instead, early childhood teachers understood the importance of supporting language development and were motivated to use this relatively teacher-friendly curriculum. However, their capacity and time to enact the program were the key barriers to implementation.

Limitations & Future Directions

Although this mixed-methods study brings together ample information sources across three studies to understand barriers to implementing an academic language curriculum supplement, there are limitations. One important limitation of this study is that the data were not drawn from exact replications, but rather from three conceptually linked studies using the Developing Talkers curriculum derivatives. Differences across these curriculum variations, such as the Spanish version in Study 2 or the additional family and literacy components in Study 3, could have affected outcomes and perceived barriers. Another limitation is our retrospective use of the TDF; this meant only some of the interview and survey questions were explicitly framed to assess all aspects of the TDF, whereas other data simply used open-ended questions about what worked versus what could be improved. Future studies could better elucidate barriers by applying this theoretical framework to all feedback gathered and by using research designs that explicitly contrast potential BCTs outlined above.

Moreover, future research designs could examine the impact of BCTs on teacher FOI. For example, a new study of the Developing Talkers curriculum uses an adaptive intervention approach to tailor intervention supports to match individual teacher response. We are using a Sequential Multiple Assignment Randomized Trial (SMART; Almirall et al., 2014) to compare the benefit of two different types of additional professional development – individual coaching and professional learning communities (PLCs). We will causally compare the added benefit of these two different sets of BCTs, which vary in implementation cost and complexity. Individual coaching will be randomly assigned to half of the teachers who are not responding to the curriculum in terms of FOI. In contrast, the more demanding PLCs will be randomly assigned for half of the teachers who are responding adequately. The current study’s identification of barriers to FOI was key to elucidating sets of BCTs that warrant further investigation within the new SMART study.

Conclusion

The implementation of evidence-based curricula and instructional practices depends on a complex set of human behaviors. This manuscript considered three studies and drew upon the TDF theory to consider possible reasons for heterogeneity in past implementation and smaller child effect sizes than expected on distal outcomes. Researchers and educators can improve classroom curriculum and PD interventions by drawing on such implementation science theories. The TDF brings together a large number of overlapping constructs of relevance to changing human behaviors and identifies potentially relevant techniques to improve behavioral targets. Amongst an array of potential techniques, we considered a set of approaches that warrant further investigation for improving implementation of curricular interventions. In future studies, we aim to causally evaluate several of these BCTs to better understand for whom and under what conditions the Developing Talkers approach is most effective. Importantly, we encourage other education researchers to use the TDF to improve interventions within schools.

Acknowledgments

We are grateful to Heather Mui, Danielle Pico, Hilary Kallin, Maria Carlo, Susan Landry, and Jessica Logan for their contributions to this research. A special thanks to Yoonkyung Oh for helping us consider the meaning of key findings.

Disclosure Statement

Tricia Zucker has received research grants from Scholastic Inc. for the Developing Talkers curriculum. Erin Jacobs and Sonia Cabell have no conflicts of interest to declare.

Supplementary Material

Supplemental data for this article can be accessed on the publisher’s website.

Additional information

Funding

This research was supported by gifts from the Dan L. Duncan Family and the Brown Foundation, Inc. Research reported in this publication was also supported by the Institute of Education Sciences (IES) under award number R305A150319 and R305A190065. The content is solely the responsibility of the authors and does not necessarily represent the official views of IES.