Scaling up early language intervention in educational settings: First steps matter

ABSTRACT

Objective

To report how improvements on a Brazilian language intervention for early childhood education settings (PROLIN) were made and evaluated.

Study Design

In the first phase, the programme layout and materials were improved. This involved redesigning the guidelines for the programme, adding videos (using a learning management system) and creating an observation checklist to monitor the fidelity of implementation. The second phase was a two-week pilot study (a seven-session intervention) involving two teachers and 22 students. Checklists and video footage were analysed to investigate implementation.

Results

Quality of implementation was generally good, but we identified additional areas for improvement. Teachers had some difficulties with aspects related to session dynamics, implementation of activities and use of techniques that reinforce learning.

Conclusions

The pilot study was instrumental in identifying obstacles for a scaled-up, high-quality implementation. The design of these materials took into consideration ways of guiding and supporting teachers to: (1) offer students adequate participation time; (2) help include children who are shy or have behaviour problems; (3) use teaching strategies properly; (4) bring sessions to a close; and (5) reach the objectives of each session. Further modification is still needed, especially in the manual, videos and supplementary materials.

KEYWORDS:

• Teacher training
• quality of implementation
• pilot projects
• early intervention
• child language

Introduction

Educational interventions that combine preschool curricula with teacher training and coaching are known to offer the best chances of success to their recipients (Yoshikawa et al., 2016, 2013). However, there is a dearth of knowledge on how best to combine materials and professional development to increase the effectiveness of interventions in disadvantaged educational settings in middle-income countries.

This study draws on experiences of a team implementing a language intervention programme in Brazil that showed minimal or null effects in two randomised controlled trials delivered in early childhood education centres (Puglisi et al., 2019, 2018). This programme was built on the Nuffield Early Language Intervention (NELI), an intervention proven to be effective in the UK (Fricke et al., 2013, 2017; West et al., 2021), and was adapted linguistically and culturally for children speaking in Brazilian Portuguese. Although no systematic data on quality of implementation were collected, we speculate, based on the research teams’ observations, informal conversations, and differences between the settings and contexts of our two previous studies (classroom vs. small-group intervention; municipalities with high vs. middle human development indices), that low fidelity of implementation was the most likely explanation of the null results. We discuss the obstacles encountered in implementing the intervention and how we addressed them in order to roll out a further trial.

The educational programme for promoting child language (PROLIN)

Language is a foundation for literacy, numeracy and more broadly education. There is therefore a strong case for the implementation of language interventions in the early school years (Law et al., 2017; Snowling & Hulme, 2021). The Nuffield Early Language Intervention programme (NELI), initially developed in 2004, has been evaluated in several trials in the UK (see Snowling et al., 2022, for a review). The most recent and largest trial, which involved an independent team, found positive effects on the language skills of children entering school who received the intervention, including those with English as an additional language (West et al., 2021). At the core of NELI is work to support the development of vocabulary, narrative and active listening. It consists of two strands: (1) training in expressive and receptive language skills throughout the 20-week programme; (2) training in pre-literacy skills (phonological awareness and recognition of letters and words) for the second 10-week period.

The Educational Programme for Child Language Promotion (Programa Educacional para a Promoção da Linguagem Infantil – PROLIN) was inspired by NELI and developed for Brazilian schools by a multidisciplinary team. Given the poor educational situation in Brazil (UNESCO, 2014) and its standing as a middle-income economy with huge socioeconomic disparities that adversely affect educational outcomes (World Bank, 2021), such a programme is much needed.

PROLIN’s principles and structure followed those of NELI and were adapted culturally and linguistically for the Brazilian context. Researchers worked together with preschool teachers to create a list of language/song activities and storybooks commonly used in their classrooms and appealing to children at this age. The language/song activities were adapted to form an ‘Active Listening’ axis (oral language activities focusing on words, songs, and phonological awareness), and the storybooks formed a ‘Vocabulary & Narrative axis’ for extending and enriching children’s expressive language skills. This procedure allowed us to keep the structure and axes of NELI while fully incorporating cultural substrates (values, beliefs and experiences) that were relevant to both children and teachers (Larson et al., 2020).

Factors related to the success of an intervention

Newbury et al. (2022) propose a model for guiding the implementation of interventions in educational settings (see Figure 1). In this model, factors external and internal to the intervention affect its acceptability, fidelity of delivery and sustainability. Although the decision to adopt evidence-based intervention programmes depends upon school and governmental policies and must be aligned with educational, linguistic, cultural and social values (both external factors), their success depends directly on how teachers and the school leadership team will receive and implement it in the classroom with their students.

Figure 1. Implementation framework for language and reading interventions in low-income educational settings (Newbury et al., 2022).

It is also important to take account of the background knowledge of the teachers who are to deliver the programme including their qualifications, attitudes, beliefs, dispositions, wellbeing, personal style and socioemotional characteristics. Personal style, in particular, strongly influences teacher responsiveness – it is the ‘how’ related to engagement, attendance and practice (LoCasale-Crouch et al., 2016), whereas teacher adherence – the ‘what’ related to delivery of intervention components as prescribed – relates to implementation fidelity (Pence et al., 2008). Both aspects (responsiveness and adherence) have been shown to influence outcomes of interventions (Bleses et al., 2018a, 2014, 2018b; Blewitt et al., 2020; Durlak & DuPre, 2008; LoCasale-Crouch et al., 2016, 2011) and are amenable to improvement through training and follow-up support (Downer et al., 2012; Hamre et al., 2012).

Another, less flexible facet of these aspects is teachers’ perceptions and beliefs. Still, once teachers perceive the value of an intervention early in the process, their engagement is likely to grow. An overt approach to interventions, where teachers are made consciously aware of the potential benefits of a programme – and its evidence-base – may positively correlate with their degree of adherence and engagement, thus affecting outcomes (LoCasale-Crouch et al., 2016, 2011).

In summary, both the quality of the materials and the nature of the support provided to teachers delivering the intervention are the basis of teacher motivation and programme fidelity. In this light, the design of materials becomes crucial: they need to (i) be aligned with the local culture; (ii) be acceptable to policy makers and fit with the current climate in schools; (iii) be easy to implement often in time-limited sessions and to groups with differing needs; and (iv) embed sufficient guidance and strategies to enable school staff with different degrees of experience and background knowledge to deliver the programme effectively. Thus, in designing an intervention, it is important to keep in mind the goals of acceptability and fidelity of delivery if it is to be taken up in practice and be sustainable.

Earlier versions of PROLIN have carefully taken into consideration aspects related to the local culture (designing materials and activities that were culturally and linguistically appropriate) and leadership support (securing acceptability by the secretary of education and school principal). However, while the rationale for PROLIN was based on robust scientific evidence emanating from the UK, its lack of efficacy (or some success limited to few schools) in a culturally diverse setting suggested that materials needed modification both to address the needs of teachers and children who would receive the programme if fidelity of delivery was to be ensured.

The present study

This small ‘proof-of-principle’ study aimed to explore if improvements to training materials and the introduction of a system for engaging and communicating with teachers to offer support within PROLIN could increase fidelity of delivery. ‘Prototyping with intent’ involves testing whether key intervention components produce the expected results and represents a crucial step for large-scale studies (Hulleman & Cordray, 2009; LoCasale-Crouch et al., 2016). Our approach involved scrutiny of PROLIN to identify parts thought to require revision and to make changes (Phase 1) and a further pilot study to monitor and evaluate the implementation of an updated version of the programme (Phase 2).

According to Williams and Beidas (2019), the explicit sequence, scope and structure of materials are important for the success of an intervention. Further, semi scripted and skill-focused curricula can benefit children without prior systematic instructional focus on language or preliteracy skills (Bleses et al., 2018a, 2018b). To facilitate programme delivery and improve instructional interactions, teachers need training and coaching support (Downer et al., 2012; Hamre et al., 2012). With structured materials, teacher training can be optimised both at pre- and during-intervention stages. Fidelity to the intervention model (verification of key intervention components through checklists) can provide measurable verification of application (Fixsen et al., 2005). It can be examined via component-specific observer ratings throughout implementation, and secured by offering adequate instructional support to teachers through materials (key components) (LoCasale-Crouch et al., 2011; Pianta et al., 2012).

Method

First phase: improving the programme

Phase 1 of the study, which led to improvements in the programme, involved a multidisciplinary focus group of one educator and four speech and language therapists, chosen for their knowledge of child language, language difficulties and pedagogy. There was no prior involvement of members of the focus group or the school in our previous studies. The lead researcher was the single element of connection between previous trials and the present study and gave the following briefing to the group: ‘I am going to give you written instructions of different types of activities [included in the original version of PROLIN] and would like you to point out, for each of them: 1) if it is clearly explained; 2) if there is enough information about how to deliver it; 3) if not clear, which aspects need revision; 4) how you would manage group dynamics for this activity; and 5) whether you would use specific teaching techniques in case children demonstrate difficulty performing the task?’. These initial observations were taken as a starting point for the revision of each activity, which was carried out through discussion until the group reached a consensus on how changes could be implemented. The group met for 20 weekly meetings of 120 to 180 minutes and undertook a complete review of the entire programme manual.

The materials to be revised were agreed as: (i) teacher guidelines; (ii) teacher training programme; (iii) implementation videos; (iv) learning management system (LMS) setup and design; and (v) observation report for monitoring the fidelity of implementation. Box 1 shows a summary of original materials, the rationale for change, how they were revised and a feasibility check for adaptability to different wealth contexts.

Box 1. Condensed revisions made to teachers’ and students’ materials and their feasibility in relation to cost (high/medium/low ) and implementation (easy/medium/difficult ) considering diverse wealth contexts and local capabilities.

	Materials	Original	Revision	Rationale for change	Feasibility
TEACHERS	Guidelines	Descriptive instructions	Clear divisions for skills, objective, instructions and application for two versions (simple & complex) and variations	Highlight objectives for teachers to readily grasp what students should be able to do (Burchinal et al., 2008)	Low Easy
	Training programme PPT	None	For leaders: state objectives, goals, and outcomes For teachers: rationale and step-by-step implementation	Introduce the programme’s rationale (Piasta et al., 2012)	Low Medium
	Implementation videos	None	1’ to 3’ videos on how activities should be delivered	Enhance teacher implementation (Piasta et al., 2012)	High Medium
	Learning management system	None	Edmodo to host videos and for instant messaging and engagement	Provide a means of communication for teachers and team during training and coaching; monitor the implementation (Hulleman & Cordray, 2009)	Low Medium
	Observation report	Less than 15 questions with yes/no answer	22-29 questions (depending on the session) to rate delivery with a three-point Likert scale (ranging 0 to 2)	Rate teachers’ implementation, quality of teacher-student relationship, delivery, student engagement, and fidelity. (LoCasale-Crouch et al., 2016)	Low Medium
STUDENTS	Visuals	None	A5/A6 cards and A4 boards	Support children’s engagement during implementation	Medium Easy

Notes: Cost was classified as low, medium or high and computed according to the following criteria: low – not dependent on outsourced materials/means; medium – dependent on outsourced materials with a modest budget (under US$ 100); high – dependent on outsourced materials with a more robust budget (above US$ 100). Ease of implementation was classified as easy, medium and difficult and was computed according to the following criteria: easy – requiring short time and effort to implement; medium – requiring moderate time and effort to implement; difficult – requiring long time and effort to implement.

Teacher guidelines

Guidelines were modified to highlight the aim of each activity. Objectives were made explicit so that teachers could readily grasp what students should be able to do. For instance, the third activity in the Active Listening axis is the ‘Word Battle’, a card game that requires children to identify the word that contains more syllables, among three or more options. The objective of the activity was clearly highlighted for teachers on the top of the page (i.e. segment and count the number of syllables) and the manual provided precise information on how to deliver the activity. Instructions were aided by a visually attractive, clear framework for delivery for all the activities in the manual (see Figure 2 for a sample page of the new layout of teachers’ guidelines).

Figure 2. Layout of the improved teachers’ guidelines.

Teacher training programme

Two slide presentations were created to introduce the programme to different stakeholders. The first was intended for school leadership teams and aimed to introduce the programme’s objectives, overall goals, and outcomes. The second was aimed at teachers taking part in the programme and consisted of a summarised version of what was presented to school leaders together with a step-by-step timeline of the training programme (see online Appendix 1).

Implementation videos

Videos emphasising the goals of each activity and providing detailed instructions to teachers on how to deliver them were considered an important addition to support and facilitate the training of teachers involved in the programme. The videos (1–3 minutes in length) were scripted and filmed by the research team. They provided examples of each type of activity to be carried out by teachers and could be viewed at any time.

Learning management system

A Learning Management System (LMS) was implemented using Edmodo, a simple-to-use, freely available platform that allowed settings to be programmed in Brazilian Portuguese. The design is intuitive and easy to manage both by providers and users. The intent of the LMS was to provide a means of communication between teachers and trainer in charge and to monitor the implementation, allowing specific feedback to teachers based on data from observations. The LMS also served to host the videos, thus allowing easy viewing by teachers. Coaching to ensure quality of implementation was monitored via instant messaging and user access for the purpose of engagement. Posts and folders were password protected and could only be accessed by those in the programme.

Observation protocols

In order to monitor and enhance fidelity of delivery, observation protocols were introduced for each day of delivery. These protocols included the expected achievements of each type of session in the programme: auditory and phonological skills (termed Active Listening), vocabulary building and story retelling (termed Vocabulary & Narrative) and the Consolidation Session. Each protocol contained 22–29 questions (depending on the day of the programme) regarding the programme delivery with a three-point Likert scale (ranging from 0 to 2) for the observer to rate teachers’ attitudes, as well as an open comment section in which observers could make qualitative comments. The observation protocols were given to teachers as a guide for each day of the implementation (as a form of self-calibration). In addition, they helped the research team gauge the quality of implementation concerning teacher-student relationship, delivery of the intervention, student engagement, and teacher fidelity.

Programme materials for children

The team created new visuals for the programme’s materials such as cards and boards because the previous versions lacked a visual identity and were not systematically designed. To improve the professional look, they were centred into frames that were uniform (A4 for boards and either A5 or A6 for picture cards) and laminated for improved usability.

The expectation based on the findings of Phase 1 was that changes to the original programme, coupled with the additional support for teachers, would afford language gains and better programme outcomes.

Second phase: implementing the changes in a pilot study

Given the changes made to the original programme and materials, coupled with additional support for teachers via LMS and media channels, we expected the programme to have more impact and that we would observe better language outcomes for children receiving the programme. We proceeded to pilot seven sessions in a proof-of-concept study. After the final session, we gathered feedback, via interviews with teachers and the school director, to reflect on the success of the implementation and remaining obstacles for delivery. The study was approved by the Ethics Committee of the Federal University of São Paulo (CAAE: 29401920.8.0000.5505; Review nb: 3.903.532).

Participants

This pilot study was implemented in a small private school in the city of São Paulo. In 2018, around 60% children in the uppermost socioeconomic quintiles in Brazil were enrolled in private preschools while around 64% of lower quintiles were in state preschools (OECD, 2021). In Brazil, children between 1 and 3 years old attend an early education development programme (creches); while 4–6 year-olds receive compulsory, pre-primary education (pré-escola). Out of this last tier, 75% of children attend state schools and 25% attend private institutions, which are classified into two different types: government-dependent (over 50% government-funded), and independent private, as the one enrolled in this study.

Two teachers working in this small private school piloted the programme, each one in charge of a kindergarten class (K1 and K2; 4 and 5 year-olds, respectively). An assistant teacher helped support the most challenging students in each group. Teacher 1 is a 20-year experienced female early education teacher with a double degree in psycho-pedagogy and an employee at the school since her practicing internship. Teacher 2 is an early education teacher with 7 years’ experience. The variability in the teachers’ academic backgrounds and experience reflects a typical characteristic of both private and state schools. Each teacher gave informed consent to be involved in the pilot programme without fees or remuneration. The school management agreed to exempt both teachers from planning time during the intervention weeks.

Each teacher had a class of 11 children, ages ranging from 3.7 to 5.8 (n = 22; MM = 4.2; SD = 0.46; 12 girls). This number is close to the average class size for the public sector (21 students: OECD, 2017). Two children (one in each group) had Autism Spectrum Disorder (ASD). The study was opt-in with no exclusionary criteria.

Materials

Seven activities were selected for piloting from each of the two main axes of PROLIN (Active Listening; Vocabulary & Narrative). The structure of each session of the pilot study was as follows: Simple and Complex versions of one Active Listening activity followed by one Vocabulary & Narrative activity.

Active Listening

Each Active Listening activity had a simple and a complex version, which differed in terms of the targeted verbal stimuli (word length, number of words and syllables to be retrieved/identified) and the processing demands (possibility to use cues; amount of information to be processed simultaneously). Teachers were asked to deliver the two versions in each session.

Examples of the simple and complex versions of the same Active Listening activity were as follows: Word within the Word. The objective is to identify a smaller word contained within a larger word. The task instruction is described as: ‘This is an activity in which children should identify which “small” words (e.g. ice) are within “large” words (e.g. rice). For the SIMPLE version, select the pairs of words (small and large, found in the complementary materials set) and arrange the figures so that the large words stay on the left and the small words on the right, all facing down. Then, turn all the small figures up, but only one large figure at a time. Children should look for which of the small figures is “inside” the large figure. Encourage children to analyze and compare figures to judge which word is within which. For the COMPLEX version, use only the figures for the large words. Turn each large figure at a time and elicit from children, off the top of their heads, which smaller words are within the large word illustrated’.

Vocabulary & Narrative

The Vocabulary & Narrative activities were based around the first two storybooks of PROLIN. Each book was used for three days: the first day focused on understanding the story; the second day explored vocabulary; the third day consisted of story retelling, and a final (fourth) day was a Consolidation session (see Figure 3 for an illustration of the structure of the Vocabulary & Narrative sessions).

Figure 3. Illustration of the structure of the vocabulary & narrative sessions.

Participating teachers received four resources: (1) a printed version of the abridged and updated Teachers’ Manual containing the theoretical basis, techniques and activities selected; (2) explanatory videos of the selected activities, which could be downloaded from LMS in bulk or viewed individually as required; (3) complementary materials (lists of stimuli/prompts) and picture cards for children engagement; and (4) the two storybooks selected for Vocabulary & Narrative. Teachers also had access to the slides used for the training session through the library assortment within the LMS.

Procedures

Training

Initially, we contacted and briefed the school principal and teachers on objectives, details, and procedures of the study. We also informed them about the scope of the intervention and the content of the training. Before implementation, there were two training sessions: a one-hour session with leadership and a three-hour teacher-training session led by a member of the multidisciplinary group, who presented the manual, activities, and complementary materials. Teachers were also shown how to download and use the LMS (Edmodo) to serve as a means of communication via instant messaging and email for questions or advice as needed, and to access the video repository for all the activities in the programme. Although teachers were not expected to post in the LMS, they could use it at their own discretion and feedback would be always provided. Teachers were instructed on our use of this resource to monitor implementation.

Implementation

Each teacher delivered one intervention session per day within the regular class during the school day: one Active Listening activity (20 minutes) (both simple and complex versions) and one Vocabulary & Narrative activity (25 minutes). There were seven days of intervention over a two-week period totalling 45 minutes per day. All activities contained the modifications made during the first phase of the study.

For each session and each teacher, a member of the multidisciplinary team observed the activities while completing a checklist containing component-specific items for session delivery. Video footage was also obtained for double-checking the quality of implementation. Online Appendix 2 presents example items from the Observation Checklists. The average score represented a quantitative measure of quality for each session (higher scores denote better quality). Total scores were extracted and averaged for the seven sessions attended by the research group.

Results

Observation scores

Scores on the checklists assessed quality of implementation, which was taken as a proxy for the improvements in the programme’s materials. Table 1 summarises the average score of each teacher on the main components of each type of activity. As there were only two teachers in this proof-of-principle study, it is problematic to infer reasons for differences in their delivery of the programme and interpretation should proceed cautiously.

Table 1. Average score of each teacher on the main components of each type of activity.

Type of Activity		Component										Total
		Teachers’ preparedness and familiarity		Dynamics of the activity		Teachers’ use of techniques and strategies		Students’ attentiveness and engagement		Activity closure
		T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2
Active Listening	Simple	1.9	1.4	1.9	1.9	1.9	1.6	2.0	1.8	1.1	0.3	1.8	1.4
	Complex	1.9	1.4	1.9	1.9	1.8	1.4	1.8	1.5	1.1	0.3	1.7	1.3
Vocabulary & Narrative	Day1	2.0	1.5	2.0	2.0	2.0	1.5	2.0	2.0	2.0	0.0	2.0	1.4
	Day2	2.0	1.3	2.0	1.5	1.7	1.2	1.8	1.5	1.5	1.0	1.8	1.3
	Day3	2.0	1.5	1.8	1.8	1.4	1.3	1.8	2.0	1.0	1.5	1.6	1.6
	Cons.	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0	2.0
	Total	2.0	1.5	2.0	1.9	1.8	1.5	1.9	1.8	1.5	0.9	1.8	1.5

Notes: T1 = Teacher 1; T2 = Teacher 2; Cons. = Consolidation session. Values represent the average scores per component of the intervention, for each teacher, in a Likert scale ranging from 0–2.

On average, Teacher 1 scored higher (M = 1.8, 90%) than Teacher 2 (M = 1.5, 75%), possibly reflecting the difference in teaching experience between the teachers, which was more evident on the components ‘teachers’ preparedness and familiarity’ (environment preparation to start activities) and ‘activity closure’ (summary to close off the activity). Ending sessions appeared to be very difficult for both teachers, highlighting difficulties in closing and recapping on activities. Both teachers also found it difficult to encourage the shyest/struggling children to participate.

For Active Listening sessions, the dynamics were generally good, though children were often not given sufficient time to participate. Complex activities were considered to need better presentation for students. In Vocabulary & Narrative sessions, there were more differences between the teachers. For example, on Day 1, Teacher 1 obtained the maximum score while Teacher 2 did not check understanding of the story adequately and made no summary to close off the activity/story. Similarly, in Vocabulary & Narrative Day 2 sessions targeting vocabulary, Teacher 2 did not prepare the environment adequately and was not familiar with the target words, struggling to define them for the students.

Both teachers had difficulties using teaching techniques (i.e. expansion; extension; imitation, recapping, evocation, analogy, synonym and antonym, and definition) even though such techniques were included in the training session and flagged in the manual. Neither teacher made sufficient use of evocation questions, and each forgot to elaborate on children’s speech. Teacher 2 used analogies infrequently and applied synonyms and antonyms sparingly, thereby failing to keep to the prescribed approach.

On Day 3 of Vocabulary & Narrative, teachers had to ask children to retell the stories they had been read. Teacher 2 once again scored lower than Teacher 1 in their preparation of the environment, in using complementary material (pictures) to support the session and in encouraging the dynamics of the session. Teacher 1 failed to engage children successfully and did not include everyone equitably. Both teachers struggled to use techniques with a low application of extension and expansion, and poor use of imitation.

Both teachers showed excellent performance in the Consolidation session in using activities aimed at employing target vocabulary in different contexts. However, despite high scores in these sessions, the Edmodo messaging app suggested that teachers felt insecure regarding materials, guidelines, objectives and expected outcomes. In summary, they had doubts on how to structure and implement this activity, indicating that additional support would be valuable.

Feedback

Alongside the checklists, we asked teachers for feedback. These post-implementation sessions highlighted that: (a) teachers needed more planning time, e.g. for reading the manual; familiarising themselves with the materials; and watching the videos; (b) more opportunities were needed for teachers to simulate or view role plays during the training period; (c) teachers perceived programme benefits for all students (with or without difficulties) in language and social interaction; (d) teachers got confused with the number of cards and figures provided (targeted vocabulary and active listening cards) and suggested ways to better organise the materials for each activity; and (e) for activities for which children receiving the programme have no previous knowledge (e.g. phonological awareness), ‘warm-up’ practice items would be helpful.

Based on these findings, it can be concluded that further changes to PROLIN materials (manual, videos and supplementary materials) would be required before scale-up to provide teachers with extra guidance and to address the weaknesses identified in the pilot study (Box 2).

Box 2. Issues and resolutions for improving the quality and fidelity of implementation.

Type	Issue	Solutions
		What	How
Session dynamics	Difficulty offering students adequate time to participate in the activity or respond to what was proposed.	A short pause is required after requesting something so that students can process what was requested and organise their thoughts to respond properly.	Inclusion of such an instruction will be provisioned in implementation videos.
	Difficulties engaging all children in the proposed activity, including maladaptive profiles.	Strategies or techniques can be used to engage all students and enable effective inclusion and differentiation.	Ideas and tips for using such strategies will be provided in the manual.
Activity implementation	Difficulty understanding the purpose of some activities.	Teachers need to fully understand the objectives of each activity.	Stating objectives clearly in the teachers’ manual for each activity.
	Difficulties understanding the how to deliver some activities (e.g. complex AL, consolidation).	Need of step-by-step instructions for implementation of every activity.	Although activity instructions have been provided, careful scripted suggestions for instructional delivery will be now included (video samples and supplementary materials).
Reinforcing learning	Difficulty performing a closing for each activity.	Closing off activities help children consolidate their knowledge after performance.	Inclusion of a specific section in the manual explain why and how closing can the performed (examples, alternatives and possible video samples will be provided).
	Low adherence to strategy/ technique usage for each activity (specially V&N).	Teachers should use techniques to help children make deeper semantic connections between words and concepts, and not only name target words.	A ‘Teaching techniques’ section (classification, definition, and examples) will be added in the manual, with recommendations for each type of activity. A full example of how to strengthen semantic connections based on one storybook will be provided in videos. Supplementary materials (infographics, roadmaps) will also be presented.

By proceeding in an iterative manner it is possible to refine an evidence-based programme to ensure acceptability and fidelity of delivery, if sustainability is to be achieved.

Discussion

This study reports how aspects of a research-led educational intervention, evaluated in the UK, were prepared and refined for scale-up in a diverse urban setting. This early years language intervention, PROLIN, was the subject of this case study following an initial set of trials, which had not shown efficacy. We describe how weaknesses in the programme, specifically relating to the content, materials and the professional training of teachers to deliver the programme were identified. Our results showed that the quality of implementation of a refined version was generally good but identified additional areas for modification.

We used a framework for implementation of educational interventions as theoretical background to the study (Figure 1) (Newbury et al., 2022; Snowling et al., 2022). In this model, external factors (such as quality of teaching materials and teachers’ background knowledge) affect internal factors (e.g. teacher training and support), which in turn critically influence the intervention’s acceptability, fidelity of delivery and sustainability.

Our findings showed that adding structured materials to the programme and improving training resources had in general good acceptability, but still had some faults. Teachers showed some difficulties with aspects related to session dynamics (offering students adequate time; engaging all students), implementation of activities (understanding the objectives of each activity and how to reach them) and use of techniques that reinforce learning (using strategies to improve learning). These weak points should be thus considered for further modification of PROLIN’s materials, especially in the manual, videos and supplementary materials (Box 2).

These changes should start from refining the guidelines to make the sequence and scope of activities clearer in the layout, as suggested by Williams and Beidas (2019). Infographics, tables and roadmaps that are visually appealing and easy to understand may bridge the gap from theory to practice and increase implementation fidelity. Also, adopting streamlined and consistent layout in each section may help signal to teachers how instructions should be applied and what sequence to follow (e.g. appropriate closing was lacking in almost all sessions).

Another point for further improvement is the need to provide real examples on how to deliver the activities, which may help increase fidelity of implementation and adherence to the programme (Pence et al., 2008). Activities may be clarified by providing in-loco practice videos to support teachers in understanding the activity and applying it correctly. A granular, step-by-step explanation and guidance in relation to the complementary material (visuals) and instructional delivery (techniques) could also bring clarity during implementation (LoCasale-Crouch et al., 2016; Pence et al., 2008).

Among all activities, teachers needed more support in delivering the Vocabulary & Narrative sessions, demonstrating difficulties to expand children’s vocabulary and assist them to build semantic networks. Tips on where and how such vocabulary could be explored must be inserted in teachers’ materials. Additionally, strategic support from our speech-language therapist’s team could deepen teachers’ comprehension on how to use language-learning techniques (Pence et al., 2008). For example, use of ‘open-ended questions’ – questions that allow the child to answer using more words besides ‘yes’ or ‘no’ (Dickinson & Smith, 1994; Wasik & Hindman, 2011) – to promote opportunities for meaningful language use will be included. It was also evident that the objectives for storybook activities and instructions for consolidation sessions were not described well enough, differently from the Active Listening activities. Guidance for effective instructional support (Ansari & Pianta, 2018) with explicit information on what and how children are expected to learn are needed (Piasta et al., 2012). Consolidation sessions, in particular, need a deep revision to enable teachers to work with recurring semantic categories in a flexible and consistent way.

It was also clear that teachers requested more practice during training to feel confident throughout implementation. This extra time was also needed to help them become familiarised with materials (e.g. recurring word categories in the consolidation session). Mastering the components and aims of the programme is a crucial step not only to increase fidelity of implementation, but also to enable teachers to promote qualitative interactions with children during the activities (Burchinal et al., 2008). In this pilot, we noticed that teachers did not offer students ample time to participate in the activity or respond appropriately, which could mean that they were so focused on performing the activity as instructed that they could not fully pay attention to the children’s behaviours. With clear objectives in mind, teachers may be able to deliver the intervention with high fidelity and adjust activities according to their students’ needs. And by noticing that their contributions are welcomed, their reliability on the programme’s components also grows, further increasing adherence (Downer et al., 2012). Investing in adherence is essential as when it varies, children’s outcomes may fluctuate (O’Donnell, 2008).

Finally, it is important to highlight that school support and teachers’ educational experience influence their motivation to deliver the intervention in real-world settings (Newbury et al., 2022). Durlak and DuPre (2008) found larger effect sizes on outcomes when implementation was intentionally supported and monitored (high fidelity). Similarly, Hulleman and Cordray (2009) found that applied interventions have to take in contextual variables – like teachers’ availability for training, leadership support and internet access. These aspects, together with future improvements in PROLIN’s materials and training have the potential, to yield positive effects on the next clinical trial study.

Conclusion

Taken together, findings from the pilot study were instrumental in signposting changes needed in the programme for future upscaling. PROLIN, and possibly similar adapted intervention programmes for diverse settings, may benefit from further changes in materials related to the session dynamics, implementation of activities and use of techniques that reinforce learning. Although small, the pilot study was effective in bridging the gap between previous unsuccessful trials and research findings at large. The chance to prototype with intent optimises the time-costly process of developing a programme to implementing it in real contexts (Fixsen et al., 2005). In searching for key components to go from ideal to actual implementation, first steps matter. And when it comes to implementing interventions on a large scale, pilot studies are essential to develop sustainable, effective culturally diverse language intervention programmes suitable for children learning in wealth diverse settings.

Authors’ contributions

M.C.C.R., H.S., H.G.S. and M.L.P. worked on the intervention and designed the pilot study. M.C.C.R., H.S. and H.G.S. carried out the pilot study. M.C.C.R., H.S. and M.L.P. contributed to the results’ presentation and interpretation. M.C.C.R. wrote the manuscript. M.J.S., M.L.P., H.S. and H.G.S revised the manuscript. C.H., M.J.S, D.F.N and M.L.P. provided critical feedback on the research and manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/03054985.2022.2088488

Additional information

Funding

This work was supported by the British Academy [ECE190048, NIFAL19\190036 and NA21/210021].

Notes on contributors

Mirela C. C. Ramacciotti

Mirela C. C. Ramacciotti is a PhD Candidate in Human Communication Disorders at the Federal University of São Paulo (UNIFESP), MSc Interdisciplinary Education at Johns Hopkins University. Education researcher and consultant.

Helena Sousa

Helena Sousa is a Speech-Language Therapist, specialist on Child Language, Master’s student in Human Communication Disorders at UNIFESP.

Heloisa G. Silveira

Heloisa G. Silveira is a Speech-Language Therapist, Master’s degree and PhD Candidate in Human Communication Disorders at UNIFESP.

Charles Hulme

Charles Hulme is a Professor of Psychology and Education at the University of Oxford. Recipient of the Feitelson Research Award from the International Reading Association (1998), the Marion Welchman International Award for Contributions to the study of Dyslexia from the British Dyslexia Association (2016) and the Society for the Scientific Studies of Reading Distinguished Scientific Contributions Award (2019). Fellow of the British Academy and of the Academy of Social Sciences and member of Academia Europea.

Margaret J. Snowling

Margaret J. Snowling is President of St. John’s College and Professor of Psychology at the University of Oxford. Awarded the British Psychological Society Presidents’ Award (2003) and the Samuel T Orton Award of the International Dyslexia Association (2005). Holds honorary doctorates from Goldsmiths London, University College London, Warwick and Bristol Universities for contributions to the science of reading and dyslexia. Fellow of the British Academy, the Academy of Medical Sciences and the Academy of Social Sciences. Appointed CBE for services to science and the understanding of dyslexia in 2016.

Dianne F. Newbury

Dianne F. Newbury is a Molecular geneticist, Senior Lecturer and Lab Chief at Oxford Brookes University. Research funded by the Leverhulme Trust, the ESRC and the British Academy on the genetic contributions to childhood neurodevelopmental disorders.

Marina L. Puglisi

Marina L. Puglisi is an Adjunct Professor at the Department of Communication Sciences at the Federal University of São Paulo (UNIFESP). PhD in Communication Sciences at the University of São Paulo (USP); Post-doctorate at USP and University of Oxford and former Newton International Fellow.