Adapting the preschool environment to the needs of children on the autism spectrum in Sweden: A quasi-experimental study

Abstract

Background

Children on the autism spectrum may receive Early Intensive Behavioural Intervention (EIBI) in Sweden to support development by providing learning opportunities. However, research suggests a need to improve the quality of their learning environment.

Aim

This pre-registered study (#NCT03634761) aimed to evaluate the Swedish Autism Program Environment Rating Scale (APERS-P-SE) as a means to promote the quality of the pre-school learning environment for children on the autism spectrum, along with outcomes for these children, and their pre-school staff.

Material and methods

In a quasi-experimental study, pre-school staff (n = 35) conducted either EIBI supported by in-service training and on-site coaching based on APERS-P-SE assessment (at k = 9 preschools) or EIBI only (k = 8), during an 8-months period.

Results

The quality of the immediate learning environment (primary outcome) increased significantly in the EIBI/APERS-P-SE preschools. However, changes in child and pre-school staff measures were not significant.

Conclusions and significance

In-service training and on-site coaching based on APERS-P-SE assessments can increase the quality of the learning environment in Swedish pre-school. Studies of longer duration and larger sample size are required to determine if using the APERS-P-SE can also achieve desired behavioural change for children-, and impact pre-school staff applying EIBI.

Keywords:

• Autism spectrum disorder
• coaching
• early intensive behavioural intervention
• early intervention
• environmental enrichment
• learning environment

Introduction

Autism spectrum disorder (ASD) is a neurodevelopmental disability with diagnostic criteria including altered social communication and interaction, alongside repetitive, stereotypical behaviours, restricted interests, and atypical sensory processing [1]. The rates of ASD diagnoses have increased globally to about 1–3 percent of all children in high income countries [2]. In Stockholm, Sweden, the number of pre-school-aged children diagnosed with ASD has doubled during the last five years [3]. Most children on the autism spectrum can be identified reliably at 24 months of age [4,5], and epidemiological data shows that children on the autism spectrum are not only diagnosed more often today, but also earlier for each cohort [6]. Particularly in many high-income countries, early detection today allows young children diagnosed with ASD to get access to clinical and educational services that involve Early Intensive Behavioural Intervention (EIBI).

EIBI is a systematic, highly structured, comprehensive, and evidence-based child-focussed approach for teaching pre-school children socially significant behaviours, based on the principles of applied behaviour analysis (ABA; [7,8]). Although actual practice may differ between programs, EIBI involves several core components. Programs typically address various domains of skills that are challenging for children on the autism spectrum (e.g. language, motor, imitation). Interventions are individualized and supervised by trained professionals, with positive reinforcement used to promote new skills and reduce problem behaviours. Different evidence-based practices are used, which can be both child- and adult-initiated [7]. One example of a commonly used practice is Discrete Trial Training (DTT; [9]), to promote children’s skill acquisition using a planned, controlled, and highly structured one-to-one approach. Other examples are reinforcement [10], and prompting [11], which are applied to reinforce socially significant behaviours, while providing the child with enough support (i.e. prompt) to successfully perform the behaviour. EIBI can be implemented in different settings such as preschools, clinical centres, and by children’s parents at home. EIBI supervision may be provided monthly, or multiple hours weekly, depending on the setting and context [12]. Ideally, EIBI supervision should be provided by practitioners with extensive knowledge of ASD, the principles and procedures of ABA, as well as EIBI approaches [13]. Also, data on children’s learning are usually continuously collected for evaluation, for setting goals, and for choosing teaching procedures [14,15]). Starting with EIBI at a younger age may facilitate favourable developmental outcomes [16–23]. However, the increasing number of ASD diagnoses also challenges the capacities of service providers, such as preschools (education sector) and habilitation centres (health care sector), which predominantly provide these services in Sweden [24].

Sweden has a universal and tax subsidized pre-school system which almost all children attend, and children with disabilities (such as ASD) are no exception [25,26]. The publicly funded habilitation centres provide child-focussed intervention to children on the autism spectrum using a community-based EIBI model. The intervention follows national recommendations outlined by the Swedish Board of Habilitation Directors [27]. The recommendations suggest a minimum of 25-hours of weekly instruction, during two years, for children at the youngest age possible, as early intake and high intensity is associated with better outcomes [8,19,20,28]. The EIBI model focuses mainly on supporting children’s learning and functioning in the areas of social interaction, functional spontaneous communication, problem behaviours, cognitive skills, and personal independence. Typically, adult-initiated one-to-one instruction is provided, using previously described strategies such as DTT, reinforcement, and prompting. In this delivery model, EIBI is provided mainly at the pre-school by a paraprofessional, but also at home by children’s parents. The paraprofessional is usually employed or assigned by the pre-school principal to work specifically and intensively with the child on the autism spectrum, for whom the pre-school has applied for and received financial resources from the municipality, and brief introductory training at the habilitation centre. EIBI supervisors [29] provide supervision weekly or bi-weekly to the paraprofessional and the parents, over a period of two years. Supervision sessions are typically about 90 min long, and provided at the habilitation centre, with occasional visits to the pre-school. In these sessions, the EIBI supervisor commonly introduces new instructions and procedures for the child to promote behaviours, such as motor imitation, social imitation or joint attention. The paraprofessional and parents can then practice providing instruction with the support of the supervisor, before continuing to provide the instruction at the pre-school or home setting. The EIBI supervisors may have different professional backgrounds (e.g. occupational therapists, psychologists, speech and language pathologists, and teachers in special education), and may have varying knowledge about ABA. For an overview of the different stakeholders at organizational and individual level in this service delivery model, see Figure 1.

Figure 1. Overview of stakeholders at organizational and individual level.

In Sweden, special needs in preschools are defined as having physical, cognitive, or other developmental needs that warrant special support [30]. Special needs are assessed by pre-school staff in collaboration with children’s parents, and do not require a formal diagnosis [31]. The national Swedish curriculum for pre-school [32] emphasizes the importance of supporting [33] and including [34] children with special needs based on their individual needs, but a recent report from the Swedish School Inspectorate [35] suggests actual practices differ significantly from these guidelines. Only 30% of preschools were found to provide adequate support to special needs children. In 2019, a new national Swedish curriculum for pre-school [32] was introduced, with a stronger focus on the pre-school’s responsibility in providing support to children with special needs to promote their optimal development, which additionally reinforces the demands on habilitation and preschools to meet the needs of children on the autism spectrum. However, as noted by several groups of researchers [17,36,37], implementing evidence-based intervention programs such as EIBI in poor quality learning environments is difficult and likely to fail. Thus, physical, instructional, and social features of educational settings (i.e. the learning environment) are of importance [38]. These include structural features (e.g. class size and staff’s education), and procedural features (e.g. teacher and child- interaction) [39,40], which have been associated with children’s developmental outcomes, including social, cognitive, and language development [41,42], as well as children’s engagement [43] and performance in various tasks [42].

In Sweden, and for successful EIBI implementation in particular, findings of previous studies on important features of autistic children’s pre-school learning environments are worrisome. They have indicated that pre-school staff are inexperienced, lack formal education in working with children on the autism spectrum, have negative attitudes towards EIBI, and neutral to negative attitudes towards inclusion of children on the autism spectrum in mainstream classrooms [44–46]. All of these environmental factors may negatively affect intervention quality, child engagement, inclusion, developmental trajectories, and long-term learning outcomes for children on the autism spectrum and other children with special needs in pre-school [32,46,47].

In order to address similar environmental challenges in a United States (U.S.) context, the Autism Program Environment Rating Scale (APERS) was developed by the U.S. National Professional Development Centre on ASD [36]. It was designed to assess the quality of educational programs for children on the autism spectrum. The instrument has shown high internal consistency (Cronbach’s Alpha r = 0.94) for the total scale, and moderate internal consistency for its subdomains (Cronbach’s Alphas averaging r = 0.70). It has also shown sensitivity for improvements in autism program environment quality, as well as a single factor loading conceptualized as ‘autism program environment quality’, using an exploratory principal component analysis [38]. The information generated by the APERS allows assessors to identify areas of strengths and weaknesses in the pre-school or school setting. Subsequently, an action plan can be developed in collaboration with pre-school or school staff to improve the program quality of the child on the autism spectrum in the class. Thus, APERS serves two purposes; first to assess the current learning environment, and second, as a means for intervention planning by highlighting areas with potential for improvement. Assessments and intervention planning can then be used to initiate and facilitate refinement in the immediate learning environment for children on the autism spectrum.

The theory of change by the APERS authors specifies that targeting and improving program quality features for children on the autism spectrum (i.e. as measured by the APERS) sets the stage for implementation of child-focussed evidence-based practices (e.g. behavioural interventions, as found in EIBI), thus facilitating positive outcomes for children on the autism spectrum [36]. The APERS authors have used information about evidence-based practices to develop and evaluate a competence model of in-service training and on-site coaching [48] in 132 pre-school and school intervention programs for children on the autism spectrum in 12 states in the U.S. to improve pre-school and school learning environments. The APERS evaluation found improvements in key program areas, such as structure and set-up of the environment, curriculum and instruction, teaming, as well as children’s goal attainment, and staff usage of evidence-based practices [36].

In conclusion, there is a need to evaluate and develop the pre-school learning environment for children on the autism spectrum receiving EIBI in inclusive and mainstream pre-school settings in Sweden. Providing a pre-school environment-focussed intervention in addition to more conventional child-focussed intervention (such as EIBI) includes targeting different aspects of the immediate learning environment. These include various aspects as operationalized in the APERS, including staff knowledge on ASD, set-up and structure, curriculum and instruction, how pre-school staff interacts with children and their families, how pre-school staff support children’s independence and communication, how pre-school staff cooperate, and how pre-school staff promote children’s adaptive behaviours. It also comprises the use of evidence-based practices for children on the autism spectrum, which can be defined as focussed interventions based on scientific evidence of their effectiveness [49]. This extended target for intervention is in line with recent research exploring the context of function of ASD, using the World Health Organization International Classification of Functioning, Disability and Health (ICF; [50]). It is based on a bio-psychosocial model of functioning, where functioning and disability are conceptualized as the outcome of complex interactions between personal and environmental factors. The ICF thus demonstrates that a multitude of factors outside of the individual may facilitate children on the autism spectrum to meet their activity and participation potential [51]. Examples of such environmental factors from the second-level ICF categories in the comprehensive ICF core set for individuals on the autism spectrum across the entire lifespan [51], that correspond with how the APERS operationalize a high-quality learning environment for children on the autism spectrum, include products and technology for communication (e125), products and technology for education (e130), education and training services, systems, and policies (e585), light (e240) and sound (e250), friends (e320) and acquaintances, peers, colleagues, neighbours, and community members (e325), social norms, practices, and ideologies (e465), and people in positions of authority (e330).

Serving this purpose of assessing and improving pre-school learning environment quality for children on the autism spectrum in Sweden, Bejnö et al. [52] adapted and validated the Swedish pre-school version of the APERS (APERS-P-SE). During the process of matching the instrument to Swedish education and health care sectors, several revisions were made. The scale was modified to be used only for pre-school, and not both pre-school- and elementary school, as these have different curricula, and differ conceptually in Sweden. Furthermore, three items were omitted as deemed nonrelevant, and the remaining 56 items were modified when needed, to be relevant for the Swedish pre-school and early intervention context.

The overarching aim of the current study was to investigate the effect of sharing APERS-P-SE information, and providing in-service training and coaching in preschools on the quality of the pre-school program based on initial APERS-P-SE assessments, in addition to typical implementation of EIBI. The primary objective was to change (improve) pre-school learning environment quality for children on the autism spectrum. The secondary objective was to examine if there was an effect of the intervention on children’s adaptive behaviours-, engagement-, and general clinical impression. The third objective was to examine the if there was an effect of the intervention on pre-school staff’s applied knowledge of ASD and evidence-based practices, their self-efficacy, and their attitudes towards EIBI. We hypothesized that applying in-service training and on-site coaching derived from APERS-P-SE assessment in addition to EIBI would be superior to using EIBI only for both proximal (i.e. pre-school learning environment quality for children on the autism spectrum) and distal (i.e. outcome measures for children and pre-school staff) outcomes.

Material and methods

Study design

We conducted a study with a non-equivalent pretest-posttest group design [53]. At three different timepoints, over a total of three cohorts, preschools/EIBI intervention sites were assigned to either the experimental group or the control group. In each pre-school, at least one child on the autism spectrum was enrolled in an EIBI program. The experimental group received EIBI plus APERS-P-SE-based in-service training and on-site coaching (EIBI/APERS-P-SE preschools) while the control group received EIBI only (EIBI only group). The cohorts were recruited in Region Stockholm, Sweden, between September 2017 and May 2019. The first cohort of preschools (k = 6) was non-randomly allocated to experimental or control conditions, based on three matching variables. This purposeful group assignment ensured that the first cohort of preschools were divided into comparable experimental and control groups regarding (i) supervisors’ experience of supervising EIBI programs, (ii) the baseline level of APERS-P-SE determined level of pre-school environment quality, and (iii) children’s adaptive behaviour (Vineland Adaptive Behaviour Scales-II ratings) scores at baseline. The subsequent second and third cohorts of participating preschools (k = 11) were divided into matched pairs (based only on supervisor experience on EIBI), and with each pair randomly assigned to either the control or the experimental group. Group allocation was done by the first and second author, following first APERS-P-SE assessment, using an online randomization tool (https://randomizer.org). Neither researchers or the participants were blind to the group allocation. In all participating preschools, pre-intervention data including APERS-P-SE assessments were collected before the onset of intervention, and post-data was collected about 8 months after baseline measures. Thus, for each cohort of participants, the total duration of the intervention was about 8 months.

Ethical considerations

The study was approved by the regional ethical vetting board in Stockholm (2016/2498-31/5), and pre-registered with the U.S. National Institutes of Health (ClinicalTrials.gov; #NCT03634761). All participating preschools in the study’s control condition were provided with APERS-P-SE feedback, and were also offered in-service training equivalent to what was provided to the EIBI/APERS-P-SE preschools, about three months following the post-data collection.

Participants

Inclusion criteria

Inclusion criteria for participating children were: age 2–5 years, a documented community diagnosis of ASD according to ICD-10 [54], DSM-IV or DSM-5 [1], and having recently been enrolled in an EIBI pre-school program planned and supervised by a habilitation centre specialized in young children on the autism spectrum. Inclusion criterion for preschools was that a child fulfilling the above-mentioned criteria was enrolled in an EIBI program in one of the pre-school’s classrooms. Inclusion criterion for participating pre-school staff was to work in the same pre-school classroom as one of the participating children, either as a paraprofessional, pre-school teacher, or other pre-school staff (such as pre-school teacher assistant). To avoid supervisors providing supervision to pre-school staff in both group conditions, children recently enrolled in EIBI programs supervised by habilitation supervisors already in the study were excluded.

Recruitment

Parents whose children had recently started to receive the EIBI program were informed about the study in the context of a supervision meeting at the habilitation centre. Following their informed consent, pre-school principals were contacted and informed about the study. Following the pre-school principals’ informed consent on the preschools’ participation, pre-school staff were informed about the study by the research team, and informed consent was collected. In total, 52 individuals (i.e. 17 child/parent dyads and 35 associated pre-school staff) were recruited, over 17 different preschools/EIBI intervention sites. These sites were served by 17 habilitation supervisors.

Participating children

Participating children were aged between 3 and 5 years at the onset of study, with a mean age of 4 years and 8 months in both the EIBI/APERS-P-SE preschools and EIBI only preschools. Children had previously been diagnosed with ASD within the Child and Adolescent Mental Health Services in Stockholm by multidisciplinary assessment teams consisting of psychiatrists, psychologists, and sometimes other professions such as occupational therapists, speech-language pathologists, and social workers. ICD-10 criteria [54] and regional clinical guidelines for the assessment and treatment of ASD were used [24], including standardized diagnostic instruments, particularly the Autism Diagnostic Observation Schedule [55]. A minority of the children (4 out of 17) had received an additional neurodevelopmental disorder diagnosis, such as intellectual disability, speech disorder, or attention-deficit hyperactivity disorder (ADHD). All children in the study attended inclusive and mainstream community-based preschools, except for one child in the EIBI only group who attended a special education group for children with disabilities in a community-based pre-school.

Participating preschools

Preschools were situated in different parts of Stockholm County, representing a variety of socio-economic conditions. A small number of preschools had defined pedagogical profiles such as ‘Reggio Emilia-inspired’, and different outdoors profiles (i.e. spending a substantial part of the day outdoors). However, the majority of preschools did not have any specific pedagogical profile other than following the national Swedish curriculum [32]. Eight out of the 17 preschools were publicly funded independent preschools, while the remaining nine preschools were municipal preschools.

Participating pre-school staff

Participating pre-school staff had different educational backgrounds, with a minority (34%) having a formal pre-school teacher education. More than half of the participating pre-school staff had been working in preschools for more than 5 years before the onset of the study, and the vast majority had known the child diagnosed with ASD for at least 6 months or longer.

Participating habilitation supervisors

EIBI supervisors employed by the habilitation centres had various professional backgrounds such as psychologists (n = 4), speech-language pathologists (n = 10), and special educators (n = 3). However, a majority of the supervisors had very limited experience of supervising EIBI programs (less than two previously supervised comprehensive programs) before the onset of the study.

APERS-P-SE

The APERS-P-SE scale [52] is designed to assess the pre-school learning environment of children on the autism spectrum in Swedish preschools. It can be used as a baseline measure of quality, to guide and promote improvements in the learning environment, and as a follow-up measure of potential change in the quality of the learning environment. It is a translated and culturally adapted Swedish version of the original APERS [38]. It consists of 56 items scored on a 5-point Likert-scale ranging from poor quality (1) to high quality (5) of different aspects of the learning environment, grouped into 10 main domains. The domains are: Learning Environments, Positive Learning Climate, Assessment and Individual Education Plan (IEP) Development, Curriculum and Instruction, Communication, Social Competence, Personal Independence and Competence, Functional Behaviour, Family Involvement, and Teaming. All domains target different aspects of the learning environment for children on the autism spectrum in their pre-school program. For example, ‘communication’ targets how the learning environment is arranged to support children’s spontaneous communication, responsiveness to children’s communication, and the use of augmentative and alternative communication. As another example, ‘social competence’ addresses how the pre-school staff arrange opportunities for children on the autism spectrum to improve social competence, as well as teaching and modelling, and the potential use of peer-buddies.

The scale is comprehensive and takes about 6–7 h to administer. When using APERS-P-SE, the rater gathers information about the pre-school from direct observations of activities in which the child on the autism spectrum participates, reviews of the child’s pre-school action plans, and interviews with pre-school staff and parents. All this information is then combined and used to rate the scale’s 56 items. The ratings yield a global score for the overall pre-school learning environment quality, but also scores for individual items, subdomains and domains. The scale has previously been assessed as having a high level of content validity, across individual items, domains, as well as for the whole instrument [52]. However, other psychometric properties such as prognostic validity and test-retest validity are yet to be empirically evaluated. It can be used in both self-contained and inclusive settings, by professionals with expertise in ASD who have undergone training in using the instrument. The training includes a theoretical introduction, preparatory workshops, pre-school observations, and subsequent ratings with pertaining written feedback, supported by an experienced APERS user and trainer. The study’s authors can provide a copy of the adapted Swedish version, and information about training resources, under copyright and authorization by the original U.S. authors.

In the current study, all APERS-P-SE ratings were done by the first author H.B. For 21% of the APERS-P-SE assessments, a master’s student in psychology trained in using APERS-P-SE did independent assessments, in order to assess the inter-rater reliability of the instrument. These ratings were however not used to assess the quality of the learning environment for children on the autism spectrum in the current study. Agreement between two raters was calculated using a two-way, absolute agreement, single-measure intraclass correlation (ICC; [56,57]). The resulting ICC over a total of 370 rated items was r_ICC = 0.77 (p < 0.001, 95% CI [0.72, 0.81]), indicating good reliability [58].

Procedure

EIBI only and EIBI/APERS-P-SE preschools

In this study, the APERS-P-SE was administered to assess strengths and weaknesses of the pre-school learning environment, and corresponding results were provided to pre-school staff in all preschools prior to group allocation. Thus, all participating preschools received individualized feedback based on their baseline APERS-P-SE ratings, both in written and oral form, on areas of strengths and weaker areas targeting to improve the pre-school learning environment for the children on the autism spectrum. However, in the EIBI only preschools no support (i.e. in-service training and on-site coaching) was provided in how to transform APERS-P-SE results to best practice.

Both EIBI only preschools and EIBI/APERS-P-SE preschools received EIBI supervision as usual over the 8 months long APERS-P-SE-based intervention, in accordance with the previously described national Swedish habilitation guidelines [27]. The EIBI program was implemented in the children’s pre-school and home settings. In both groups, EIBI supervision was provided weekly, or sometimes bi-weekly at the habilitation centre, with occasional visits to the pre-school where supervision was provided to the children’s paraprofessional and parents. The EIBI supervision was mainly focussed on children’s learning, development, and functioning. As shown in Table 1, the EIBI only group received an average of 17.25 supervision sessions during the 8 months, while the EIBI/APERS-P-SE preschools group received 18.22.

Table 1. Median of the total number of EIBI programs supervised by habilitation supervisors before study participation (supervisor experience). Means and standard deviation of total number of supervision sessions during the study (EIBI supervision sessions).

	Supervisors’ experience	EIBI supervision sessions	EIBI total implementation hours	EIBI pre-school implementation hours
EIBI only (n = 8)	1.5	17.25 (3.85)	17.50 (3.54)	12.13 (2.96)
EIBI/APERS-P-SE preschools (n = 9)	1	18.22 (3.23)	19.06 (6.47)	12.33 (4.26)

Means and standard deviation of weekly EIBI implementation hours reported for home- and pre-school setting combined (EIBI total implementation hours), and specifically for the pre-school setting (EIBI pre-school implementation hours) during the course of the study.

In both of the study’s group contingencies, participating children were in pre-school groups with 2 to 5-year-old children, with an average of about 18 children in each group. In general, the staff-to-child ratio was a bit less than 4 children per pre-school staff. All children on the autism spectrum who received EIBI had an assigned paraprofessional.

In both the EIBI only group and the EIBI/APERS-P-SE preschools, less time (18.28 h/week) than the recommended 25 h of weekly intervention was reported over all participating preschools and children, with the majority of intervention (12.23 h/week) taking part in the pre-school setting. During the 8 months of intervention, the habilitation centre staff provided an average number of 17.7 supervision sessions for the participating child. There were no significant differences between the EIBI only group and the EIBI/APERS-P-SE preschools regarding reported EIBI training hours, number of supervision sessions, or supervisors' experience in supervising EIBI programs (Table 1).

EIBI/APERS-P-SE preschools

In the EIBI/APERS-P-SE preschools, results from initial APERS-P-SE assessments were used as the foundation of building a learning environment-focussed intervention. In-service training to pre-school staff (typically, the assigned paraprofessional and a pre-school teacher), and on-site coaching was provided. The training was based on evidence-based practices in ASD, in order to promote further development of strengths (e.g. teaming, positive learning climate) and reduce weaknesses (e.g. curriculum and instruction) of the pre-school context, and support children’s goal attainment. This extra support using the APERS-P-SE was provided by the habilitation supervisors who already supervised the implementation of regular standard EIBI intervention to the child on the autism spectrum. Serving this purpose, all habilitation supervisors to the study’s EIBI/APERS-P-SE preschools group had, at the onset of the study, received a one-day workshop in the basic tenets of using of APERS-P-SE including: coaching procedures, selected evidence-based autism practices, inclusion and children’s engagement. The habilitation supervisors also received a half-day booster work-shop on inclusive evidence-based practices (i.e. peer-mediation and naturalistic interventions), to enhance children’s engagement and inclusion. Furthermore, these habilitation supervisors also received the APERS-P-SE feedback that had previously been provided to the APERS-P-SE pre-school, to inform and guide their subsequent on-site coaching and EIBI supervision. The habilitation supervisors in the EIBI/APERS-P-SE preschools group also had access to continuous support from the research team during the course of the study in how to best translate the results of the APERS-P-SE into best practice.

In the EIBI/APERS-P-SE preschools, an assigned paraprofessional and a pre-school teacher from each pre-school participated in a one-day in-service training session including: selected evidence-based autism practices, children’s inclusion and engagement, quality features of pre-school learning environment for children on the autism spectrum, and goal scaling. Subsequently, habilitation supervisors made monthly, one-hour long visits to the pre-school, where coaching sessions were performed with one or two of the pre-school staff, during a period of 8 months. On-site coaching was occasionally supplemented with communication via telephone, or in association with regular EIBI supervision meetings at the habilitation centre. The coaching sessions included identifying, formulating, and scaling goals based on initial APERS-P-SE feedback with the use of a goal attainment scale (GAS; [59]), selecting relevant evidence-based practices and then provide support in implementing these to promote children’s goal attainment, and to improve other environmental quality features such as the physical set-up of the environment and the use of visual supports. Specifically, the coaching sessions followed the guidelines by Kucharczyk et al. [48], including a pre-observational meeting, observations and post-observation meetings, and the use of implementation checklists. Examples of children’s goals in the pre-school setting were for children to actively participate during circle time, to independently eat lunch among peers, to independently initiate and engage in role-play with peers, and to independently choose and initiate an activity with the support of visual cues. Examples of evidence-based practices used were reinforcement, prompting, visual supports, naturalistic intervention, and peer-mediated instruction and intervention [23]. The habilitation supervisors were asked to complete a questionnaire during the study that described their latest on-site coaching session at random time points. The questionnaire included five statements rated on a six-point Likert-scale (0 = I do not agree at all, 5 = I completely agree).

Midway through the intervention, pre-school staff in the EIBI/APERS-P-SE preschools were invited to a half-day seminar to follow up on their work in their preschools, to reflect on their use of evidence-based practices and children’s progress, in collaboration with members of the research team and the participating staff from other EIBI/APERS-P-SE preschools. Thus, supervision as usual was supplemented with continuous in-service training to a minimum of two pre-school staff, and on-site coaching was provided monthly (without the children’s parents) to two of the pre-school staff. For an overview of the differences between the two groups, see Table 2.

Table 2. Overview of usual practice, and the addition of the intervention provided.

EIBI Only	EIBI/APERS-P-SE
APERS-P-SE feedback	APERS-P-SE feedback
EIBI supervision	EIBI supervision
	In-service training based on APERS-P-SE assessments concerning ASD, improving learning environment, goal scaling, and evidence-based practices to support learning and participation
	On-site coaching based on APERS-P-SE assessments to improve learning environment and support children’s learning and participation

In contrast to the child-focussed EIBI program, where all instructions typically are provided to the pre-school’s paraprofessional from the habilitation supervisor, in the EIBI/APERS-P-SE preschools the specific goals and aims for the on-site coaching were not provided unidirectional by the habilitation supervisor but rather developed in collaboration between pre-school staff and supervisors based on the initial APERS-P-SE assessment, with the help of GAS. Furthermore, instead of mainly focussing on individual skill acquisition, the additional on-site coaching focussed more on improving the learning environment, supporting inclusion and participation for the child on the autism spectrum, promoting interaction with peers, and creating naturalistic learning occasions. For example, one focus could be to support a child’s previously mentioned participation and learning in circle time by: (i) adapting learning environment, (ii) creating physical delineations, (iii) changing contents of the activity to match interests of the child, (iv) adapting child group size as well as duration of activity, and (v) use peer-mediated instruction, rather than providing adult-initiated one-to-one instruction in an adjacent room during circle time. Other foci could be to implement visual supports in the learning environment, to prepare for and support transitions between activities (using a visual schedule), support independency on tasks such as hand washing, independently putting on clothes before going out to the playground (using activity schedules), and in engaging the child in independently choosing a preferred activity (using a choice board).

Outcome measures

Our primary outcome measure was change in APERS-P-SE total scores from pre- to post- measure. In face of a study with limited sample size, this was chosen as we expected the instrument to be the most sensitive to the intended outcomes of improving quality in the learning environment for children on the autism spectrum in Swedish pre-school settings, in the study’s experimental group. Furthermore, the instrument is based on expert ratings, and it is closely connected to the intervention itself.

Secondary exploratory outcome measures were APERS-P-SE domain scores, and pre-school child and pre-school staff behaviours and experiences (see Table 3). The child outcomes included Clinical Global Impression-Autism (CGI-AUT [63]), Vineland Adaptive Behaviour Scales – Second Edition (Vineland-II) – Teacher ratings [60], Children’s Engagement Questionnaire (CEQ [43,61,62,70]), and for the children in the experimental group also the Goal Attainment Scale (GAS [59]).

Table 3. Overview of secondary outcome measures.

Instrument	Type of scale	Description
Pre-school learning environment
APERS-P-SE	Likert scale (1–5) where 1 is ‘poorest quality’ and 5 is ‘highest quality’, comprising 56 items.	A scale designed to assess the pre-school learning environment for children on the autism spectrum in Sweden. The scale is translated and cross-culturally validated, showing high levels of content validity (CVIs = 0.97 − 1.0).
Children’s outcomes
Vineland Adaptive Behaviour Scales – Second Edition (Vineland-II) – Teacher ratings	Likert scale (0–2) where 0 is ‘never occurs’, and 2 is ‘occurs often’, comprising 177 items in the current study.	A scale designed to assess children’s adaptive behaviours within various domains such as Independence and Communication skills. The scale is translated to Swedish and cross-culturally validated, but raw data are not normalised for the teacher ratings. In the current study all domains are rated except for the motor skills domain
Children's Engagement Questionnaire (CEQ)	Likert scale (1–4), where 1 is ‘not at all typical’, and 4 is ‘very typical’, comprising 29 items.	A scale designed to assess children's level of engagement in various situations. It has previously been translated and validated in Sweden [61,62] showing high internal consistency (α = 0.86–0.93). In the current study one version is rated by pre-school staff about the level of children’s engagement in the pre-school setting, and another version is rated by parents about the level of children’s engagement in the same situations outside of the pre-school setting (e.g. at home).
Clinical Global Impression-Autism (CGI-AUT)	Likert scale (1–7), where 1 is ‘no autism symptoms’, and 7 is ‘maximum symptom severity’, comprising 1 item.	An instrument designed to assess the symptom severity of pre-school age children with ASD. It has previously been validated in a Swedish context [63] with moderate intraclass correlations found among 16 experts (rit.= 0.59 − 0.72).
Goal Attainment Scale (GAS)	Ordinal scale (0–4), where 0 is baseline measure, 1 is ‘meets initial objective’ 2 is ‘meets secondary objective’, 3 is ‘meets annual goal’, and 4 is ‘exceeds annual goal’.	A scale originally designed to be used for treatment of psychiatric disorders which is now used in a broad range of areas. The scale has been translated to Swedish, and in the current study’s EIBI/APERS-P-SE preschools three goals for each child were decided upon in collaboration between participating pre-school staff, habilitation supervisors, and children’s parents.
Pre-school staff’s outcomes
Ethics, Values and Allegiance (EVA)	Likert scale (0–4) where 0 is ‘I do not agree’ and 4 is ‘I completely agree’, comprising 11 items in the current study.	A scale designed to assess allegiance towards EIBI. Cronbach’s alpha for the internal consistency of the scale was found to be high (α = 0.90) in previous research using the scale [45,47].
Evidence Based Practices Questionnaire (EBP-Q)	Likert scale (0–3) where 0 is ‘not all all/never/not at all’, and 3 is ‘very well/very often/completely’, comprising 15 items.	A scale designed for pre-school staff to self-assess knowledge about, use of, and implementation skills of 15 evidence-based practices for children with ASD. The practices were chosen from the 27 evidences-based practices identified by Odom, Cox & Brock [36], and Hall [64], because of their relevance for working with pre-school age children in pre-school settings. It is translated from the original English version, and no psychometric properties of the instrument are currently established.
Autism Knowledge Questionnaire (AKQ)	A questionnaire comprising 18 statements which can be answered ‘true’, ‘false’, or ‘I don’t know’.	A questionnaire designed to assess pre-school staff’s knowledge about ASD. Example of statements include ‘All children with ASD are uninterested in playing with other children’. The questionnaire is a revised edition of an earlier version used in a study on knowledge about ASD among grandparents to children with ASD receiving education at a habilitation centre [65].  No psychometric properties of the instrument are currently established.
Self-Efficacy, Stress and Satisfaction Questionnaire (SESSQ)	Likert scale (0–5) where 0 is ‘I do not agree at all/no stress/not at all/I do not agree at all’ and 5 is ‘I completely agree/extreme stress/very much/completely/I completely agree’, comprising a total of 34 items/statements.	A scale designed to assess self-efficacy for working with children in pre-school, self-efficacy for working with children with ASD, and perceived levels of work-related stress and satisfaction, among pre-school teachers, paraprofessionals, and other pre-school staff. It contains the translated short, 12-item version of the Ohio State Teacher Efficacy Scale (OSTES; [66]), a translated and abbreviated version (14 items) of the Autism Self-Efficacy Scale for Teachers (ASSET; [67]), translated version of the three item Job Satisfaction Survey [68], as well as five translated items about stress from the Teacher Stress Inventory [69]. ASSET has in its original English version shown high internal consistency (α = 0.96), and items loading on one dominant factor, the internal consistency of modified version used in the current study was (α = 0.89). OSTES has in its original, short form, displayed a high level of internal consistency (α = 0.90), as well as sound factor loading, and satisfying construct validity. Internal consistency found in the current study was (α = 0.94).
Social Validity Scale (SVS)	Likert scale (0–5), where 0 is ‘I don’t not agree at all’ and 5 is ‘I completely agree’, comprising 5 statements/items.	A scale developed for the current study which is designed for pre-school staff in the EIBI/APERS-P-SE preschools to assess the relevance and acceptability of the goals targeted within the study, of the methods used, of the APERS-P-SE feedback provided, the contents of the in-service training and coaching, and the usefulness of the knowledge and skills acquired when working with other children in the future. One example of items is ‘The procedure and methods we have used have been helpful for working with children with ASD in the pre-school’ No psychometric properties of the instrument are currently established.

The pre-school staff outcomes included Ethics, Values and Allegiance (EVA [47]), Evidence Based Practices Questionnaire (EBP-Q [36,64]), Autism Knowledge Questionnaire (AKQ), Self-Efficacy, Stress and Satisfaction Questionnaire (SESSQ [66–69]), and for pre-school staff in the EIBI/APERS-P-SE preschools also Social Validity Scale (SVS).

Data analysis

All analyses were conducted using an intention-to-treat approach [71], including all preschools, children, and pre-school staff. Prior to intervention, student’s t tests [53] were used to determine if the two groups differed at baseline. To ensure that identified group differences did not confound with the APERS-P-SE-based intervention effects, we examined if these variables were associated with our primary and secondary outcome measures using Pearson correlation coefficient [53]. Based on these analyses, the influence of EVA was covaried for in the analysis of APERS-P-SE total score, and the influence of EVA, SESSQ-Satisfaction, and SESSQ-Stress were covaried for in the analyses of APERS-P-SE domain scores.

To investigate changes in APERS-P-SE total score between pre- and post- measures, we used a two-factor mixed-design analysis of covariance (ANCOVA; [53]) with APERS-P-SE total score (i.e. overall quality of learning environment for children on the autism spectrum) as the dependent variable, using an alpha level of p = 0.05. To address changes in APERS-P-SE domain scores between pre- and post- measures, we used a two-factor mixed-design multivariate analysis of covariance (MANCOVA; [72]) with APERS-P-SE domain scores as the dependent variable, and time and group condition as independent variables, using a Bonferroni corrected alpha level of p = 0.005 for all univariate outcomes.

Additionally, to address changes in our exploratory secondary outcomes on child level, we used a two-factor mixed-design multivariate analysis of variance (MANOVA; [53]) over all dependent variables (Vineland-II, CGI-AUT, CEQ-Pre-school, CEQ-Parent), with time and group condition as independent variables.

To address changes in our exploratory secondary outcomes on pre-school staff level, we used a two-factor mixed-design MANOVA over all dependent variables (EVA, EBP-Q, AKQ, SESSQ), with time and group condition as independent variables. For the MANOVAs and the MANCOVA conducted, not all statistical assumptions were met. First, most but not all dependent variables were normally distributed. Secondly, in the MANCOVA of APERS domains, the number of dependent variables (10) exceeded the sample sizes (9 and 8). However, F-tests have proven robust to violations of non-normality in data [73,74]. SPSS Statistics 26 [75] was used to analyse all data.

Additionally, paired t-tests were used to explore within-group changes for all outcomes [53], with Bonferroni corrected alpha levels of p = 0.005 for the APERS-P-SE domains. All effect sizes were computed with partial eta squared [53].

Results

Baseline group differences

Prior to intervention the EIBI only group pre-school staff reported higher work-related stress (SESSQ-Stress; [69], p = 0.004), lower work-related satisfaction (SESSQ-Satisfaction; [68], p = 0.04), and lower allegiance towards EIBI (EVA; [47], p = 0.007) compared to the EIBI/APERS-P-SE group.

Association between identified group differences and intervention outcomes

EVA had a moderate positive correlation with APERS-P-SE total score at follow-up (r = 0.49; p = 0.045). Furthermore, for our secondary outcome measures, we found that EVA had moderate positive correlations with the APERS-P-SE domains Learning Environments (r = 0.49; p = 0.049), Personal Independence and Competence (r = 0.52; p = 0.035), Functional Behaviour (r = 0.49; p = 0.048), and Family Involvement (r = 0.49; p = 0.048) at follow-up. We also found that SESSQ-Satisfaction had moderate to strong positive correlations with the APERS-P-SE domains Learning Environments (r = 0.54; p = 0.024), Social Competence (r = 0.70; p = 0.002), and Personal Independence and Competence (r = 0.49; p = 0.046) at follow-up. Finally, we found that SESSQ-Stress had negative moderate correlations with the APERS-P-SE domains Learning Environments (r = −0.49; p = 0.048), Assessment and IEP Development (r = −0.55; p = 0.024), and Personal Independence and Competence (r = 0.50; p = 0.040) at follow-up.

Pre-school learning environment (APERS-P-SE)

The group by time interaction effect (i.e. pre to post intervention) indicated a significantly larger change in pre-school learning environment quality total score (APERS-P-SE) for the EIBI/APERS-P-SE preschools (F = 12.42; p = 0.003; η² = 0.47) compared to the EIBI only group, indicating a large effect size in favour of the EIBI/APERS-P-SE preschools (Table 4).

Table 4. APERS-P-SE (primary outcome measure) total score (calculated with two-factor mixed-design ANCOVA) and domain scores (calculated with two-factor mixed-design MANCOVA) at baseline and post-intervention in the control group (EIBI only) and experimental (EIBI/APERS-P-SE preschools).

APERS-P-SE total score and domain scores (min – max)	Mean (SD)		Group difference in change score	p value	Effect size (Partial η^2)*
	EIBI only (n = 8)	EIBI/APERS-P-SE preschools (n = 9)
APERS-P-SE total (1–5)
Baseline	2.87 (0.37)	2.82 (0.30)
Post-intervention	2.94 (0.25)	3.73 (0.65)**
Change score	0.07	0.91	0.84	0.003	0.47
Domain Learning Environments (1–5)
Baseline	3.18 (0.42)	2.81 (0.53)
Post-intervention	2.94 (0.34)	3.74 (0.70)**
Change score	–0.24	0.93	1.17	0.002	0.51
Domain Positive Learning Climate (1–5)
Baseline	3.39 (0.82)	3.72 (0.38)
Post-intervention	3.66 (0.38)	4.31 (0.56)
Change score	0.27	0.59	0.32	0.188	0.14
Domain Assessment and IEP Development (1–5)
Baseline	3.82 (0.48)	3.50 (0.66)
Post-intervention	3.78 (0.25)	4.16 (0.46)
Change score	–0.04	0.66	0.70	0.192	0.14
Domain Curriculum and Instructions (1–5)
Baseline	2.23 (0.48)	2.31 (0.48)
Post-intervention	2.48 (0.32)	3.54 (1.07)**
Change score	0.25	1.23	0.98	0.094	0.22
Domain Communication (1–5)
Baseline	2.19 (0.65)	2.47 (0.64)
Post-intervention	2.49 (0.53)	3.43 (1.22)
Change score	0.30	0.96	0.66	0.88	0.02
Domain Social Competence (1–5)
Baseline	3.06 (0.66)	3.42 (0.67)
Post-intervention	3.16 (0.79)	4.39 (0.47)**
Change score	0.10	0.97	0.87	0.24	0.11
Domain Personal Independence (1–5)
Baseline	2.50 (0.53)	1.93 (0.52)
Post-intervention	2.33 (0.25)	3.11 (0.55)**
Change score	–0.17	1.18	1.35	0.004	0.51
Domain Functional Behaviour (1–5)
Baseline	2.43 (1.03)	1.98 (0.60)
Post-intervention	2.31 (0.44)	3.12 (1.10)**
Change score	–0.12	1.14	1.26	0.034	0.32
Domain Family Involvement (1–5)
Baseline	4.67 (0.44)	4.81 (0.37)
Post-intervention	4.50 (0.36)	4.82 (0.24)
Change score	–0.17	0.01	0.18	0.70	0.01
Domain Teaming (1–5)
Baseline	2.83 (0.53)	2.80 (0.39)
Post-intervention	2.82 (0.52)	3.42 (0.69)
Change score	–0.01	0.62	0.63	0.962	0.01

Within-group differences calculated with paired t-tests.

*Positive effect sizes favour EIBI/APERS-P-SE preschools.

**Within-group change was statistically significant (p < 0.005) after false discovery rate correction for multiple comparisons.

For the APERS-P-SE domain scores, there was no significant multivariate group by time interaction effect. However, univariate results indicated significant group by time interaction effects for the domains Learning Environments (F = 14.74; p = 0.002; η² = 0.55) and Personal Independence and Competence (F = 12.30; p = 0.004; η² = 0.51), in favour of the EIBI/APERS-P-SE Preschools.

Within-group changes indicated significant improvements in APERS-P-SE total score in the EIBI/APERS-P-SE preschools (t [8] = −5.82; p = 0.001), as well as significant improvements for the domains Learning Environments (t [8] = −6.42; p = 0.001), Curriculum and Instruction (t [8] = −4.27; p = 0.003), Social Competence (t [8] = −4.63; p = 0.002), Personal Independence and Competence (t [8] = −6.75; p = 0.001), and Functional Behaviour (t [8] = −4.13; p = 0.003). No significant within-group changes were identified for the EIBI only group.

Child outcome measures

On child level, there was no significant multivariate interaction effect of group condition and time, reflecting that no significant univariate interaction effects were found for children’s adaptive behaviours (Vineland-II), global clinical impression (CGI-AUT), or children’s engagement (CEQ-pre-school/parent; Table 5). Within-group changes indicated significant improvement for children’s adaptive behaviours (Vineland-II) in the EIBI/APERS-P-SE preschools (t [8] = −2.48; p = 0.038). No significant within-group changes were identified for the EIBI only group. For children’s goal attainment in the EIBI/APERS-P-SE preschools, progress was made on 89% of targeted goals, and 63% of the goals were either met or exceeded. Mean level of goal attainment across all children in the experimental group was 2.59 (SD = 1.42).

Table 5. Child and pre-school staff secondary outcome measures (calculated with two-factor mixed-design MANOVA) at baseline and post-intervention in the control group (EIBI only) and experimental (EIBI/APERS-P-SE preschools).

Measure (min – max)	Mean (SD)	EIBI/APERS-P-SE preschools	Group difference in change score	p	Effect size (Partial η^2)^a
	EIBI only	EIBI/APERS-P-SE preschools
Children’s outcomes	(n = 8)	(n = 9)
Vineland-II (0–354)
Baseline	44.88 (30.88)	55.33 (39.88)
Post-intervention	65.50 (43.50)	87.22 (57.02)*
Change score	20.62	31.89	11.27	0.56	0.03
CGI-AUT (1–7)
Baseline	5.88 (0.97)	5.11 (0.78)
Post-intervention	5.38 (0.99)	4.87 (0.97)
Change score	–0.5	–0.24	–0.26	0.759	–0.006
CEQ-Pre-school (29–116)
Baseline	53.38 (14.01)	62.00 (18.55)
Post-intervention	57.13 (16.28)	67.67 (20.62)
Change score	3.75	5.67	1.92	0.808	0.004
CEQ-Parent (29–116)
Baseline	67.63 (15.14)	69.13 (13.37)
Post-intervention	71.75 (18.71)	76.87 (21.18)
Change score	4.12	7.74	3.62	0.544	0.027
GAS (0–4)
Baseline	–	0.00 (0.00)
Post-intervention	–	2.59 (1.42)		–	–
Pre-school teachers’ outcomes	(n = 16)	(n = 19)
EBP-Q (0–3)
Baseline	1.68 (0.80)	1.66 (0.55)
Post-intervention	1.82 (0.52)	1.92 (0.61)*
Change score	0.14	0.26	0.12	0.45	0.039
EVA (0–44)
Baseline	28.50 (6.41)	34.79 (6.40)
Post-intervention	28.63 (12.49)	32.53 (9.14)
Change score	0.13	–2.26	−2.39	0.493	−0.032
SESSQ-S-E (0–5)
Baseline	3.73 (0.55)	4.08 (0.71)
Post-intervention	3.63 (4.8)	4.08 (0.62)
Change score	–0.10	0	–0.10	0.55	0.024
SESSQ-S-E-ASD (0–5)
Baseline	3.49 (0.68)	3.72 (0.48)
Post-intervention	3.53 (0.67)	3.92 (0.74)
Change score	0.04	0.20	0.16	0.487	0.033
SESSQ-Stress (0–5)
Baseline	3.39 (0.70)	2.42 (1.07)
Post-intervention	3.31 (0.70)	2.51 (0.97)
Change score	–0.08	0.09	0.17	0.636	−0.015
SESSQ-Satisfaction (0–5)
Baseline	4.48 (0.57)	4.84 (0.37)
Post-intervention	4.40 (0.70)	4.73 (0.54)
Change score	–0.08	–0.11	−0.03	0.882	−0.002
AKQ (0–18)
Baseline	13.94 (3.44)	14.26 (2.96)
Post-intervention	14.25 (3.27)	15.37 (3.13)
Change score	0.31	1.11	0.80	0.466	0.036
SVS (0–5)
Baseline	–	–
Post-intervention	–	4.74 (0.11)		–	–

Within-group differences calculated with paired t-tests.

Note. Vineland-II. Vineland Adaptive Behaviour Scales – second edition – teacher ratings, CGI-AUT. Clinical Global Impression – autism, CEQ-Pre-school. Children's Engagement Questionnaire – pre-school version, CEQ-Parent. Children’s Engagement Questionnaire – parent version, GAS. Goal Attainment Scale, EBP-Q. Evidence-Based Practices Questionnaire, EVA. Ethics, Values and Allegiance, SESSQ-S-E. Self-Efficacy, Stress and Satisfaction Questionnaire – self efficacy for working with children in general, SESSQ-S-E-ASD. Self-efficacy, Stress and Satisfaction Questionnaire – Self Efficacy for Working with Children with ASD, SESSQ-Stress. Self-Efficacy, Stress and Satisfaction Questionnaire – Work Related Stress, SESSQ-Satisfaction. Self-Efficacy, Stress and Satisfaction Questionnaire – Work Related Satisfaction, AKQ. Autism Knowledge Questionnaire, SVS. Social Validity Scale, ^aPositive effect sizes favour EIBI/APERS-P-SE preschools.

*Within-group change was statistically significant (p < 0.05).

Pre-school staff outcome measures

On pre-school staff level, there was no significant multivariate interaction effect of group condition and time, reflecting that no significant univariate interaction effects were found for pre-school staff’s allegiance towards EIBI (EVA), knowledge about, use of, and implementation of evidence-based practices (EBP-Q), autism knowledge (AKQ), and self-efficacy, work-related stress and satisfaction (SESSQ; Table 5). Within-group changes indicated significant improvement for knowledge about, use of, and implementation of evidence-based practices (EBP-Q) in the EIBI/APERS-P-SE preschools (t [18] = −3.61; p = 0.002). No significant within-group changes were identified for the EIBI only group. Mean ratings on social validity (SVS) completed by the APERS-P-SE pre-school group were 4.8 (SD = 0.42) for in-service training relevance, 4.6 (SD = 0.52) for APERS-feedback relevance, 4.7 (SD = 0.48) for APERS-goals relevance, 4.9 (SD = 0.32) for helpfulness of procedures and methods used, and 4.7 (SD = 0.67) for usefulness of acquired teaching skills with other children.

Treatment fidelity

Dosage

On average, a total of 6.78 (SD = 1.48) coaching sessions were provided for each pre-school in the EIBI/APERS-P-SE preschools group, with a range between 5 and 8 sessions. Sometimes, to compensate for cancelled coaching sessions due to illness, subsequent coaching sessions were extended in time. On average, a total of 7.19 h of coaching (SD = 0.97) was provided to each pre-school. All preschools in the EIBI/APERS-P-SE pre-school group attended the preparatory in-service training, and the half-day follow-up seminar. All pre-school staff in the EIBI/APERS-P-SE group received APERS-P-SE feedback as the foundation of improving the learning environment for children on the autism spectrum.

Integrity

In total, the questionnaires that described habilitation supervisors’ latest on-site coaching session were completed following 24 coaching sessions, distributed over all supervisors, at different time points during the study. The average ratings were 4.00 (SD = 1.17) for ‘I followed the coaching procedure’ (pre-observation conference, observation, and post-observation conference), 4.65 (SD = 0.92) for ‘I perceived the pre-school staff as engaged in the coaching session’, 4.25 (SD = 1.07) for ‘The coaching session proceeded without any major disruptions or incidents’, 4.63 (SD = 0.82) for ‘The coaching session focussed on one or more APERS-goals’, and finally 4.58 (SD = 0.72) for ‘I perceived the coaching session as helpful for the pre-school staff in their work’.

Attrition and adverse effects

In total, 12 of the 35 pre-school staff did not remain in their pre-school group by the time of post-data collection. Of those, 9 pertained to the study’s EIBI/APERS-P-SE preschools, and 3 to the study’s EIBI only group. Reasons for attrition were extensive sick leave, change of workplace, and relocation to another pre-school within the pre-school unit. No reason for attrition appeared to be related to the study or intervention. Staff who left their positions were replaced in the pre-school. However, replaced staff did not receive additional in-service training within the framework of the study with the exception of participating in the half-day booster seminar, and receiving on-site coaching if possible, depending on when the previous staff had left during the course of the study. There was no child and pre-school attrition. No adverse events were reported during the study.

Discussion

Our results showed that applying the learning environment-focussed APERS-P-SE in addition to EIBI significantly increased the quality of the learning environment for children on the autism spectrum in the target preschools compared to preschools using EIBI only. Specifically, significant improvements between the groups were found in how the pre-school staff arranged the set-up of their physical environment and prepared for transitions between settings (APERS-P-SE domain Learning Environments), and in how the pre-school staff supported the personal independence (APERS-P-SE domain Personal Independence and Competence) of the child on the autism spectrum in his or her daily activities. This is to the authors’ best knowledge the first time children on the autism spectrum’s pre-school learning environment has been evaluated and improved within a research study in a European context.

In the current study, intervention was focussed on contextual aspects that support function [51], rather than more traditional client-factors related to a biomedical approach [76]. Within the framework of the current study, and the improvements made in the learning environment for children on the autism spectrum, one may conclude that changes were made in second-level ICF categories from the comprehensive ICF core set for individuals on the autism spectrum. For example, training staff, making accommodations in the environment in regards to physical delineations, using visual supports, preparing for transitions, arranging environments to promote children’s communication, adapting group size to limit visual and auditory stimuli, and implementing peer-mediated intervention, connects to ICF categories such as products and technology for communication (e125), products and technology for education (e130), education and training services, systems, and policies (e585), sound (e250), friends (e320) and acquaintances, peers, colleagues, neighbours, and community members (e325) [51]. These finding maps onto other research examining the effect of contextual functional circumstances of ASD, such as accommodations in the learning environment to support school performance, functioning, and quality of life among children on the autism spectrum [40], and the association between quality features of EIBI instruction and intervention outcomes [77].

Additionally, although we could not contrast children’s personal goal attainment between the EIBI only group and the EIBI/APERS-P-SE preschools group, as GAS was only used in the EIBI/APERS-P-SE preschools, the children for which goals were formulated, did substantially improve (mean goal attainment 2.59). These GAS improvements are consistent with the experimental condition in a similar study conducted by Sam et al. [78], where service as usual condition improved less. It should however be noted that this was a secondary and exploratory outcome in our study, not intended for between-group comparisons. Nevertheless, it does indicate that children in the EIBI/APERS-P-SE preschools either met or exceeded their annual goals, indicating that they made progress in important areas of development such as independence, communication, and social interaction.

Furthermore, participating pre-school staff provided very high overall ratings for the social validity of the intervention (SVS). This might indicate that pre-school staff, who may not always feel competent to support children on the autism spectrum [79,80], found the APERS-P-SE-based intervention helpful.

However, during the study period, changes in the learning environment (i.e. as measured by APERS-P-SE) were not accompanied by improvements on any of the child and pre-school staff measures compared to the EIBI only group. This suggest that the group difference in APERS-P-SE improvements over time did not directly translate or generalize to broader improvements on a child or pre-school staff level. Thus, it is not possible to conclude that the environmental changes in second-level ICF categories influenced individual functioning, compared to the EIBI group.

Overall, our results are consistent with the findings of Odom, Cox, and Brock [36], and the findings of Sam et al. [78] where the evaluation of using the original APERS, outside of Europe, indicated that using the APERS to guide intervention was successful in promoting quality in learning environment for children on the autism spectrum in pre-school and school settings, as well as individual children’s goal attainment. Furthermore, within-group changes in the EIBI/APERS-P-SE preschools indicated that EIBI in combination with APERS-P-SE-based feedback, in-service training, and on-site coaching was successful in promoting children’s adaptive behaviours, and pre-school staff’s use, knowledge and implementation of evidence-based practices for children on the autism spectrum. However, it should be noted we did not identify any significant differences between the EIBI/APERS-P-SE preschools and the EIBI only group for our secondary outcome measures. Nevertheless, descriptively, small to medium effect sizes were found in favour of the EIBI/APERS-P-SE preschools group for some of the child and staff outcomes such as Vineland-II, CEQ-Parent, EBP-Q, SESSQ-SE, and SESSQ-SE-ASD, and in favour of the EIBI group, EVA, that might have reached significance in a larger sample.

The current study included four children who had received an additional neurodevelopmental diagnosis, as well as ASD. The study also included preschools with different pedagogical profiles, as well as both independent and municipal governed preschools. Although, by impression, we did not experience any obvious connection between additional diagnosis, pedagogical profile, independent or municipally governed pre-school, and outcomes, it cannot be ruled out that such relations may be identified in a larger scale study that allows meaningful subgroup or moderator analyses. Furthermore, EIBI was supervised by a heterogenous group of habilitation supervisors in regard to EIBI supervision experience, where many of the supervisors had a very limited experience. Also, EIBI was in general provided with less intensity (i.e. 18.28 h/week) than recommended by habilitation guidelines [27] (i.e. 25 h/week), although almost equivalent for the two group conditions. It can thus, in line with the conclusions by Eikeseth [13], be hypothesized that EIBI supervisors with limited experience and competence may have produced suboptimal outcomes for children receiving EIBI, in comparison to supervisors with more experience and competence. It could also be hypothesized that the lower intensity of EIBI implementation compared to the habilitation guidelines [27], as well as international guidelines and recommendations [14,17,81], may have limited the effect of EIBI in both groups in this study.

There are several other limitations to our study. First, although we did see a large and expected effect on the improvement of quality in learning environment for children on the autism spectrum in the EIBI/APERS-P-SE preschools group compared to the EIBI only group, we did not do any follow-up assessments after the termination of the intervention, meaning that is unclear if observed improvements in learning environment remained stable over time. Furthermore, we could not evaluate if improvements in the quality of the learning environment may have produced an effect on secondary outcomes at a later time point.

Another limitation is attrition, as 34% of all participating pre-school staff had left their pre-school group when post-data was collected. However, importantly, staff attrition was reportedly not related to the use of APERS-P-SE or the study, but high staff fluctuation is currently a quite usual phenomenon in Swedish pre-school context [82]. Still, the larger proportion of attrition in the experimental group compared to the control group may have limited the impact of the intervention, as well as the identified between-group effect sizes [83]. Unfortunately, this also led on some instances to different pre-school staff assessing the child outcomes on pre- and post-measures, which may have limited their accurateness and validity, as well as the identified between-group effect sizes on the child level outcome measures. Other limitations to the present study are a small sample size and lack of blinding. The small sample size increases the risk of type II-errors [84], thus limiting the statistical power for the variety of outcome measures used. Also, due to limited resources and the extensive training needed to conduct APERS-P-SE assessments, neither the participants nor the research team, including H.B who did the APERS-P-SE ratings, were blind to the participants’ group allocation. This is known to potentially influence and bias results from both external observations and self-rated scales [85], and may have influenced the ratings. Furthermore, only 2 out of 17 participating children were girls. This girl/boy ratio of 1:8.5 is not equal to the ASD population estimates of about 1:3 [86], and limits the generalizability of the study’s findings to boys. An additional limitation is that we do not have the exact details of how many times habilitation supervisors visited their preschools for EIBI supervision, in each of the study’s conditions. Although anecdotal data indicate that the two groups did not differ, we cannot evaluate how potential differences between the groups in frequency of pre-school visits may have influenced the study’s outcomes. Another limitation is that we did not collect data on the fidelity of the pre-school staffs’ use of evidence-based practices, neither in the EIBI/APERS-P-SE preschools group or the EIBI group. Thus, it is not possible to rule out that a low fidelity of implementing evidence-based practices may have limited the effect of the intervention on children’s development. We did collect self-reported data on implementation integrity of habilitation supervisors’ coaching sessions. However, one could argue that it is a methodological limitation to use self-reported measures instead of externally observed treatment integrity ratings.

There are also limitations in the research design of the current study. Because of the small sample size, proper stratification was not feasible to ensure full baseline equivalence. The group assignment was conducted using quasi randomization, which may have potentially introduced bias, increasing the risk for caveats like selection-regression threat [87,88]. Nonetheless, this was deemed necessary to ensure as equivalent groups as possible. Furthermore, masking was not applied in the randomization of matched pairs for the second and third cohort. Additionally, GAS was only used in the EIBI/APERS-P-SE preschools for goal scaling and to evaluate children’s goal attainment, as it was not feasible to assure pre-intervention equivalence of GAS descriptions between the two groups [89]. As a result, we could not compare the groups for goal attainment, or assess if the EIBI only preschools would have experienced improvements in this regard. Finally, some form of biasing effect by using the same tool for learning environment quality assessment and intervention purposes cannot be ruled out, as this could potentially increase the likelihood of observing improvements. However, we did not find a generic effect between the EIBI/APERS-P-SE preschools and the EIBI only group for all of the APERS-P-SE domains, but rather differential and specific effects on two domains (Learning Environments, and Personal Independence and Competence), which may indicate no such a generalized bias of using the APERS-P-SE for both assessments and intervention planning.

Future research should further explore the association between quality of learning environment for children on the autism spectrum and EIBI outcomes by using larger sample sizes, true randomization, blinded raters, equivalence tested GAS in both experimental and control preschools [89], and by doing long term follow-up ratings to assess both possible later emerging positive effects and maintenance of potential improvements. Further research should also investigate different stakeholder perceptions (i.e. pre-school staff, parents, supervisors, pre-school principals) of the APERS-P-SE implementation model. Further research could also explore the use of APERS-P-SE in addition to focussed interventions instead of EIBI, as for example EIBI may not always be accessible for all children on the autism spectrum. Finally, further research could also evaluate the effect of an APERS-P-SE-based model on other children in the pre-school, who may also benefit from this environmental approach.

In conclusion, almost all children in Sweden attend mainstream pre-school, including children on the autism spectrum who receive early intervention [25,26]. However, research to date suggests that there is a need to improve the quality of the learning environment in the Swedish preschools for children with special needs, including those on the autism spectrum receiving EIBI [35,44-46]. Therefore, the current study evaluated the effect of using the APERS-P-SE, an assessment aiming to improve the quality of learning environment, and derived in-service training and coaching, as an add-on and complement to child focussed EIBI in Swedish pre-school settings using a non-equivalent pretest-posttest group design [53]. Our results indicated that the use of APERS-P-SE did support the targeted pre-school staff in improving the overall quality of the learning environment for children on the autism spectrum receiving EIBI. However, these changes did not directly transfer to improvements for child- and pre-school staff outcomes, in comparison to the study’s control group.

Acknowledgements

The authors would like to thank the families, preschool staff, and habilitation staff, without whom this research would not have been possible.

Disclosure statement

The authors have no conflict of interest to report in relation to this article. Hampus Bejnö discloses that he receives royalties for a textbook chapter from Natur & Kultur. Lars Klintwall discloses that he receives royalties from a text book from Natur & Kultur. Ulrika Långh discloses that she receives royalties for a textbook chapter from Natur & Kultur. Samuel Odom discloses that the APERS assessment will be published by Brookes Publishing Company and will generate royalties. Sven Bölte discloses that he has in the last 3 years acted as an author, consultant or lecturer for Medice, and Roche. He receives royalties for text books and diagnostic tools from Huber/Hogrefe, Kohlhammer and UTB publishers.

Additional information

Funding

This work was supported by a grant from the Swedish Research Council [Vetenskapsrådet, 2015-01212].