Domains of quality in early childhood education and care: A scoping review of the extent and consistency of the literature

ABSTRACT

Early childhood education and care programmes provide opportunities to enhance children’s learning and development, especially when high-quality learning experiences and educator-child interactions are embedded within them. However, the quality of early childhood education programmes varies greatly. Quality in early childhood education and care is conceptualised in three domains: structural, process and system. Understanding how to drive quality improvements in early childhood education and care relies on clear, consistent evidence concerning each of these domains, however, the current literature is not comprehensive. This scoping review maps the extent and consistency of the research literature in each domain of quality to identify knowledge gaps and inform future research. Through a search of the peer-reviewed literature, 85 meta-analyses and systematic reviews meeting our inclusion criteria were identified. We found a wide variation in the number of included studies in each domain and sub-domain of quality. We found the greatest number of meta-analyses and systematic reviews related to programmes, interventions, and curricula (process quality) and professional development and support (structural quality). The literature included in this scoping review is heterogeneous and of varying methodological quality, with inconsistent or contradictory findings. The research is most consistent in relation to pedagogy, professional development and support, and programmes, interventions, and curricula (process quality) and learning environments (structural quality). Interactions between the different domains of quality are complex and future research should focus on the associations between different features of quality in early childhood education programmes and practices that are critical to implementing successful continuous improvement initiatives.

KEYWORDS:

• Early childhood education
• quality
• structural
• process

Introduction

The body of international research on the early years of human development provides a compelling argument for the provision of high-quality early learning experiences in the years before school. Children’s earliest experiences provide the momentum for substantial development and growth and lay a foundation for life-long learning (Phillips & Shonkoff, 2000). These early experiences are vastly different depending on the environments in which children live, grow, and learn, and can be influenced by relatively small shifts in the actions and interactions of parents, carers and educators (Asbury & Plomin, 2013).

Children’s learning environments include those in early childhood education and care (ECEC) services. ECEC is defined, in this review, as any early childhood education and care service provided to groups of children outside of their home environment in the years before they start formal school (Sincovich et al., 2020). This includes kindergarten or preschool providing an education programme, and long-day care and family-day care for infants, toddlers and young children. The chosen definition is broad enough to ensure that international literature is captured in the review, regardless of the contextual factors in different countries. The evidence is clear that these high-quality early learning experiences benefit children’s cognitive, language and social development in the short- and long-term, particularly for children from disadvantaged backgrounds (Campbell et al., 2008; Fox & Geddes, 2016; Melhuish et al., 2015; Ramey & Ramey, 2005; Taggart et al., 2015).

However, quality in early childhood education contexts is a multidimensional concept and at the centre of much debate amongst academics, researchers, policy makers, and practitioners (Wysłowska & Slot, 2020). Sociocultural perspectives (Vygotsky, 1978) and bioecological theory (Bronfenbrenner & Morris, 2007) provide frameworks to identify the features of quality. Internationally, features of quality in ECEC have been characterised as falling within two domains: “structural quality” and “process quality” (Dowsett et al., 2008; Mashburn et al., 2008; Vandell & Wolfe, 2000). Other taxonomies of quality in ECEC have been proposed which include operational features, educational concepts and the classroom environment in the structural domain (OECD, 2006; Pianta et al., 2016), and interactions, parent engagement and experiences in the process domain (OECD, 2006; Pianta et al., 2016; Urban et al., 2012). More recently Torii et al. (2017) identified “system quality” as a third domain. In this review we adopt this three-domain approach, in order to clearly articulate the difference between “system” level supports to quality such as funding and regulatory standards and the “structural” supports to quality such as staff qualifications, group size and ratio of staff to children. Arguably, despite differences in terminology, the broad domains of structural, process, and system quality can encapsulate the full range of quality features.

Structural quality is typically understood to include features such as the learning environment, educator qualifications, and educator-child ratios. Also included within structural quality are professional development activities, learning frameworks to guide educational programming and practice, and supports that build learning opportunities in the home environment (OECD, 2006; Pianta et al., 2016; Torii et al., 2017). Process quality encompasses children’s experiences within ECEC, with a focus on educator pedagogy and effective teaching strategies, educator-child interactions, programmes, curricula and learning interventions and social-emotional support. It relates to the participation of children in ECEC programmes, including the daily back-and-forth exchanges children have with educators and their peers (Burchinal et al., 2008; Pianta et al., 2016; Torii et al., 2017). The importance of curriculum in driving quality improvement and children’s learning and development has also been described as a feature of process quality (OECD, 2006). System quality consists of factors such as governance and regulatory standards and the degree of investment in services and programmes (Torii et al., 2017).

While the value of high-quality practice across each domain is clear, the provision of high-quality ECEC is still lacking in many jurisdictions and services (Torii et al., 2017). This is particularly the case when taken from the view of an individual child’s typical experience in ECEC, where the frequency and quality of interactions with educators has been found to be relatively low (Pianta et al., 2016; Tayler et al., 2013). As such, a better understanding of how to drive quality improvements is imperative for enhancing quality ECEC and learning and development outcomes for all children.

A wide range of approaches to continuous improvement in each quality domain have been applied worldwide (Navarro-Cruz & Luschei, 2018; OECD, 2006; Torii et al., 2017; Urban et al., 2012). These have included changes to system quality, for example through improvements to national policies, funding allocations and regulatory or evaluation frameworks. Changes to structural quality have included enhanced educator training, working conditions and career pathways. Improvements to process quality have developed enhanced curriculum programmes designed to improve educator-child interactions and improved pedagogical leadership. Prior research on the outcomes of these initiatives indicates variable effects of such interventions, while also identifying which specific features may be most effective at improving quality and supporting children’s learning and development (European Commission, 2020; Melhuish et al., 2015; Pianta et al., 2016).

Making the policy decisions and practice changes required to continue to drive quality improvement in ECEC is a complex undertaking. It relies on clear, consistent, and high-quality evidence within and across all domains of quality. It requires an understanding of the nuances in the research literature regarding which domains of quality are the key drivers of young children’s learning and development, what the key factors and thresholds for impact on educators and/or children are within each domain, and how the domains are interrelated.

While existing research provides indications of the most effective levers of change, for example pedagogy and interactions (Melhuish et al., 2015; Pianta et al., 2016), professional development (European Commission, 2020), and system level accreditation processes (European Commission, 2020), the current body of literature is not comprehensive. There is significant variation in the amount of research which has been undertaken in each domain of quality. The evidence that does exist is heterogeneous in nature and of varying methodological quality, with inconsistent and at times, contradictory findings. As a result, it is challenging for policy initiatives to target or achieve change in the key drivers of quality improvement.

In addition, the literature in the field has not been comprehensively synthesised. The presence of a number of meta-analyses or systematic reviews alone does not provide overarching insights into the extent of the evidence in a given field (van de Glind et al., 2013). It is not until these findings are drawn together that insights helpful to researchers, policy makers or practitioners become accessible. A scoping method is an appropriate approach to achieve this, as outlined below.

The current study

This scoping review will map the extent and consistency of the literature in ECEC on each domain and sub-domain of quality in order to identify gaps in knowledge and to inform a future research agenda. Scoping reviews are useful for clarifying working definitions, summarising key concepts, and identifying the breadth of and gaps in the evidence in a field (Arksey & O’Malley, 2005; Levac et al., 2010; Munn et al., 2018; Peters et al., 2015). While scoping reviews are more commonly used in health research than in educational research, they are particularly helpful for understanding topics such as this, in which the evidence is heterogeneous, unevenly distributed and not amenable to a traditional systematic review (Munn et al., 2018; Peters et al., 2015).

The research questions for this scoping review are broad to allow for a wide-ranging summary of the breadth of literature in the field. To maintain clarity and focus, however, they are accompanied by an articulated scope of enquiry clearly linked to the purpose of the study (Levac et al., 2010).

The research questions for this study are aligned to the purpose of mapping the literature and informing a future research agenda. They are:

1. What is the extent of the research literature in the domains of quality in ECEC?

2. How consistent are the key findings of the research literature reviewed in each sub-domain of quality in ECEC?

Method

The scope of this study is concerned with ECEC settings and services, the educators who work in these services, and the children who access them. Throughout the paper we use “educator” as the broad term to include all adults, qualified teachers and educators, working with children in ECEC settings. The core concept of interest is the quality of ECEC programmes and practice, which is defined according to the structure, process, and system domains, as outlined above. The key outcomes of interest are closely tied to these domains of quality: changes to educators’ practice and changes to children’s outcomes. To ensure that the findings are meaningful and address our research questions, only rigorous peer reviewed meta-analyses and systematic reviews were included.

The methods used in this review were informed by the guidance for scoping reviews provided by Peters et al. (2015) and scoping study methods outlined by Levac et al. (2010). In line with these approaches, the following steps were taken in this scoping review: (1) identification of the review purpose, research question and scope of the enquiry; (2) definition of eligibility criteria; (3) search of the literature; (4) selection of included studies; (5) extraction of results; (6) analysis; and (7) presentation of results. The presentation of results in this article was informed by the revised PRISMA guidelines for scoping reviews (Tricco et al., 2018). The purpose of the research and the research questions are outlined above (step 1). The definition of the inclusion and exclusion criteria for the study, the search and screening strategy and the planned approach to data extraction and analysis are outlined below (steps 2–6), followed by a presentation of results (step 7). The protocol for the broader study was registered with the OSF Registries (URL: https://osf.io/rdc5j/).

Search strategy

A multi-disciplinary team contributed to the development of the following search strategy for this study, aligned closely with the research questions. A method akin to a rapid evidence assessment (Barends et al., 2017), the search strategy for a scoping study is comprehensive yet practical and feasible. However, the limits set for the search strategy must not compromise the ability of the search to answer the research questions (Levac et al., 2010).

In education as well as in medicine, policy makers and practitioners look to secondary sources such as meta-analyses and systematic reviews to summarise and critically appraise available evidence (van de Glind et al., 2013). Due to their use of explicit and reproducible methods to systematically search, critically appraise, and synthesise literature on a specific issue, systematic reviews and meta-analyses provide reliable estimates of effects and reduce biases and random errors that may be present in individual studies (Gopalakrishnan & Ganeshkumar, 2013). Consequently, they provide a comprehensive and reliable summary of multiple studies. In acknowledgement of the value of such syntheses (Joanna Briggs Institute, 2013; Murad et al., 2016), our search strategy commenced with a search of academic databases for systematic literature reviews and meta-analyses relating to quality in ECEC published between 2015 and 2020. Limiting our search to this 5-year period ensured that the included syntheses were current and the overall number manageable, whilst also highlighting that the studies included within each of the syntheses resulted in a much wider time-span of literature that was reviewed.

In 2020 searches were completed in the following academic databases: ERIC (EBSCO), Education Databases (ProQuest), PsycINFO, Education Research Complete, A+ Education, Educational Administration Abstracts, Socindex with full text, Web of Science, Google Scholar.

Searches of academic databases used the following terms: Early childhood education (including kindergarten, pre-kindergarten, child care, day care, family day care, nursery, early learning, preschool or playgroup); and review or meta-analysis; and/or quality. Synonyms, alternative spellings, and thesaurus terms were used for each term. Specific search terms, filters and strategies were defined to align with the conventions for each database and Boolean operators “OR” and “AND” were used to combine the terms. Hand searches of reference lists and grey literature were used to augment database searches. Refer to Appendix A for the search strategy.

Eligibility criteria

The following inclusion criteria were applied to the initial screening of titles and abstracts in line with the purpose and scope of the study: (i) the population of interest for this review was children aged birth to six years old attending kindergarten, preschool or ECEC services, and/or educators of those children; (ii) articles were included if they pertained to an ECEC service or programme, as defined above and there was no restriction on the geographical area; (iii) studies were included if they reported an outcome relating to any domain of quality, including outcomes such as improved service environments (structural quality), teaching practices (process quality), system improvements or children’s learning and development outcomes; (iv) meta-analyses and systematic literature reviews were included; (v) peer-reviewed articles published in English between 2015 and 2020.

During the full-text screening of records an assessment of methodological quality was completed according to the relevant National Institutes of Health (NIH) Study Quality and Assessment tool pertaining to the study methodology (National Institute of Health Research, 2019). Articles were excluded if they were rated as “poor” quality using these criteria. A quality review is not required for scoping reviews (Peters et al., 2015); however, to interpret the findings in relation to the research questions for this study, it was deemed appropriate to exclude poor-quality studies or articles (Levac et al., 2010).

Selection of included studies

The title and abstract of each record sourced through the search strategy was screened for inclusion by members of the research team independently and identified for full-text screening if they met the inclusion criteria above. Subsequently, the full text of each identified article was screened for inclusion by two members of the research team independently with reference to the complete set of inclusion and exclusion criteria described above. Inter-rater agreement at this stage was classified as substantial to near-perfect with Cohen’s Kappa ranging from 0.61 to 0.85 between reviewers (McHugh, 2012). Discrepancies were resolved by a third member of the research team, if necessary, at each stage. Referencing software was used to compile search results and Covidence systematic review software was used to record and track selection decisions.

Data extraction

The research team collectively determined which variables to extract or “chart” (Levac et al., 2010) for each included study. These included: (i) study details (including: authors, year, country/s, study type); (ii) research question/s and/or objective/s; (iii) domains of quality considered/included (see Table 1); (iv) study specific inclusion criteria and key characteristics of studies; (v) number of included studies; and (vi) key findings in relation to domains of quality.

Table 1. Definitions of domains and subdomains of quality used.

Domain	Subdomain	Definition of sub-domain
Process quality	Interactions	The shared experiences that educators and children have together as part of their relationships in ECEC settings.
	Learning programmes, interventions, and curricula	Learning experiences and activities for children in ECEC, sometimes including discussion of curricula. We define intervention as a programme or package that targets a specific population (e.g. children from socially disadvantaged backgrounds) and/or targets specific outcomes (e.g. improve educator-child interactions, improve child language).
	Pedagogy	The instructional techniques and strategies used by educators to support children’s learning and development.
	Parent and community engagement	The involvement of parents, families, and community members in the education of young children.
Structural quality	Ratios and group size	A measure of the number of children for whom each educator is responsible; group size refers to the number of children in a given ECEC, preschool or kindergarten room.
	Qualifications and pre-service training	Initial teacher education, training or qualifications obtained by educators prior to commencing their role, such as certificate or degree programmes.
	Professional development and support	Professional learning opportunities and professional support provided while employed in an ECEC service, for example, in-service training, coaching, and mentoring.
	Learning environments	The diverse physical locations, contexts, and cultures in which children learn.
System quality	Funding, regulation, and frameworks	The systems that support, organise, and maintain ECEC. These include funding and investment from State and Federal governments, the regulations, frameworks, and policies that determine the provision and implementation of ECEC services.

Data were extracted using Covidence systematic review software. Ten per cent of records were screened by at least two authors independently prior to commencing data extraction from remaining records. This allowed the research team to discuss discrepancies and refine the process for the variables to be extracted by individual researchers.

Included studies were independently rated as either of “good” or “fair” methodological quality by two authors using the NIH Study Quality and Assessment Tool for Systematic Reviews and Meta-Analyses (National Institute of Health Research, 2019). Studies rated as “poor” had already been excluded. Discrepancies in the rating were resolved by a third author where necessary.

Data analysis

Following data extraction, descriptive statistics were used to illustrate the extent of the literature pertaining to each domain and sub-domain of quality. The domains and sub-domains of quality used as analytic categories are detailed in Table 1 and are based on the work of Torii et al. (2017), the OECD (2006), Pianta et al. (2016) and Urban et al. (2012). This allowed the identification of gaps in the research pertaining to each domain and sub-domain.

A consistency ranking was then determined to apply to studies in each sub-domain of quality. Using an approach similar to that described by the Education Endowment Foundation (2018) a consistency ranking of high (3), moderate (2), or low (1) pertaining to each sub-domain of quality was generated following a qualitative review of the key findings in included studies conducted independently by two authors. Discrepancies were resolved via discussion.

Consistency was defined as the “consistency of the estimated impacts across the studies that have been synthesised” (EEF, 2018, p. 30). In this scoping review, a “high” degree of consistency referred to studies in which findings almost universally indicated the positive impact of a given sub-domain on ECEC quality or children’s learning and development outcomes, or, conversely, evidence which almost universally indicated negative or no impact on quality or children’s outcomes. A rating of “moderate” consistency indicates a general trend in findings in relation to impact, but with a reasonable proportion of studies indicating the opposite result or inconclusive findings. A rating of a “low” degree of consistency indicates that there is inconclusive evidence about the impact of a sub-domain of quality, or mixed evidence as to whether the impact was positive, negative, or null. Descriptive statistics were used to summarise the consistency of the findings in each of the sub-domains of quality.

The results were synthesised to achieve the objectives of the study. That is, the results were discussed in relation to the scope and gaps in the evidence base pertaining to each domain of quality and the consistency of findings.

Results

Eighty-five meta-analyses and systematic reviews were included in the review. Figure 1 presents the results of the search and screening process.

Figure 1. Flow chart of screening and selection processes.

Characteristics of included studies

The characteristics of included meta-analyses and systematic reviews are outlined in Table 2. Of the included studies, 28 were systematic reviews, 33 meta-analyses, and 14 were both meta-analyses and systematic reviews. Meta-analyses and systematic reviews were published between 2015 and mid-2020 – the full range of our inclusion criteria. This included seven in 2015, 17 in 2016, 16 in 2017, 13 in 2018, 19 in 2019 and 13 in 2020. Studies included in the meta-analyses and systematic reviews were published between 1971 and 2019. Of the 55 meta-analyses or systematic reviews that stated the countries of their included studies, most (48) were in the USA alongside a wide range of other, mostly English-speaking, countries. Eighty-one meta-analyses and systematic reviews focussed on children, 28 on professionals and 9 on parents (noting that several studies targeted more than one of these groups, as can be seen in Table 2). Meta-analyses and systematic reviews included a median of 27 papers in their analyses (range: 6–272). Of the included studies 40 were rated as “fair” and 45 rated as “good” quality.

Table 2. Characteristics of included studies.

Citation	Primary domain of quality	Study Type	Target participants of interest	Countries of included studies	Earliest publication date of included study	Most recent publication date of included study	Number of included studies
Albritton et al. (2019)	Programmes/interventions/curricula	SR	Children, professionals and parents/carers	USA	2006	2016	13
Alhassan et al. (2018)	Programmes/interventions/curricula	SR	Children	USA	2006	2016	10
Ang et al. (2017)	Learning environments	SR	Children	UK, US, Australia, New Zealand, and Canada	1983	2013	20
Averett et al. (2017)	Parent and community engagement	SR	Children, professionals and parents/carers	USA	1990	2012	20
Barnes et al. (2018)	Programmes/interventions/curricula	MA	Children	NS	1978	2016	31
Baron et al. (2017)	Programmes/interventions/curricula	SR	Children	USA	2008	2014	6
Barton et al. (2020)	Programmes/interventions/curricula	SR	Children	NS	1985	2015	27
Berti et al. (2019)	Learning environments	SR	Children, professionals and parents/carers	USA (42), Norway, Canada, Turkey, UK, Australia, Brazil, Sweden, China, Finland, NZ, South Korea, Denmark, Egypt, Cyprus, Greece, Iran, Israel, Italy, Oman. Some data collection in India, Indonesia, South Africa.	1974	2018	88
Blewitt et al. (2020)	Programmes/interventions/curricula	SR	Professionals	USA (13), Turkey (2)	2004	2019	15
Blewitt et al. (2021)	Programmes/interventions/curricula	SR	Children	USA, Australia (1), China (1), Canada (1), Belgium (1)	2002	2018	19
Blewitt et al. (2018)	Programmes/interventions/curricula	SR and MA	Children	Africa, Australia, Europe, Middle East, North America, South America.	1995	2017	142
Bowman-Perrott et al. (2016)	Programmes/interventions/curricula	MA	Children	USA, Spain (1), Sudan (1), Canada (1), Belize (1)	1969	2013	21
Bowne et al. (2017)	Ratios and group size	MA	Children	NS	1965	2007	38
Brock and Carter (2017)	Professional development and support	MA	Professionals	USA	1980	2015	12
Brunsek et al. (2020)	Professional development and support	SR and MA	Children	USA, UK, Canada, NZ, Belgium, Jamaica, Denmark, Australia, Chile	1993	2018	64
Camargo et al. (2016)	Programmes/interventions/curricula	MA	Children	NS	1996	2008	19
Castro et al. (2015)	Parent and community engagement	MA	Children and parents/carers	USA, Mexico, Korea, Egypt, Iceland, Greece, Cambodia, Israel	2000	2013	37
Chambers et al. (2016)	Programmes/interventions/curricula	SR and MA	Children	USA	1990	2015	32
Ciesielski and Creaghead (2020)	Professional development and support	SR	Children and professionals	USA	2003	2019	23
Dalziel et al. (2015)	System	SR	Children	USA, England	1985	2011	13
Davenport et al. (2019)	Programmes/interventions/curricula	SR	Children	NS	2004	2017	14
Dunst et al. (2019)	Qualifications & pre-service training	MA	Children and professionals	NS	1979	2018	130
Eddy et al. (2019)	Programmes/interventions/curricula	SR	Children	NS	2013	2017	23
Egert et al. (2020)	Professional development and support	MA	Professionals	USA (14) and Chile (1)	2008	2017	15
Egert et al. (2018)	Professional development and support	MA	Children and professionals	USA, Canada (4)	1992	2011	45
Falenchuk et al. (2017)	Qualifications and pre-service training	SR and MA	Children and professionals	USA, Canada (1), Multiple (1).	1972	2014	39
Fancher et al. (2018)	Programmes/interventions/curricula	SR	Children	NS	2000	2017	15
Fukkink et al. (2017)	Programmes/interventions/curricula	MA	Children	Netherlands	2000	2015	21
García-Carrión and Villardón-Gallego (2016)	Interactions	SR	Children and professionals	USA, UK, New Zealand, Finland	2005	2015	11
Graham, Liu, Aitken, et al. (2018)	Programmes/interventions/curricula	MA	Children	NS	1985	2016	47
Graham, Liu, Bartlett, et al. (2018)	Programmes/interventions/curricula	MA	Children	United States (49%), Canada (23%), the United Kingdom (9%), Spain (4%), Netherlands (4%), Australia (2%), France (2%), Hong Kong (2%), Norway (2%), Portugal (2%), Taiwan (2%)	1980	2016	54
Grindal et al. (2016)	Parent and community engagement	MA	Children and parents/carers	NS	1965	2008	272
Gunning et al. (2019)	Programmes/interventions/curricula	SR	Children	NS	1986	2017	31
Jasmin et al. (2018)	Programmes/interventions/curricula	SR	Children, professionals and parents/carers	USA (14), Canada (4), Australia (2), Israel (2), Germany (1).	2000	2014	23
Jensen and Rasmussen (2019)	Professional development and support	MA	Children and professionals	Netherlands, France, Germany, Wales, Denmark (Inclusion criteria was European studies)	2009	2017	9
Joo et al. (2020)	Parent and community engagement	MA	Children	USA	1969	2007	38
Kossyvaki and Papoudi (2016)	Programmes/interventions/curricula	SR	Children	NS	2003	2014	14
Kraft et al. (2018)	Professional development and support	MA	Children and professionals	USA, Chile, (2), Canada (3)	2006	2017	60
Lippard et al. (2017)	Interactions	SR	Children	USA (23), Israel (2), Australia (2)	2000	2015	27
Luckner and Movahedazarhouligh (2019)	Programmes/interventions/curricula	SR	Children	USA (6), Canary Islands (1), Australia (2) Turkey (1), Iran (1)	1996	2014	11
Ma et al. (2016)	Parent and community engagement	MA	Children and parents/carers	NS	1993	2012	46
Magnuson et al. (2016)	Programmes/interventions/curricula	MA	Children	US	1960	2007	36
Manning et al. (2017)	Qualifications and pre-service training	SR and MA	Children and professionals	NS	1987	2015	48
Manning et al. (2019)	Qualifications and pre-service training	MA	Children	NS	1987	2015	49
Markussen-Brown et al. (2017)	Professional development and support	MA	Children and professionals	NS	1995	2014	27
McMullen et al. (2020)	Other	Both SR and MA	Children	NS	2002	2018	32
Murano et al. (2020)	Programmes/interventions/curricula	MA	Children	Criteria states studies from inside & outside USA included (no other reference to countries of origin)	1982	2017	48
Nelson and McMaster (2019)	Programmes/interventions/curricula	MA	Children	21 studies in the USA; 13 unspecified	1984	2016	34
Nocita et al. (2020)	Qualifications and pre-service training	SR and MA	Children	USA	1972	2018	16
O’Connor et al. (2017)	Parent and community engagement	SR	Children and parents/carers	USA, UK, Australia, Netherlands, Sweden, Hong Kong	2005	2015	21
Oh-Young and Filler (2015)	Learning environments	MA	Children	USA	1984	2011	24
Peden et al. (2018)	Professional development and support	SR	Children and professionals	USA, Australia, UK, Switzerland	2008	2016	11
Peleman et al. (2018)	Professional development and support	SR	Children and professionals	EU countries	1993	2013	39
Perlman et al. (2016)	Interactions	SR and MA	Children	USA	2006	2015	35
Perlman et al. (2017)	Ratios and group size	SR and MA	Children	Germany, USA, Korea, Bermuda	1980	2015	29
Pesco and Gagné (2017)	Pedagogy	MA	Children	NS	1982	2012	15
Phelps (2019)	Programmes/interventions/curricula	MA	Children	Barbados (2), Belgium (3), Canada (22), China (1), Germany (2), Israel (16), Korea (3), Mexico (6), UK (16), USA (436)	1971	2010	149
Pokorski et al. (2017)	Pedagogy	SR	Children	NS	1971	2013	10
Razak et al. (2019)	Learning environments	SR	Children, professionals and parents/carers	USA (13), Canada (3,), Australia (2), Europe, multiple (1)	2007	2017	19
Rogers et al. (2020)	Professional development and support	SR	Children and professionals	USA (22), Canada (1), NZ (1)	2006	2015	24
Roorda et al. (2017)	Interactions	Both sys MA	Children and professionals	NS	1990	2016	179
Ross and Joseph (2019)	Programmes/interventions/curricula	SR	Children	USA	1998	2016	10
Schindler et al. (2015)	Programmes/interventions/curricula	MA	Children	NS	NS	NS	31
See and Gorard (2015)	Parent and community engagement	SR	Children and parents/carers	N/A	NS	NS	127
Shepley and Grisham-Brown (2019)	Programmes/interventions/curricula	SR and MA	Children and professionals	NS	2007	2016	16
Shepley et al. (2020)	Programmes/interventions/curricula	SR and MA	Children and professionals	NS	2007	2017	19
Sills et al. (2016)	Programmes/interventions/curricula	SR	Children	NS	1986	2015	21
Sim et al. (2018)	Programmes/interventions/curricula	SR	Children	USA, Denmark, Netherlands, India, France, UK, Portugal, Canada.	2008	2018	108
Snell et al. (2019)	Professional development and support	SR	Children and professionals	NS	2001	2017	11
Snyder et al. (2015)	Pedagogy	SR	Children	NS	1981	2013	43
Soto et al. (2019)	Programmes/interventions/curricula	SR	Children	NS	2002	2014	15
Suggate (2016)	Programmes/interventions/curricula	MA	Children	NS. Mostly English speaking (60.6%)	1980	2013	71
Therrien et al. (2016)	Programmes/interventions/curricula	SR	Children	NS	1988	2013	19
Tonge et al. (2016)	Learning environments	SR	Children	USA (56%), Canada, Sweden, Netherlands, Belgium, Denmark, Australia.	1992	2015	27
Tupou et al. (2019)	Programmes/interventions/curricula	SR	Children and professionals	USA (12), Italy (4)	2002	2017	16
Ulferts et al. (2019)	Pedagogy	MA	Children	Of 15 Programmes: Greece, Portugal (2), Germany (6), Spain, Northern Ireland, England, Finland, Netherlands (2), Scotland	2002	2016	17
van Huizen and Plantenga (2018)	System	MA	Children	Canada (3), USA (9), England (2), Australia (1), Germany (7), Norway (3), France (1). Spain, (1).	2005	2017	27
Vandenbroucke et al. (2018)	Interactions	SR and MA	Children	USA, Ecuador, Belgium, Portugal, Italy, Netherlands, Turkey, Chile.	2009	2017	28
Verhoeven et al. (2020)	Programmes/interventions/curricula	MA	Children	USA, Turkey, UK, Netherlands, Canada, Finland, Australia, Israel.	1995	2017	59
Vermeer et al. (2016)	Ratios and group size	MA	Children and professionals	Bangladesh, China, Jordan, Singapore, South Korea, Australia, NZ, Austria, Germany, Greece, Ireland, Italy, Netherland, Portugal, Spain, Sp. Basque Country, Sweden, UK, Canada, Neth. Antilles, USA, Brazil, Chile	1989	2012	72
Wang et al. (2016)	Programmes/interventions/curricula	MA	Children	NS	2002	2012	29
Ward et al. (2015)	Programmes/interventions/curricula	SR	Children	USA & Netherlands	2008	2015	15
Wasik et al. (2016)	Programmes/interventions/curricula	SR	Children and professionals	NS	1994	2014	31
Werner et al. (2016)	Interactions	MA	Children and professionals	USA, Jamaica (1), The Netherlands (2), Canada (2)	2003	2012	18
Wick et al. (2017)	Programmes/interventions/curricula	SR and MA	Children	USA (12), European (12), Other (Iran, Australia, Taiwan)	NS	NS	30

Notes: NS = not stated: N/A = not applicable; SR = Systematic Review; MA = Meta-Analysis.

Synthesis of results

Research question 1: What is the extent of the literature in the domains of quality in ECEC?

Sixty-four percent of the literature related to the process quality domain, with the structural quality domain representing another 31%. Six studies (5%) focussed on the system quality domain (see Table 3). The most common sub-domain focus in the literature was on programmes, interventions, and curricula within the process quality domain (52 included studies), followed by professional development and support within the structural quality domain (16 included studies). The remaining sub-domains were the focus of between 1 and 9 meta-analyses or systematic reviews.

Table 3. Quality domains and subdomains addressed in included studies.

Quality domain	Sub-domain	Total (n)	Proportion of total (%)
Process		75	64.6
	Programmes/interventions/curricula	52	44.8
	Interactions	9	7.75
	Parent and community engagement	9	7.75
	Pedagogy	5	4.3
Structure		35	30.2
	Professional development and support	16	13.8
	Qualifications/pre-service training	7	6
	Learning environments	7	6
	Ratios and group size^a	4	3.4
	Other (years of experience)	1	0.1
System		6	5.2
Total		116	100

^aThree studies were found on group size; four on ratios.

Research question 2: How consistent are the key findings of the review literature in the domains on quality in ECEC?

Following the ranking of consistency in included studies, we found that four sub-domains were highly consistent: Learning environments (structural), professional development and support (structural), programmes, interventions, and curricula (process) and pedagogy (process). The findings in relation to interactions (process), and parent and community engagement (process) were moderately consistent. These two sub-domains, each had the same number of reviews included (nine) and were given the same consistency rating, and therefore, are represented by the same colour (yellow) in Figure 2. The findings in relation to ratios and group size (structural), and qualifications and pre-service training (structural) were inconsistent. We were unable to assign a consistency rating to system quality as the studies were too heterogeneous in their topics of study.

Figure 2. Matrix of number and consistency of studies in sub-domains.

Structural quality

There was limited and inconsistent evidence about the impact of educator-child ratios or group size on quality, with only four meta-analyses or systematic reviews on this topic identified. The findings from these studies indicate that while group size may not be critical, lower ratios may have some bearing on improved process quality (Vermeer et al., 2016), particularly when the ratios are very low, such as 7:1 and lower (Bowne et al., 2017). In contrast, Perlman et al. (2017) found that there was no evidence of a relationship between educator-child ratios and child outcomes.

Evidence from seven included meta-analyses and systematic reviews indicated that the learning environment – including both the physical environment and social context – of ECEC both affect quality. Findings from the included studies indicated that the physical environment can affect children’s behaviour, cognition, and emotion (Tonge et al., 2016). Influencing factors include the size of the play space and the opportunity to play outside (Tonge et al., 2016). There is overlap with the findings in this domain and findings in relation to system quality, including that the social contexts in which children from disadvantaged backgrounds and children with disabilities are included are important (Oh-Young & Filler, 2015; van Huizen & Plantenga, 2018). There were indications that the greatest gains for children were to be found within disadvantaged community contexts (van Huizen & Plantenga, 2018). There was limited evidence about whether home-based or centre-based care was associated with quality (Ang et al., 2017).

Seven included studies investigated whether educator qualifications were related to ECEC quality, but the findings were inconsistent. For example, Manning and colleagues found that educator qualification was associated with higher ECEC quality (Manning et al., 2017; Manning et al., 2019), yet Falenchuk et al. (2017) and Nocita et al. (2020) found that educators’ qualifications or specialist early childhood training was not related or only weakly associated with positive child outcomes. In addition, the effectiveness of professional development interventions was found to be unrelated to educators’ previous qualifications (Egert et al., 2018, 2020). In related findings, McMullen et al. (2020) found that educators’ years of experience was not associated with quality.

Of the 16 meta-analyses and systematic reviews which investigated professional development and support, 12 indicated positive associations between professional development and either teaching practices (Albritton et al., 2019; Brock & Carter, 2017; Egert et al., 2018, 2020; Peleman et al., 2018) or children’s learning and development outcomes (e.g. Joo et al., 2020). One study found that the inclusion of professional development alongside an intervention made no difference (Joo et al., 2020) and one found mixed results (Ciesielski & Creaghead, 2020). The included studies indicated that professional development was most effective when it included features such as modelling and performance feedback (Brock & Carter, 2017). There were mixed results on issues of duration or frequency (dose) of professional development required to detect a change in practice (e.g. Brock & Carter, 2017; Rogers et al., 2020) but clear indications that forms of ongoing professional development which were integrated into practice such as coaching were likely to be effective (e.g. Kraft et al., 2018; Peleman et al., 2018; Rogers et al., 2020).

Process quality

Among the five included meta-analyses or systematic reviews, four found that pedagogical approaches such as sustained shared thinking (García-Carrión & Villardón-Gallego, 2016; Pesco & Gagné, 2017; Ulferts et al., 2019) and naturalistic instruction for children with disabilities (Snyder et al., 2015) have generally had lasting benefits on quality of educator practices and children’s learning and development. The fifth study found limited research on their pedagogical approach of interest (Pokorski et al., 2017). We identified a clear overlap between the findings related to pedagogy and other sub-domains within process quality, such as interactions.

Nine included studies (7.75%) investigated the relationship between educator-child interactions and quality of programmes and practice with four reviews indicating a positive relationship (García-Carrión & Villardón-Gallego, 2016; Roorda et al., 2017; Werner et al., 2016). One systematic review (Perlman et al., 2016) indicated that high-quality interactions were not associated with positive child outcomes but noted that the findings were inconclusive. Another found that interactions were related to quality, but this differed depending on the measure of quality used (Vandenbroucke et al., 2018). Six of the included studies found either a limited number of studies or poor methodology in the studies they included in their analyses.

Fifty-two meta-analyses or systematic reviews were identified that evaluated the effect of programmes, interventions, or curricula. Many of these findings were also included in summaries of other domains, as the interventions related to other quality areas, such as professional development and support (Egert et al., 2018; Jensen & Rasmussen, 2019; Kraft et al., 2018; Markussen-Brown et al., 2017) and educator-child interactions (García-Carrión & Villardón-Gallego, 2016). In the studies included in this domain most focussed on gains for children rather than quality as an up-stream measure (Blewitt et al., 2020). Of these, 37 reviews identified that the programmes, intervention, or curricula studied can positively influence children’s learning and development outcomes (Blewitt et al., 2018; Graham, Liu, Aitken, et al., 2018; Gunning et al., 2019; Joo et al., 2020; Murano et al., 2020; Sim et al., 2018; Wang et al., 2016) or behaviour (Bowman-Perrott et al., 2016). However, several of these reviews or meta-analyses found studies of low quality (Baron et al., 2017; Davenport et al., 2019; Sim et al., 2018; Tupou et al., 2019) or found few studies to include in their analyses (Alhassan et al., 2018; Kossyvaki & Papoudi, 2016; Luckner & Movahedazarhouligh, 2019). Other included meta-analyses and systematic reviews found mixed effects (Soto et al., 2019) or no evidence of a relationship between the intervention and positive outcomes (Fukkink et al., 2017; Ross & Joseph, 2019; Shepley et al., 2020).

Nine meta-analyses or reviews (7.75%) focussed on the topic of parent or community engagement, with inconsistent results. For example, Grindal et al. (2016) found no differences in impact between programmes that included parent education and those that did not, while others found that the inclusion of parent programmes (Joo et al., 2020) or parental engagement (Ma et al., 2016) can lead to better outcomes for children. See and Gorard (2015) found mixed results among their included studies.

System quality

We found only six meta-analyses or systematic reviews which investigated the relationship of system features to quality. The application of a consistency rating was not applicable to these studies as the elements of the system in focus varied widely between studies, from investment in ECEC (Dalziel et al., 2015; Razak et al., 2019) to the integration of services for children with additional needs (Jasmin et al., 2018; Oh-Young & Filler, 2015) and programme structure (Ang et al., 2017; van Huizen & Plantenga, 2018). A meta-analysis by van Huizen and Plantenga (2018) found that full-time rather than part-time enrolment in preschool programmes may produce significant gains, but that age of enrolment does not appear to be important. There is also evidence that integrated services can support the learning and development of children with disabilities (Oh-Young & Filler, 2015). Having adequate funds to enable services to purchase equipment and resources may be important (Razak et al., 2019), however, the evidence on return on investment to wider society was found to be equivocal (Dalziel et al., 2015).

Discussion

The purpose of this scoping review was to determine the extent of the research literature in the domains of quality (structural, process and system) in ECEC and identify the consistency of key findings of the literature included in the review in each sub-domain of quality. The sub-domain of programmes, interventions and curricula within the process quality domain included the greatest number of studies by far, followed by professional development and support in the structural quality domain. Most sub-domains included a small number of studies (i.e. ≤9).

Based on our scoping review, there was high consistency in findings for several sub-domains (i.e. programmes, interventions and curricula; professional development and support; pedagogy and learning environments) and moderate consistency for interactions and parent and community engagement. However, many meta-analyses and systematic reviews presented contradictory evidence related to outcomes for either educators or children. It is difficult to reach an overarching conclusion regarding the considerable amount of literature for programmes, interventions and curricula, given the significant diversity that exists between studies. While the evidence consistently suggests that programmes and interventions are effective tools to drive children’s learning outcomes, further research is needed to identify if there are global characteristics or active ingredients of successful programmes, interventions, or curricula. Conclusions drawn from other sub-domains of quality may assist in identifying these characteristics, as some studies have identified particular foci embedded in programmes or interventions that demonstrate moderately consistent and positive outcomes, such as educator-child interactions.

Our findings suggest that the sub-domains of structural quality: professional development and learning environments consistently influence early childhood education programmes and practice. However, we also noted that most of the evidence has been gathered from centre-based programmes and that there is a need for further research exploring the impact of quality in home-based (family day) care compared to centre-based care. Other structural features, such as staff qualifications, pre-service training and ratios and group size were studied less frequently and contributed inconsistent and contradictory evidence regarding high-quality early childhood education. While there is consensus regarding minimum standards for qualifications, educator-child ratios and group size necessary for provision of high-quality preschool programmes (ACECQA, 2018) the evidence in these sub-domains of quality remains under-researched and equivocal. Furthermore, process quality must be supported by structural and system quality. Features of system and structural quality are understood as having more indirect impacts on children’s outcomes via supporting efforts to improve process quality (Burchinal, 2018; Urban et al., 2012).

Consequently, structural features provide the levers by which process quality can be the focus of quality improvement initiatives within early childhood education programmes. Several studies attest to the fact that sub-domains in structural quality have an impact on educator practice, if efforts to improve process quality are also embedded, for example targeting pedagogy and interactions within structural quality professional development initiatives. Equally, improving facilities and reducing ratios and group sizes have been found to allow for increases in high-quality educator-child interactions (European Commission, 2020; Melhuish et al., 2015).

The consistency of the evidence for the structural quality sub-domain: Professional Development was high in this review. Common features of professional development programmes emerging from the evidence were (i) include multiple learning components, (ii) practice-based, (iii) include coaching, (iv) adopt a collaborative approach, and (v) allow time for critical reflection on the implementation of new strategies. Professional development has been found to be effective in a range of service systems and settings, based on the range of international contexts included in the reviews for our analyses. Urban et al. (2012) emphasise the development of competent early childhood professionals as a key lever to drive quality. However, they see this competence not as something resting solely with professional learning of individuals and their knowledge, skills, and attitudes, but as a characteristic of the ECEC system. In this way, individual competence is interdependent with and supported by system and structural features of quality in early childhood education; a “competent system”.

There are important caveats to be noted regarding the conclusions in this review. First, clear delineation between the domains of quality is not always possible and, in many studies, there is considerable overlap between the sub-domains. For our purposes, we used the primary focus when coding meta-analyses and systematic reviews to a domain and sub-domain. For example, a number of studies included the sub-domains of learning programmes or interventions, and pedagogy. Equally, evidence related to professional development often incorporates working with educators to improve the quality of their interactions with children. The “stacking” of several different quality features within evaluation studies can make it difficult to interpret outcomes within different (sub)-domains of quality. Potentially, professional learning programmes may have an impact on educator practice irrespective of the content knowledge focus (e.g. interactions) within them, however, understanding the independent impact of more than one subdomain of quality included in a study is often not feasible given the methodological approach adopted.

Importantly, this review highlighted that different study methodologies and the measurement of quality within studies contribute significantly to the contradictory evidence in several of the sub-domains of quality. Synthesising the evidence was further complicated by contested approaches to measuring quality (Burchinal et al., 2016; Caronongan et al., 2016) across many studies. How quality is defined and understood in the literature is reflected in the tools used to measure and assess quality; where some tools and regulatory practices focus more on structural features of quality and other tools prioritise process features. System and structural quality parameters are often addressed through accreditation or regulatory documents and standards of practice developed for national contexts, while measures of process quality have emerged through observational studies of educator-child interactions and children’s experiences within programmes (Pianta et al., 2008).

Limitations

In the context of this study, it is important to recognise that the literature notes multiple features related to the child, their family and community, as well as early education settings that interact to promote young children’s learning during the years prior to school. The focus of this review is on the features of high-quality early childhood education that can maximise the learning opportunities for all children participating in ECEC. Hence, this review and the included studies do not account for influences outside of ECEC, or for children not attending early childhood programmes.

It is important to acknowledge that any scoping review is limited by the inclusion and exclusion criteria applied to the published literature (Levac et al., 2010). By focusing on published meta-analyses and systematic reviews we may have missed the most recent examples of research in the field and relevant grey literature. Qualitative research is underrepresented in meta-analyses and systematic reviews, and consequently in this scoping review. Finally, we acknowledge that publication bias can influence the research literature that represents certain fields of study, and this could be the case in early childhood as it is in other fields that include intervention studies.

Conclusion

In this scoping review, we included meta-analyses and systematic reviews to determine the extent and consistency of the evidence of quality in early childhood education and care programmes. Results indicated that most studies pertained to programmes, interventions, and curricula (within process quality) and professional development and support (within structural quality). The evidence was most consistent in the following sub-domains of quality: pedagogy; professional development and support; programmes, interventions, and curricula (process quality); and learning environments (structural quality).

High-quality early childhood education and care has the potential to optimise all children’s learning and development prior to formal school (Melhuish et al., 2015). Implementing high-quality programmes that integrate intentional teaching within play-based, developmentally appropriate learning experiences is complex and requires investment in ongoing and differentiated professional support. This scoping review identified that there is limited research evidence for the impact of several domains of quality on both educator practice and children’s learning outcomes. Future research needs to build a more nuanced understanding of the relationships between the domains of quality, such as comparisons of different features of structural (e.g. home-based or centre-based care) and process quality (e.g. the active ingredients in early childhood play-based programmes and interventions) which lead to improved learning outcomes for children.

The extent and consistency of evidence pertaining to what constitutes high-quality early childhood education programmes varies across the domains of structure, process and system. When synthesised, the research literature suggests that first and foremost “quality” is not a singular concept. Quality features (or domains) are highly interdependent and interact together in complex and context-dependent ways. This scoping review goes some way towards understanding this complexity and future research should focus on the associations that are critical to implementing successful continuous quality improvement initiatives.

Acknowledgement

The authors wish to thank Ms Eileen Cini and Dr Amy Watts for their input during the screening of papers and with the final manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Appendix A.

Sample search strategy for stage 1

Table

Database	Search date	Search terms OR search string	Search strategy	Limiters	Records returned
ERIC (EBSCO)	24 April 2020	TI “Child Development Centre*” OR “Child Development Center*” OR “Child Care Center*” OR “Child care centre*” OR “Early Childhood Education” OR nursery or kinder* or preschool* OR “pre-school*” OR “early edu*” OR childcare OR “child care” OR “child-care” OR daycare OR “day care” OR “day-care” OR playgroup* OR “play-group*” OR “play group*” AB “Child Development Centre*” OR “Child Development Center*” OR “Child Care Center*” OR “Child care centre*” OR “Early Childhood Education” OR nursery or kinder* or preschool* OR “pre-school*” OR “early edu*” OR childcare OR “child care” OR “child-care” OR daycare OR “day care” OR “day-care” OR playgroup* OR “play-group*” OR “play group*” SU “Child Development Centre*” OR “Child Development Center*” OR “Child Care Center*” OR “Child care centre*” OR “Early Childhood Education” OR nursery or kinder* or preschool* OR “pre-school*” OR “early edu*” OR childcare OR “child care” OR “child-care” OR daycare OR “day care” OR “day-care” OR playgroup* OR “play-group*” OR “play group*” TI (review or meta*) AND AB (review or meta*) AND SU (review or meta*)	1 AND 2 AND 3	2015–2020	143