Scoping reviews: Their development and application in environmental and sustainability education research

Figures & data

Table 1. Topics to consider for systematic and scoping reviews.

Feature	Expectation	Systematic Review	Scoping Review
Rationale	“Describe the rationale for the review in the context of what is already known” (Liberati et al., 2009). Explain and justify the alignment of the research objectives with the types of review selected.	Researchers should specify the importance of elaborating the review considering current evidence and knowledge gaps.
		Researchers should describe “what the systematic review may add to what is already known” (Liberati et al., 2009), including previous systematic reviews elaborated in the same field.	Researchers should describe why a scoping review is required aligned to specific purposes (Levac et al., 2010)
Objectives	Specify research question(s) and objectives.	The research question should be stated explicitly to understand the scope of the review (Liberati et al., 2009).
		The research question(s) should describe the “Participants, Interventions, Comparisons, Outcomes, and Study design (PICOS)” (see Liberati et al., 2009)	The research question(s) should describe “the Population, Concept, and Context” (PCC) (Peters, Godfrey, et al., 2020). Scoping reviews could have only one research question if this one completely addresses the PCC components.
Search	Determine the sources, search strategies, and the data items to be sought. Select the evidence that will be analysed based on inclusion/exclusion criteria.	Researchers should specify the search and source selection justifying each methodological decision.
		Researchers should specify the method that evaluates whether evidence fits the criteria (Liberati et al., 2009). Likewise, researchers should describe how the risk of bias will be assessed.	The search should “be as comprehensive as possible within the constraints of time and resources” (Peters, Godfrey, et al., 2020). Resources could come from grey literature.
Critical appraisal (risk of bias)	Researchers should justify the reliability and certainty of the evidence. Quality assessment: identification of evidence’ strengths, limitations, and paucities.	Considering that systematic reviews serve to inform evidence-based decisions, researchers should report the risk of bias per each study and the “assessments of certainty (or confidence) in the body of evidence for each outcome assessed” (Liberati et al., 2009).	This step is not mandatory in a scoping review. Therefore, we reiterate, if researchers are more interested in assessing quality concerns for a particular intervention or practice, then they should evaluate if a systematic review is more pertinent Munn et al. (2018). If done, researchers should justify why a critical appraisal was required (Pollock et al., 2021).
Synthesis and/or analysis	Prepare and present results.	This step will depend on the type of systematic review undertaken (see Aromataris & Munn, 2020). Generally, systematic reviews might include a statistical synthesis to determine heterogeneity. However, if qualitative research is studied, researchers should implement an appropriate interpretative approach (Lockwood et al., 2020).	Researchers should estimate frequency counts of topics or categories. Pollock et al. (2021) argue that scoping reviews do not involve synthesising results to inform decision-making. Instead, researchers might use descriptive qualitative content analysis to content summary.
Limitations	Discuss limitations of the review.	Limitations could list the exclusions of the search. For instance, when the data is limited to a specific population (Liberati et al., 2009)
		When a systematic review is elaborated, two types of limitations should be described: “at study and outcome level (such as the risk of bias), and at review level (such as incomplete retrieval of identified research, reporting bias)” (Liberati et al., 2009).	Although the risk of bias across studies is not applicable in scoping reviews (Tricco et al., 2018), the limitations should include methodological aspects and constraints related to the available evidence (Pollock et al., 2021).
Guidelines and Checklists	Use authoritative guidelines and checklists to enhance the review process’s quality, rigour, and transparency.	Aromataris and Munn (2020); Liberati et al. (2009)	Peters, Godfrey, et al. (2020); Tricco et al. (2018)

Figure 1. Critical steps for scoping reviews.

Note: * to indicate this step is often underdeveloped in the guidance and practice of scoping reviews.

Table 2. Examples of research questions of scoping reviews elaborated in ESE.

Features related to research objectives	Purported Purpose	Research Questions Offered	Example
Scope	To identify the focus and purpose of studies of ESE in tertiary institutions.	“How do HEIs promote sustainable development?”	(Berchin et al., 2021, p. 4)
Characteristics and scope	To analyse the applicability and implementation of photovoice in environmental, sustainability, and conservation context. Additionally, this review explores the benefits and challenges of this method reported in the literature.	“In our review of the literature to better understand our own application of photovoice within youth participatory research in environmental contexts, we wondered how and in what ways the method was being used and how it was being adapted for use in diverse social and academic environments.”	(Derr & Simons, 2020, p. 362)
Scope, characteristics, nature, and spectrum	To analyse the literature in engineering education for sustainable development to determine their approach to educational initiatives, drivers, and barriers in specific contextual settings. From the evidence, this review proposed a classification of educational approaches via an integral framework.	“What is the nature of the sustainability initiatives implemented in engineering education in tertiary institutions? What strategies and methodologies are used in each of these initiatives? How was the context understood in terms of relationships, spaces, and time influenced SE initiatives? What are the main differences between the expected and actual students’ outcomes of SE initiatives in this instance of engineering education?”	(Gutierrez-Bucheli et al., 2022) [p.3 of pdf]

Table 3. Examples of inclusion and exclusion criteria used in ESE scoping reviews.

Feature related to the inclusion and exclusion criteria	Description	Example
Type of sources	Only peer-reviewed papers published in journals.	(Berchin et al., 2021)
Type of sources and primary focus of study	Only peer-reviewed papers published in journals. Studies were included if they focused on “science education, environmental education, place-based education, sustainability education, or conservation.”	(Derr & Simons, 2020, p. 363)
Types of sources, study purpose, aim, and kind of population.	Grey literature was included alongside peer-reviewed papers. Candidate publications had to show they related to a higher education program. Studies were excluded if they did not have an educational purpose explicitly related to sustainability, e.g., as defined by Unesco.	(Gutierrez-Bucheli et al., 2022)

Table

Label	Description	Search	Appraisal	Synthesis	Analysis
Critical review	Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include a degree of analysis and conceptual innovation. Typically results in hypothesis or mode	Seeks to identify the most significant items in the field	No formal quality assessment. Attempts to evaluate according to contribution	Typically narrative, perhaps conceptual or chronological	Significant component: seeks to identify conceptual contribution to embody existing or derive a new theory
Literature review	Generic term: published materials that provide an examination of recent or current literature. Can cover a wide range of subjects at various levels of completeness and comprehensiveness. May include research findings	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative	Analysis may be chronological, conceptual, thematic, etc.
Mapping review/ systematic map	Map out and categorise existing literature from which to commission further reviews and/or primary research by identifying gaps in the research literature	Completeness of searching determined by time/scope constraints	No formal quality assessment	May be graphical and tabular	Characterises quantity and quality of literature, perhaps by study design and other key features. May identify a need for primary or secondary research
Meta-analysis	Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results	Aims for exhaustive, comprehensive searching. May use funnel plot to assess completeness	Quality assessment may determine inclusion/ exclusion and/or sensitivity analyses	Graphical and tabular with narrative commentary	Numerical analysis of measures of effect assuming absence of heterogeneity
Mixed studies review/mixed methods review	Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies	Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies	Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists	Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies	Analysis may characterise both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other
Overview	Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics	May or may not include comprehensive searching (depends whether systematic overview or not)	May or may not include quality assessment (depends whether systematic overview or not)	Synthesis depends on whether systematic or not. Typically narrative but may include tabular features	Analysis may be chronological, conceptual, thematic, etc.
Qualitative systematic review/qualitative evidence synthesis	Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies	May employ selective or purposive sampling	Quality assessment typically used to mediate messages not for inclusion/exclusion	Qualitative, narrative synthesis	Thematic analysis, may include conceptual models
Rapid review	Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research	Completeness of searching determined by time constraints	Time-limited formal quality assessment	Typically narrative and tabular	Quantities of literature and overall quality/direction of effect of literature
Realist review	Examines the outcomes of (social) complex intervention programs. From a deep analysis of interconnections between Mechanisms, Outcomes, and Context (MOC), realist review seeks to understand why an intervention program worked or did not work.	Iterative search that includes a broad range of data sources required to construct (or reject/refine) a theory	Appraisal is done to different fragments of the evidence based on relevance to the theory and rigor of the method (Wong et al., 2013)	Present key features of the evidence including their outcomes. Examples and quotes could be used to describe theory	Analyse data using MOC patterns. Additional analytic techniques could be implemented and should be justified
Scoping review	Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research)	Completeness of searching determined by time/scope constraints. May include research in progress	No formal quality assessment	Typically tabular with some narrative commentary	Characterises quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review
State-of-the-art review	Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives	Aims for comprehensive searching of current literature	No formal quality assessment	Typically narrative, may have tabular accompaniment	Current state of knowledge and priorities for future investigation and research
Systematic review	Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review	Aims for exhaustive, comprehensive searching	Quality assessment may determine inclusion/exclusion	Typically narrative with tabular accompaniment	What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research
Systematic search and review	Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’	Aims for exhaustive, comprehensive searching	May or may not include quality assessment	Minimal narrative, tabular summary of studies	What is known; recommendations for practice. Limitations
Systematised review	Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative with tabular accompaniment	What is known; uncertainty around findings; limitations of methodology
Umbrella review	Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad conditions or problem for which there are competing interventions and highlights reviews that address these interventions and their results	Identification of component reviews, but no search for primary studies	Quality assessment of studies within component reviews and/or of reviews themselves	Graphical and tabular with narrative commentary	What is known; recommendations for practice. What remains unknown; recommendations for future research

Table

Review Type	Description	Approximate Timeframe	No. of Reviewers	Expertise in topic area required	Urgency of review	Number of references	Complete literature	Consequences of incomplete literature review	A priori review protocol	Explicit, transparent, peer-reviewed search strategy	Research question	Research question format	Data extraction	Analysis	Risk of bias statement	Resources	Example
																	Study	Research Question(s)
Systematic review	Seeks to systematically search for, appraise and synthesise research evidence in order to aid decision-making and determine best practices. Systematic reviews can vary in their approach and are often specific to the type of study: studies of effectiveness, qualitative research, economic evaluation, prevalence, aetiology or risk, diagnostic test accuracy, etc.	8 months to 2 years	2 or more	High	Low	Small	Yes	High	Yes	Yes (full)	Specific	PICO* format, specific considerations exist for each type of systematic review	Highly detailed	Detailed	Required	High	(Gómez-Olmedo et al., 2020)	P	Non-clinical population
																		I	Mindfulness programs
																		C	Sustainable lifestyles
																		O	Socioemotional competencies
Scoping review	Assesses the potential scope of the research literature on a particular topic. Helps determine gaps in the research.	12 months to 2 years	1-2	High	Medium	Medium	Yes	Medium	Yes (sometimes)	Yes (partial)	Rather specific	PCC** format	Medium-highly detailed	Any	Recommended	Medium	(Gutierrez-Bucheli et al., 2022)	P	Engineering education
																		C	Sustainability education initiatives
																		C	Tertiary institutions
Rapid review	Assesses what is known about an issue by using a systematic review method to search and appraise research and determine best practices.	2-6 weeks	2	High	High	Medium	Yes	High	Yes	Yes (partial)	Specific	Narrow question, PICO* or PCC** may be used	Medium-highly detailed	Detailed	Required	Medium	(Trendafilova & McCullough, 2018)	P	Sport sector
																		C	Current efforts
																		C	Environmental sustainability research
Traditional (narrative) literature review	A generic review which identifies, and reviews published literature on a topic, which may be broad. Typically employs a narrative approach to reporting the review findings. Can include a wide range of related subjects.	1-4 weeks	1	Low	Low	Any	No	Low	No	No (unstructured)	Any	No specific search questions. However, the argument of the literature review is aligned with the project’s research question(s).	Any	Any	Recommended	Low-medium	(Heimlich & Ardoin, 2008)	To assess how insights from behavioural psychology could influence ESE research
Realist review	Seeks to identify theory that explains why social intervention programs worked or did not work identifying mechanism, outcomes, and context patterns.	6 months to 1 year	1 or more	High	Low	Small	Yes	High	No	Yes (full)	Specific	Includes elements such as “What works, how, why, for whom, to what extent and in what circumstances?” (Wong et al., 2013, p. 6)	Highly detailed	Detailed	Required	Low-medium	(Rickinson et al., 2016)	To understand the program’s elements that influence the program’s outcomes

* PICO refers to Patient/Problem or Person (P), Intervention/Exposure/Therapy or Treatment (I), Comparison (C), and Outcomes (O)

** PCC refers to Population (P), Concept (C), and Context (C)

Table

Category aspects	Considerations	Recommendations
Design, purpose, and scope	Identify research question(s) using PCC format. Consider and delimit the review’s scope and purpose. Develop the review protocol.	Cross-check the review’s purpose with the aims ascribed to scoping reviews. Examine PRISMA-ScR and JBI recommendations.
Relevant literature and evidence search	Define search strategies following the protocol according to the scope. Identify sources, search terms, and limitations. Define inclusion and exclusion criteria. Monitor the comprehensiveness and feasibility of the review. Evaluate the value of incorporating elements of the grey literature	Consult a librarian to determine the most relevant databases in the field and the correct design of the search terms. Evaluate the reliability of the search terms using text mining. Justify all the decisions related to the evidence search, including the limitations.
Evidence selection	Select the evidence that will be analysed to answer the research question(s) according to inclusion and exclusion criteria. Elaborate the screening process in two stages: (a) title, abstract, and keywords and (b) full text.	Develop a PRISMA flowchart describing the steps and results of this process. Consider involving a multidisciplinary team. Use a specific software for reference screening management.
Data extraction	Prepare the extraction framework defined in the protocol or plan. Test the extraction form before the extraction properly happens.	Elaborate a standardised extraction form.
Summary and report	Map intentions via descriptive qualitative analysis. Present results in charts or tables. Detailed narrative summary where the research question(s) are answered. Analyse implications for the current literature, policy, and practice.	Consider using a specific systematic method.
Consultation and review	Consult experts from the planning stage to ensure the rigor of the scoping review. Check that the data prioritised in the review is enough to answer the research question(s).	Ensure effective communication of findings.

Table

Further research proposed by	Topic	Possible ESE topics or areas for scoping reviews
Ardoin et al. (2013)	Collective learning and action	In what ways have hybrid learning configurations been used in a particular context related to environmental and sustainability education (ESE)? What is the ‘nature’ of place-based education initiatives implemented in a named education sector or stage?
	Learning in social-ecological communities	What kinds of education initiatives have been promoted to address particular ethical dimensions of sustainability? What strategies have been identified as effective in ESE that integrate sustainability into everyday life considerations?
	Urban and diverse populations	What city-based ESE initiatives have been implemented in a particular and diverse location? How has ESE incorporated indigenous perspectives?
	Technology	In what ways have specific technologies been used in ESE to enhance the student experience? What evidence underpins claims to best practices for technology adoption in ESE?
Clark et al. (2020)	Intended learning outcomes	What are the education initiatives that have been promoted in ESE to develop environmental knowledge? What is the state-of-the-art of teaching and learning strategies in ESE that improve social and cultural aspects?
Heimlich and Ardoin (2008)	Behaviour change	What kinds of strategies have been promoted in ESE that are shown to encourage conservation-related actions? Which drivers are identified and documented in ESE research as those that foster a behaviour change in students?
Feinstein et al. (2013)	ESE general: clarifying concepts	What is the research available on climate change education (CCE) in a particular context? In what way does it differentiate from education for sustainable development?
	Governance and policy	What does research show is the state-of-the-art of ESE policy development for a particular context/location/topic?

Liberati, A., D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Gotzsche, J. P. A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher. 2009. “The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Healthcare Interventions: Explanation and Elaboration.” BMJ (Clinical Research ed.) 339: b2700. doi:https://doi.org/10.1136/bmj.b2700.

 PubMed Web of Science ®Google Scholar

Levac, D., H. Colquhoun, and K. K. O’Brien. 2010. “Scoping Studies: Advancing the Methodology.” Implementation Science 5 (1): 69. doi:https://doi.org/10.1186/1748-5908-5-69.

 PubMed Web of Science ®Google Scholar

Peters, M. D. J., C. M. Godfrey, P. Mclnerney, Z. Munn, A. C. Tricco, and H. Khalil. 2020. “Chapter 11: Scoping Reviews.” In JBI Manual for Evidence Synthesis, edited by E. Aromataris and Z. Munn. South Australia: JBI doi:https://doi.org/10.46658/JBIMES-20-12.

 Google Scholar

Munn, Z., M. D. J. Peters, C. Stern, C. Tufanaru, A. McArthur, and E. Aromataris. 2018. “Systematic Review or Scoping Review? Guidance for Authors When Choosing between a Systematic or Scoping Review Approach.” BMC Medical Research Methodology 18 (1). article number 143. doi:https://doi.org/10.1186/s12874-018-0611-x.

 Web of Science ®Google Scholar

Pollock, D., E. L. Davies, M. D. J. Peters, A. C. Tricco, L. Alexander, P. McInerney, C. M. Godfrey, H. Khalil, and Z. Munn. 2021. “Undertaking a Scoping Review: A Practical Guide for Nursing and Midwifery Students, Clinicians, Researchers, and Academics.” Journal of Advanced Nursing 77 (4): 2102–2113. doi:https://doi.org/10.1111/jan.14743.

 PubMed Web of Science ®Google Scholar

Aromataris, E., and Z. Munn. 2020. “Chapter 1: JBI Systematic Reviews.” In JBI Manual for Evidence Synthesis, edited by E. Aromataris and Z. Munn. JBI. doi:https://doi.org/10.46658/JBIMES-20-02.

 Google Scholar

Lockwood, C., K. Porrit, Z. Munn, L. Rittenmeyer, S. Salmond, M. Bjerrum, H. Loveday, J. Carrier, and D. Stannard. 2020. “Chapter 2: Systematic Reviews of Qualitative Evidence.” In JBI Manual for Evidence Synthesis, edited by E. Aromataris and Z. Munn. JBI. doi:https://doi.org/10.46658/JBIMES-20-03.

 Google Scholar

Berchin, I. I., A. R. Aguiar Dutra, and J. B. S. O. d Guerra. 2021. “How Do Higher Education Institutions Promote Sustainable Development? A Literature Review.” Sustainable Development (Bradford, West Yorkshire, England) 29 (6): 1204–1222. doi:https://doi.org/10.1002/sd.2219.

 Web of Science ®Google Scholar

Derr, V., and J. Simons. 2020. “A Review of Photovoice Applications in Environment, Sustainability, and Conservation Contexts: Is the Method Maintaining Its Emancipatory Intents?” Environmental Education Research 26 (3): 359–380. doi:https://doi.org/10.1080/13504622.2019.1693511.

 Web of Science ®Google Scholar

Gutierrez-Bucheli, L., G. Kidman, and A. Reid. 2022. “Sustainability in Engineering Education: A Review of Learning Outcomes.” Journal of Cleaner Production 330: 129734. doi:https://doi.org/10.1016/j.jclepro.2021.129734.

 Web of Science ®Google Scholar

Wong, G., T. Greenhalgh, G. Westhorp, J. Buckingham, and R. Pawson. 2013. “RAMESES Publication Standards: Realist Syntheses.” BMC Medicine 11 (1): 21. doi:https://doi.org/10.1186/1741-7015-11-21.

 PubMed Web of Science ®Google Scholar

Gómez-Olmedo, A. M., C. Valor, and I. Carrero. 2020. “Mindfulness in Education for Sustainable Development to Nurture Socioemotional Competencies: A Systematic Review and Meta-Analysis.” Environmental Education Research 26 (11): 1527–1555. doi:https://doi.org/10.1080/13504622.2020.1777264.

 Web of Science ®Google Scholar

Trendafilova, S., and B. P. McCullough. 2018. “Environmental Sustainability Scholarship and the Efforts of the Sport Sector: A Rapid Review of Literature.” Cogent Social Sciences 4 (1): 1467256. doi:https://doi.org/10.1080/23311886.2018.1467256.

 Web of Science ®Google Scholar

Heimlich, J. E., and N. M. Ardoin. 2008. “Understanding Behavior to Understand Behavior Change: A Literature Review.” Environmental Education Research 14 (3): 215–237. doi:https://doi.org/10.1080/13504620802148881.

 Web of Science ®Google Scholar

Rickinson, M., M. Hall, and A. Reid. 2016. “Sustainable Schools Programmes: What Influence on Schools and How Do we Know?” Environmental Education Research 22 (3): 360–389. doi:https://doi.org/10.1080/13504622.2015.1077505.

 Web of Science ®Google Scholar

Ardoin, N. M., C. Clark, and E. Kelsey. 2013. “An Exploration of Future Trends in Environmental Education Research.” Environmental Education Research 19 (4): 499–520. doi:https://doi.org/10.1080/13504622.2012.709823.

 Web of Science ®Google Scholar

Clark, C. R., J. E. Heimlich, N. M. Ardoin, and J. Braus. 2020. “Using a Delphi Study to Clarify the Landscape and Core Outcomes in Environmental Education.” Environmental Education Research 26 (3): 381–399. doi:https://doi.org/10.1080/13504622.2020.1727859.

 Web of Science ®Google Scholar

Feinstein, N. W., J. Laessøe, N. Blum, and D. Chambers. 2013. “Challenging the Premises of International Policy Reviews: An Introduction to the Review Symposium.” Environmental Education Research 19 (2): 198–205. doi:https://doi.org/10.1080/13504622.2013.768603.

 Web of Science ®Google Scholar