Environmental education outcomes of community and citizen science: a systematic review of empirical research

Figures & data

Figure 1. PRISMA flow diagram of systematic review methods based on Moher et al. (2009).

Figure 2. Keyword Boolean code search criteria.

Table 1. Example main codes for analysis of empirical articles.

Code	Sample sub-codes
Audience	Pre-K, Elementary School, Middle School, High School, College/ University, Adult 18+
Participation in Main Activities of Scientific Research	Choose/define research question, gather information and resources, formulate hypotheses, design of data collection methods, collect data/samples, analyze samples, documenting/reporting of local ecological knowledge (LEK), analyze data, interpret data and draw conclusions, disseminate results and/or translate into action, discuss results and ask new questions
Environmental Topics	Insects, mammals, plants, birds, air quality, water quality, terrestrial biodiversity, environmental health, environmental social science, invasive species
Research Design	Case study/ies, quasi-experimental, qualitative, Community-based participatory research, ethnography, participatory action research, evaluation, mixed methods, action research
Data Collection Methods	Survey, focus group, interview, observation, community meetings/listening sessions, document/artifact review
Learning Outcomes Reported	Environmental science content knowledge, science inquiry skills, knowledge of process & nature of science, positive attitudes toward science or environment, interest in science or environment, self-efficacy toward science or environment, place values or connection to nature, community connectedness and cooperation, environmental behavior & stewardship (definitions provided in Table 6)

Table 2. Environmental education learning outcomes and definitions.

EE learning outcome	Definition
Environmental Science Content Knowledge	‘knowledge of science content refers to understanding of subject matter, i.e. facts or concepts’ (Phillips et al. 2018, 9), which we consider to include environmental science subject matter.
Science Inquiry Skills	‘observable practices that can be transferred to daily life, such as asking and answering questions; collecting data; developing and using models; planning and carrying out investigations; reasoning about, analyzing, and interpreting data; constructing explanations; communicating information; and using evidence in argumentation’ (Phillips et al. 2018; NGSS Lead States 2013) .
Community-Level Outcomes	Outcomes related to how people relate to each other and groups of individuals, as opposed to other outcomes which focus on individual-level outcomes, including such outcomes as community capacity, networking and connecting with others, improving dialogue, improved governance, community ownership of knowledge and process, social connectedness, group belonging, social learning (collective sense-making, trust between political actors and stakeholders), collaboration (Jadallah & Ballard, 2021; Jordan et al. 2012).
Environmental Behavior & Stewardship	Measurable actions resulting from engagement in citizen science, but external to the protocol activities and the specific project-based skills of the citizen science project. This includes observable actions to directly maintain, restore, improve, or educate about the health of an ecosystem, or participation in civic, governmental, or cultural affairs to solve problems at the local, regional, or national level. (Phillips et al. 2018).
Knowledge of Process and Nature of Science*	‘understanding the methodologies that scientists use to conduct research (for example, the “scientific method”)’ (Phillips et al. 2018, 9), and knowledge of the Nature of Science (NOS) refers to understanding the epistemological underpinnings of scientific knowledge and how it is generated, sometimes presented from a post-positivist perspective (Phillips et al. 2018, 9).
Positive Attitudes Toward Science & Environment	‘measurable demonstration of assessment of, change in, or exercise of attitude toward a particular scientific topic, concept, phenomena, theory, or careers central to the project or one’s capabilities relative to these areas’ (Phillips et al. 2018, 3).
Self-efficacy with science or environment	a person’s beliefs about his/her capabilities to learn specific content and to perform particular behaviors (Bandura, 1997), and in the context of citizen science, the extent to which a learner has confidence in his or her ability to participate in a science or environmental activity (Phillips et al. 2018).
Place Values and Connection to Nature	includes a group of distinct but related constructs (Gosling & Williams, 2010) including sense of place, place meaning (symbolic meaning ascribed to places), place identity, place attachment (a positive bond between people and a particular place), or connection to the local community or environment (Kudryavtsev et al. 2012), or feeling a part of nature (Schultz, 2001)
Interest in science or environment*	The degree to which an individual assigns personal relevance to a science or environmental topic or endeavor, characterized by positive feelings and an increasing investment in learning more about a particular topic, and included interest in science or the environment, environmental engagement, environment relatedness & engagement (Phillips et al. 2018).
Identity & Agency with science or environment	Identity (how learners view themselves; National Academies, 2017) and Agency (a person enacting meaningful change in their life; Holland et al. 1998)) with respect to science or environmental knowledge and skills.

Figure 3. Number of articles reporting participants taking part in of the main scientific research activities of CCS.

Table 3. Number of articles with participants engaging in clustered scientific research activities of CCS.

Main scientific research activities	Clustered scientific research activities in CCS	No. of articles in each cluster
Choose/Define the Research Question	Planning	18
Gather Information & Resources
Formulate Hypotheses
Design of Data Collection Methods
Collect Data/Samples	Data Collection	97
Analyze Samples
Documenting/Reporting LEK
Analyze Data	Data Analysis and Interpretation	33
Interpret Data & Draw Conclusions
Disseminate Results and/or Translate into Action	Reporting	25
Discuss Results & Ask New Questions

Table 4. Number of articles reporting learning outcomes (all positive outcomes except for *knowledge of nature of science and interest in science or environment).

Learning outcome	# of articles reporting positive outcomes	# of articles reporting negative or no change	Total # of articles examining this outcome
Environmental Science Content Knowledge	56	0	56
Science Inquiry Skills	32	0	32
Community-Level Outcomes	30	0	30
Environmental Behavior & Stewardship	29	0	29
Knowledge of Process and Nature of Science*	15	2	17
Positive Attitudes Toward Science & Environment	16	0	16
Self-efficacy with science or environment	11	0	11
Place Values and Connection to Nature	9	0	9
Interest in science or environment*	6	2	8
Identity & Agency with science or environment	4	0	4

Table 5. EE learning outcomes by clustered scientific research activities of CCS.

	Number of articles
Number of articles (n = 100)	Planning (n = 18)	Data collection (n = 97)	Data analysis (n = 33)	Reporting (n = 25)
Science Content Knowledge	12	55	20	15
Science Inquiry Skills	6	30	16	11
Community-level outcomes	9	30	12	9
Environmental Behavior & Stewardship	7	28	10	11
Attitudes Toward Science & Environment	0	16	1)	1
Process and Nature of Science	1	17	4	4
Self-efficacy toward Science and Environment	3	11	4	3
Place Values	3	9	5	5
Interest in Science and Environment	1	68	1	2
Identity & Agency with Science and Environment	3	4	2	2

Figure 4. Number of empirical articles on CCS for EE published 2003–2018.

Table 6. Research designs used in studies of CCS for EE.

Research design
Case Study/ies	27
Quasi-experimental	26
Qualitative	14
CBPR	10
PAR	8
Mixed Methods	6
Other	5
Ethnography	4
Evaluation	5
AR	3

Table 7. Data collection methods used (n = 100).

Methods of data collection	Examples	Number of articles^a
Structured survey instruments	Validated scales, knowledge tests, self-reported assessments	54
Interviews	Structured interviews, semi-structured interviews, in-depth conversations, informal discussions	45
Observation	Participant observation, researcher interaction, structured or unstructured observation protocols	25
Focus Groups	Structured focus groups	21
Community meetings/ Listening sessions	Participatory, facilitated or informal group discussions	21
Other	Other methods used or not clearly defined	14
Document/ Artifact Review	Documents or other artifacts produced by participants assessed for learning outcomes	10
Quality Control Checks	Instructor/leader/third party review or auditing of data quality	6

^aThe sum is greater than 100 because many articles reported on more than one method of data collection.

Moher, D., A. Liberati, J. Tetzlaff, and D. G. Altman, PRISMA Group. 2009. “Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement.” PLoS Medicine 6 (7): e1000097. https://doi.org/10.1371/journal.pmed.1000097.

 PubMed Web of Science ®Google Scholar

Phillips, T., N. Porticella, M. Constas, and R. Bonney. 2018. “A Framework for Articulating and Measuring Individual Learning Outcomes from Participation in Citizen Science.” Citizen Science: Theory and Practice 3 (2): 3. https://doi.org/10.5334/cstp.126.

 Google Scholar

NGSS Lead States. 2013. Next Generation Science Standards: For States, by States. Washington, D.C.: National Academies Press. https://doi.org/10.17226/18290.

 Google Scholar

Jadallah, C., and H. L. Ballard. 2021. “Social Learning in Participatory Approaches to Conservation and Natural Resource Management: Taking a Sociocultural Perspective.” Ecology and Society 26 (4): 34. https://doi.org/10.5751/ES-12654-260437.

 Google Scholar

Jordan, R. C., H. L. Ballard, and T. B. Phillips. 2012. “Key Issues and New Approaches for Evaluating Citizen-Science Learning Outcomes.” Frontiers in Ecology and the Environment 10 (6): 307–309. https://doi.org/10.1890/110280.

 Google Scholar

Bandura, A. 1997. Self-Efficacy: The Exercise of Control. New York, NY: W. H. Freeman.

 Google Scholar

Gosling, E., and K. J. Williams. 2010. “Connectedness to Nature, Place Attachment and Conservation Behaviour: Testing Connectedness Theory among Farmers.” Journal of Environmental Psychology 30 (3): 298–304. https://doi.org/10.1016/j.jenvp.2010.01.005.

 Web of Science ®Google Scholar

Kudryavtsev, A., R. C. Stedman, and M. E. Krasny. 2012. “Sense of Place in Environmental Education.” Environmental Education Research 18 (2): 229–250. https://doi.org/10.1080/13504622.2011.609615.

 Web of Science ®Google Scholar

Schultz, P. W. 2001. “The Structure of Environmental Concern: Concern for Self, Other People, and the Biosphere.” Journal of Environmental Psychology 21 (4): 327–339.

 Web of Science ®Google Scholar

Holland, D., W. Lachicotte, D. Skinner, and C. Cain. 1998. Identity and Agency in Cultural Worlds. Cambridge, MA: Harvard University Press.

 Google Scholar