“Stickier” learning through gameplay: An effective approach to climate change education

Abstract

As the impacts of climate change grow, we need better ways to raise awareness and motivate action. Here we assess the effectiveness of an Arctic climate change card game in comparison with the more conventional approach of reading an illustrated article. In-person assessments with control/reading and treatment/game groups (N = 41), were followed four weeks later with a survey. The game was found to be as effective as the article in teaching content of the impacts of climate change over the short term, and was more effective than the article in long-term retention of new information. Game players also had higher levels of engagement and perceptions that they knew ways to help protect Arctic ecosystems. They were also more likely to recommend the game to friends or family than those in the control group were likely to recommend the article to friends or family. As we consider ways to broaden engagement with climate change, we should include games in our portfolio of approaches.

Keywords:

• Climate change
• education
• game
• Arctic

Purpose and learning goals

The past years saw one climate record broken after another. Based on our 140-year record of world climate information, the five warmest years have all occurred since 2015 (National Oceanic and Atmospheric Administration (NOAA), 2020). The Arctic is experiencing the most drastic change, twice the warming seen by the rest of the world, with concomitant melting of glaciers and ice sheets, as well as loss of sea ice (National Oceanic and Atmospheric Administration (NOAA), 2018). Due to anthropogenic emissions, greenhouse gases are increasing in the atmosphere; in 2016, the concentration of carbon dioxide did not fall below 400 ppm for the first time since before the Ice Age (Tans & Keeling, 2017). According to the Intergovernmental Panel on Climate Change:

Continued emission of greenhouse gases will cause further warming and long-lasting changes in all components of the climate system, increasing the likelihood of severe, pervasive and irreversible impacts for people and ecosystems. Limiting climate change would require substantial and sustained reductions in greenhouse gas emissions which, together with adaptation, can limit climate change risks. (Intergovernmental Panel on Climate Change (IPCC), 2014)

While there is a consensus regarding the evidence for climate change, moderating the impacts remains a multifaceted, collective and global challenge. Compounding the challenge is the topic itself: an area that is scientifically complex, laden with popular misinformation and misdirection, and often emotionally and politically charged. As a result, the general public continues to express uncertainty about climate change (Leiserowitz et al., 2015). With this uncertainty and large gaps in knowledge about what can be done, communicators of climate change and policy leaders constantly seek ways to better educate and motivate people.

An important approach to tackling this uncertainty is to talk with friends and family more about climate change. Maibach et al. (2016) found that while six in ten Americans say that the global warming issue is “extremely” (9%), “very” (17%), or “moderately” (35%) important to them, seven in ten Americans report that they “rarely” (36%) or “never” (32%) discuss global warming with family and friends. Yet a 2016 survey by Hamilton (2016) indicates that friends and family are second only to scientists as trusted sources of information about climate change. This suggests that conversation starters, which increase the frequency and quality of communication between friends and families, can lead to opportunities to voice opinions that would otherwise be kept to oneself (Priest, 2016). This can, in turn, broaden learning impact and influence (Priest, 2016), beyond individuals who originally learn the new information.

There is an urgent need for more effective strategies that promote better understanding of climate change, as well as critical thinking skills, empathy and new perspective taking (Simmons, 2011), and conversations. One such strategy, emphasized in the behavioral sciences and in recent policy initiatives (e.g., Zaval & Cornwell, 2016), suggests that games on the topic of climate change may serve as uniquely effective tools in teaching diverse audiences to understand and take action on climate-related issues (Wu & Lee, 2015). “Climate change games” are defined as games (including simulations) that involve climate change as their central theme, and focus on its scientific processes, social and physical impacts, mitigation and adaptation options, and the potential role of human behavior. These games allow players to experience some of the complexities of the climate system, and envision as well as rehearse the future, preparing people for what may come (e.g., Mendler de Suarez et al., 2012).

Here we analyze learning from a climate change card game focused on impacts associated with Arctic sea ice loss in comparison with reading an illustrated article on the same topic. The EcoChains: Arctic Crisis card game (Deaton, 2015, Lee, 2020, Turrin et al., 2020; Wu & Lee, 2015), was designed to demonstrate the key role of Arctic sea ice, increase understanding about the effects of climate change on the Arctic marine food web, and explain links between carbon pollution, anthropogenic climate change, and sea ice loss. Our learning goals for the game were for players to increase their knowledge of the Arctic marine ecosystem and to engage in considering mitigation options. The game was developed to promote engagement with understanding climate change through interacting with family and friends, thus was created to be appropriate for ages 10 and up.

We explore to what extent a game-based approach is engaging to participants and how it affects knowledge – both actual and perceived – compared with a more conventional learning approach (see Wouters et al., 2013). Our hypotheses are that:

1. This educational game will be as effective as the conventional, text-based approach at increasing knowledge, in the short term.

2. Newly learned information will be retained longer by game players than by those who read the illustrated article.

3. The game will stimulate greater levels of attention, interest, and enjoyment than will the article.

4. Game players will increase their willingness to talk about climate change with friends and family.

5. Game players will be more likely to recommend the game to others than readers would be likely to recommend the article to others.

Literature context

Games have rapidly exploded in growth in recent years and are now used in innovative ways, leading to growing interest in increasing their use for making a difference in climate education, awareness, and engagement (Wu & Lee, 2015). We now have “games for social change” (Games for Change, 2016); “games for health” (Kato, 2010); “persuasive games” (Bogost, 2010); “epistemic games” (Shaffer & Gee, 2008); “serious games” (Charsky, 2010); “game-based learning” (Gee, 2007; Prensky, 2007); and “games for social impact” (Stokes et al., 2016). The popularity and widespread use of digital and non-digital games are staggering; nearly half (49%) of Americans play games regularly (Pew Internet, 2015). In 2019, global digital game revenues were $152.1 billion, a + 9.6% one-year increase (Newzoo, 2019) and tabletop games have a $1.55B market in the US and Canada alone (Griepp, 2018). From 2013-2017, hobby/tabletop game sales more than doubled, with hobby card and dice games growing 15% between 2016 to 2017 (Griepp, 2018).

Games are played by all population cohorts, including adult learners. Adult learners, be they community leaders, the general public, or college students, are today’s decision makers and are more likely to make informed choices if they understand the social, economic, and environmental consequences of climate change. However, most adults are not in school and so cannot be reached through formal education approaches; therefore, informal educational resources – television, film, museums, aquariums, science centers, the Internet, and potentially also games – are critical in helping adults learn scientific content (Bell et al., 2009). Also, inherent to games are aspects known to be important in adult education (Forrest & Peterson, 2006): learning occurs experientially through a problem-centered, performance-oriented activity, and players are ready to learn and self-directed because learning has immediate value as it assists strategic game play. Gamer demographics show broad appeal across age and gender: the average game player is 33 years old, and 46% of gamers are female (Entertainment Software Association (ESA), 2019). While different types of games attract different types of people, Panagiotopoulos (2019) found that overall gamers’ political demographics match the general population. Hobby board gamers, also known as tabletop gamers, who responded to a survey (Nicole, 2016) were predominately white, more than 66% had some higher education, 24% were female, and 42% were 25-34 while 33% were 35-44 years old. Almost all (98%) played some kind of board game as a child.

Beyond reaching multiple age ranges, games have several other advantages over various educational approaches. Due to their fun and engaging properties, and ability to deliver challenges and problems to be solved (Gee, 2007), learning scientists have touted the value of well-designed games as robust learning environments (Gee, 2007; Shute & Ke, 2012; Squire, 2006). Increasingly, teachers are looking to include gameplay in their classroom (e.g., Ouariachi et al., 2019), providing active learning opportunities (Monroe et al., 2019). As noted by Martindale and Weiss (2019), tabletop games tend to be cheaper than digital games, and do not have issues with technology compatibility, so they are easier to incorporate in a wide range of learning environments.

Regarding the potential to learn from games, Heath et al. (2007) stated that “The first problem of communication is getting people's attention” (p. 64). Games focus attention as players need to respond dynamically to changes as games evolve, making multiple decisions and seeing their outcomes. Once attention is focused, the next challenge is retention. Gladwell (2000) states “And the specific quality that a message needs to be successful is the quality of “stickiness.” Is the message memorable? Is it so memorable, in fact, that it can create change, that it can spur someone to action?” (p. 92). Integral to playing tabletop games, is several people gathered together using multiple approaches: visual to read text and examine images, haptic as they play, auditory as they take their turn, and often repeating game play. Small group, multisensoral experiences are exactly the elements that lead to sticky learning (Inglis et al., 2014). Similarly, engaging multiple intelligences has been shown to lead to greater learning, and two strengths that students work from are social/interpersonal and body movement/kinesthetic (Griggs et al., 2009).

Research indeed indicates that students demonstrate increased engagement, as well as improved learning and performance on tests following participation in game-based learning activities (e.g., Papastergiou, 2009; Ricci et al., 1996; Tüzün et al., 2009; Wilson et al., 2009). A meta-analysis of serious games by Wouters et al. (2013) as well as the literature review by Boyle et al. (2016) of mostly digital games, found significant gains for both knowledge and retention. The Wouters et al. (2013) analysis concluded that learning is enhanced when games are played in groups, gameplay is supplemented with other instructional methods, and there are multiple training sessions for people to get used to the game.

Turning to climate change, games can allow players to practice 21st century skills ( Dondlinger, 2007; Shute & Ke, 2012; Partnership for 21^st Century Skills, 2015, Boyle et al., 2016, Qian & Clark, 2016 ) that are essential to addressing this complex issue. Qian and Clark (2016) review of game-based learning found moderate to large effect sizes for “collaboration, competition, complexity, exploration and discovery, role play, self-expression and interactivity” (p. 53). In addition, compared to documentaries, written text or games that focus on drilling practice exercise for improving content knowledge, there has been an increasing interest in designing to promote important skills such as empathy (McGonigal, 2011) and systems thinking, and to deliver powerful experiences from new perspectives (Wu & Lee, 2015). For example, Zec and Porter (2017) describe Tree, an immersive virtual reality game where players experience the impacts of climate change from the perspective of a tree in the rainforest. Similarly, Young et al. (2018) found that empathy and compassion for animals correlates with understanding their needs and has the potential to motivate conservation behavior.

Most people are not currently personally observing local effects of climate change that are drastic enough to motivate them to take action, but through games they can experience impacts first-hand. By positioning the player in various climate-centered scenarios, a game can provide “designed experiences” where players can learn through doing and being, rather than simply absorbing information from descriptive presentations or textbooks alone. Incorporating a level of personal control that is simply not possible in the real world, games are able to simulate complex scientific models. This is particularly helpful when the topic involves the global climate system that would otherwise be impossible to experience concretely in real life (Dörner et al., 2016).

First-hand experience is a much stronger motivator for action compared with analytically focused, descriptive information (Loewenstein et al., 2001; Reckien & Petkova, 2019). Assessing the influences on reactions to climate change, Priest (2016) observed “… factors that are particularly important include patterns of trust, a belief that action matters (perceived efficacy), and a sense of responsibility. Each of these offers opportunities for improving communication that could motivate a commitment to change …” (p. 54). As summarized by Ouariachi et al. (2017) climate games can be used to advance climate change action by avoiding psychological distance (Van Pelt et al., 2015), making climate change more real and memorable, and providing a sense of agency (see also Ouariachi et al., 2019). Coming back to the Gladwell (2000) quote above, such sticky messages can be so memorable that they can “spur someone to action” (p. 92). While we cannot assess action in this study, the proposed memorable aspect of games provides the foundation for hypothesis 2: Newly learned information will be retained longer by game players than by those who read the illustrated article.

Furthermore, because games are anticipated to be fun, climate games can also defuse some of the conflict around climate change perceptions and actions (Eisenack, 2013). Known to be useful in opening people up leading to questions and discussion (Depping et al., 2016; Petranek, 1994), playing a game together creates a common experience and terminology that can support discussion on the complex issue of climate change and related environmental problems (Reckien & Eisenack, 2010). When played with family and friends, climate games create an opportunity to voice opinions within a small group (Priest, 2016), thus extending conversation as recommended by Maibach et al. (2016) and Hamilton (2016). This research leads to hypotheses 4: Game players will increase their willingness to talk about climate change with friends and family and hypothesis 5: Game players will be more likely to recommend the game to others than readers would be likely to recommend the article to others.

Despite the recent increase in number of games, including climate games (Reckien & Eisenack, 2013; Wu & Lee, 2015) as well as the assumed advantages outlined above, only a few randomized control studies have attempted to test the effectiveness of games using an informative media comparison (Ouariachi et al., 2017; Soekarjo & van Oostendorp, 2015; Wouters et al., 2013). Further research is needed to explore how effective gameplay can be a medium for social, educational, or persuasive impact in comparison to more conventional learning methods, such as an article or text, particularly for the challenges of climate change education and communication.

Materials and implementation

Materials for the study consisted of the EcoChains: Arctic Crisis card game (described further below and in Deaton, 2015; Lee, 2020; Turrin et al., 2020; Wu & Lee, 2015), a six-minute instructional video about EcoChains produced to explain the rules and goals of the card game to treatment participants before playing it, the control intervention (illustrated article), three survey instruments (pre-survey, post-survey, and four-week follow-up survey), and a brief (6-question) discussion guide. The game used for the study was a limited research edition of a card game titled EcoChains: Arctic Crisis developed under the Polar Learning and Responding: PoLAR Climate Change Education Partnership (Pfirman, 2018). A 2-player version called EcoChains: Arctic Life is included in Supplemental Materials as a printable pdf, and is available for free download at https://ecochainsgame.com/free-print-and-play-game-deck.html. Through an iterative design process, our interdisciplinary team of environmental scientists, game designers, learning scientists, artists, and Arctic content experts created this game with three objectives: (1) using a simple design that is accessible, fun and easy to learn; (2) enabling a short gameplay suitable for both formal and informal learning environments; and (3) communicating the most important aspects of climate change in the Arctic marine ecosystem. Specifically the game addresses: the causes of carbon pollution, the linkages between the use of fossil fuel energy sources and the effects of global warming on the ecosystem, the diversity of species that form the Arctic marine food web and their predator-prey relationships, the reliance of some species on Arctic sea ice and therefore the effects of climate change on Arctic species beyond the iconic polar bear (Born, 2019), and steps people can take to reduce negative impacts on the Arctic.

Gameplay

In the two-to-four player EcoChains card game, each player is the steward of their part of the Arctic ecosystem, with responsibility for building their food web and responding to changes (see video in Deaton, 2015). The mission is to “play cards right” for a healthy Arctic. Players begin with a starter food chain that includes a predator that feeds off a prey source (see Figure 1), in this case, a ice copepod (predator) that eats ice algae (prey). As players take turns drawing species cards from the center of the table, they place the cards into their existing food chains based on their predator-prey relationships, indicated by representative species icons, or start a new chain. Some species cards also have a sea ice requirement that must be met before they can be played. For example, Arctic cod requires only one sea ice card at the base of the food web, while polar bears require three sea ice cards. Only if a player has three sea ice cards can the polar bear join the food chain.

Figure 1. Example of a linked predator-prey food chain in the card game EcoChains. The number in the small white circle on the bottom right indicates how many sea ice cards are required for each species. Event cards are played immediately and melt sea ice, while Action cards are played strategically to restore ice.

When an Event card appears, it impacts all players immediately. If a Carbon Pollution Event card is drawn, all players must “melt” sea ice by turning over two sea ice cards to reveal ocean water (melted ice) on the other side (see video in Deaton, 2015). This loss of ice impacts their food chain; species in the food chain that no longer have the requisite sea ice to survive must “migrate” to other food chains. In the example shown in Figure 1, a player must remove the polar bear and ringed seal from their affected food chain which no longer has sufficient sea ice to support the survival of these species, and ask the other players if the polar bear and seal can migrate to their food chains. If the species cannot find a new habitat, they are removed from the game. However, players can then choose to strategically play Action cards to reduce warming through the development of alternative energy, emissions reductions, and other events, and thereby restore sea ice. To restore ice in the game, sea ice cards are flipped back from the ocean side to the ice side.

Final scores are based on the number of species in viable food chains, the number of Action cards played, and for EcoChains: Arctic Life the cumulative number of sea ice requirements. The player with the highest number of points wins.

Evaluation

Design and strategy

A lab-based randomized controlled experiment was conducted to identify the impacts of EcoChains: Arctic Crisis on knowledge, attitudes, and beliefs of adult players (ages 18 and up), in comparison to a text-based control. Adults were recruited for the experiment as we wanted to assess communication options for current decision-makers, including parents and lifelong learners. We chose to compare the effects of two learning tools rather than using a “true control” (i.e., treatment vs. no treatment of any kind). Recall, EcoChains was created with the goal of providing an innovative educational resource that moved beyond the conventional approaches to teaching about climate change.

The control intervention was an article adapted and pilot tested specifically for this study. Originally a chapter in a book by the PoLAR PI Pfirman (2009), the control article “Changes in the Far North” (see Supplemental Materials), was re-written by the PoLAR Project Manager Brunacini to include information similar to that on the cards, including incorporation of some of the images used on the cards. The article is three pages long, and includes five images. We chose a magazine article format as the control because it is a form of media commonly used by the general population to acquire information. Adults not engaged in formal education obtain most of their information from informal sources such as television, newspapers, magazine articles and online sources (Rosenstiel et al., 2014). The magazine article format allowed us to retain greater control over the content design, allowing us to match as closely as possible the content provided by EcoChains.

Data sources

Evaluation participants completed three surveys: pre-, immediate post-, and four-week follow-up, to identify knowledge of and attitudes toward climate change and the Arctic region before and after the intervention. In addition, the post-survey examined participants’ impressions of the game-play/article-reading experience, and the follow-up survey included questions to identify longer-term retention of knowledge.

Questions that addressed knowledge, attitudes, and beliefs were included on the surveys to allow for analysis of change over time, including short term (immediately after the intervention) and longer term (four weeks after the intervention). Survey questions addressed:

1. Actual and perceived knowledge of climate change and the Arctic region

2. Attitudes and beliefs about climate change and its impacts

3. Impressions of and engagement with the intervention experience

Participants were asked to list up to three species that lived in the Arctic at three points in time: 1) at the beginning of the study (i.e., before the treatment or control intervention), 2) immediately after the study, and 3) four weeks later. At the immediate post and four-week follow-up, participants were specifically asked to list species they discovered while playing the game or reading the article. Responses were limited to three species for the sake of time and participant fatigue, as well as for efficiency of coding. The program developers and research team agreed that the top three responses were sufficient to capture participants’ ideas about Arctic species.

Data collection

In preparation for the study, Goodman Research Group, Inc. (GRG) collaborated with the EcoChains research and development team to design and pilot test instruments and the study design. During these discussions, we refined pre-, post-, and follow-up surveys, the six-minute instructional video, and the control article.

We pilot tested the surveys and the treatment and control intervention materials with students at Teachers College, Columbia University. At the end of each pilot test, a debriefing session was conducted with participants to obtain feedback on their understanding of the questions and any issues associated with their ability to complete the data collection instrument. Between the pilot study and GRG’s study, the study instruments and the treatment and control intervention materials were finalized.

Study population, setting, and procedure

After pilot testing in the New York area, GRG recruited a convenience sample of 50 adults in the Greater Boston, Massachusetts area using both GRG’s internal participant database and social media. The flyer that was distributed (see Supplemental Materials), sought people to participate in a group session to help with the development of a new educational game created at Columbia University. The flyer noted participation would involve reviewing materials with a group of other adults as well as answering questions about their experience and completing a follow-up survey online about one month later.

We asked prospective participants to list two times and dates they were available. Using that information, and no other descriptive data, we randomly assigned them into one of two groups: treatment and control. Due to no-shows, the study had a total of 41 participants.

Among the full sample (N = 41; treatment: N = 21; control: N = 20), the study participants ranged in age from 18 to 67 years, with an average age of 35 years. Over half (63%) were women and 37% were men. Two thirds (66%) identified themselves as Democrats and the remainder identified themselves as Independents (20%) or “other” (14%). No participants selected the option to identify themselves as Republican. With such a political identification participants’ political affiliations were similar to those in the Greater Boston Area. In February 2016, among the 382,946 registered voters in Boston, 54% were registered as Democrats, 7% as Republicans, less than 1% each were registered as Independent or Green-Rainbow Party, and 39% were not enrolled in a political party (Carraggi, 2016). We did not collect race/ethnicity data.

At the GRG office in Cambridge, we facilitated six group sessions of up to eight or ten participants over two weekends (see Supplemental Materials for a Master Facilitation Guide that includes the full pre- and post-surveys). The procedure started with an introduction to the study and consent forms (read by facilitators), followed by a pre-survey (non-facilitated). Participants sat at computers in the office, and completed the online pre-survey that was queued on each individual computer.

They then moved into a small conference room for the treatment or control intervention experience. The control group read the article individually for 20 minutes and then had five minutes to discuss their impressions amongst themselves, without the facilitator in the room. The treatment group watched a six-minute instructional video about rules and goals of the game, then were facilitated in playing the EcoChains game in groups of three to four. Research facilitators were instructed to conduct "hands-on facilitation" during game play. They circulated around the room and answered participant questions as they arose. After playing the game, which took 40 minutes, plus an additional five minutes for tallying points, participants had five minutes to discuss the experience amongst themselves without the facilitator in the room.

Each group then went back to the computers to complete the online post-surveys. Following survey completion, facilitators led a brief five-minute group discussion for participants to reflect on the experience. Questions were displayed on a slide and one facilitator led the discussion while a second facilitator took notes that were later coded and summarized. Facilitators spent five more minutes reading a conclusion to the session, explaining the purpose was to examine how audiences responded to the same content presented with different resources: a game or an article. Facilitators noted they would send a more detailed debrief after completion of a final online follow-up survey. Four weeks after the study, all participants completed the online follow-up survey and were provided a full debrief of the study.

Results

Pre-intervention comparison

We compared pre-intervention data between treatment and control groups using one-way ANOVA to assess participants’ self-reported knowledge, attitudes, and behaviors before the intervention. There was no statistical difference in knowledge or attitudes between the two groups, indicating that baseline knowledge and attitudes of the two groups were similar regarding the content of interest. On the pre-survey, all participants in both groups believed that climate change is happening, and the vast majority believed climate change is caused mostly by humans (73%).

Pre-post intervention comparison

Survey data were analyzed to assess change over time, as well as differences between the treatment group (intervention = EcoChains game) and the control group (intervention = article). As the primary analysis, repeated measures ANOVAs were conducted to examine change over time, by group, and any interactions between the two. Significant differences revealed at the p<.05 level are displayed and described.

Attitudes about climate change

On average, participants in both groups disagreed with the statement “Climate change impacts have been greatly exaggerated" (a question frequently used on surveys, e.g., Dugan, 2014) and they agreed it is likely to have a major impact on themselves and/or their household (Figure 2). There were no significant differences between groups and minor changes over time in each group were not statistically significant. Participants tended to believe the actions of a single individual could make a difference (i.e., disagreed that actions of a single individual won’t make a difference), and they were less certain about whether or not climate change is caused by a hole in the ozone layer. The majority of participants agreed that they contribute to climate change when they heat and cool their homes and drive their cars.

Figure 2. Climate change beliefs among all participants. Question: “How much do you agree or disagree with the following statements?” N = 41 Scale: 1 (Strongly disagree), 2 (Disagree), 3 (Neither disagree nor agree), 4 (Agree), 5 (Strongly agree).

Before the intervention, 73% of the full sample of participants agreed that climate change is happening and it is mostly caused by humans. Four weeks later, 85% reported that they held that belief. There were no significant differences between groups. For both the treatment and control groups, participation in the study confirmed or reinforced their existing attitudes toward climate change and the Arctic region. During facilitated group discussion following the intervention, participants were asked, “Do you think your attitudes toward the Arctic may have changed from [playing this game/reading this article]? Why or why not?” Participants in both groups explained while their attitudes did not change as they were “already aware” of climate change and its causes, the intervention reinforced their knowledge and beliefs, and encouraged them to learn more. One treatment group participant went on to express, “if I played it a few more times, would have stuck much more.”

Knowledge of arctic species

Participants were asked to list up to three species that lived in the Arctic at three points in time: 1) at the beginning of the study (i.e., before the treatment or control intervention), 2) immediately after the study, and 3) four weeks later. There were no significant differences between groups (treatment vs. control) either before or immediately after the intervention (Figure 3).

Figure 3. Species listed by study participants. N = 41. Species categories displayed were listed more than once (i.e., by more than one participant), while species categories that were only listed by one participant are grouped as “Other.” Before intervention question a) “Please list up to three species that live in the Arctic.” After intervention and four week follow-up question b,c,d,e) “Please list up to three species that live in the Arctic that you discovered by playing EcoChains/reading article.” a) Species categories listed by participants (full sample) on pre-survey before intervention. b) Species categories listed by participants (control: article) on immediate post intervention survey. c) Species categories listed by participants (treatment: article) on immediate post intervention survey. d) Species categories listed by participants (control: article) on four week follow-up survey. e) Species categories listed by participants (treatment: game) on four week follow-up survey.

We created 35 “species categories” based on pre, post, and follow-up responses. At pre, the 114 species listed by participants fell into 20 of those 35 categories (see Supplemental Materials). For example, algae was a category that we created, but was not mentioned at pre, only at post. Most participants (n = 36, 88%) listed three species, while two (5%) listed two species, another two (5%) listed one species, and one participant (2%) listed no species. The most frequently mentioned species were polar bears (32%), seals (18%), and (incorrectly, for the Arctic region) penguins (10%).

In the immediate post-survey, across both groups, participants listed a total of 116 species that fell under 20 of the species category labels, suggesting that they learned at least as many new species from the game/article as they originally listed on the pre-survey (Figure 3). Because participants were limited to listing three species on all surveys, we cannot comment on their full knowledge base. However, participants were asked specifically to list species they learned from the intervention experience; up to three was considered unlikely to be too limiting a boundary. The most frequently mentioned species were krill (16%), algae (15%), cod (9%), and plankton (9%), all of which were highlighted in the intervention materials (i.e., the game and the article).

Treatment and control participants performed similarly on this question in the immediate post-survey. The 21 treatment participants mentioned 57 species that fell under 16 species categories, and the 20 control participants mentioned 59 species that fell under 16 species categories. In the treatment group, 86% of participants listed three species, and in the control group, 95% listed three species. Compared to the pre-survey results wherein two species, polar bears and seals, accounted for half of the 114 species listed by participants, the immediate post-survey results suggest diversification of knowledge as the top two species listed were krill and algae; these accounted for less than a third of the 116 species listed by participants. During facilitated group discussion following the intervention, participants were asked, “Do you think you learned anything by [playing this game/reading this article]? If so, what do you think you learned?” Open-ended responses were coded and summarized; those in the treatment group self-reported believing that they learned about Arctic species more than did those in the control group. Treatment group participants explained many species were new to them, particularly the “lower-level species” and they learned more about the “interdependence of species” in the Arctic ecosystem. Control group participants, in contrast, highlighted learning about humans as another species that are “actually being harmed and becoming unhealthy now.” This was a new perspective for participants.

Asked on the four-week follow-up survey to recall species that they learned from the game/article, the 41 participants listed a total of 110 species that fell under 27 species categories, suggesting that they retained the diversity of knowledge demonstrated immediately after the intervention (Figure 3). The most frequently mentioned species among the full sample were polar bears (15%), plankton (13%), algae (12%), and seals (12%).

Between groups, treatment and control participants listed similar numbers of species and species categories at follow-up (treatment: 56 species, 21 species categories; control: 54 species, 19 species categories). However, the top two species recalled by the treatment group at four-week follow-up (plankton 20% and algae 13%) were species they had learned from the game and did not know prior to the game, whereas the top two species recalled by the control group at follow-up (polar bears 22% and seals 17%) were the top two species the group knew prior to reading the article; a much smaller proportion of participants had learned them from the article. This means that participants who played EcoChains demonstrated greater longer-term retention of new knowledge about Arctic species than did those who read the article.

Self-reported knowledge

Immediately after the intervention on the post-survey, participants reported their knowledge of particular content areas related to climate change generally, and the Arctic region and Arctic food chains, as well as how much they believed they knew before the intervention. This retrospective pretest (RPT) method was included to reduce potential response-shift bias, wherein there is a change in participants’ metric for answering questions from pre- to post-survey due to their new understanding of the concept being investigated (Klatt & Taylor-Powell, 2005), in this case, climate change, the Arctic region, and Arctic food chains. When self-reporting knowledge or attitudes, participants often use a different internal standard between ratings. The retrospective pretest method has been shown to reduce this shift, and leads to more accurate assessments of an actual intervention effect, relative to the traditional pretest-posttest method (Evalu-ATE, 2015).

When asked, on the post-survey, to reflect back to before the intervention (i.e., retrospective pretest questions), participants in both groups considered themselves fairly knowledgeable overall about climate change and less knowledgeable about the Arctic region specifically. Analyses revealed a statistically significant group by time interaction for two items. Both groups self-reported large perceived gains in knowledge of Arctic food chains after the intervention (p=.000, d'=.166); and there was a modest but reliable advantage for people in the game group (p=.035, d'=.08.) with their self-perceived gains increasing a little more than those in the article group. The effect size for change from before to after the intervention is considered large, while the effect size for the interaction is small, or modest, but statistically significant and reliable (Figure 4).

Figure 4. Perceived knowledge about the Arctic marine ecosystem. N = 41 (treatment: N = 21; control: N = 20). Question: How much do you know about each of the following topics: Arctic food chains?” Scale: 1 (Nothing at all), 2 (Only a little), 3 (A moderate amount), 4 (A great deal).

Participants also rated their perceived knowledge about ways in which they themselves can help protect the Arctic ecosystem (Figure 5). Those who played the game reported increased knowledge immediately after playing, while those in the control group reported less knowledge after reading the article (p=.037, d’=.07).

Figure 5. Perceived knowledge of ways to help protect the Arctic ecosystem. N = 41 (treatment: N = 21; control: N = 20). Question: How much would you say you now know about the following topics: ways in which you can help protect the Arctic ecosystem? Scale: 1 (Nothing at all), 2 (Only a little), 3 (A moderate amount), 4 (A great deal).

Engagement with the intervention experience

On the immediate post-survey, participants in both groups rated their agreement with several statements about their impressions of the game or article and how the experience made them feel (Figure 6). Analyses revealed statistically significant differences between the treatment and control groups in several areas, particularly those related to how engaging the experiences were. On a scale from 1 (Strongly disagree) to 5 (Strongly agree), more participants in the treatment group than the control group agreed that the content held their interest, and that they lost track of time. Those in the treatment group found the game more fun to play than the control group found the article interesting to read. While we recognize that “fun” and “interesting” are not the same, most people do not colloquially refer to a nonfiction article as being "fun" to read. “Interesting” is the more usual comparative language for enjoyment. In fact, in the brief discussion after the intervention, participants responded to the question, “Was the [game fun or interesting/article fun or interesting to read]?” Treatment group participants said the game was “fun” and they “would play again,” whereas most participants in the control group differentiated between the two words and explained the article was “Interesting to read. Not fun, but interesting.”

Figure 6. Experience with the intervention on immediate post-survey: significant differences by group. N = 41 (treatment: N = 21; control: N = 20) Question: “Please indicate your agreement with the following about this game/this article.” Scale: 1 (strongly disagree), 2 (disagree), 3 (neither disagree nor agree), 4 (agree), 5 (strongly agree). * indicates group difference was statistically significant (p < .05).

The treatment group also felt happier while playing the game than did the control group while reading the article, and less angry. Importantly, those who played EcoChains were also more likely to recommend the game to friends or family than those in the control group were to recommend the article to friends or family. Averages on the 5-point scale were statistically significant: 2.86 for the treatment group and 1.90 for the control group.

Overall, across all these items, the average ratings were 3.80 for the treatment group and 2.97 for the control group, (p=.00, d'=1.61), indicating a large and reliable difference between groups. Because no one identified as Republican we cannot assess differences across their political party viewpoint but analysis of those identified as Democrat vs. Independent vs. Other indicated no significant differences by group.

Discussion

A key finding of this study is that the EcoChains game was as effective as the article in teaching participants about climate change and the Arctic region ecosystems, and in some respects more effective in advancing knowledge – both actual and perceived. This indicates that gameplay can be at least as valuable as reading for teaching and learning about climate change. Considering the recent increase and popularity in gameplay, this finding is important. Playing a climate change game has the potential to reach different people than does reading an article, and to move beyond the already concerned public (Leiserowitz et al., 2010). Therefore, integrating educational games into climate communication and education strategies can potentially enhance learning within both formal and informal settings.

Regarding the learning observed in this study, both the treatment/game and control/article groups gained knowledge: learning about Arctic species, species interdependence, and environmental influences were similar between the two groups. On the post-survey, when asked to list species they discovered from the intervention and not just species they knew, both groups demonstrated similar gains in knowledge of Arctic species. Both groups also self-reported that their knowledge of topics such as climate change, food chains, and the Arctic increased after participation in the study. The treatment group perceived larger learning gains: on the immediate post-survey, the treatment group self-reported greater gains in knowledge about Arctic food chains. The treatment group also self-reported learning more about Arctic species during group discussion following the intervention than did the control group. Perception of learning has been found to be associated with liking and perceived usefulness of the educational intervention (Velada & Caetano, 2007).

A perception of usefulness, which is related to perceptions of efficacy and agency, was also seen in the responses of the game players to the question about knowing what they can do to address climate change impacts. Those in the treatment group self-reported greater gains in knowledge about ways to help protect Arctic ecosystems than did the control group. The option to play Action cards that restore sea ice by reducing carbon emissions or sequestering carbon from the atmosphere, comes up during gameplay. The images on the Action cards are of things people can do, such as efficient light bulbs, walking, and solar panels, which have been shown to promote personal efficacy (O’Neill & Nicholson-Cole, 2009). We did not ask specifically about feeling a sense of responsibility for action (e.g., Priest, 2016) or empathic concern (Young et al., 2018). However, when Event cards are played and players lose sea ice and therefore species dependent on ice, they frequently utter an exclamation, such as “Oh no” or “Awww” indicating that they were invested in the food web they had created. Physically playing the Action cards in response may give players a greater sense of agency (Ouariachi et al., 2019), in comparison with article readers who engaged with information about solutions in a more passive way. Also, positive empathy research indicates that believing an action will help a recipient, may result in people being more likely to act (Young et al., 2018).

On the four-week follow-up survey, when asked to list species they recalled learning from the game/article, the treatment group demonstrated higher long-term retention of the new species learned from the study, than did the control group. This increase in long-term retention of new information, as well as perception of agency, on the part of game players demonstrate that learning through playing EcoChains was indeed stickier (Gladwell, 2000; Inglis et al., 2014), than learning by reading the article. Furthermore, in the follow-up survey, the top two species recalled by treatment group participants as new species learned from the study were plankton and algae, compared with polar bears and seals for control group participants. Interestingly, the top two species remembered by the treatment group were the two species at the base of the food chains in the game, potentially because the gameplay required those species to be present before upper trophic levels could be added. Therefore these species were meaningful to game players, and their role made sense (e.g., Priest, 2016). While many people associate polar bears with melting ice due to warming climate (e.g., Born, 2019), this game broadened perspectives to include less-charismatic species at lower levels of the food chain. Thus game-based approaches may be useful for other ecosystems where the public focus is on the more well-known top predators.

The game-based educational approach of EcoChains was found to be more engaging than the traditional educational approach of the control article. Compared with the control group’s assessment of the article, the treatment group’s assessment suggests that the game is more fun vs. interesting for the article, and more engrossing than the article. This has implications for including games like EcoChains in both formal and informal learning settings where variations in pedagogical approaches are valued by both teachers and students (Martindale & Weiss, 2019).

The fact that people felt happier and less angry while playing the game than while reading an article is important, as it is likely to lead to repeat gameplay, which Wouters et al. (2013) found led to greater learning. Some participants noted during reflection in the group discussion immediately after the intervention, that they would like to play the game multiple times and believed they would learn more with each experience. Through observing thousands of EcoChains game plays (outside of this evaluation study), we have seen that strategy emerges with repeat game play which could mean that when not distracted by figuring out the rules, players have more time to focus on the content. Eisenack (2013) similarly observed that as groups played the KEEP COOL game repeatedly, the institutional arrangements they established became more complex which could reflect deeper understanding. This is another essential difference between playing a game and reading an article: people tend to play fun games repeatedly, which is likely to deepen learning through repetition and exploration of alternative outcomes. An article, while sometimes shared with others, is usually only read once by an individual.

Because increasing conversation with family and friends is important in broadening awareness of climate issues and actions (Hamilton, 2016; Maibach et al., 2016), it is noteworthy that people who played the game were more likely to recommend the game to others than people who read the article were likely to recommend the article to others. This means that engaging people in playing climate games like EcoChains could catalyze talking with others, beyond the game players, which would extend the impact of the experience (Priest, 2016).

Limitations

Climate change is one of the most polarizing political issues of our time (Hamilton, 2015), and a limitation to this experiment is that participants were skewed toward a Democratic party affiliation due to the demographics of the location where it was conducted. The study used a sample of convenience in the Metro Boston Area. The percentage of participants who agreed that climate change is happening and is mostly caused by humans was higher than the national average. Hamilton (2016) noted that the national average for agreeing climate change is happening and is mostly caused by humans is 63%. Additionally, Hamilton has found that these beliefs vary by political affiliation; Democrats more than Republicans tend to agree with the response/s: climate change is happening/mostly caused by humans. Since the study sample primarily identified as Democrat or Independent, it is not surprising to see the responses above national average responses. The study design did not intend to target this demographic and, while it may have influenced overall receptiveness to new information for the full sample, we focus here on differential effects of treatment and control interventions.

Another issue with recruitment is that the recruitment language asked for people “to help with the development of a new educational game.” Therefore, participants were likely predisposed to believe that educational games have value. An indication that participants were familiar with playing games is that there was not a statistically significant difference in the average rating on “The game was easy to learn/The article was easy to read.” On a scale from 1-5, the rating was 3.33 for the treatment group, and 3.80 for the control group. In addition, because the recruitment language mentioned one task would be “Reviewing materials with a small group of other adults,” the participants who responded were likely open to interacting with strangers.

Another factor is that Martindale and Weiss (2019) found a significant difference in enjoyment of and perceived levels of learning from an in class board game among racial groups: white students rated the experience higher, potentially due to more familiarity with tabletop games (Nicole, 2016). Therefore, in order to assess overall receptivity to climate-oriented games it is necessary to conduct a larger study with more neutral recruitment language, and a more diverse racial, ethnic and political participant group. Further research with a more diverse sample would provide more generalizable results, with participants likely to have more varied experiences with gameplay, background knowledge and political party affiliation.

The time on task was 20 minutes for reading the control text and double that at 40 minutes for the game-play intervention, plus 5 minutes for scoring. The additional time on task playing the game could help with learning. However, game players were also asked to do additional tasks during the 40 minutes as they needed to learn to play an unfamiliar game as well as interact with strangers, versus reading a brief article individually.

An additional issue is that this study relied on self-reported knowledge alongside questions about actual content knowledge. The content knowledge questions were adapted from previously used and validated surveys (Hamilton, 2015, 2016). While this is a common practice when collecting data via surveys, and self-reported knowledge may be a useful way to assess the construct of self-efficacy, a survey based more heavily on actual content knowledge assessment would strengthen the argument for knowledge gain attributable to the EcoChains game-playing experience.

Implications

As we consider ways to broaden engagement with climate change, we should include games in our portfolio of approaches. Games can promote long-term learning, they are designed to be engaging, they reach different audiences than lectures or publications, and they are often played in social situations. In addition to learning, well-designed games can promote empathy, personal agency, and spark conversations on climate change.

Currently, more than 3,000 EcoChains decks are in circulation and being used by families, schools, museums, and science centers. We have directly facilitated use with thousands of people ranging from 7 years olds to seniors, in informal settings such as museums and science fairs to the formal settings of middle, high, undergraduate and graduate school (see middle and high school guides in Supplemental Material, Turrin et al., 2020). Regarding use in schools, Martindale and Weiss (2019) observed that both students and educators appreciate the variety that games bring into the classroom, and Prince (2004) found that pairing brief activities with lecture, helps students remember more. For classroom use, as noted by Wouters and Van Oostendorp (2013), we recommend supplementing EcoChains gameplay with contextual instruction to address multiple learning styles (see also Griggs et al., 2009) and reinforce learning. For example, the control article “Changes in the Far North” could be assigned as reading. We also recommend having students play the game more than once, potentially in more than one group. Debriefing could include an immediate oral discussion, followed by a written reflection on what they learned through game play across groups and in relation to other resources, for example assigned text, media stories, etc. (Petranek, 1994). Furthermore, asking students to “mod” the game, developing some cards and rules of their own, would engage their creativity, potentially leading to enhanced motivation, deeper learning and strategy use, perceived competence (e.g., Vos et al., 2011), and higher order thinking as per the revised Bloom’s Taxonomy of applying, analyzing, evaluating, creating (Anderson et al., 2001). For example, students could research and develop Ocean Acidification (hint: would impact more than one species) or Overharvesting Event cards to explore their implications, as well as other Action cards such as Solar Radiation Management (hint: could bring back sea ice but ocean acidification would continue).

We also recommend using climate change games like EcoChains, as icebreakers (Eisenack, 2013) when getting together with relatives and at other social gatherings, as well at formal and informal events. In addition to being fun and creating a base of common knowledge (Reckien & Eisenack, 2010), games offer a venue for conversation that is helpful for broadening engagement in controversial topics such as climate change (Hamilton, 2016; Maibach et al., 2016), and, by definition, playing games together is a collective endeavor, which is important for scaling climate action (Priest, 2016).

Declaration of conflicting interests

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Acknowledgments

We thank Lisa Zaval for providing helpful feedback and suggestions. This research is based primarily on work supported by the National Science Foundation under grant numbers 1239783 and 1928235. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. D.R. wants to acknowledge support by the German Research Foundation (RE 2927/2-1). T.O. and D.R.: Research design and data collection for this article was undertaken whilst at the Center for Research on Environmental Decisions, Columbia University, New York, NY 10027, USA.