Visualizing climate change through LIFE-AMDRYC4: a VR mobile-based video game to educate adult audiences on sustainable agricultural practices

Abstract

LIFE-AMDRYC4 (2021) is a VR application designed for mobile phones combining informational design and gamification to stimulate critical reflection and awareness about agricultural practices and their effects on the environment. Agile production frameworks and researchers’ input in the design are discussed. A Likert-based questionnaire was employed to measure attitudes from academics (environmental sciences) and professionals (farmers). Qualitative data showed that academics were more concerned about the representational aspects of the game while professionals were more concerned about its design and performance. Positive significative correlation among items, including the dimensions of ‘enjoyment’ and ‘realism,’ was also part of the findings.

Introduction

Digital technologies, including those employing different forms of video games, are seen as a tool of great potential for environmental education and, particularly, for raising awareness of climate change and its consequences (Martínez-Rodríguez & Fernández-Herrería, 2022; Whitehouse, 2008; Wok, 2019; Wu & Lee, 2015). Research on educational video games has emphasized their immersive nature through interfaces, such as virtual reality (VR), augmented reality (AR), or any form of extended reality (XR). Combining the real world with multimedia allows the user to access more information about educational content while maintaining a sense of presence and greater user engagement (Lee & Hu-Au, 2021; Raybourn et al., 2019; Stanney et al., 2021).

In recent years, the fight against climate change has taken center stage in many environmental research initiatives. Most somber predictions about climate change include the disappearance of Mediterranean rainfed agriculture, with serious consequences of loss of arable land, desertification, migration, and famine. The role of environmental education in tackling climate change has been clearly outlined, without forgetting the ethical dimensions and the impact on international cooperation and the development of less favored nations (UNESCO, 2009, 2017).

The main aim of the LIFE AMDRYC4 project (LIFE16 CCA/ES/000123) is to study the adaptation to climate change of rainfed agricultural systems in the Mediterranean area. The pilot project is being carried out in the Murcia province of southeastern Spain (Figure 1), which has one of the highest risks of desertification compared to other areas in Europe. This is an international project involving stakeholders from Italy, Spain, and Portugal, and is informed by a perspective of Ecosystem-based Adaptation (EbA). The final objective of this project is to carry out organic agriculture in rainfed crops, which involves conservative agricultural practices, traditional approaches to biodiversity conservation, and ecosystems for sustainable socio-economic development.

Figure 1. (Left) Map of the area framed for the LIFE AMDRYC4 pilot project (LIFE16 CCA/ES/000123). (Right) Areas selected for the intervention and sampling were located in the region of Murcia (SE Spain).

The design of the app is inspired by common methods of digital design: agile methodologies (McInerney & Maurer, 2005) and Participatory Design (Khaled & Vasalou, 2014; Vieira et al., 2022). In contrast with other types of production frameworks, such as User Center Design (UCD), the agile frameworks define the creative product as the result of the dialog between the stakeholders - researchers in this case - and the designers. Evaluation is also part of the process but, unlike UCD methodologies, does not take place before the design process and does not condition it (McInerney & Maurer, 2005). Instead, agile methodologies are organized around iterations or ‘scrums,’ which are short meetings with the project managers and occasionally involve stakeholders.

Participatory Design (PD) employs audiences’ input into the design by employing different techniques, such as storyboards and brainstorming (Khaled & Vasalou, 2014). PD has been traditionally discouraged in relation to serious games in younger audiences because the lack of knowledge on areas related to game content (i.e. environmental sciences) and game design (Khaled & Vasalou, 2014, p. 93). On this occasion, however, only researchers and designers were directly involved in the PD. User evaluation was performed with the aim of adequately assessing the reception of the last version of the prototype, as well as informing future iterations of the app.

Design and development of the LIFE-AMDRYC4 application

Narrative design and gameplay

After some preliminary meetings with the researchers, a concept proposal for the product was pitched (March 2020). Our virtual experience, or video game, would present the user with a landscape that would change as a result of players’ decisions. Digital storytelling would be developed using a sequence of playable scenes and ellipsis. Parallel to this, the gameplay would simulate five years over four turns, a total of twenty years of landscape transformations. In this experience, the user can choose up to a maximum of four actions before skipping the turn. After each turn and at the end of the game, the player adopts the role of a farm manager and is informed of the consequences that their decisions have had on the environment, as well as the social and economic outcomes.

Level design

The game’s design consisted of a single level that combines realistic 3D representations and stylistic conventions from video games. Digital Elevation Models (DEM) of the terrain - a common tool in generating simulations (Henry, 2018; Kolb et al., 2018) - were used to reproduce the topography and vegetation of the chosen landscape. The level recreates one of the plots from the LIFE AMDRYC4 pilot project. In this plot, the following spaces are distributed to dynamically reflect the economic, environmental, and social transformations consequent to the user’s decisions:

• Crop fields: This section is populated with trees and lateral vegetation.

• Residence: A house designed in blocks reflects the economic development of the inhabitants of the farm.

• Livestock corral: This area allows the display of different species of livestock, and also, depending on the decisions taken, tourists visiting rural areas.

• Other facilities: These include waste containers, pruning, and common areas.

During the design process, the advice of the researchers was frequent and informed each scrum. They decisively influenced, for example, the asset design (e.g. animal manure containers) and the selection of photo-realistic textures (e.g. moist ground or soil rich in iron, plots after a desertification process, etc.).

To create a sense of presence in the user, the field of view (FOV) of the game camera was adopted to 110°, emulating first-person shooters and other first-person simulator games. Here, however, immersion is limited to three degrees of freedom (3DoF), an experience similar to that provided by 360° videos. To deal with this limitation, the areas where the game objects spawn had to be allocated closer to the camera and the user. Conversely, the environmental design had to achieve a compromise between accuracy and iconicity in the representation of these spaces (Figure 2). Their distribution obeys both story-world and gameplay.

Figure 2. Screenshot of the gameplay. The current game state displays a deteriorated residence as a reflection of the poor level of the variables related to socioeconomic development. Source: LIFE AMDRYC4 (Universidad de Murcia, 2021b).

Interfaces and UI design

The prototype can be played in two different ways: the tactile mode, interacting with device screens (mobile or tablet), and the virtual reality (VR) mode, in which the user employs an animated grid to interact with each menu, by holding their gaze for a few seconds. The application is designed to be used with inexpensive and easily accessible VR systems, such as Google Cardboard.

The user interface (UI) was designed according to the project identity guidelines as well as the derivative educational and promotional materials (LIFE AMDRYC4, 2020). Some elements were extra-diegetic or presented as a complement to the level design. For example, the year is indicative of the turn the user is playing while a thermometer on the screen displays the level of carbon sequestration as a direct cause of climate change.

Educational content

Research on educational strategies for climate change has identified ‘facts’ and ‘actions’ as a way to differentiate ‘science educators’ from ‘environmental educators’ as well as the different attitudes they may elicit in their audiences (Monroe et al., 2019, p. 792). In this sense, our research tries to work as a bridge between these two paradigms, using facts resulting from previous research conducted on soil contamination, and actions examined through the pilot studies under the LIFE AmdryC4 project, to address a bigger picture of the consequences of climate change. The educational component is conveyed through the development of individuals’ responsibility through the game, but also the visual and narrative representations of actual research data.

The choices available to the user (Figure 3) are based on dryland agriculture strategies (‘actions’) previously evaluated by the LIFE AMDRYC4 research project:

Figure 3. Screenshot of the ‘actions’ menu. Icons are designed for easy understanding of the agricultural strategies studied through the LIFE AmdryC4 pilot project. Source: LIFE AMDRYC4 (Universidad de Murcia, 2021b).

• Livestock waste: use of composted animal excrement.

• Sludge from treatment plant: use of composted solid waste from domestic water treatment plants.

• Pruning: Incorporation of crushed plant remains from pruning.

• Fallow: vegetative cycle without cultivating.

• Vegetation in lateral strips: planting of steppe herbaceous vegetation, such as esparto grass.

• Platforms: Modification of the elevated slopes through stratified terraces.

• Green fertilizer: use of fast seasonal growing plants and cultivation between rows of trees.

• When there are different modalities between the user options, as in the case of plow depth (deep/superficial), plow direction (perpendicular/sloping), and grazing (soft/intensive), these options are mutually exclusive.

Users are introduced to these options and their consequences through a visual dictionary shown in the main menu (Figure 4). This submenu also includes a description of the project goals that are intended to inspire the player’s journey:

Figure 4. Screenshot of the ‘visual dictionary’ that gives information about the actions available for the user and briefly introduces potential consequences of the gameplay. Source: LIFE AMDRYC4 (Universidad de Murcia, 2021b).

• 4/1000 Initiative: represents the organic carbon content in the soil (44p1000, 2018).

• Ecosystem services: refers to the benefits that nature brings to society.

• Circular economy: reducing the use of raw materials and the generation of waste.

• Mitigation of climate change: actions that reduce and limit greenhouse gas emissions.

• Adaptation to climate change: implement measures that reduce the vulnerability of an ecosystem to the adverse effects of climate change.

Variability and randomness

Simulation games are considered instances of a conceptual model that embeds a sociological system (Barreteau et al., 2007) in the sense that they emulate stories or experiences of social nature (e.g. a football match, the management of resources in a military campaign, etc.). However, in this case, more than simulating social constructs or experiences, this application focuses on disseminating a scientific model that explains the effects of agricultural practices on the environment and, eventually, on society as a whole. This information was delivered by employing visual metaphors, landscapes inspired by real geographical settings, and the same variables employed in the real scientific pilot research. Conversely, environmental games employing natural resource management (NRM) have been addressed as part of simulation game mechanics (Barreteau et al., 2007). We employed variables that are hidden from the users, so the results of their decisions throughout the game are communicated in terms of consequences (positive and negative) to the environment rather than presenting a complex table of results. The use of these variables are also reflected in emergent digital storytelling, adopting the following forms:

• Short-term effect variables: These are visible and direct consequences of user actions in one turn. For example, when a user chooses the direction of plowing, it changes on stage as the turn passes.

• Long-term variables: Several effects can take place in one or more turns. Their magnitude is determined by the interaction with other actions and their corresponding effects. For example, the use of sewage sludge and livestock waste both contribute to an increase in organic matter.

In both cases, the effect of these variables was designed in collaboration with the researchers and inspired by the hypotheses of the pilot project in three areas:

• Environmental: increase in organic matter, increase in soil moisture, carbon sequestration, and level of biodiversity.

• Social: increase in population and appearance of ecotourism.

• Economics: agricultural production and quality of work.

Finally, to create an illusion of complexity and plausibility in the narrative world, random variables were incorporated, such as the time of day, torrential rain, and the landing position of flocks of birds on the game level, among others.

Evaluation of the prototype and distribution

Following the agile methodology, results led to modifications on each iteration of the prototype. For example, the dictionary was improved through illustrations that follow the guidelines of the corporate identity of the project. Extra feedback messages at the end of each turn were added. These were all suggestions offered by researchers in different scrum meetings, creating, in each case, a new milestone for the development of the project.

While the experience was originally designed for mobile devices with the Android operating system, in a later phase of the project, it was decided to deploy it to iOS devices as well. In addition, the application was exported to WebGL, enabling the user to access the experience directly from the official website of the project. In June 2021, after a few months in the mobile application markets (Universidad de Murcia, 2021a, 2021b), the mobile application was evaluated using a questionnaire.

Methodology: user evaluation

Study design

The user evaluation was aimed to collect information about attitudes toward the product and its potential effectiveness, employing a Likert-type questionnaire, a common tool to evaluate serious games (Knol & De Vries, 2011; Patchen et al., 2020; Schneider & Schaal, 2018). Items were designed to reflect the immersive nature of the tool, its enjoyability, its playability, and ease of access. This questionnaire (Table 1) also included qualitative items to collect the users’ insights concerning these dimensions.

Table 1. Frequencies of the eight items included in the Likert-style questionnaire, which was employed as a user evaluation of the mobile application (N = 84).

	Academics (N = 64)					Professionals (N = 20)					Stats
Items	Totally agree	Agree	Neutral	Disagree	Totally disagree	Totally agree	Agree	Neutral	Disagree	Totally disagree	U	Z	p
1. I have enjoyed using the app. I would like to play again.	12	35	8	9	0	4	16	0	0	0	295.000	−0.451	.652
2. The instructions for use of the application (game tutorial) are clear enough.	24	27	9	4	0	8	8	4	0	0	309.000	−0.187	.852
3. I have learned something new as an effect of this 20-min play or as effect of previous play.	16	31	12	5	0	6	12	2	0	0	266.000	−0.925	.355
4. In relation to my knowledge of the different agricultural practices, I consider that the actions and consequences portrayed are plausible.	16	31	13	4	0	4	14	2	0	0	290.000	−0.516	.606
5. After this experience, I have a greater interest in knowing more about the environmental consequences of agricultural practices.	22	21	21	0	0	12	4	4	0	0	238.000	−1.377	.168
6. After this experience, I have a greater interest in knowing more about the economic consequences of these practices.	18	26	17	3	0	8	8	4	0	0	265.000	−0.922	.357
7. In relation to my previous knowledge of the objects and scenarios represented through the game, I consider that their representation is accurate or realistic.	13	27	18	6	0	6	10	4	0	0	250.000	−1.174	.240
8. In relation to my previous knowledge of the represented space (SE of Spain), I consider that the virtual experience (using cardboard) is immersive. That is, it is similar to the experience of being there.	9	21	23	9	2	2	6	12	0	0	312.000	−0.133	.894

The Mann–Whitney test revealed no significant (two-tailed) statistical differences among the groups.

The sample consisted of 84 volunteers, who were distributed according to their professional status into two groups (‘academics’ and ‘professionals’). The group of scholars included students, researchers, and volunteers from the Environmental Science Department (n = 64). The group of professionals (n = 20) consisted of professionals working in primary industries. These were contacted through research stakeholders, such as agricultural associations. For both groups, only individuals with low experience with VR devices (i.e. VR headsets) were considered while the rest were discarded. Following previous research (Patchen et al., 2020; Schneider et al., 2020), Cronbach’s alpha reliability test was used to judge the consistency of each item. The Cronbach alpha value was 0.823 for the eight items, indicating a high level of reliability for our survey. This questionnaire (Table 1) also included qualitative items that sought to collect the insights of the users concerning these same dimensions.

Hypotheses and data analysis

The questionnaire was intended to test whether there were any significant differences between the two groups (academics and professionals) regarding the following variables: immersion (item 8), realism (item 7), plausibility or the quality of seeming reasonable regarding the world logic (item 4), learning (item 3), ease of use (item 1), enjoyment (item 2), and its ability to arouse interest in socioeconomic (item 6) and environmental (item 5) problems. Therefore, the H0 (null hypothesis) for each item would be the following: ‘Belonging to a group doesn’t affect the variable.’ Likert scales were encoded and analyzed in terms of frequency, mode, and median. The Mann–Whitney method was employed to test these hypotheses.

Procedure for the prototype evaluation

Subjects could fill out the survey if they confirm that they had played the application for one entire game—four turns and ∼15 min. The results were then sent to the researchers by email for collection and subsequent analysis.

Results

General results

The opinions concerning the game among the individuals of both groups were mostly positive. Quantitative and qualitative data indicated positive results in terms of enjoyment and appreciation of the tool’s focus, especially in the dimensions of realism and plausibility of the simulations (Figure 5). The mode or most repeated value for most of the items of the scale was 4 (items 1–4 and 6–7), and was 5 for item 5 and 3 for item 8, which is equivalent to ‘Agree,’ ‘Totally agree,’ and ‘Neutral,’ respectively. The percentages of answers confirmed this (Figure 5).

Figure 5. Percentages of responses re-encoded to show differences among groups.

Answers were later re-encoded in terms of agreement/disagreement and neutrality, following common procedures for the re-encoding of Likert-based scales (Knol & De Vries, 2011). The opinions (Figure 5) were still mostly positive among groups although some items showed a higher level of disagreement (item 4) and neutrality (items 5, 7, and 8). However, the statistical test employed failed in pointing out differences between groups. Particularly, the Mann–Whitney U test didn’t show differences between groups (Table 1). On the other hand, there were significant correlations among the responses when both groups were considered (N = 84). The highest values include a positive appreciation of the general enjoyment of the experience (item 1) with the representational realism (item 7) (r = .49; p = .00); consideration of economic consequences (item 6) (r = .47; p = .00), consideration of environmental consequences (item 5) (r = .43; p = .00) and self-assessed learning (item 3) (r = .47; p = .00). Interest in economic and environmental aspects - responses to items 5 and 6 - also correlated positively (r = .75; p = .00). Finally, no correlation was found for items 7 and 8 with the rest of the survey.

User comments

Qualitative data was collected through user insights. After coding, a content analysis made from these comments (N = 119) was performed assigning data to nine different categories. Most of the comments can be understood in a similar way to users’ reviews and refer to problems the user has faced when using the tool. A significant percentage of the comments among groups (Figure 6) were related to the Performance of the System (11.87%), which may denote differences among users, depending on the platform (iOS, Android, or WebGL) and the interface used (desktop, mobile, or Cardboard). These qualitative results offer many useful insights for researchers and designers of this tool, not only in the form of explicit ‘suggestions’ (3.39%) but also through the different dimensions explored. For example, academics showed particular interest in discussing the plausibility of the tool (21.65%), commenting on processes such as carbon sequestration, and particularly, in the need for reflecting a wider diversity of positive and negative consequences to enrich future iterations of this environmental simulation. Qualitative insights on professionals obtained through the questionnaire showed an increase in awareness about the social and economic implications of these agricultural practices (‘May sound silly, but before this, I have never thought about how much the final quality of the soil - and its contents - could depend on previous practices’).

Figure 6. Number of comments found in the questionnaires (N = 119) among professionals (n = 22) and academics (n = 97).

Discussion

Professionals valued the tool very positively, although most of their comments tend to focus on the general experience rather than particular aspects of the educational tool. They highlighted the ‘novelty’ of the game and its capacity to effectively represent agricultural practices (Table 2). When considered as percentages, the responses among groups led to interesting insights for the researchers. Specifically, it can be seen a trend toward neutrality among professionals that avoid expressing disagreement. This is congruent with research that questions the use of Likert scales, pointing out how they may amplify social desirability and reduce the respondents’ willingness to differentiate among items responses. Some studies on attitudes employing Likert scales have found a positive bias in those scales in which the ‘agreement’ is presented at the left, or at the beginning in Western languages, and is considered a manifestation of the ‘primacy’ effect, or of acquiescence (Chyung et al., 2018; Keusch & Yang, 2018). This makes us consider balancing the design of future scales for the prototype evaluations. The use of Likert scales could also be replaced by conducting the data collection through qualitative methods, such as interviews or consumer panels (Bellotti et al., 2013). Relevant differences were found through the thematic analysis of users’ comments among groups. Academics showed more interest in making comments related to the levels of plausibility of the application, while professionals were more concerned about the UX (user experience) design, their access, familiarity with the different interfaces, and the general value of the experience provided by the tool.

Table 2. Distribution of comments found regarding the categories of analysis.

Topics	Professionals %	Academics %	Examples
User experience (UX) design and interfaces	18.18	5.15	‘Sometimes, I have problems finding the way to close the menus.’
Accuracy of the 3D models	9.09	8.25	‘It feels quite real. However, in the game, I had problems in assessing this aspect when the gameplay was set up during the night.’
General value of the experience	18.18	15.46	‘I found it quite didactic and a good way to explore the impact of these consequences in the region.’
Clarity of instructions	9.09	13.40	‘(Instructions) should be clearer and elaborate’
Plausibility	9.09	21.65	‘It feels as if there are no negative consequences of many of the actions.’
Performance of the system	9.09	13.40	‘My device (Android) isn’t powerful enough to gain full access to the experience.’
Personal interest in the topic	9.09	13.40	‘I’m more interested in the economic consequences of these practices.’
Access and familiarity with interfaces (virtual reality/tactile)	9.09	6.19	‘The VR mode made me feel dizzy. Those options meant I had to make extreme movements with the head to select the right options.’
Suggestions for future versions	9.09	3.09	‘A story mode would be a great addition, adding dialogues and greater elaboration as regards the actions chosen.’
Total	100 (N = 22)	100 (N = 97)

An example from the coding is included to better describe these groups of insights.

In this sense, it may be worth further examining the relationship between realism on representation (item 7) and general enjoyment of the experience (item 1), as well as the self-assessment of learning and the world’s plausibility (item 4). While correlations between these and other items were strong, such explorations may also justify taking different methodological approaches for a deeper exploration. This may be using purpose-designed tests (Kolek et al., 2021) or other qualitative approaches, such as interviews or focus groups (Bernhaupt, 2010).

This tendency towards neutrality through some of the items (e.g. item 8 on ‘immersion’) may also indicate a problem for the users in fully understanding the terminology used through the survey. For instance, the term ‘immersion’ originated in scholarship related to video games and virtual environments referring to ‘the experience of being transported to an elaborately simulated place’ (Murray, 1997, pp. 98–99). While the term has been popularized over the years, especially due to the rise and increasing access to the so-called ‘immersive technologies’ such as VR domestic devices, this does not necessarily imply that potential users of the tool will be familiar with it. The complexity of this kind of experience and the development of a larger corpus of studies on digital media have attracted some criticism of the term, and some authors suggest replacing it with more specific descriptions of the user’s experience in terms of engagement, presence, realism, plausibility, etc. (McMahan, 2013).

Conclusions

This article aims to report and critically reflect on the design, production, and evaluation of a mobile VR application experience as part of the LIFE AMDRYC4 Project research. The involvement of expert adult audiences (academics and professionals) in the design of this educational tool was examined using two complementary strategies. The first introduces the use of agile project management to inform game design using results from a previous research project. The second approach consisted of analyzing the results of the user evaluation questionnaires. The objective of these questionnaires was to measure acceptance by target audiences, explore their capacity to stimulate discussion about environmental problems, and collect qualitative information that could help future iterations of this prototype. The potential social impact of these approaches seems especially relevant in the case of the wider research project due to the involvement of multiple social agents and audiences, and overall, the global significance of the problems associated with the economic, social, and environmental consequences of climate change.

Future studies should examine more deeply the differences among expert audiences in terms of ideology and attitudes to key environmental issues (e.g. ‘climate change’). They can also consider previous knowledge about environmental issues to characterize in more detail the differences between professionals and academics. While this research narrates the process of designing an educational tool, the evaluation was conducted with informed audiences (professionals and academics). The next steps should include testing the application in higher education environments, such as universities, and particularly in courses on environmental studies. This would help to improve future versions of the tool and stimulate further discussions on sustainable agricultural practices.

Additional information

Funding

This research is part of the LIFE AMDRYC4 project which is an initiative funded by the European Commission (LIFE16 CCA/ES/000123).