Toward a Mid-Range Design Theory for Developing Pedagogically Effective Serious Games

ABSTRACT

Serious games (SGs) represent promising digital learning tools. However, SG design frameworks frequently lack a comprehensive integration of pedagogical considerations, which might explain the observed variance in SG effectiveness. To address this research gap, we use a design science research approach to develop a mid-range design theory that translates pedagogy into SG design. We exemplify and empirically evaluate the application of our design theory with an SG designed for intercultural competency development with good results.

KEYWORDS:

• Serious games
• design science research
• design theory
• experiential learning theory
• theory of multiple foci of intelligence
• flow theory

Introduction

Video games and gamification are experiencing ever-increasing popularity (Böckle et al., 2021; Hollebeek et al., 2021; Teng et al., 2022, 2022). Serious games (SGs) represent a subgroup of video games that are particularly noteworthy due to their high compound annual growth rate (18,5 %, Allied Market, 2022) and associated, continuously growing scientific interest (Zhonggen, 2019). They are largely educational video games (Carvalho et al., 2015) and have been used in various areas, including information systems (Calderón & Ruiz, 2015; Grund & Schelkle, 2019), management (Calabor et al., 2019), and medicine (Graafland et al., 2012). Furthermore, SGs are able to improve learning outcomes (Sitzmann, 2011; Vogel et al., 2006) while enabling a motivating (Grund & Schelkle, 2019; Wouters & Van Oostendorp, 2017) and active (Boyle et al., 2016) learning experience, which is also why ‘most agree SGs to have strong potential for learning’ (Arnab et al., 2012, pp. 159–160).

However, while the evidence regarding their educational prowess seems promising (De Freitas, 2018), SGs are not effective by default (Watt & Smith, 2021). For example, meta-analytic research has revealed insignificant study effect sizes when using some SGs in a university context in the past (Lamb et al., 2018), and there is also evidence that they are not necessarily effective in developing motivation and communicative learning outcomes (Wouters et al., 2009), although other studies investigating single SGs have found them to be highly motivating (e.g., Treviño-Guzmán & Pomales-García, 2014) and helpful in terms of developing communication skills (e.g., Guillén-Nieto & Aleson-Carbonell, 2012). Given that SGs are very costly to develop (Nadolski et al., 2008), it is thus important to design them in a way that they achieve their intended learning outcomes (Watt & Smith, 2021), which requires integrating pedagogical considerations into the design process (Ravyse et al., 2017). While the connection between pedagogical (e.g., how people learn) and game-related (e.g., how games are designed) considerations has been investigated by research in the past (e.g., Arnab et al., 2015; Starks, 2014), a great deal of research addressing SG design neglects it (e.g., Abeele et al., 2011; Barbosa et al., 2014), and even research emphasizing the importance of pedagogy in the SG design process is frequently too generic (e.g., Kiili, 2005; Starks, 2014), lacks proper learning theoretical foundations (e.g., Nadolski et al., 2008; Westera et al., 2008) or is not empirically evaluated (e.g., Arnab et al., 2015; Bellotti et al., 2011; De Freitas & Neumann, 2009).

Therefore, the main objective of this paper is to advance explanations in relation to learning effective SG design by introducing and empirically testing a design theory (Gregor & Jones, 2007) that translates learning theoretical insights into game design principles. The design theory was developed using a design science research (DSR) approach, which is a promising but underexplored avenue in SG research (Braad et al., 2016; Grund & Schelkle, 2019). DSR is a problem-solving process in which design artifacts or theories (amongst other outcomes) are developed and evaluated in order to improve organizational capabilities (Baskerville et al., 2018; Hevner et al., 2004). It has to be added—for the sake of clarity—that we will use the word “artifact” exclusively when referring to material artifacts such as a piece of hardware or software (Baskerville et al., 2018), although design theory and models are also sometimes referred to as (abstract) artifacts (Gregor & Jones, 2007). Furthermore, design theories are usually mid-range theories or ‘conceptual intermediaries between the highly abstract space of potential problem solutions suggested by kernel theories or insights and the concrete problem solution of the implemented artifact’ (Kuechler & Vaishnavi, 2012, p. 398).

In order to represent and test our design theory (Gregor & Jones, 2007), this article also introduces an SG in the field of intercultural competency development as the expository instantiation of our design theory. Intercultural competencies are defined as the ability to function effectively in another culture (Bird et al., 2010), and we chose to develop an SG in this area for two reasons. First, intercultural competencies are an important success factor, not only for introducing information systems internationally (Kummer et al., 2012), but also in the fields of global software development (Vallon et al., 2018) and global IT outsourcing (Sahay et al., 2003), and for understanding technology adaptation (Zhang et al., 2018) and e-commerce consumption behaviors across borders (Gupta et al., 2019). This makes a culturally competent workforce a necessity for all organizations working in this environment (Dowling et al., 2017). Second, intercultural competencies comprise cognitive, meta-cognitive, affective, and behavioral competency dimensions (Ang & Van Dyne, 2015), thereby allowing us to test the efficacy of our design theory with an SG aiming to develop a highly complex skill involving multiple and very different competency dimensions (Kolb et al., 1986).

Summarizing, this article aims at contributing to the debate about how to integrate pedagogy into SG design (e.g., Kiili, 2005; Watt & Smith, 2021). However, instead of translating single pedagogical principles into game mechanics (Arnab et al., 2015), teaching pedagogical experts about game design (Theodosiou & Karasavvidis, 2015), or developing generic and rather abstract SG design models and principles (Kiili, 2005; Ravyse et al., 2017; Watt & Smith, 2021), we introduce a comprehensive mid-range design theory that importantly accounts for the specific learning objectives an SG sets out to achieve (Romero et al., 2015). In addition, we also evaluate it empirically, which most of the related work to date has not done (e.g., Ravyse et al., 2017; Watt & Smith, 2021).

The remainder of the paper is structured as follows. We begin by presenting the research background to our study. Next, we explain the underlying methodology used in this paper and present our design theory. The following section describes the SG we developed, and thereafter we provide empirical game evaluation results based on a qualitative study, in order to outline and discuss to what extent the intended learning goals, in line with the principles of design science, have been reached (Baskerville et al., 2018; Hevner et al., 2004; Venable et al., 2016). Last, we discuss our contributions, limitations, and implications for future research before presenting some practical insights and concluding the article.

Research background

In the following, we provide an overview of the relevant literature by discussing the different learning outcomes of SGs as well as existing SG development frameworks, all of which help understand the possibilities and limitations of both what an SG can contribute to education as well as our current knowledge about how it can be designed.

Serious games and learning outcomes

SGs can focus on cognitive, affective, and behavioral learning outcomes (Garris et al., 2002; Lamb et al., 2018)—as meta-analytic studies indicate (Boyle et al., 2016; Clark et al., 2016; Connolly et al., 2012; Lamb et al., 2018; Riopel et al., 2019; Wouters et al., 2013; Zhonggen, 2019) These studies demonstrate that SGs are effective in improving players’ cognitive abilities and in achieving affective learning outcomes (Zhonggen, 2019) while also inducing behavioral changes and leading to hard and soft skills development (Boyle et al., 2016). However, there is also some evidence suggesting that behavioral outcomes might be weaker than the cognitive and affective outcomes of playing SGs, although they still exist (Lamb et al., 2018). Given that SGs can target highly differentiated learning outcomes, ranging from compassion (Rončević Zubković et al., 2022) and software programming (Miljanovic & Bradbury, 2018), to decision-making capabilities in the context of climate change (Flood et al., 2018) or even creativity (Agogué et al., 2015), they are highly relevant for the educational environment in general and also for learning in the field of information science in particular.

Serious game development frameworks and learning

Several design frameworks exist to develop SGs (e.g., Arnab et al., 2015; Carvalho et al., 2015; Kiili, 2005). While most of them underline the importance of matching learning objectives and pedagogical insights with game design, it still remains unclear how to systematically tie both aspects together (Watt & Smith, 2021). Existing research has focused on the interplay between pedagogy and SG design by elaborating what game designers can learn from insights into experiential learning theory (Rooney, 2012) or by mapping learning and game mechanics for SG analysis and design (Arnab et al., 2015). While this research contributes to our understanding of how to translate single pedagogical principles into game design, they are very generalized in their scope, and thus they neglect the role that the specific type of learning objective plays in this regard (Watt & Smith, 2021). This is an important factor, as research shows that the learning goals of a game affect its characteristics. For example, SGs seeking to increase the player’s creativity include expressive choices (e.g., choosing an avatar with certain characteristics), while those looking to develop critical thinking skills feature tactical choices (e.g., making decisions) (Romero et al., 2015). Further, the majority of existing SG development frameworks do not consider theoretical models of psychological development (Ravyse et al., 2017), thereby neglecting important insights from learning research, which in turn might cause those SGs to be ineffective or less effective than they could be (Watt & Smith, 2021; Westera, 2019): ‘[…] the ambition to make learning more attractive and joyful should never go at the expense of learning effectiveness […]’, which is why ‘[…] serious game design should explicitly base its design on advances and evidence in learning sciences research’ (Westera, 2019, p. 65). A comprehensive review of how existing SG design frameworks incorporate a learning theoretical foundation as well as learning objectives can be found in Appendix A, which confirms the aforementioned lack of integrated pedagogical considerations (Watt & Smith, 2021). There are, however, three exceptions that need further analysis. The first is the experiential gaming model developed by Kiili (2005), which utilizes experiential learning theory to describe a cyclical gameplay process also involving flow theory. However, while this model emphasizes the importance of learning research, it is very generic and reduces SG design flexibility by elaborating a predetermined gameplay process. Second, the cognitive behavioral game design model by Starks (2014) is rooted in social cognitive theory and outlines game attributes such as narratives, goals, or graphics. While this model provides a neat overview that supports the analysis of SG design, it lacks a clear structure that can help inform game development. Last, Watt and Smith (2021) use social learning theory to deduce serious game design factors such as cooperative play, but it is strictly limited to multiplayer SGs, which ultimately limits the scope of their insights. Thus, new insights are needed that systematically integrate learning theory as well as learning objectives and can help game design researchers and practitioners understand how to develop effective SGs.

Methodology

Before we move on to introducing our design theory, it is important to clarify what exactly we aim to contribute with this article as well as how the DSR process unfolded in this project (Baskerville et al., 2018; Sonnenberg & Vom Brocke, 2012).

Contributions in DSR

While DSR contributions are varied (Peffers et al., 2018) and can encompass science and technology evolutions, design artifacts, design theories, and DSR processes and impacts (Baskerville et al., 2018), it is particularly the nature of design theories that has been subject to debate in the DSR community and beyond (e.g., Gregor, 2006; Iivari, 2020; Venable, 2013), leading researchers to conclude that ‘the concept of “theory” is a nebulous one’ in DSR (Iivari, 2020, p. 502). Thus, in a first step, it is important to specify what we mean when we speak of a “design theory.” At this point, we would also like to acknowledge the debate surrounding whether or not a design theory should be called a “theory” or merely “design knowledge” (Gregor & Hevner, 2013; Iivari, 2020; Weber, 2012), which—for the sake of ease and clarity—we will not add to and instead continue talking about a design theory as our primary contribution.

IS theories can be divided into theories for analyzing, explaining, predicting, explaining and predicting, and for design and action (Gregor, 2006), with the latter encompassing design theories in the context of DSR (Baskerville et al., 2018; Gregor & Jones, 2007). However, there are different interpretations of what exactly constitutes a design theory (Iivari, 2020). For example, the work by Walls et al. (1992) can be considered foundational for design theorizing (Baskerville et al., 2018; Gregor & Jones, 2007; Iivari, 2020; Peffers et al., 2018), and it presents kernel theories, meta-requirements, and meta-design as important components of a design theory. Other conceptions of theories (Iivari, 2020) include the works by Venable (2006), Gregor and Jones (2007), Baskerville and Pries-Heje (2010), and Niehaves and Ortbach (2016), all of which introduce a slightly different version of what a design theory should entail, involving different components (e.g., Gregor & Jones, 2007) or a different focus (e.g., Venable, 2006) than discussed in Walls et al. (1992). In this article, we have chosen to incorporate the perspective of Gregor and Jones (2007), because it is one of the most comprehensive conceptualizations of design theories incorporating not only kernel theories and artifact design, but also the effects of the developed artifacts (Iivari, 2020). In addition, it has been foundational for other DSR-related research focusing on DSR evaluation (Sonnenberg & Vom Brocke, 2012) or on structuring associated articles (Gregor & Hevner, 2013), making it an established “anatomy” for design theories (Gregor & Jones, 2007). Rooted in the work by Walls et al. (1992), Gregor and Jones (2007) propose eight components of a design theory: The (1) purpose and scope describe the set of meta-requirements or goals that specify the type of artifact to which the theory applies, (2) constructs represent the entities of interest in the theory, and (3) the principles of form and function cover the “blueprint” or general architecture of an artifact. In addition, (4) the artifact mutability encompasses the degree of artifact change that is encapsulated in the theory, while truth statements about the design theory are included as (5) testable propositions. Furthermore, theoretical foundations for the design theory are introduced as (6) justificatory knowledge, and the last two are optional components, including (7) principles of implementation, e.g., how to implement the theory, and (8) expository instantiation, e.g., a physical instantiation or prototype of the theory.

The DSR process

In order to develop our design theory, our research involved multiple design cycles relating to artifact design, implementation, evaluation, and refinement, and it was rooted in the build-evaluate pattern presented by Sonnenberg and Vom Brocke (2012). This pattern comprises four DSR activities, namely, problem identification, design, construction, and usage, with each DSR activity being followed by an evaluation. We followed this pattern in our research project, starting with problem identification, which we found in some SGs being ineffective (Lamb et al., 2018) and which can be attributed to the described disconnect between SG design frameworks and pedagogical considerations (Watt & Smith, 2021) (problem identification). After identifying this problem and corroborating its relevance by taking a closer look at the existing literature, we set out to develop a comprehensive design theory rooted in the work of Gregor and Jones (2007) that would help develop pedagogically effective SGs (design). Subsequently, the development phase (construction) involved creating an expository instantiation in the form of an SG aiming to develop intercultural competency, as well as multiple formative evaluations (Venable et al., 2016) with students and fellow researchers. These evaluations sought to improve the quality of the game and comprised an early alpha test after three weeks of development to evaluate the interface design and early representation of the game world, an alpha test focusing on usefulness (e.g., Did students perceive the combination of virtual experiences in a structured setting helpful for their concept understanding?) and user experience (e.g., How enjoyable was playing the game? Which emotions came up?), and multiple beta tests aiming at further improving effectiveness, user experience, and technical stability. These testing cycles were instrumental not only in improving the quality of the SG, but also in confirming that developing games with the help of our provided design knowledge seemed to help in creating pedagogically effective SGs. The last step in the build-evaluate pattern comprised implementation of the SG into different courses (use), the respective summative evaluation (Venable et al., 2016) for which is presented later on in this paper.

A design theory of pedagogically effective serious games

In the following section, we present the first seven components of our design theory (Gregor & Jones, 2007), before we introduce our expository instantiation in more depth in section 5.

Purpose and scope

The main purpose of our design theory is to provide design knowledge that helps create pedagogically effective SGs, by which we mean SGs that achieve their intended learning outcomes. By doing so, we hope to support researchers as well as practitioners in developing SGs that can then be used for different educational settings (e.g., in higher education) or to conduct further research about serious game-based competency development. This overarching purpose necessarily requires taking a closer look at three different aspects: (i) how learning generally unfolds and how individuals acquire specific competencies (Kolb, 1984; Kolb et al., 1986), (ii) how this is reflected in the different game elements (e.g., game mechanics or dynamics) (Hunicke et al., 2004), and (iii) how SGs are designed so that users are actually able to engage with them, because they can focus on playing the game or enjoy playing it (de Almeida & dos Santos Machado, 2021). Based on these three aspects, as well as by incorporating additional literature addressing game-based learning (e.g. Böckle et al., 2021; Romero et al., 2015), we derived a set of three meta-requirements, presented in Table 1. These meta-requirements typify a whole class of artifacts—pedagogically effective SGs— (Gregor & Jones, 2007) and they encompass not only how learning unfolds and how this is reflected in different game elements (Starks, 2014), but also account for the user experience, given that SGs that do not appeal to users in one way or another will likely undermine their potential in achieving desirable learning outcomes (e.g., de Almeida & dos Santos Machado, 2021).

Table 1. Meta-requirements.

Meta-requirement 1	The SG should enable a learning experience that is in line with our knowledge of how learning unfolds.
Meta-requirement 2	The SG should consist of different game elements that are best suited to achieve intended learning outcomes.
Meta-requirement 3	The SG should provide a pleasant user experience that allows players to focus on and enjoy playing it.

As such, the meta-requirements reflect existing research about the learning effectiveness of SGs, which suggests that ‘[…] the key challenge for effective learning with games is for the learner to be engaged, motivated, supported and interested but also importantly for the learning to be undertaken in relation to clear learning outcomes […]’ (De Freitas, 2006, p. 5) and that ‘[…] serious game design should explicitly base its design on advances and evidence in learning sciences research’ (Westera, 2019, p. 65). We also incorporated into our meta-requirements the perspective of Romero et al. (2015), who concluded that ‘[…] certain skills are better developed within certain categories of games’, meaning that there is a close interrelationship between the type of learning outcomes one wants to achieve with an SG and the game’s characteristics (e.g., game elements or genre). In addition, the meta-requirements are also closely aligned to requirements formulated in related fields, such as gamification-based competency development (Böckle et al., 2021), as well as to the proposed elements that make e-learning effective (Brown & Voltz, 2005).

In sum, the class of artifacts defined by our meta-requirements spans many different contexts, including various educational stages as well as different types of learning outcomes, which makes our theory applicable and relevant for numerous areas. However, while it can be applied to developing SGs for learners in different educational stages, this will have substantial implications for the actual game design (e.g., pace of progression, complexity), which needs to be accounted for in the design process.

Justificatory knowledge

This section introduces the “kernel” theories (Walls et al., 1992), or the justificatory knowledge (Gregor & Jones, 2007) we have integrated into our design theory, in order to provide an ‘explanation of why an artifact is constructed as it is and why it works’ (Gregor & Jones, 2007, p. 328). We chose these theories because of their capability to meet the meta-requirements that we introduced in the previous section and they are summarized in Table 2. With the help of these theories, we can deduce design requirements, which are foundational for developing design principles and the artifact’s blueprint in section 4.3 (Böckle et al., 2021; Gregor & Jones, 2007). Before presenting the design requirements, however, we would like to briefly provide a rationale as to why we have chosen them, as they are not the only ones suitable to meeting our meta-requirements.

Table 2. Kernel theories.

Theoretical foundations	Theory description and usage in the development of our design theory
Experiential learning theory (ELT)	ELT was developed by Kolb (1984). It defines learning as the process whereby knowledge is created through the transformation of experience, and it is frequently associated with SG-induced learning, although this connection is underexplored. We have used the six characteristics of ELT in order to develop design requirements and design principles for SG design.
Theory of multiple foci of intelligence	The theory of multiple foci of intelligence, developed by Sternberg and Detterman (1986), offers a theoretical perspective on intelligence that incorporates four complementary dimensions that make up an individual’s intelligence: meta-cognitive intelligence, cognitive intelligence, motivational intelligence, and behavioral intelligence. We used this theory in combination with ELT in order to relate existing game components to the learning outcomes of an SG.
Flow theory	Proposed by Csikszentmihalyi (1990), flow theory describes a subjective state or experience in which individuals are fully focused on and enjoy pursuing an activity. As such, research suggests that flow constitutes an important indicator for the quality of an SG’s user experience (Kiili et al., 2012), which is why we incorporated flow theory in order to account for the user experience when developing SGs.

Starting with experiential learning theory (ELT), which has been described as a ‘holistic integrative perspective on learning that combines experience, perception, cognition and behavior’ (Kolb, 1984, p. 21), we have chosen a learning theory that is very influential in SG design research (Gouveia et al., 2011) and beyond (Kayes, 2002). While other learning theories such as social (Watt & Smith, 2021) or situated learning theory (Krath et al., 2021) have also been described as important for SG design, ELT is considered foundational for many other learning theories (Rooney, 2012), thus making it relevant beyond the theory itself. In addition, ELT has been characterized as holistic, as it encompasses the learning modes of experiencing, reflecting, thinking, and acting, which learners can—but do not usually—rely on in their entirety (Li et al., 2013). This implies that ELT is able to capture the individual learning process on a generic level, without neglecting learning preference diversity (Kolb, 1984), which is particularly important when designing SGs seeking to appeal to a diverse audience.

Next, the theory of multiple foci of intelligence introduces four complementary ways of conceptualizing individual-level intelligence, and it has been developed by integrating different views on intelligence (Sternberg & Detterman, 1986). It consists of (1) cognitive intelligence, which encompasses the actual knowledge and knowledge structures of an individual, (2) meta-cognitive intelligence, which represents the knowledge and control of cognition, also including an individual’s ability to develop mental strategies for solving problems, (3) motivational intelligence, or an individual’s motivation to engage in a specific task or endeavor, and (4) behavioral intelligence, e.g., the capabilities of an individual on the behavioral level (e.g., actions) (Sternberg & Detterman, 1986). We chose this theory primarily because the intelligence dimensions cover highly different types of intelligence (e.g., cognitive and behavioral intelligence), thereby making it applicable for various learning outcomes; moreover, it is broader than other theories covering learning outcomes (e.g., Krathwohl, 2002). In addition, all of the four intelligence dimensions introduced by Sternberg and Detterman (1986) can be developed (e.g., Ang & Van Dyne, 2015), thereby allowing them to be targeted effectively with educational SGs.

Last, we chose flow theory as the theoretical basis for ensuring that the designed SG would provide a good user experience (Kiili et al., 2012) in which users are able to focus on and enjoy playing the SG (de Almeida & dos Santos Machado, 2021). According to Kiili et al. (2012), the user experience emerges from the interplay between three different elements, namely users, an artifact, and a task: the user has certain characteristics (e.g., emotions and values) and uses an artifact (e.g., SG) to accomplish a specific task or goal (e.g., challenges in the game). These elements together create the user experience, and if tasks and artifacts are designed so that they meet the user’s needs, these users are able to enter a state of flow when playing an SG (Kiili et al., 2012), which is also why ‘flow theory can be adopted for measuring user experience and analyzing the quality of serious game designs’ (Perttula et al., 2017, p. 57). Flow is described as a pleasurable state of optimal functioning whereby individuals are fully absorbed by an activity (Csikszentmihalyi, 1990), and it has been found to contribute to learning gains when using SGs (Brom et al., 2014) while also representing a good indicator for evaluating the overall quality of the gameplay (Kiili et al., 2014). In addition, flow has been suggested to result from the right balance between challenge and skill (Kiili et al., 2012), and players that experience a state of flow while playing SGs are highly focused, feel a sense of control, have a distorted experience of time, and perceive the activity as innately rewarding (Brom et al., 2014). Given that experiencing a state of flow is not only predicated upon good usability (e.g., clarity of interface design), but also accounts for the emotional side of using a product (Kiili et al., 2012), it is also particularly suitable for a context that is well known for arousing various emotions, such as playing a video game (Granic et al., 2014). Thus, it constitutes a good proxy for evaluating the user experience of an SG (Kiili, 2005); in the following, we therefore use ELT, the theory of multiple foci of intelligence, and flow theory to deduce the design requirements for our design theory (Böckle et al., 2021).

According to ELT, learning can be characterized by six propositions (Kolb, 1984). First, it is best conceived as a process and should not be viewed in terms of outcomes. This proposition implies that learning represents an ongoing activity in which individuals constantly modify ideas and habits, instead of acquiring fixed ideas about certain phenomena, which according to Kolb (1984) is even maladaptive. Instead, in line with the perspectives of Freire (1970) and Bruner (1966), Kolb (1984) emphasizes that learning outcomes only represent historical records and not knowledge of the future, which suggests that every single learning experience precedes another. Simultaneously, the second proposition of ELT highlights the continuity of learning by describing it as continuous process that is grounded in experience (Kolb, 1984), meaning that while every learning experience precedes another, it simultaneously also builds upon one. Applying these two propositions to SG design entails designing them in such a way that they integrate past learnings (e.g., different levels of experience) (e.g., Hendrix et al., 2018) while also not limiting the player’s capabilities to learn from future experiences (e.g., experiences made after playing the SG) (Zumbach et al., 2020).

Design requirement 1: An SG should support players in their individual learning process by accounting for the player’s past experiences as well as by not limiting their capabilities to learn from future experiences.

Importantly, however, while conceptualizing learning primarily as a process, ELT does not deny the existence of learning outcomes (Kolb, 1984). In fact, its sixth proposition even suggests that learning is the process of creating knowledge, which is a measurable learning outcome, and given the substantial amount of research emphasizing the importance of defining clear learning outcomes when developing SGs (e.g., Romero et al., 2015; Watt & Smith, 2021), this should also be reflected in one of the design requirements. Given the fact that SGs can achieve very diverse learning outcomes, including, for example, behavioral change (Boyle et al., 2016), proposition six of ELT (Kolb, 1984) is too narrow when considering its pronounced focus on advancing knowledge. Instead, we have chosen to incorporate the theory of multiple foci of intelligence (Sternberg & Detterman, 1986), not only because it also integrates affective and behavioral learning outcomes, thereby having a broader scope than ELT (Kolb, 1984), but also because it still allows to uphold a procedural perspective on learning, thereby complying with the propositions of ELT (Kolb, 1984). For example, SGs have been shown to be able to develop intercultural awareness (Guillén-Nieto & Aleson-Carbonell, 2012), which represents a facet of meta-cognitive intelligence (e.g., understanding that communities have different value orientations) (Ang & Van Dyne, 2015; Sternberg & Detterman, 1986). Utilizing a procedural approach to developing it means (a) accounting for the player’s previous learnings (e.g., degree of intercultural experience, existing intercultural awareness), (b) helping them to make sense of the game while playing it (e.g., use intercultural awareness in conversations with other characters), and (c) enabling them to learn from future intercultural encounters (e.g., when traveling to a country, where one does not know the culture), all of which can be done in SGs (e.g., Lopes & Bidarra, 2011; Zumbach et al., 2020).

Design requirement 2: An SG should have clearly defined learning outcomes, which can be defined with the help of the theory of multiple foci of intelligence.

Next, proposition 3 of ELT suggests that learning requires the resolution of conflicts between dialectically opposed modes of adaptation to the world (Kolb, 1984). According to Kolb (1984), there are four different learning modes, namely concrete experience (feeling), reflective observation (observing), abstract conceptualization (thinking), and active experimentation (acting), and learning tends to be specialized around the learning modes with which an individual engages. For example, an individual engaging abstract conceptualization (e.g., reading a scientific text) is more likely to develop thinking skills or cognitive intelligence, while engaging active experimentation (e.g., talking to another person) is more prone to developing behavioral intelligence (Kolb et al., 1986; Sternberg & Detterman, 1986). Thus, SGs should enable players to employ those learning modes that are best suited to developing their desired type of intelligence. For example, a game aiming to improve a player’s knowledge about different database management systems (cognitive intelligence) should allow students to think about them (abstract conceptualization), for instance by incorporating a task in an SG in which the player has to decide which database management system to implement in a company.

Design requirement 3: Depending on the exact learning outcomes an SG sets out to achieve, it should enable players to engage the different learning modes accordingly.

Proposition 4 of ELT suggests that learning is an adaptation process in which individuals constantly adjust to a changing world (Kolb, 1984). For example, the artificial intelligence chatbot chatGPT has been recently described as having substantial power to advance academia and librarianship (Lund & Wang, 2023) and to transform healthcare (Goodman et al., 2023) in both ‘anxiety-provoking and exciting new ways’ (Lund & Wang, 2023, p. 1), thus reflecting a potentially profound change process to which individuals have to adapt (e.g., learning to work with chatGPT). These type of adaptation processes should also be mirrored in the SG experience, such as an SG requiring players to adapt to a new gameplay reality (e.g., player’s character is fired from his/her company) by changing the rules of the SG (e.g., player can no longer spend money on luxury goods in the game).

Design requirement 4: An SG should allow players to adapt to changing circumstances within the game world.

The fifth proposition of ELT is that learning encompasses transactions between the person and the game environment (Kolb, 1984). Instead of viewing learning as an intraindividual process (as is frequently done in conventional education settings), ELT views it as a transactional process in which the learner and the environment constantly interrelate. For example, a programming expert (or multiple programming experts) can develop AI solutions such as chatGPT, which is capable of fixing software bugs through analyzing and correcting existing code (Sobania et al., 2023), thereby having the potential to change how coding is conducted and also what one needs to learn about coding in order to be an effective programmer in the future. This transaction between individual action and environmental conditions should also be reflected in the game so that the game environment and the player are intricately intertwined.

Design requirement 5: An SG should have transactions between the person and the game environment that reciprocally influence each other.

Last, in order for SGs to become effective, they should also provide a good user experience that motivates players to engage with them (e.g. de Almeida & dos Santos Machado, 2021). Given that research suggests flow to represent a useful lens through which to understand and optimize the user experience (Kiili et al., 2012), an SG should be designed toward increasing the likelihood of inducing a state of flow in players.

Design requirement 6: An SG should include game elements that foster the experience of flow.

Constructs

Based on these design requirements, we now introduce the constructs for our design theory, which represent ‘[…] the most basic level in any theory’ and ‘[…] should therefore be as clearly defined as possible’ (Gregor & Jones, 2007, p. 325). In our design theory, these constructs are “learning outcomes” (e.g., Sternberg & Detterman, 1986), “game components”, “game mechanics”, and “game dynamics”, with the latter three representing different types of game elements (Werbach & Hunter, 2012). According to Werbach and Hunter (2012), these game elements form the basic building blocks of any game, and while there is some interrelationship between them, they can be analyzed and defined independently (Hunicke et al., 2004; Werbach & Hunter, 2012). Game dynamics capture the dynamic, changing, or evolving aspects of a game encompassing, for example, the state of the game (e.g., evolving narrative) or the player’s affective state (e.g., enjoyment) (Werbach & Hunter, 2012). Game mechanics represent the various actions, behaviors, and control mechanisms afforded to players (Hunicke et al., 2004; Werbach & Hunter, 2012), such as the ability to click on objects in the game world (Hunicke et al., 2004), whilst game components are the static building blocks that make up a game, such as avatars, different levels, or quests (Werbach & Hunter, 2012). In addition, learning outcomes are what players are expected to be able to do at the end of playing an SG (Prøitz, 2010).

Principles of form and function

These constructs, and the design requirements, inform the blueprint of our design theory, which is composed of five different layers, as shown in Figure 1. The lowest or first layer reflects the philosophical base for how learning should be conceived in SG-based learning contexts (Kolb, 1984), followed by the second layer, which encapsulates considerations regarding learning outcomes. Layers three, four, and five contain the game elements, namely the game’s components, mechanics, and dynamics (Werbach & Hunter, 2012).

Figure 1. Blueprint of our design theory.

Note. SG = serious game, DP = design principle

We have chosen to depict our blueprint in this pyramid structure, because while all layers can still be observed independently, they are also built upon each other in specific ways. For example, in order to formulate learning outcomes (layer 2), it is important to understand how learning unfolds (layer 1) (Kolb, 1984). Similarly, players use game mechanics (e.g., clicking on interaction symbols, layer 4) to engage with existing game components (e.g., objects, layer 3), which—over time—makes a game dynamic (e.g., objects change, layer 5) (Werbach & Hunter, 2012).

This blueprint incorporates seven design principles, which we now present in the following by employing the structure proposed by Gregor et al. (2020). The first design principle is related to design requirement 1 and reflects the conception of the learning process that underlies this article (Kolb, 1984). It stipulates that an SG should (a) account for the players’ previous learnings while (b) not compromising their learning process after playing it. Starting with (a), Lopes and Bidarra (2011) discuss five game elements that can be adapted to reflect different levels of player experience: (i) the game world and its objects, (ii) gameplay mechanics, (iii) non-playing characters and AI, (iv) game narratives, and (v) game scenarios or quests. For example, game mechanics can support inexperienced players by allowing them to use a helping tool that provides important tips and hints and which is not available to experienced players (Lopes & Bidarra, 2011). In addition, there exist various approaches to making SGs adaptive (e.g. Peirce et al., 2008; Streicher & Smeddinck, 2016). For example, Peirce et al. (2008) developed a so-called “ALIGN system architecture,” which uses gameplay data to evaluate the proficiency of players and to then adapt different game elements accordingly. They demonstrate their approach with the SG ELEKTRA, which teaches players about the physics of optics, and if they repeatedly fail to complete a specific task (e.g., getting a marble into a specific area), they receive encouraging feedback by a non-player character that stands next to their game character (Peirce et al., 2008). The other facet of design principle 1 is that SG-based learning should not inhibit future learning (b). According to Kolb (1984), this can happen in behaviorist theories of learning as well as in idealist approaches to education, each of which presumes and teaches the existence of fixed ideas that always remain the same. Thus, SGs built upon ELT should teach ideas in such a manner that learners are equipped to modify them, if necessary. This primarily means not designing SGs in a drill-and-practice manner (e.g., constantly repeating fixed ideas) as well as teaching concepts so that students know when to apply or when to change them in the future. For example, an SG aiming to develop leadership skills should not only focus on teaching traditional leadership concepts (e.g., transformational leadership, Bass & Riggio, 2006), but also incorporate intercultural perspectives, because the cultural context strongly influences perceptions about what makes leaders effective (House et al., 2004), implying that individuals need to be able to adapt their behaviors if they want to lead effectively across cultures (House et al., 2013).

Design principle 1: For SG designers to develop pedagogically effective SGs, they need to ensure that the SG accounts for different levels of player experience and does not limit the player’s ability to learn after playing the game, as learning represents a continuous process according to experiential learning theory.

The next layer of the blueprint revolves around defining learning outcomes, which should be done by using the theory of multiple foci of intelligence, the design principle of which is related to design requirement 2. As very few competency constructs employ the theory of multiple foci of intelligence to distinguish between different components of a given competency, this step usually entails using it to break down a competency into its cognitive, meta-cognitive, motivational, and/or behavioral subdimensions (Sternberg & Detterman, 1986). For example, analytic problem-solving (such as identifying the shortest path between a set of locations) requires finding a single solution to a problem (Fischer et al., 2015) and involves primarily cognitive and meta-cognitive intelligence. In this regard, individuals need to know how they can solve the problem, which requires strategizing or meta-cognitive intelligence, and they also need to have the appropriate knowledge or cognitive intelligence (e.g., mathematical knowledge) to implement it (Sternberg & Detterman, 1986). As illustrated in this example, it will frequently be the case that multiple dimensions of intelligence are reflected in one learning outcome. Given that learning outcomes and game design are intricately linked (Romero et al., 2015), it is therefore important to use the theory of multiple foci of intelligence conscientiously when defining learning outcomes.

Design principle 2: For SG designers to develop pedagogically effective SGs, the SG should have clearly defined learning outcomes, which should be achieved by using the theory of multiple foci of intelligence, because research suggests that clearly defining learning outcomes is critical for developing effective SGs.

The next design principle is related to design requirement 3 and connects layer 2 in the blueprint with layers 3, 4, and 5, thereby demonstrating which game elements are best suited for achieving intended learning outcomes. An overview of these interrelationships is depicted in Table 3 and is based on existing work about game elements (Hunicke et al., 2004; Werbach & Hunter, 2012), linking game characteristics to learning outcomes (Bedwell et al., 2012; Romero et al., 2015) and competency development more generally (Kolb et al., 1986; Sternberg & Detterman, 1986).

Table 3. Matching intelligence types to game elements.

	Cognitive Intelligence	Meta-cognitive Intelligence	Motivational Intelligence	Behavioral Intelligence
Game Components	Feedback (Locations) (Non-Player Characters) (Objects) (Player Characters/Avatars) Quests/Tasks Rules (Turns)	Feedback (Locations) (Non-Player Characters) (Objects) (Player Characters/Avatars) Quests/Tasks Rules (Turns)	(Cut Scenes) (Locations) Feedback Non-Player Characters Player Characters/Avatars Quests/Tasks Rewards Story	Feedback Locations (Non-Player Characters) (Objects) Player Characters/Avatars Quests/Tasks Rules (Turns)
Game Mechanics	Choices: Strategic or Tactical Puzzles Resource Acquisition Resource Management Transactions	Choices: Strategic or Tactical Design/Create/Edit Puzzles Resource Acquisition Resource Management Selecting/Collecting/Finding Transactions	Choices: Expressive (Competition) (Communication) (Cooperation) Design/Create/Edit Quick Time Events Role-Playing	Choices: Expressive, Strategic or Tactical (Cooperation) Communication Movement Quick Time Events Resource Management Role-Playing Transactions
Game Dynamics	Constraints	Constraints Narrative Development	Narrative Development (Relationships)	Relationships

Note. The game elements in brackets are found less frequently in SGs aiming to develop a particular intelligence dimension, or they require a specific game genre. For example, turns are only relevant in turn-based games (e.g., where one player makes a decision and only thereafter can another player make a decision), albeit these are not the only type of game that can successfully develop cognitive intelligence.

For example, reflective thinking is an important skill for solving complex problems (Kalelioğlu, 2015) and is also relevant in software programming (Bergin et al., 2005). Furthermore, it is a part of meta-cognitive intelligence (Sternberg & Detterman, 1986), and an SG aiming to develop reflective thinking capabilities in students could look like this: (1) The player plays a young woman (player-character/avatar) and her manager asks her to find out what exactly has corrupted the intranet of the company she is working for (task). To do so, she can only use the company’s intranet and her knowledge of the company’s IT infrastructure (rules). (2) During the game, she has to collect information from the intranet (collect) and reflect about how to structure this information in order to understand its meaning (puzzle). As the game evolves, her manager also allows the young woman to hack into different company computers, in order to gain all the necessary information, but limits her access to two computers in total. Thus, she has to decide which computers carry the most important information (choices: tactical), which she finds out by solving the puzzles. (3) In a big story twist (narrative development), the player understands that the intranet was never corrupted but that the manager who gave her the task actually perceived her as a threat and wanted to generate evidence to fire her (hacking into other computers). Now, the player has to think about and find ways to prove that she is actually innocent.

This brief outline exemplifies how an SG can use different game elements in order to help players mature their reflective thinking skills or meta-cognitive intelligence, which happens at various stages throughout the game, e.g., employing the perspective of a young woman, collecting information and selecting what is important, structuring this information correctly, choosing which computers to hack into, experiencing a story twist, planning how to deal with the situation, thinking about the type of information the player needs to prove her innocence, etc.

Design principle 3: In order for an SG to achieve its learning outcomes, the developer must ensure that the game elements allow players to engage the learning modes that are required to develop specific intelligence dimensions, because experiential learning theory suggests that the type of learning modes in which a player engages influences which capabilities he or she develops.

Design principle 3a: If the SG aims to develop cognitive intelligence, it should have game elements that modify or develop the player’s knowledge and/or cognition according to experiential learning theory and the theory of multiple foci of intelligence.

Design principle 3b: If the SG aims to develop meta-cognitive intelligence, it should have game elements that modify or develop the player’s awareness of knowledge and/or control of cognition according to experiential learning theory and the theory of multiple foci of intelligence.

Design principle 3c: If the SG aims to develop motivational intelligence, it should have game elements that modify or develop the player’s level and/or direction of motivation according to experiential learning theory and the theory of multiple foci of intelligence.

Design principle 3d: If the SG aims to develop behavioral intelligence, it should have game elements that modify or develop the player’s academic, social, and/or practical behavior according to experiential learning theory and the theory of multiple foci of intelligence.

The next design principle is related to design requirement 4 and is concerned with changing circumstances in the game world as well as the player’s capability to adapt to them. Changing circumstances are encapsulated in the game’s dynamics (Werbach & Hunter, 2012), and there are different ways through which they can be realized. For example, a game can constrain the player’s access to different locations in the game and only unlock these if the player has made enough progress (e.g., by having successfully completed a given amount of learning challenges). Similarly, the narrative of a game can limit or enrich a player’s portfolio of conceivable actions. For example, a game could include different non-player characters that are all friendly toward the player’s character up until a certain point in the story, when the player makes a big mistake and loses all of their trust. Now, these non-player characters refuse to help the player, and so they have to find other ways to gather information and to make progress. In addition, in multi-player games (involving more than one human player), players develop relationships by engaging with others. Here, change can be realized, for example, by unlocking different tools to communicate across time. For instance, at the beginning of an SG, players might only be able to communicate with each other by using emotes, while later on they may also use written chat or even voice or video chat.

Design principle 4: For SG designers to develop pedagogically effective SGs, the SG should incorporate game dynamics that allow players to adapt to changing circumstances in the game world, because experiential learning theory suggests that learning is a holistic adaptation process that requires learners to constantly adapt to a changing world.

The next two design principles are related to design requirement 5, thereby reflecting the transactional character between the person and the environment (Kolb, 1984). First, environmental conditions should influence the repertoire of available behaviors or actions, meaning that the dynamic state of the game world (e.g., progression in the narrative) and the existence of specific game components should both affect what the player can do. For example, if a player found a screwdriver (game component) in the game, there should be a game mechanic allowing them to use it (e.g., opening a secret tunnel). In addition, using these game mechanics should in turn also affect the components and the dynamics of a game. For example, if the player has used the screwdriver to open the secret tunnel but forgotten to disguise it again after entering, this might get detected by non-player characters in the game and influence their actions or reactions to the player (e.g., acting nervously, refusing to talk to the player).

Design principle 5: For SG designers to develop pedagogically effective SGs, game mechanics should have consequences for the game’s dynamics and components, because experiential learning theory suggests that any given action has the ability to change objective conditions.

Design principle 6: For SG designers to develop pedagogically effective SGs, the state of the game world (e.g., game components and dynamics) determines the game mechanics that are available to players, because experiential learning theory suggests that objective conditions have a significant effect on the subjective experience and the repertoire of available behaviors.

The last design principle related to design requirement 6 specifies the game components and mechanics that have to be integrated in order for players to experience flow. Due to its emotional component, flow can be characterized as part of the game’s dynamics (Werbach & Hunter, 2012), and according to Kiili et al. (2012), there are five different antecedents that help players experience it. First, the SG should have clear goals detailing what exactly the player needs to accomplish, which should also be related to the game’s learning objectives and can be split into different sub-goals (Kiili et al., 2012). Second, the game should provide cognitive and immediate feedback, thus helping inform the player about his or her progression toward the goal, with the latter referring to feedback that follows immediately after the player does something (e.g., reaction to an action), while cognitive feedback primarily aims to support the learning experience by providing feedback, explanations, or debriefings related to the learning contents. Third, the game should have good playability, meaning that controlling it (e.g., using game mechanics) should be easy (spontaneous and automatic) while simultaneously incentivizing players to consciously and willingly interact with its educational content (Kiili et al., 2012). Fourth, the game should have an appropriate level of challenge, meaning that the level of difficulty should reflect a player’s capabilities, which can be achieved by manipulating game components or mechanics in respective ways (e.g., integrating more feedback for inexperienced players). Last, the player should have a sense of control, meaning that they feel they can beat the game and master its contents, which is usually realized through incorporating game mechanics such as choice or allowing players to manage the direction of activity (e.g., what exactly to do in a game) (Garris et al., 2002).

Design principle 7: For SG designers to improve the likelihood of players experiencing flow while playing an SG, the SG should have clear goals, provide cognitive and immediate feedback, have good playability, an appropriate level of challenge, and engender a sense of control in players, because research suggests these are the antecedents for flow in the context of SGs.

Artifact mutability

Artifact mutability comprises the ways in which IT artifacts evolve over time and in different contexts (Gregor & Jones, 2007). In general, it is to be expected that an IT artifact such as an SG changes over time, as most video games experience some sort of change due to patches or updates: ‘[…] the fact that [games] change over time leads to a situation where, depending on the time of purchase, the platform, the territory, the number of updates and patches applied, different players playing ostensibly the same game might necessarily enjoy different experiences in markedly, or subtly, different worlds’ (Newman, 2012, p. 140). These changes can result from players offering feedback after the SG has been released, which game developers might incorporate, thereby also potentially changing the original architecture of the SG.

In addition, while learning processes (employing ELT) will likely remain similar—although, admittedly, new discoveries could change our understanding of how learning unfolds—SG design choices are also affected by the platform for which an SG is developed. For example, mobile gaming sessions are—on average—shorter than PC gaming sessions (Paik et al., 2017), making ‘designing a game for the quick and short mobile usage sessions […] different from creating a typical computer or console video game’ (Mäyrä, 2015, p. 1). Similarly, other platforms such as augmented or virtual reality also differ in terms of player controls (e.g., using specific controllers) or the degree of immersion (e.g., 360° immersion vs. being immersed in a computer display) compared to conventional computer or console video games, with potentially important implications for SG design.

Last, while our blueprint has been developed with a generalist perspective on SG-induced learning in mind (e.g., developing SGs for a variety of learning outcomes and for different target groups), target group preferences should also be accounted for in the design process. For example, younger players tend to play video games because of fantasy motives (e.g., pretending to be someone else), while older players are more likely to pursue gaming activities for competitive motives (e.g., willingness to win) (Greenberg et al., 2010). Similarly, men and women have been found to favor different types of games (Greenberg et al., 2010), meaning that the composition of an SG’s primary target group will make some game elements more or less suitable than if it were a different target group.

Testable propositions

Testable propositions constitute so-called “truth statements” about a design theory and can be evaluated through an instantiation of the design theory (Gregor & Jones, 2007). While these propositions ‘can vary in their degree of generality – from claims that a design works all the time and in many contexts […] to claims that a design proposition is only an approximation to what will work in different contexts’ (Gregor & Jones, 2007, p. 327), they are important for determining whether or not the meta-design or design method for an artifact is actually effective (Gregor & Jones, 2007; Walls et al., 1992). Given that the design theory focuses on learning effective SG design while also enabling players to experience flow while playing the SG (Csikszentmihalyi, 1990), there are two major testable propositions:

Proposition 1:

An SG that is designed based on the proposed design theory will achieve its intended learning outcomes and contribute to cognitive, meta-cognitive, motivational, and/or behavioral intelligence development in players.

Proposition 2:

An SG that is designed based on the proposed design theory will enable players to experience flow while playing the SG.

It is important to add that both of these propositions only represent approximations of what an SG designed with our design theory is likely to achieve. Thus, while we do not expect every single player to achieve intended learning outcomes, or to experience a state of flow while playing the SG (which would also be unrealistic, given the diverse and idiosyncratic preferences or interests of players), we expect that both of these propositions are—on average—significant and true.

Principles of implementation

This section introduces some principles of implementation that support the realization of our SG architecture as shown in Figure 1. The main goal of this section is to provide guidance for putting our SG design theory into practice (Gregor & Jones, 2007) and for developing learning effective SGs. Thus, we have formulated implementation principles for each of the five different layers, which are depicted in Table 4.

Table 4. Implementation principles.

Implementation Principle	Description
Implementation principle 1: Different levels of experience	There are various ways through which a game can adapt to different levels of player experience. For example, game analytics can help adjust the difficulty of a game (Ninaus & Nebel, 2021), which can be realized with the help of established approaches focusing on data-driven difficulty adaptation (Peirce et al., 2008). This could be done, for example, by including a test at the beginning of the game, measuring the proficiency of the player in a given intelligence dimension, and then adapting the game’s difficulty (e.g., limited time to answer questions, more difficult questions) according to these test results. Other measures to evaluate the player’s proficiency and experience include analyzing their affective state during gameplay and then adapting game mechanics (e.g., speed) accordingly (Steinrücke et al., 2019). This has also been described as ‘emotional adaptivity’ and can be assessed by using self-reported or physiology-based data (Tijs et al., 2008).
Implementation principle 2: Learning outcomes	It is important to clarify what exactly players can expect to learn when playing an SG, and this should also be reflected in the SG’s goals (Kiili et al., 2012). For example, game developers might integrate a quest or a task window that players can access in order to find out which tasks they have to fulfill, where they are at in the game, and also how these tasks are related to the learning outcomes. In addition, the game should be clear as to what exactly the player has to do and how to play it. For example, in a lot of story-driven games (e.g., Telltale Games), players employ the perspective of a character and have to decide how to act in different situations. This can mean either making decisions on behalf of that character (incorporating the character’s personality or values into the decision-making) or acting as if they were the characters themselves (decisions based on personal preferences). While both of these alternatives sound similar, note that they train different facets of meta-cognitive intelligence, in that the former requires developing awareness about other people (e.g., through observing and listening to the character) while the latter contributes to self-awareness (e.g., How do I want the player character to behave? What are my personal preferences?).
Implementation principle 3: Game components	When integrating different game components, ensure to carefully reflect about whether or not it is necessary to integrate them. As a rule of thumb, SG developers should only integrate game components that have a concrete benefit, as, otherwise, the game might be overloaded with components that ultimately confuse players (e.g., confusion about which ones are important to pay attention to). Questions that might help to find out whether or not game components are important include, for example: Is a specific non-player character essential for the game’s narrative or the tasks that a player needs to fulfill? Is there enough feedback or too little? Does this game really need objects that a player can interact with, or does this simply distract players from reaching the actual objectives in the SG? In addition, some SGs might require a lot of different game components. In these cases, a handbook might help players maintain an overview thereof, which is particularly important at the beginning of an SG, when players are not yet familiar with all the game components.
Implementation principle 4: Game mechanics	Game mechanics represent the game elements that connect the game to the players and through which the latter can change the state of the game world (Werbach & Hunter, 2012). As such, they should be tested extensively, so that players have a good experience playing the game. Similar to the game components—they should not be included arbitrarily. For example, a lot of adventure games require players to find hidden objects in order to advance the narrative or in order to unlock a location that was previously closed. While this game mechanic might be entertaining for some players, others might find it tedious and boring. Thus, if such a game mechanic is included, it should be linked to the learning outcomes of the SG (Kiili et al., 2012) and tested with players as well. In addition, in order to familiarize players with the different game mechanics, the game should include a tutorial at the beginning of the game that allows players to get to know these mechanics and experiment with them.
Implementation principle 5: Game dynamics	The SG should be dynamic, so that players find it interesting and not too repetitive. Thus, an SG should be able to evoke different emotions (curiosity and joy), have challenges and puzzles that differ from each other, allow dynamic interactions with other players, or include a narrative with twists. While trying to enable dynamic gameplay experiences in different areas might be very costly (e.g., a narrative has to be written, different puzzles have to be designed and coded), SG developers should prioritize dynamics according to the learning outcomes (design principle 3) in order to not exceed the budget, which can happen quite easily in complex SG development projects.

Overview of design theory

Table 5 presents a summarized overview of the different components of our design theory (Gregor & Jones, 2007). The first seven components were discussed in this section and the next section will take a closer look at the expository instantiation (component 8).

Table 5. Components of the developed design theory.

Component	Description
1) Purpose and scope	The scope of the proposed design theory is to provide design knowledge for the development of serious games that effectively achieve intended learning outcomes.
2) Constructs	Learning outcomes, game components, game mechanics, game dynamics
3) Principle of form and function	The means of designing a serious game that is pedagogically effective is visualized in a blueprint and explained with the help of seven design principles.
4) Artifact mutability	The artifacts are likely to change over time, due to user feedback, and across contexts, due to different requirements of target groups (e.g., age or gender) and peculiarities of platforms (e.g., mobile gaming vs. virtual reality gaming).
5) Testable propositions	Proposition 1: An SG that is designed based on the proposed design theory will achieve its intended learning outcomes and contribute to cognitive, meta-cognitive, motivational, and/or behavioral intelligence development in players. Proposition 2: An SG that is designed based on the proposed design theory will enable players to experience flow while playing the SG.
6) Justificatory knowledge	Experiential learning theory, theory of multiple foci of intelligence, and flow theory helped deduce the design requirements.
7) Principles of implementation	Five principles of implementation are defined regarding (i) SG adaptivity, (ii) learning outcomes, (iii) game components, (iv) game mechanics, and (v) game dynamics.
8) Expository instantiation	Development and evaluation of a cultural learning game

Expository instantiation: presenting and evaluating a cultural learning SG

In the following, we present the SG we developed as part of our design theory. The SG aims to develop cultural intelligence (CQ) in players, thereby implementing design principle 2 that encapsulates defining clear learning outcomes (in this case CQ) for SG development projects. CQ is a construct measuring an individual’s intercultural competency and is defined as ‘an individual’s capability to function and manage effectively in culturally diverse settings’ (Ang et al., 2007, p. 337). It comprises four different dimensions in line with the theory of multiple foci of intelligence (Ang & Van Dyne, 2015; Sternberg & Detterman, 1986). The cognitive dimension comprises intercultural knowledge of norms, practices, and conventions, while meta-cognitive CQ reflects higher-order cognitive processes. Motivational CQ measures an individual’s ability to direct attention and energy toward intercultural situations, and behavioral CQ covers the degree to which individuals exhibit appropriate verbal and non-verbal actions when interacting with people from different cultures (Ang & Van Dyne, 2015). Given that all types of individual-level intelligence (Sternberg & Detterman, 1986) are covered by CQ, it seems like a promising construct with which to test our design theory.

However, the ability to evaluate CQ, and its connection to the theory of multiple foci of intelligence, was not the only reason why we decided to develop a cultural learning SG. Another reason was that most cultural learning SGs lack proper evaluation, making it impossible to conclude how effective they actually are. Related publications mostly elaborate in a descriptive way on the cornerstones of developed cultural learning SGs (e.g., Bohn et al., 2014), but they primarily evaluate single aspects of intercultural competency development, such as increasing players’ intercultural awareness (e.g., El Kechaï & Pierrot, 2015), which is summarized in an overview in Appendix B. However, looking at more complex constructs such as CQ, which represents the most influential intercultural competency construct in cross-cultural psychology and management research (Andresen & Bergdolt, 2017), we find that it consists of multiple subdimensions, and since these are interdependent (Ang & Van Dyne, 2015), employees need to be adept in all of them, in order to successfully pursue global IT projects (e.g., Kummer et al., 2012).

The second major drawback of existing SGs focusing on cultural issues is that they were designed years ago and utilize partly outdated technology, such as old-fashioned 3D graphics and animations (e.g., Mortara et al., 2014), as well as outdated cultural models, such as the cultural dimensions approach by Hofstede (An et al., 2019). This perspective can lead to sophisticated stereotyping by representing an overly simplified depiction of reality that might be of limited value (Zhu & Bargiela-Chiappini, 2013) and neglects recent developments in cultural research (e.g., Fischer & Schwartz, 2010; Gelfand et al., 2017). Thus, both of these drawbacks motivated us to develop a new cultural learning SG.

Implementation, graphics, & controls

The SG we developed is a 2D point-and-click adventure, whereby the user takes control of an avatar and has to unravel a plot that endangers not only the future of said avatar, but also the future of the whole company around which the story is centered. This was an important foundation for implementing various design principles into the SG: through the avatar, the player can interact with different game components and influence the game’s dynamics, for example by talking to different non-player characters (design principle 5), thereby making it possible to control the fate of that avatar, which should allow players to feel a sense of control while playing the game (design principle 7). A detailed description of how the blueprint of our design theory (Figure 1) informed the design of our instantiation is provided in Appendix C.

The learning contents are seamlessly embedded into the narrative of the game, encompassing six episodes, with each one taking approximately 60 minutes to complete. The game was realized by using the Unity game engine and the Unity extension Adventure Creator (https://www.adventurecreator.org/), as Unity arguably represents one of the most widespread engines in video game development and runs reliably (Craighead et al., 2008). In addition, Adventure Creator allowed us to implement efficiently the basic building blocks of adventure games, such as conversations, navigation, and quick time events, or to cut scenes into the game and by doing so enabled us to set aside more time in the tight development schedule for testing and optimizing. We consciously decided to develop the game with 2D graphics (Figure 2) that do not have to compete with sophisticated 3D graphics in current AAA games, which users potentially treat as a standard to compare the game against, and which is adequate, given that research has elaborated that it might even be counterproductive to use visually sophisticated graphics (Rooney, 2012).

Figure 2. SG interface.

The avatar is controlled by using the mouse curser, and we added a soundtrack as well as FX sounds to make the experience as immersive as possible. We also integrated a dialogue system into the game that allows the player to talk to various non-player characters. In addition, we recorded voiceovers from professional actors and actresses for the dialogue, in order to underline the different personalities of the characters and non-player characters.

We set out to develop the game as a Windows application and published a MacOS version as soon as some teething problems (e.g., technical stability, bugs) were solved. However, publishing the game as a desktop version for the two operating systems led to too many additional technical problems, for instance because of continuous updates to MacOS that required updating the game in order for it to run smoothly (e.g., Catalina no longer supported 32 bit apps). As a result, we decided to publish the game as a web-based browser version, in order to increase independence from operating system updates and to reduce the frequency with which updates needed to be run.

Evaluation

In the following, we present our evaluation of the SG after it has been implemented and initially applied, which represents a facet of the exterior mode of DSR that is concerned with generating descriptive knowledge about the artifact in use (Sonnenberg & Vom Brocke, 2012). To do so, we evaluate both of our testable propositions:

Proposition 1:

An SG that is designed based on the proposed design theory will achieve its intended learning outcomes and contribute to cognitive, meta-cognitive, motivational, and/or behavioral intelligence development in players.

Proposition 2:

An SG that is designed based on the proposed design theory will enable players to experience flow while playing the SG.

Evaluating these propositions is also in line with existing literature suggesting that effectiveness and the impact on its users (Sonnenberg & Vom Brocke, 2012), usefulness and ease of use (Prat et al., 2015), and the evaluation of human risk and effectiveness (Venable et al., 2016) constitute important evaluation criteria for DSR projects. More specifically, the human risk and effectiveness evaluation strategy is recommended in DSR projects in which the major design risk is social (Venable et al., 2016), which is the case in our design theory. Thus, we followed this strategy and conducted multiple formative evaluations in the form of alpha and beta tests that helped estimate the artifact’s impact on users (e.g., How did players experience playing the SG, including negative experiences such as boredom or anxiety?) as well as improve the SG in terms of achieving learning outcomes and in terms of its ability to induce flow in players. In addition, we also conducted a summative evaluation of the SG after we finalized its development (Venable et al., 2016), which we present in this section.

The summative evaluation allowed us to test whether or not the introduced propositions hold in regards to the specific instantiation of our design theory (Venable et al., 2016). Given that our SG aims to strengthen CQ in players, we thus evaluated the SG’s ability to improve the players’ cognitive, meta-cognitive, motivational, and behavioral CQ, thereby contributing important insights as to whether or not SGs designed with the help of our design theory are in fact able to target cognitive, meta-cognitive, motivational, and behavioral intelligence development in players (proposition 1). In addition, flow has been defined as a pleasurable state (enjoyment) of optimal functioning, whereby individuals are fully absorbed (attention) by an activity (Csikszentmihalyi, 1990), which is why we operationalized and evaluated proposition 2 by analyzing the degree to which players reported enjoyment and attention when playing the game (Kiili et al., 2012).

It has to be added that an important discussion surrounding the evaluation of design artifacts is that it should not only focus on an artifact’s usefulness, but also address its design fitness (e.g., an artifact’s malleability or decomposability, which allow it to evolve over time in response to changing environmental demands, for example) (Gill & Hevner, 2013). While our summative evaluation covered the user experience and the degree to which the cultural learning SG was able to develop cultural intelligence in players, we would like to emphasize that the formative evaluations in the form of alpha and beta tests were instrumental in creating an effective cultural learning SG. In one of our early tests, for example, we noted that players frequently overlooked the information that new learning material was available in the respective interface. In order to increase the likelihood that players received this information and were able to read the learning material in good time, we therefore included a sound and a specific symbol in the respective interface menu as soon as the new learning material was unlocked, in order to increase its detectability. Thus, while the implementation of the aforementioned design principles remained exactly the same across all formative evaluations, many details changed during these evaluations, and we believe that these changes were instrumental in improving the SG. Thus, SGs developed with the help of our design theory should always undergo multiple formative evaluations, thereby translating the more static blueprint of pedagogically effective SGs (Figure 1) into effective and unique SGs, which also helps them comply with user demands, were they to change in the future (Gill & Hevner, 2013).

Participants, methods, and design

For the summative evaluation, we implemented the SG in different intercultural management courses involving a diverse group (nationalities and gender) of 73 Master in Management students. All courses were structured in the same way, comprising six hours of gameplay, seven hours of debriefings, and two hours of additional information (e.g., introduction).

To evaluate the SG’s ability to induce flow, and to achieve the intended learning outcomes, we asked these students to reflect in an essay on their experience of playing the game (instructions can be found in Appendix D). Reflective essays help gain in-depth perspectives on perceptions, and they have been utilized in various studies (Poldner et al., 2012). Particularly in experiential education, reflective essays have been said to measure ‘knowledge and cognitive growth in a way that a multiple-choice test or a self-reported survey cannot’ (Bennion et al., 2020, p. 39), thereby allowing us to understand how students actually experienced the game, which also enabled us to gain insights into whether or not our design theory proved beneficial when designing the SG. While we could have also used established CQ questionnaires (e.g., Ang et al., 2007) to evaluate the players’ CQ development, we consciously decided to base our evaluation on reflective essays, as the SG was embedded in a course format that prevented utilizing a longitudinal quantitative research design to carve out its learning effects, and interpretive approaches help better isolate the game’s learning effects when implemented in a real environment—also described as “naturalistic evaluation” (Venable et al., 2016): ‘To the extent that naturalistic evaluation is affected by confounding variables or misinterpretations, evaluation results may not correspond to real use [and] the dominant interpretive paradigm brings to naturalistic DSR evaluation the benefits of stronger internal validity’ (Gummesson, 2000; Venable et al., 2016, p. 5). In addition, some of the items relating to established CQ measures are very specific (e.g., one item asks individuals to rate the degree to which they know the marriage systems of other cultures (Ang et al., 2007)), and given that we wanted to understand actual CQ development more broadly instead of focusing on how much students developed specific CQ items, we also decided against a randomized experimental research design, which would have otherwise limited the impact of confounding variables as well (Campbell & Stanley, 2015) but would have required us to utilize existing CQ questionnaires (e.g., Ang et al., 2007).

It is important to add that in class and in the essay questions, we encouraged students to think about their actual experience of playing the game, in order to decrease the likelihood of receiving feedback that might suffer from a social desirability bias (Nederhof, 1985). This entailed giving specific instructions on how to answer the essay questions, in case the students felt they had learned nothing from playing the game or did not enjoy it. In addition, the essay questions were formulated very openly, in order to grant students the freedom to choose which areas they wanted to write about. When asking about their learning experience, for instance, they could—but did not have to—reflect on how the game improved their knowledge, motivation, and behavior; instead, they were allowed to focus on single aspects thereof, which would help improve the accuracy of responses, as they could decide to write about areas where they were more certain about the game’s impact.

In the essay, we let students reflect on their experience of flow, which was done by zooming in on their level of enjoyment and attention when playing as well as about their learning experience in terms of their knowledge about intercultural management, their motivation to engage with people from other cultures, or their actual behavioral competency in interactions, thereby evaluating the usefulness of playing the game (Yusoff et al., 2010) in terms of developing different facets of CQ.

To analyze systematically the response patterns in the essays, we conducted a qualitative content analysis (Mayring, 2004). In order to ensure validity and reliability within our research process, we based our research on the phenomenological epoché strategy, which supports researchers in withholding their own theories and prejudices when collecting and analyzing qualitative data (Sandberg, 2005). In order to ensure preliminary communicative validity (e.g., consistent interpretations of the categories described by students), an independent researcher coded 19 randomly chosen essays (26%), without being given instructions, and compared the deduced codes (e.g., statements about enjoyment or motivational CQ), achieving highly satisfactory results of 96.6% agreement and a Krippendorf’s alpha of 0.858 (LeBreton & Senter, 2008). Next, we looked for the responses’ structural features and derived a categorization scheme, in order to differentiate the various opinions on the reported phenomena (e.g., high enjoyment when playing the game vs. low enjoyment) (for evidence see Appendix E). To test the accuracy of our categorization scheme, an independent researcher again coded 20 randomly chosen essays (27%) with our categorization scheme and compared structural features to our own, achieving 87.5% agreement and a Krippendorf’s alpha of 0.808. This was an important test for our research, as our categorization scheme seemed to allow for strong interrater reliability (LeBreton & Senter, 2008). Last, we analyzed all essay responses again and compared the codes to our categorization scheme, agreeing upon a common interpretation in cases where said categorization was more ambiguous.

Results

Flow: enjoyment and attention

We decided to differentiate between high, mediocre, low, or no levels of enjoyment and attention when coding the responses. It was deemed “high” when it was exclusively positive and did not include relativizing statements such as “in general”. If students mentioned some negative aspects, but the experience was still described as generally positive, we coded the response as “mediocre.” In contrast, if the experience was volatile or changeable throughout, we coded this as “low.” As no student mentioned “no enjoyment or attention,” we did not have to define the requirements for this category.

Table 6 reveals that the large majority of students described their enjoyment as “high,” followed by a considerably smaller proportion rating it as “mediocre” or “low.” Thus, we can conclude—with some exceptions—that our students enjoyed playing the game. Similarly, the majority indicated a satisfactory level in terms of their ability to focus on the game (Table 7).

Table 6. Degree of enjoyment when playing the game.

Enjoyment	Frequency (n = 73)	Exemplary Statements
High	n = 53 (73%)	‘To me, playing the game was a really enjoyable experience’.
Mediocre	n = 15 (20%)	‘In general, I did enjoy playing the game, because of four main reasons’.
Low	n = 5 (7%)	‘I found the game enjoyable to a certain extent’.

Table 7. Degree of attention when playing the game.

Attention	Frequency (n = 73)	Exemplary Statements
High	n = 35 (48%)	‘Another thing I really liked about the game was how it captured my attention for the whole hour…’
Mediocre	n = 32 (44%)	‘ … I’d say I was focused for 85% of the time’.
Low	n = 6 (8%)	‘My level of attention throughout the game could be pictured as a rollercoaster. In the morning, I found myself extremely focused, but I would gradually lose my concentration throughout the day’.

Learning outcomes

In terms of students’ indicated improvement in their cognitive, meta-cognitive, motivational, or behavioral CQ, we decided to differentiate between improved, somewhat improved, not improved, or not mentioned. The latter was necessary, as some students did not report on the degree of improvement with regard to some facets of CQ. We coded the degree of improvement as “not improved” when the students explicitly mentioned that they did not feel the game helped them improve on the respective facet of CQ. For example, the statement ‘I wouldn’t say that the game motivated me to engage with people from other cultures, as I am already an open-minded person talking to everyone I meet’ was coded as “no improvement” in terms of motivational CQ. Moreover, we decided to differentiate between improved and somewhat improved, in order to distinguish statements like ‘Furthermore, I now feel more motivated to engage with people from other cultures’ (improvement in motivational CQ) from statements like ‘The area I improved the least is the motivation to interact with people from other cultures’ (some improvement in motivational CQ), as the latter signifies an improvement, albeit on a smaller scale than the first one.

Table 8 summarizes the frequencies with which the students indicated improvements in their cognitive, meta-cognitive, behavioral, or motivational CQ. As evidenced, the majority felt that playing the game substantially improved their cognitive CQ, with similar results for their meta-cognitive CQ development. Their perception of behavioral competency was more ambiguous, as some reported an improvement, a few others indicated that the game did not help them train their behavior, and some did not write about it at all.

Table 8. Frequency of reported learning outcomes (n = 73).

CQ Dimensions	Degree of Improvement
	Improved	Somewhat improved	Not improved		Not mentioned
Cognitive	63 (86%)	8 (11%)	0 (0%)	2 (3%)
Meta-cognitive	55 (75%)	1 (1%)	0 (0%)	17 (24%)
Behavioral	32 (44%)	5 (7%)	8 (11%)	28 (38%)
Motivational	17 (24%)	4 (5%)	24 (33%)	28 (38%)

In terms of motivational CQ development, 17 (24%) of the students reported that they felt more motivated to interact with people from other cultures. In contrast, 24 (33%) indicated that the game had no effect on motivation, with the primary reason being that their motivation was already very high before playing, which is unsurprising given the pronounced international orientation of the study programs. It has to be added that the essay questions asked specifically about the degree to which students felt more motivated to interact with people from other cultures. However, motivational CQ encompasses the motivation to seek out intercultural experiences (e.g., to interact with people from other cultures) as well as the self-efficacy to handle intercultural experiences (e.g., dealing with the stresses of adjusting to another culture) (Ang & Van Dyne, 2015). Thus, given that the latter was not addressed in the essay questions, our analysis of motivational CQ development only reflects the game’s ability to improve the player’s intercultural motivation and not its potential in terms of strengthening perceived intercultural self-efficacy.

In addition, we implemented the SG in different intercultural management courses in Executive Education programs with managers and professionals aged 30 to 45, as well as in companies with young professionals. Although we did not systematically evaluate the flow experience and learning outcomes of our SG in these contexts, we received promising feedback, thereby providing preliminary evidence that the SG might also be an effective learning experience across contexts.

Discussion and conclusion

Our research contributes to the discussion about how to integrate pedagogical considerations successfully into SG design (e.g., Arnab et al., 2015; Theodosiou & Karasavvidis, 2015; Watt & Smith, 2021) in three distinct ways. First, by introducing our design theory for developing pedagogically effective SGs, we contribute a mid-range design theory on SG design (Baskerville et al., 2018; Watt & Smith, 2021) in an area that is currently underexplored (Watt & Smith, 2021). While there are various frameworks available that help design SGs (e.g., Arnab et al., 2015; Carvalho et al., 2015; Kiili, 2005), most of them neglect pedagogical considerations (Ravyse et al., 2017), although research frequently emphasizes the importance of incorporating pedagogy into the design process (Watt & Smith, 2021; Westera, 2019). In addition, most of the SG design frameworks are not evaluated empirically (e.g., Arnab et al., 2015; Bellotti et al., 2011; De Freitas & Neumann, 2009), and given that we did not find a single framework that has been developed using a DSR approach, these frameworks also lack information about artifact mutability and implementation principles, or they do not have an expository instantiation with which to test design knowledge. Thus, while the application domain of SG design has a high degree of maturity, given the various SG development frameworks that exist, the solution maturity—particularly in regards to developing pedagogically effective SGs—can be considered low as pedagogical considerations are frequently neglected in SG development research. This makes our design theory contribution an improvement according to the DSR knowledge contribution framework introduced by Gregor and Hevner (2013). As such, we believe that our research also constitutes an important foundation upon which future research endeavors could be built. For example, future research could take a closer look at how SGs develop across time and contexts (Newman, 2012), thereby improving our understanding of the mutability of pedagogically effective SGs (Gregor & Jones, 2007) and their fitness (Gill & Hevner, 2013), or increase its practical utility by conducting further research about implementation principles.

Second, the results described above indicate that the SG we developed proved useful in helping players develop their CQ, thereby contributing a useful and novel design artifact in line with established guidelines on design science research objectives (Baskerville et al., 2018). While the game seems to influence more strongly the cognitive and meta-cognitive dimensions of CQ, we also observed a positive effect on self-reported behavioral CQ development. The only dimension that produced mixed results with regard to the developmental possibilities of using our artifact was the motivational dimension, which can be partly explained by the high level of intercultural motivation that was already prevalent in the investigated group before playing the game, aligned with the fact that our analysis only focused on the motivation to seek out intercultural experiences and not on the self-efficacy to handle intercultural situations, both of which are encapsulated in motivational CQ (Ang & Van Dyne, 2015). All in all, this makes the SG a highly relevant educational artifact, since existing research about developing CQ has been inconclusive, and very few studies have found positive effects of an intervention for all CQ dimensions (Ott & Michailova, 2018).

Third, our design theory supports practitioners in developing pedagogically effective SGs. The design principles and the blueprint of our design theory help understand the interrelationship between learning theory and actual game design and provides concrete guidance on how to incorporate theoretical learning considerations into the design process. For example, our theory specifies which game components, mechanics, and dynamics are particularly suitable for developing different types of intelligence (Sternberg & Detterman, 1986), which has not been done before. In addition, the implementation principles as well as the description of how we developed our SG also yield practical insights into SG design.

While this article contributes to existing research in the aforementioned ways, it is not without limitations. For example, while we were able to demonstrate that the SG we developed proved beneficial in terms of developing CQ and inducing flow, we only developed one SG as part of our design theory. Despite the fact that the SG we developed covered all intelligence dimensions as outlined by Sternberg and Detterman (1986), SGs are arguably too diversified and varied (Boyle et al., 2016) to draw definite conclusions, which is why we strongly recommend developing and evaluating additional SGs with the help of our design theory that focus on entirely different learning outcomes. Moreover, while we did use the SG in different educational contexts with promising results, we only systematically investigated its effectiveness with Master of Management students as a target group. Thus, future research should also take a closer look at how much the SGs developed with our design theory are capable of inducing flow and achieving intended learning outcomes in different educational contexts. In addition, we are aware that we drew on self-reported data, but unfortunately there was no other way of assessing the effectiveness of the SG in terms of developing intercultural competencies, given the setting in which the study was conducted. Since the SG was embedded in a course format with extensive debriefing sessions, using quantitative CQ scales (Ang et al., 2007) or implicit association tests (Griffith et al., 2016), for example, would not be able to uncover the learnings from the SG alone. Therefore, future research utilizing measurement instruments like these could focus on students learning from the SG only. In addition, we let the students reflect on their experience of playing the SG and related outcomes shortly after they completed it. Future research might therefore also take a closer look at the long-term learning effectiveness of developing SGs with our design theory, or possibly even extend it by applying a different theoretical lens, such as situated (Rooney, 2012) or social learning theory (Watt & Smith, 2021).

Moreover, the detailed analysis of the essay responses also revealed some practical insights that could be useful for SG design. The storyline and the interactive decision-making mechanic contributed most to the students’ enjoyment, which is in line with other research elaborating on the importance of storytelling in SGs (Padilla-Zea et al., 2014). By including cliffhangers at the end of each episode, and by integrating plot twists, students were curious about how the narrative would evolve and how their decisions would ultimately affect the storyline. Looking at what helped them focus on the game, the soundtrack, FX sounds, and voiceovers were mentioned most often. Additionally, integrating more decisions with consequences, and refining the feedback system, were mentioned as areas where we could improve.

Concluding, the main objective of this paper was to advance the pedagogical foundation in SG design by introducing a design theory that supports the development of pedagogically effective SGs. By developing and evaluating a cultural learning game as the expository instantiation of our design theory, we were able to demonstrate that creating SGs with the help of our design theory can produce highly effective SGs in terms of developing cognitive, behavioral, and—to a lesser extent—affective competency dimensions while also inducing a state of flow. More broadly, this research shows to what extent—and how—information systems and digitalization can contribute to the field of learning and teaching, with SGs still awaiting potential widespread adoption amongst learners and educational institutions.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Tobias Schumacher

Tobias Schumacher is a research assistant and PhD candidate at the Chair of Human Resource Management and Intercultural Leadership at ESCP Business School in Berlin, Germany. His research interests include serious games, game-based learning and the development of intercultural skills in game-based learning environments. In addition, he has also gathered practical experience in the development of serious games as a project leader, game designer and author for the serious games Moving Tomorrow – An Intercultural Journey and Moving Tomorrow – The Intercultural Journey Continues that he developed together with Marion Festing and waza! Games.

Marion Festing

Marion Festing (PhD) is a Professor of Human Resource Management and Intercultural Leadership at ESCP Business School in Berlin, Germany. Marion has founded ESCP’s Talent Management Institute and the Excellence Centre for Intercultural Management, Diversity and Inclusion. These topics also reflect her research interests. She has published in renowned journals such as Human Resource Management, Human Resource Management Review, Academy of Management Perspectives, International Journal of Human Resource Management, or Journal of World Business. In terms of teaching innovations she contributed to a MOOC on Intercultural Management and developed the serious game series Moving Tomorrow together with Tobias Schumacher and waza! Games.

Markus Bick

Markus Bick is a full professor and the Chair of Business Information Systems at ESCP Business School in Berlin, Germany. He earned a PhD in business information systems from the University of Duisburg-Essen (Germany). His research interests include digital competencies and digital maturity models, gamification, global knowledge management, and Web 3.0 technologies. He has published his work in renowned journals (Journal of Business Logistics, Information Systems Frontiers, Electronic Markets, Information & Management, Business Information Systems Engineering, Decision Support Systems, International Journal of Information Management, and Information Systems Management) and acts as an associate editor for Electronics Markets and a senior editor for Information Systems Management.

Appendix A:

Review of serious game design approaches

Table

Name	Description	Learning theoretical foundation	Integration of learning theory	Integration of learning objectives	Reference
Four-Dimensional Framework for GBL	Design framework (DF) is based on learner specifics, pedagogy, representation and context	Situated learning theory Constructivism Instructivism	Learning theories are briefly mentioned, but not explicitly related to the design process	Mentions targeted learning outcomes as being important	De Freitas (2006)
EMERGO	Methodology is based on ADDIE and specifies five phases for SG design (e.g. case idea or case scenario)	No learning theory mentioned	No integration of learning theory	Can be used for developing complex cognitive skills	Nadolski et al. (2008)
Framework for Reducing Design Complexity	DF aims to reduce design complexity at technical, conceptual and practical level	No learning theory mentioned	No integration of learning theory	Mentioned, but not explicitly integrated in the framework	Westera et al. (2008)
Educational Games Design Model	Model introduces game design, pedagogy and learning content modelling as primary elements for SG design	No learning theory mentioned	No integration of learning theory	Integrates Bloom’s taxonomy of learning outcomes	Ibrahim and Jaafar(2009)
Design, Play and Experience Framework	Multi-layered DF involving learning, storytelling, game play, user experience, and technology components	No learning theory mentioned	No integration of learning theory	Integrates Bloom’s taxonomy of learning outcomes	Winn (2009)
Six “I’s” of Game Design	Approach introduces six different nested elements for SG design such as identity, immersion, or interactivity	Cognitive information-processing Self-regulated learning Problem-based learning Experiential learning	Learning theories are reviewed but the author does not explicitly relate the learning theories to the design process	Not explicitly discussed	Annetta (2010)
P-iii Framework	DF introduces an iterative design process involving concept design, game design and game development	No learning theory mentioned	No integration of learning theory	Not explicitly discussed	Abeele et al. (2011)
Six Facets of Serious Game Design	Description of various design patterns related to six facets of SG design (e.g. pedagogical objectives)	No learning theory mentioned	No integration of learning theory	Learning objectives are part of pedagogical objectives	Marne et al. (2012)
Triadic Framework for SG Design	Design framework (DF) is based upon play, pedagogy and fidelity as core design principles	Situated learning theory Problem based learning theory Experiential learning theory	Learning theories are reviewed but the author does not explicitly relate the learning theories to the design process	Not explicitly discussed	Rooney (2012)
New Methodology of SG Design	Methodology outlines how different levels and learning mechanisms (e.g. quizzes) can be integrated in an SG	No learning theory mentioned	No integration of learning theory	Not explicitly discussed	Barbosa et al. (2014)
GAMED	Approach describes a cyclical process for SG design involving multiple phases of game design, testing, evaluating and risk analysis	No learning theory mentioned	No integration of learning theory	Not explicitly discussed	Aslan and Balci (2015)
Activity Theory-based Model of Serious Games	Description of SGs according to their learning activity, gaming activity and instructional activity	No learning theory mentioned	Activity theory is more of a meta-theory rather than a concrete learning theory	High-level educational objectives are discussed	Carvalho et al. (2015)
Gamification Service Framework	DF differentiates between different game activities, such as rules, control, social interaction or feedback	No learning theory mentioned	No integration of learning theory	Not explicitly discussed	Klapztein and Cipolla (2016)
Design Methodology with Graphical Notations	Methodology is based on graphical notations and describes six phases for SG design (e.g. design of chapters or design of scenes)	No learning theory mentioned	No integration of learning theory	Determining learning objectives is part of the first design phase	De Lope et al. (2017)
Generic Game-Based Design Model	Multi-layered design model combines skills with quizzes and game-related attributes	No learning theory mentioned	No integration of learning theory	Not explicitly discussed	El Arroum et al. (2020)

Note. The inclusion of SG design frameworks, models or methodologies is based upon the systematic literature review about approaches to SG design by Ávila-Pesántez et al. (2017) as well as upon additional research for further SG design approaches as some design approaches were missing in the review. In addition, the table only includes generic SG design approaches that can be used for creating games with different learning objectives as well for various genres (e.g. not only role-playing games). As such, SG design approaches for developing SGs for children with auditory impairments (Cano et al., 2016), for example, have not been included.

References

Abeele, V. V., Schutter, B. D., Geurts, L., Desmet, S., Wauters, J., Husson, J., Audenaeren, L. V. d., Broeckhoven, F. V., Annema, J.-H., & Geerts, D. (2011). P-iii: A player-centered, iterative, interdisciplinary and integrated framework for serious game design and development. In. S. Wannemacker, S. Vandercruysse, & G. Clarebout (Eds.), Joint Conference of the Interdisciplinary Research Group on Technology, Education, and Communication, and the Scientific Network on Critical Flexible Thinking, Ghent, Belgium.

Annetta, L. A. (2010). The “I’s” have it: A framework for serious educational game design. Review of General Psychology, 14(2), 105-112. https://doi.org/10.1037/a0018985

Aslan, S., & Balci, O. (2015). Gamed: Digital educational game development methodology. Simulation, 91(4), 307-319. https://doi.org/10.1177/0037549715572673

Ávila-Pesántez, D., Rivera, L. A., & Alban, M. S. (2017). Approaches for serious game design: A systematic literature review. The ASEE Computers in Education (CoED) Journal, 8(3), 1-11.

Barbosa, A. F., Pereira, P. N., Dias, J. A., & Silva, F. G. (2014). A new methodology of design and development of serious games. International Journal of Computer Games Technology, 2014(2), 1-8. https://doi.org/10.1155/2014/817167

Cano, S., Arteaga, J. M., Collazos, C. A., Gonzalez, C. S., & Zapata, S. (2016). Toward a methodology for serious games design for children with auditory impairments. IEEE Latin America Transactions, 14(5), 2511-2521. https://doi.org/10.1109/TLA.2016.7530453

Carvalho, M. B., Bellotti, F., Berta, R., De Gloria, A., Sedano, C. I., Hauge, J. B., Hu, J., & Rauterberg, M. (2015). An activity theory-based model for serious games analysis and conceptual design. Computers & Education, 87, 166-181. https://doi.org/10.1016/j.compedu.2015.03.023

De Freitas, S., & Jarvis, S. (2006). A framework for developing serious games to meet learner needs. Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida.

De Lope, R. P., Arcos, J. R. L., Medina-Medina, N., Paderewski, P., & Gutiérrez-Vela, F. (2017). Design methodology for educational games based on graphical notations: Designing Urano. Entertainment Computing, 18, 1-14. https://doi.org/10.1016/j.entcom.2016.08.005

El Arroum, F.-Z., Hanoune, M., & Zidoun, Y. (2020). Serious game design: Presenting a new generic creative reflection framework. International Journal of Emerging Technologies in Learning (iJET), 15(19), 247-255. https://doi.org/10.3991/ijet.v15i19.15603

Ibrahim, R., & Jaafar, A. (2009). Educational games (EG) design framework: Combination of game design, pedagogy and content modeling. In M. J. Nordin, K. Jumari, & Suwarno (Eds.), 2009 International Conference on Electrical Engineering and Informatics, Bangi, Malaysia.

Klapztein, S., & Cipolla, C. (2016). From game design to service design: A framework to gamify services. Simulation & Gaming, 47(5), 566-598. https://doi.org/10.1177/104687811664186

Marne, B., Wisdom, J., Huynh-Kim-Bang, B., & Labat, J.-M. (2012). The six facets of serious game design: A methodology enhanced by our design pattern library. In A. Ravenscroft, S. Leindstaedt, C. D. Kloos, & D. Hernández-Leo (Eds.), European Conference on Technology Enhanced Learning, Saarbrücken, Germany.

Nadolski, R. J., Hummel, H. G., Van Den Brink, H. J., Hoefakker, R. E., Slootmaker, A., Kurvers, H. J., & Storm, J. (2008). Emergo: A methodology and toolkit for developing serious games in higher education. Simulation & Gaming, 39(3), 338-352. https://doi.org/10.1177/1046878108319278

Rooney, P. (2012). A theoretical framework for serious game design. International Journal of Game-Based Learning, 2(4), 41-60. https://doi.org/10.4018/ijgbl.2012100103

Westera, W., Nadolski, R. J., Hummel, H. G., & Wopereis, I. G. (2008). Serious games for higher education: A framework for reducing design complexity. Journal of Computer Assisted Learning, 24(5), 420-432. https://doi.org/10.1111/j.1365-2729.2008.00279.x

Winn, B. M. (2009). The design, play, and experience framework. In R. E. Ferdig (Ed.), Handbook of research on effective electronic gaming in education (pp. 1010-1024). IGI Global.

Appendix B:

Review of culture-oriented serious games

Table

Game name	Year developed	Mentioned or derived field or discipline of reference	Documentation of Evaluation of Intercultural Competence Dimension				Key reference
			Cognitive	Meta-cognitive	Moti-vational	Beha-vioral
VECTOR	2000s	Intercultural awareness	-	-	-	-	Deaton et al. (2005)
ELECT BiLAT	Presumably late 2000s	Intercultural negotiation	+	-	-	-	e.g. Durlach et al. (2008)
TLCTS	2000s	Intercultural communication	-	-	-	-	Johnson and Valente (2009)
America’s Army	2000s	Intercultural awareness and collaboration	-	-	-	-	Raybourn (2009)
Second China	2007–2008	Intercultural awareness	-	-	-	-	Fishwick et al. (2010)
Icura	2000s	Intercultural knowledge	+	-	+	-	Froschauer et al. (2010)
It’s a deal	2009	Intercultural business communication	+	+	-	(+)	Guillén-Nieto and Aleson-Carbonell (2012)
Argonaut Online	Presumably 2009	Intercultural communication	-	-	-	-	Wiggins (2012)
EU-Topia	Presumably early 2010s	Intercultural awareness	-	+	-	-	El Kechaï and Pierrot (2015)
Oasistan	1999	Intercultural knowledge and awareness	-	-	-	-	de Jong and Warmelink (2017)
CCDTS	Presumably mid 2010s	Intercultural knowledge and awareness	+	+	-	-	An et al. (2019)

Note. When compiling this list, we only selected games that were intentionally designed for the purpose of developing the player’s intercultural competences. This meant that we excluded games that primarily focused on language learning (e.g. Croquelandia) or games that have been developed for another purpose but entail culture-relevant contents (e.g. Darfur is Dying). The table is adapted from de Jong and Warmelink (2017).

References

An, B., Brown, D., & Guerlain, S. (2019). The evaluation of a serious game to improve cross-cultural competence. IEEE Transactions on Learning Technologies, 12(3), 429-441. https://doi.org/10.1109/TLT.2018.2865411

de Jong, M., & Warmelink, H. (2017). Oasistan: An intercultural role-playing simulation game to recognize cultural dimensions. Simulation & Gaming, 48(2), 178-198. https://doi.org/10.1177/1046878117691076

Deaton, J. E., Barba, C., Santarelli, T., Rosenzweig, L., Souders, V., McCollum, C., Seip, J., Knerr, B. W., & Singer, M. J. (2005). Virtual environment cultural training for operational readiness (VECTOR). Virtual Reality, 8(3), 156-167. https://doi.org/10.1007/s10055-004-0145-x

Durlach, P. J., Wansbury, T. G., & Wilkinson, J. G. (2008). Cultural awareness and negotiation skills training: Evaluation of a prototype semi-immersive system. In J. A. Parmentola & D. Szkrybalo (Eds.), 26th Army Science Conference, Orlando, FL.

El Kechaï, H., & Pierrot, L. (2015). Participatory design in EU-TOPIA: A serious game for intercultural competences during work mobility. In D. G. Sampson, R. Huang, G.-J. Hwang, T.-C. Liu, N.-S. Chen, Kinshuk, & C.-C. Tsai (Eds.), 2015 IEEE 15th International Conference on Advanced Learning Technologies, Hualien, Taiwan.

Fishwick, P. A., Coffrey, A. J., Kamhai, R., & Henderson, J. (2010). An experimental design and preliminary results for a cultural training system simulation. In B. Johansson, S. Jain, J. Montoya-Torres, & E. Yücesan (Eds.), 2010 Winter Simulation Conference, Baltimore, MA.

Froschauer, J., Seidel, I., Gärtner, M., Berger, H., & Merkl, D. (2010, 20-23 Oct. 2010). Design and evaluation of a serious game for immersive cultural training. In N. Henze, P. Woźniak, K. Väänänen, J. Williamson, & S. Schneegass (Eds.), Proceedings of the 16th International Conference on Virtual Systems and Multimedia, VSMM 2010, Seoul, South Korea.

Guillén-Nieto, V., & Aleson-Carbonell, M. (2012). Serious games and learning effectiveness: The case of it’s a deal! Computers & Education, 58(1), 435-448. https://doi.org/10.1016/j.compedu.2011.07.015

Johnson, W. L., & Valente, A. (2009). Tactical language and culture training systems: Using AI to teach foreign languages and cultures. AI Magazine, 30(2), 72-83. https://doi.org/10.1609/aimag.v30i2.2240

Raybourn, E. M. (2009). Intercultural competence game that fosters metacognitive agility and reflection. In A. A. Ozok & P. Zaphiris (Eds.), Online communities and social computing (Vol. 5621, pp. 603-612). Springer. https://doi.org/10.1007/978-3-642-02774-1_65

Wiggins, B. E. (2012). Toward a model for intercultural communication in simulations. Simulation & Gaming, 43(4), 550-572. https://doi.org/10.1177/1046878111414486

Appendix C:

Instantiation of the design theory’s blueprint: introducing a cultural learning game

In the following, we summarize how the different parts of the design theory’s blueprint (meta-requirements, constructs, design requirements, and design principles) shaped the development of a cultural learning game, thereby exemplifying how the design theory can be applied to a concrete SG project.

1. Meta-Requirements

Meta-requirement 1: The SG should enable a learning experience that is in line with our knowledge of how learning unfolds.

We chose experiential learning theory (Kolb, 1984) as the learning theoretical foundation for our design theory, thus helping provide first insights into which types of learning experiences a player should make in a cultural learning game. According to experiential learning theory, learning unfolds in a cyclical process involving concrete experience, reflective observation, abstract conceptualization, and active experimentation. Each of these so-called learning modes has a distinct quality, and we aimed to ensure that players can engage all of them while playing the cultural learning game. For example, players can engage concrete experience by assuming the perspective of a young woman and by traveling to different countries and getting to know their cultures. In addition, players have time to reflect on these experiences (reflective observation) at any given point in the game, as it does not put any time pressure upon them. During abstract conceptualization, players form symbolic representations about the world that help them make sense of it. In the SG, this meant to include explanations that support players in their sense-making of the game events, thereby helping them understand more about the game world and about the learning contents (e.g. cross-cultural research insights). Lastly, players can actively experiment in the cultural learning game by choosing how they want the young woman they are steering to behave.

Meta-requirement 2: The SG should consist of different game elements that are best suited to achieve intended learning outcomes.

While meta-requirement 1 focuses on the learning process more generally, meta-requirement 2 ensures that game elements in the form of game mechanics, game components, and game dynamics are chosen in such a way that they help the SG achieve intended learning outcomes. As such, meta-requirement 2 helped us to think carefully about the game elements that we wanted to include in the cultural learning game early on in the development process. For example, one of the first game mechanics we chose to integrate was “decisions with consequences,” in that we wanted players to be able to choose between different courses of actions when interacting with non-player characters. Additionally, we wanted these non-player characters to react accordingly, thereby mimicking real intercultural interactions that have been shown to help individuals develop their intercultural skills (Earley & Ang, 2003).

Meta-requirement 3: The SG should provide a pleasant user experience that allows players to focus on and enjoy playing it.

This meta-requirement shaped the vision of how the cultural learning game should look and feel during the early stages of the development process. As we looked at existing SGs, we felt that a substantial amount of them were not able to compete with the quality of existing commercial video games. Thus, while commercial video games usually have budgets that greatly exceed those of SG projects, we still wanted our cultural learning SG to come as close as possible to the experience of playing a commercial video game. This meant choosing a genre in which development costs are usually not that high – 2D point-and-click adventure games – and trying to integrate as many game elements that would help players focus and enjoy playing this type of game. For example, we incorporated a soundtrack and voiceovers in order to give the cultural learning game a distinct identity (similar to the Monkey Island video game series). Moreover, players were also able to find so-called ‘Easter Eggs’ – references to pop culture such as movies – in the game, all of which constitute popular design features of commercial 2D point-and-click adventures.

2. Design Requirements and Design Principles

As design principles provide concrete guidance for design decisions and are deduced from design requirements (see Table C1), we will focus in the following on how design principles shaped the cultural learning game in very similar ways.

Table C1. Matching design requirements and design principles.

Design requirements	Design principles
Design requirement 1: An SG should support players in their individual learning process by accounting for the player’s past experiences as well as by not limiting their capabilities to learn from future experiences.	Design principle 1: For SG designers to develop pedagogically effective SGs, they need to ensure that the SG accounts for different levels of player experience and does not limit the player’s ability to learn after playing the game, as learning represents a continuous process according to experiential learning theory.
Design requirement 2: An SG should have clearly defined learning outcomes, which can be defined with the help of the theory of multiple foci of intelligence.	Design principle 2: For SG designers to develop pedagogically effective SGs, the SG should have clearly defined learning outcomes, which should be achieved by using the theory of multiple foci of intelligence, because research suggests that clearly defining learning outcomes is critical for developing effective SGs.
Design requirement 3: Depending on the exact learning outcomes an SG sets out to achieve, it should enable players to engage the different learning modes accordingly.	Design principle 3: In order for an SG to achieve its learning outcomes, the developer must ensure that the game elements allow players to engage the learning modes that are required to develop specific intelligence dimensions, because experiential learning theory suggests that the type of learning modes in which a player engages influences which capabilities he or she develops.
Design requirement 4: An SG should allow players to adapt to changing circumstances within the game world.	Design principle 4: For SG designers to develop pedagogically effective SGs, the SG should incorporate game dynamics that allow players to adapt to changing circumstances in the game world, because experiential learning theory suggests that learning is a holistic adaptation process that requires learners to constantly adapt to a changing world.
Design requirement 5: An SG should have transactions between the person and the game environment that reciprocally influence each other.	Design principle 5: For SG designers to develop pedagogically effective SGs, game mechanics should have consequences for the game’s dynamics and components, because experiential learning theory suggests that any given action has the ability to change objective conditions. Design principle 6: For SG designers to develop pedagogically effective SGs, the state of the game world (e.g., game components and dynamics) determines the game mechanics that are available to players, because experiential learning theory suggests that objective conditions have a significant effect on the subjective experience and the repertoire of available behaviors.
Design requirement 6: An SG should include game elements that foster the experience of flow.	Design principle 7: For SG designers to improve the likelihood of players experiencing flow while playing an SG, the SG should have clear goals, provide cognitive and immediate feedback, have good playability, an appropriate level of challenge, and engender a sense of control in players, because research suggests these are the antecedents for flow in the context of SGs.

Design principle 1: For SG designers to develop pedagogically effective SGs, ensure that the SG accounts for different levels of experience in players and does not limit the players capabilities to learn after playing the game, as learning represents a continuous process according to ELT.

Due to budgetary restrictions, we were not able to include adaptation mechanisms comprehensively (e.g., automated difficulty adaptation depending on player choices) into the SG. Instead, we opted for enabling players to personalize their learning experience in the game. For example, in the beginning of the game, the player gets to know the protagonist of the game and can decide some parts of her backstory, which has an effect on how some dialogues play out: Is the protagonist shy or simply enjoying time by herself? Where did she got to know her boyfriend? Did she study in Madrid or Turin? In addition, one of the most important game mechanics is making decisions in conversation with other characters that influence the other characters’ reactions as well as the overarching backstory and players can use their previous experiences to think about how they want the protagonist to behave toward others in general or in specific situations.

Concomitantly, we have ensured to integrate the learning material in a way, so that it does not hinder players to learn from future experiences or to make wrong inferences in future situations. For example, the GLOBE study is the most recent large-scale cross-cultural study that investigates the culture of 62 countries and provides insights into how much power distance or gender egalitarianism is valued in any given country (House et al., 2004). However, these are just average scores and if you meet someone from this country, she might or might not value power distance and gender egalitarianism in that way. That is why we also included scientific literature that discusses cultural heterogeneity within countries as well as cultural homogeneity across country borders (e.g. Venaik & Midgley, 2015), in order to not teach players potentially damaging sophisticated stereotypes (Zhu & Bargiela-Chiappini, 2013).

Design principle 2: For SG designers to develop pedagogically effective SGs, the SG should have clearly defined learning outcomes which should be done by using TMI, because research suggests that defining learning outcomes clearly is critical for developing effective SGs.

The SG aims to develop cultural intelligence including its cognitive, meta-cognitive, motivational and behavioral subdimensions (Ang et al., 2007).

Design principle 3: In order for an SG to achieve its learning outcomes, ensure that the game elements allow players to engage the learning modes that are required to develop specific types of intelligence, because ELT suggests that the type of learning modes a player engages influences which capabilities he or she develops.

This design principle requires to design the game components, mechanics and dynamics in accordance with the learning outcomes, which we will exemplify by providing our rationale as to why we chose some of these:

Game components: We included various non-player characters (NPCs) in each of the locations the user visits when playing the game (Berlin, Shanghai, Moscow) and developed character sheets that also included biographical and psychological information about these characters. While we utilized cross-cultural research results for designing them (e.g., House et al., 2004), it was particularly these character sheets that helped to make the characters idiosyncratic without compromising their cultural background or authenticity. This allowed to depict a diverse and rich landscape of different characters, thereby helping to develop cognitive (e.g. knowledge about cultural peculiarities in a country) and meta-cognitive CQ (e.g. integrating anomalies that deviate from the expected country culture).

Game mechanics: The most important game mechanic was to make decisions in interactions with NPCs, where players had to repeatedly think about (meta-cognitive CQ) the types of behaviors (behavioral CQ) they wanted to enact in conversation with other characters.

Game dynamics: The storyline has been designed dynamically, so that the player’s actions determine how the story develops. As part of the story, players were able to visit different countries in the virtual world of the game (Germany, Russia and China) and they also had to unravel a conspiracy that was threatening the company that the player is working for. In general, the story was created in order to provide a positive image of cross-cultural experiences (motivational CQ).

Design principle 4: For SG designers to develop pedagogically effective SGs, the SG should incorporate game dynamics that allow players to adapt to changing circumstances in the game world, because ELT suggests that learning is a holistic adaptation process that requires learners to constantly adapt to a changing world.

In the game, the players get to know three different work environments (Germany, Russia, and China) and have to competently navigate through these contexts. This requires players to adapt to different cultural customs as well as to different characters they are interacting with. In addition, there are various minor story twists in the game, that players had to adapt to. For example, in the game, players have an AI tool that supports them by providing insights or by telling them what to do next. However, in one of the scenes, this AI tool catches a computer virus, essentially making it impossible for the players to access its functions. Thus, they had to learn how to navigate the game world without the help of the AI tool.

Design principle 5: For SG designers to develop pedagogically effective SGs, the game mechanics should have consequences on the game dynamics and game components, because ELT suggests that any given action has the ability to change the objective conditions.

In order to ensure that player actions’ have an effect on the game components and dynamics, we developed a procedural storyline, where choosing one course of action over another would affect the reactions of NPCs (components) as well as the overarching story (dynamics). We realized the procedural storyline with a decision tree-like structure, whereby the decisions a player makes unlock a specific subset of a conversation with NPCs. This also entailed programming NPC reactions in a way, that once the player has lost their trust or sympathy, they would react sceptical or even hostile toward the player character.

Design principle 6: For SG designers to develop pedagogically effective SGs, the state of the game world (e.g. game components and dynamics) determines the game mechanics that are available to the players, because ELT suggests that the objective conditions have a large effect on the subjective experience and the repertoire of available behaviors.

The company that the player is working for in the game is organized as a “holacracy”, meaning that it has very flat hierarchies and is structured around a low degree of power distance (Robertson, 2015). This company structure (game component) forced players to talk to various NPCs in the beginning of the game (game mechanics) in order to find out which project they could contribute to and how to perform well in the company. Concomitantly, the evolving narrative of the game (dynamics) also affected what the player could do in the game (mechanics). For instance, the AI tool that is supporting the player is developing an application that tests the players’ knowledge with different puzzles. However, this feature only becomes available as soon as the AI tool has finished developing the app, which is at the end of episode 4.

Design principle 7: For SG designers to improve the likelihood of players experiencing flow while playing an SG, the SG should have clear goals, provide cognitive and immediate feedback, have a good playability and level of challenge as well as engender a sense of control in players, because research suggests these are the antecedents for flow in the context of SGs

There are different antecedents that a game needs to include in order to improve the likelihood that players experience flow during the game play (Kiili et al., 2012), which we have implemented as follows:

Clear goals: We have integrated a task feature in the game, that the players can constantly access in order to find out what to do next.

Cognitive feedback: At the end of each episode, we have included a review screen summarizing and debriefing the decisions that each player made during that episode.

Immediate feedback: The immediate feedback is usually provided by NPCs that react to what the player is doing (e.g. expressing disapproval when the player is doing something poorly).

Good playability: In order to ensure that the game has a good playability, we conducted multiple alpha and beta tests. These helped to improve the user interface and navigation in the game substantially and also yielded insights about how much learning contents we could integrate in order to not cognitively overload players with information.

Level of challenge: The level of challenge in the game has been evaluated in the related alpha and beta tests.

Sense of control: The player is able to move the avatar freely through the different locations in the game. There are only very rare instances, in which the game forces a player to enter a specific location, but, most of the time, the player can decide to either enter the location that is helping to progress the game or to enter another location and talk to the respective NPCs there. In addition, the player can control some parts of the storyline with the decisions that he or she is making, thereby further advancing the player’s sense of control.

3. Constructs

The following overview summarizes the learning outcomes and game elements, thereby showing how the constructs of our design theory are integrated in the cultural learning game.

Game Components

The cultural learning game encompasses the following game components:

• Feedback

• Locations

• Non-player characters

• Objects

• Player characters

• Quests

• Story

Game Mechanics

The cultural learning game encompasses the following game mechanics:

• Communication

• Choices

• Puzzles

• Selecting/Collecting/Finding

Game Dynamics

The cultural learning game encompasses the following game dynamics:

• Constraints

• Narrative development

• Relationships to non-player characters

Learning Outcomes

The cultural learning game aims to develop cognitive, meta-cognitive, motivational, and behavioral cultural intelligence in players.

References

Ang, S., Van Dyne, L., Koh, C., Ng, K. Y., Templer, K. J., Tay, C., & Chandrasekar, N. A. (2007). Cultural intelligence: Its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. Management and Organization Review, 3(3), 335-371. https://doi.org/10.1111/j.1740-8784.2007.00082.x

Earley, C. P., & Ang, S. (2003). Cultural intelligence: Individual interactions across cultures. Stanford: Stanford University Press.

House, R. J., Hanges, P. J., Javidan, M., Dorfman, P. W., & Gupta, V. (2004). Culture, leadership, and organizations: The GLOBE study of 62 societies. Newbury Park, CA: Sage Publications.

Kiili, K., de Freitas, S., Arnab, S., & Lainema, T. (2012). The design principles for flow experience in educational games. Procedia Computer Science, 15, 78-91. https://doi.org/10.1016/j.procs.2012.10.060

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development (1st Edition). Englewood Cliffs, NJ: Prentice-Hall.

Robertson, B. J. (2015). Holacracy: The new management system for a rapidly changing world. New York, NY: Henry Holt and Company.

Venaik, S., & Midgley, D. F. (2015). Mindscapes across landscapes: Archetypes of transnational and subnational culture. Journal of International Business Studies, 46(9), 1051-1079. https://doi.org/10.1057/JIBS.2015.11

Zhu, Y., & Bargiela-Chiappini, F. (2013). Balancing emic and etic: Situated learning and ethnography of communication in cross-cultural management education. Academy of Management Learning & Education, 12(3), 380-395. https://doi.org/10.5465/amle.2012.0221

Appendix D:

Instructions for the reflective essay

These are the instructions for the reflective essay. In order to ensure the anonymity of this submission, we changed some aspects that would directly or indirectly reveal the name of the game and, thereby, potentially the authors of this paper. However, the structure and the content of the questions remained the same.

1. How was your experience of playing the game? Write 700 words for block “1” of questions. is block of questions asks you about your general experience of playing the game and not about your cultural learning

1. How would you describe your level of enjoyment while playing the game? Why did you like or not like playing it? Think, for example, about positive or negative, surprising, inspiring, or tedious aspects of your expHerience.

2. How would you describe your level of attention while playing the game? Were you focused while playing, or did your surroundings (e.g. fellow students) or other things (e.g. cell phone) distract you?

3. How would you describe your experience of playing the two main characters of the game? Was it easy/difficult to identify with these characters? Why? Why not? Was your identification stable, or did your identification with these characters change over time or with specific events taking place?

2. How would you describe your experience of playing the two main characters of the game? Was it easy/difficult to identify with these characters? Why? Why not? Was your identification stable, or did your identification with these characters change over time or with specific events taking place?

a. To what extent do you feel that the game helped you to improve your intercultural competencies overall, by improving your knowledge about culture and intercultural management, by improving your behavioral skills in interacting with people from other cultures or by feeling more motivated to engage with people from other cultures? If so, which area did you feel improved the most/the least? Please answer this question in a maximum of one paragraph.

Questions 2b) and 2c) ask you to reflect about how the game components (2b) and game mechanics (2c) of the game affected your learning experience. Please read questions 2b) and 2c) before starting to answer the respective questions.

b. Look at all the game components listed in Table D1 and answer the following two questions. You don’t have to integrate all components in your response. You can choose one, some, or comment on all. Please be specific in your answers! If you feel that you didn’t learn anything from playing the game, please answer only question ii.

1. What were the game components that helped you to improve your intercultural competencies in terms of your knowledge, behavior, or motivation? Why did they help you and how, specifically?

2. Which one of these did not help you to learn as much or anything at all? Why not? What would you have needed the game components to be like to improve your learning experience?

Table D1. Overview of game components.

Game Components	Description
Avatar	You controlled two protagonists
Non-Player Characters	You got to know a diverse set of non-player characters throughout the game
Objects and Items	You could look at various artifacts in the game and also collect some of them
Game World	You explored four different locations
Storyline	You experienced an interactive storyline

c. Look at all the game mechanics listed in table Table D2 and answer the following two questions. You don’t have to integrate all components in your response. You can choose one, some or comment on all. Please be specific in your answers!

If you feel that you didn’t learn anything from playing the game, please answer only question ii.

1. What were the game mechanics that helped you to improve your intercultural competencies in terms of your knowledge, behavior, or motivation? Why did they help you and how, specifically?

2. Which one of those did not help you to learn as much or anything at all? Why not? What would you have needed the game mechanics to be like to improve your learning experience?

Table D2. Overview of game mechanics.

Game Mechanics	Description
Conversations	You had to talk to various non player-characters
Decisions with Consequences	You had to make decisions which affected the reactions of other characters in the game
Insights	You unlocked and read various Insights as you progressed through the game
Terminal Documents	You read various Terminal Documents as you progressed through the game
Puzzles	You had to solve four different Puzzles in order to proceed in the game
Review Screen	At the end of each episode, you could take a look at the overview of decisions you’ve made in that episode

3. How would you describe your experience with the debriefing format Pitch Your Insights? Please use specific and convincing examples from your professional/personal life or from the game to illustrate your arguments.

1. To what extent do you feel that Pitch Your Insights helped you to improve your intercultural competencies in terms of your knowledge, behavior, or motivation? If so, which area did you feel improved the most/the least?

2. Answer the following two questions. If you feel that you didn’t learn anything from Pitch Your Insights, please answer only question ii.

1. How specifically did Pitch Your Insights help you to improve your intercultural competencies in terms of your knowledge, behavior, or motivation? Please be specific.

2. Why did Pitch Your Insights not help you improve your intercultural competencies? How would a debriefing format need to be designed so that you would have learned more? Please be specific.

Appendix E:

Coding and categorization scheme for the essay responses

Enjoyment/Attention

As the tabular overview of our categorization scheme that you can find in this appendix shows (see Table E1), we decided to differentiate between high, mediocre, low, and no enjoyment and attention when coding the responses. It was deemed high when it was exclusively positive and had only very few negative remarks, if any. If students mentioned a couple of negative aspects that hindered their enjoyment or attention, but the experience was still described as generally or overall positive, we coded the response as mediocre. In contrast, if the experience was volatile or changeable throughout, we coded this as a low degree of enjoyment and attention, followed by no enjoyment or attention at all, if students had reported that they disliked playing the game or couldn’t focus on playing it. By doing so, we took a conservative stance on categorizing the responses, in order to make the differentiation between the responses more clear. One could also argue that an exclusively positive response reflects a very high level of enjoyment or attention, while an experience that is characterized as good in general could be categorized as high, but we felt a conservative categorization helps to distinguish better between the different experiences students had when playing the game, particularly since the reported user experience was—by and large—very positive.

Table E1. Overview categorization scheme for enjoyment/attention.

Code	Description	Expression	Example
Enjoyment	The degree to which players enjoyed playing the game	High	Playing the game was a real source of pleasure and joy.
		Mediocre	In general, I did enjoy playing the game because of four main reasons.
		Low	I found the game enjoyable to a certain extent.
		None	I didn’t enjoy playing the game.
Attention	The degree to which players were able to focus on the game	High	Another thing I really liked about the game was how it captured my attention for the whole hour.
		Mediocre	I was generally quite focused and I think the background music helped.
		Low	My level of attention throughout the game could be pictured as a rollercoaster. In the morning, I found myself extremely focused, but I would gradually lose my concentration throughout the day.
		None	I couldn’t focus on the game at all.

Learning Outcomes

In contrast to the coding of enjoyment/attention, we decided to code the degree of cultural intelligence (CQ) improvement as improved, somewhat improved, not improved, or not mentioned (see Table E2). The latter was necessary, as some students didn’t report on the degree of improvement with regard to some facets of CQ. We coded the degree of improvement as not improved when the students explicitly mentioned that they didn’t feel the game helped them to improve on the respective facet of CQ. For example, the statement “I wouldn’t say that the game motivated me to engage with people from other cultures, as I am already an open-minded person talking to everyone I meet.” was coded as no improvement in terms of motivational CQ. In addition, we decided to differentiate between improved and somewhat improved, in order to distinguish statements like “Furthermore, I now feel more motivated to engage with people from other cultures” (improvement of motivational CQ) from statements like “The area I improved the least is the motivation to interact with people from other cultures” (some improvement of motivational CQ), as the latter signifies an improvement, albeit on a smaller scale than the first statement.

Table E2. Overview categorization scheme for learning outcomes.

Code	Description	Expression	Example
Cognitive CQ	Cognitive CQ reflects general knowledge and knowledge structures about cultures and cultural differences	Improved	It was the first time that I learnt theories about culture and intercultural management. Hence, I established valuable knowledge for my interaction with people from other cultures in my professional and personal life.
		Somewhat Improved	Regarding my knowledge, except some specificities regarding the Russian and Chinese cultures as well as the concept of holocracy, I didn’t learn much.
		Not Improved	I didn’t acquire any new knowledge.
		Not Mentioned	-
Metacognitive CQ	Metacognitive CQ reflects the mental capability to acquire and evaluate cultural knowledge. It focuses on awareness and monitoring of cognitive processes.	Improved	The serious game not only provided valuable examples of tricky situations that I could find myself in when getting involved in a multicultural setting, but it also taught me how to better recognize arising issues and challenges, how to link them to the theories I learned in class, and how to potentially resolve them.
		Somewhat Improved	Then, on the softer and subtler side of learning, come the conversations and the decisions with consequences. This is because they allow the player to think about how they could behave in certain situations based on what the non-playable characters said or how they reacted.
		Not Improved	I didn’t feel that my ability to integrate new knowledge improved.
		Not Mentioned	-
Motivational CQ	Motivational CQ reflects the mental capacity to direct and sustain energy toward functioning and performing in intercultural situations.	Improved	This increased my motivation to interact with people from various countries so I could experience those cultural differences in real-life situations.
		Somewhat Improved	The area I felt improved the least is the motivation to engage with people from other cultures.
		Not Improved	The game also confirmed my genuine interest in learning about other cultures.
		Not Mentioned	-
Behavioral CQ	Behavioral CQ reflects the ability to flex behaviors to fit different cultural contexts.	Improved	I would say that the area I felt improved the most is the behavioral skills in interacting with people from other cultures.
		Somewhat Improved	While the game had a positive impact on my overall behavioral skills, I also feel that in interactions with people from other cultures, it would not be my instinct to think about the learnings from the game.
		Not Improved	Therefore, I distanced myself from taking all nationally depicted cultures in the game for a valid representation and did not allow my prior behavioral skills toward these cultures to change significantly.
		Not Mentioned	-