Regulation crash-test: applying serious games to policy design

Abstract

Successful policy solutions rely on policy addressees responding in certain ways. Policy designers need an analytical method that allows them to anticipate impact of a new intervention, while taking into account bounded rationality of policy actors and sociopolitical complexity. The article proposes using serious games at the stage of policy formulation to test the architecture of a new regulation in a safe environment. It provides a blueprint for using games in policy design, consisting of conceptual framework, design procedure, and techniques for strengthening game validity. The application is illustrated with an example of a draft regulation on rural transport in Poland. The case study points out three advantages of game method: (1) revealing mechanisms triggered by the architecture of regulation, meaning actors’ initial assumptions, decisions, and feedback loops created by actors’ responses, (2) demonstrating the consequences of mechanisms over time, that in real life would occur with a long delay, and (3) creating a risk-free environment where policy actors can verify their assumptions and experiment with ways of interpreting and responding to new regulation. The article concludes that serious games are a promising method for anticipating impact of complex policy regulation.

Keywords:

• Serious games
• ex ante evaluation
• game design
• policy games
• policy design
• regulatory impact

1. Introduction

Public policy is a set of ongoing experiments made by governments with often inadequate information (Campbell 1998). The goal of this “tireless tinkering” with designing and implementing interventions is to address socio-economic problems, satisfy the demands of the citizens, and ultimately make the world a better place (Howlett 2011; Peters 2017).

Successful policy solutions (projects, programs or regulations) rely on policy addressees responding in certain ways (Weaver, 2015). This requires policy designers to have a good understanding of the underlying mechanisms that drive the choices of policy addressees (Lasswell 1951; Datta & Mullainathan 2012), and apply that knowledge during the policy formulation stage (Hill 2009; Howlett 2011).

Most methods used for predicting policy response are grounded in the paradigm of neoclassical economy and its rational choice model. It assumes that policy actors have an unchanging set of preferences; they are guided by personal utility and make insightful, well-calculated decisions based on prior careful planning (Amadae 2007).

However, the rational choice model has limited utility for anticipating real responses of policy addressees’ in the complex policy settings. Humans, when confronted with complex choices, use a number of heuristics (rules of thumbs) that often diverge substantially from rational choice model of decision-making (Simon 1997; Kahneman 2011). Policy actors interpret new policy rules and regulations by filtering them through their mental models, that is implicit sets of assumptions about a certain phenomenon or situation, its operation, and its causal relations with the world (Held, Knauff, & Vosgerau 2006; Johnson-Laird 2009). Furthermore, choices are often constructed rather than elicited by social situations (Sunstein 2000, 1–10); responses of a particular policy actor can trigger unexpected feedback loops in behaviors of other participants. And finally, full visible impact of policy decisions often come with a long time delay, making them difficult to assess (Druckman 1971).

Therefore, there is a need for an analytical method that is more “human-centered,” responsive to bounded rationality of policy actors and sociopolitical complexity (Mayer 2009). This method could provide policy designers with more accurate insight into possible response mechanisms of policy addressees that ultimately determines the success or failure of particular policy initiatives.

In this article, we propose using serious games as a method for revealing mechanisms of policy actors responses to policy interventions. We argue that carefully designed policy games can be used at the stage of policy formulation to test the architecture of a new regulation in a safe environment. We illustrate our point with an example from Poland—the case of a draft regulation related to transportation.

2. Conceptual framework for policy games

2.1. Types of games for public policy

Games can be broadly defined as “experi(m)ent(i)al, rule-based, interactive environments, where players learn by taking actions and by experiencing their effects through feedback mechanisms that are deliberately built into and around the game” (Mayer 2009, 826). What distinguishes games from simple play is the presence of explicit rules (Dörner et al. 2016, 8).

In this article, we focus on serious games. Their primary purpose on training or experimenting in a low-risk environment makes them different from tabletop or computer games that were developed primarily for entertainment and recreation (Ma, Oikonomou, & Jain 2011).

Literature offers number of different typologies according to genre, medium used, game mechanics, main function, etc. (Michael & Chen 2005; Harteveld 2011; Mayer, Warmelink, & Zhou 2016). For the purpose of the public policy, we propose two groups of games based on the functional distinction. The first group of serious games is for teaching. Their primary focus is to help participants gain certain types of knowledge and skills. Examples include games used for continuous training of medical personnel (Graafland, Schraagen, & Schijven 2012), games that teach principles of spatial planning (e.g. Run that town, Urban Sciences, and B3—Design your marketplace), games educating public about budgetary process (e.g. Budget Hero) (Harteveld 2011), or games that target policy personnel (e.g. Knowledge Brokers teaching policy analysts effective strategies for evidence-informed policies) (Olejniczak 2017).

The second group of games can be called “games for testing.” Their primary purpose is to explore possible interactions and outcomes that could be triggered by the public intervention in question, as well as possible response scenarios of different stakeholders. These games are quite rare. One of the most known examples is the Rubber Windmill—a policy game used for testing the Thatcher government plan for restructuring a National Healthcare System in the UK (Duke & Geurts 2004, 136–44). The case discussed in this article also belongs in this category.

2.2. Blueprint for games application in policy design

There is a rich body of work on game design (Brathwaite & Schreiber 2008; Fullerton 2008; Selinker, Garfield, & Jackson 2012). However, it is highly fragmented into methods specific to the particular game genre, with very few commonalities in conceptual tools and design process (Salen & Zimmerman, 2004). Furthermore, approaches specific to policy games are often multidimensional and non-linear (Duke & Geurts 2004). This lack of coherence in literature makes the process difficult to replicate and report to a policy audience, which in turn could undermine the accountability of policy design.

Therefore for the policy design practice, we propose a blueprint that consists of three elements: (1) conceptual framework for translating policy reality into the game environment, (2) stages of game development process, and (3) techniques for ensuring the validity of game as a research method.

The conceptual framework enables the translation of complex reality (called: reference system) into a game in a systematic and structured way. We adapted the Institutional Analysis and Development Framework (Ostrom 2005). It corresponds well with the multi-level and multi-dimensional nature of public policy as well as its non-linear system dynamics. IAD is supplemented by the simplified logic of legal reasoning in which behavior of an actor results from motives, means, and opportunities (Michie, van Stralen, & West 2011). In an open system of policy games, opportunities are only roughly delineated, as an initial set of rules, and with a passing time they are constructed by actors’ responses and their interpretations of rules (rules in actions). Table 1 presents key terms, illustrating it with examples from the discussed game.

Table 1. Key concepts used in game design.

Concept	Description	Application in a game
Action arenas	Environment in which actors function and make decisions. There might be a number of different, overlaying arenas, each containing additional space for decision-making or resource management.	Geographical area based on Moniecki sub-region (northeast part of Poland) that included a network of towns, villages, schools and roads.
		Public tenders as parallel, formal action arena.
Actors and their characteristics	List of stakeholders substantially involved or affected by an intervention. They are characterized by three dimensions: Motives—goals, priorities and preferences that drive actors behaviors Means—individual attributes of an actor such as its type, strength, and resources at its disposal. Behaviors—decisions, choices and actions undertaken by actor	(1) Self-government—municipal level (gminas) Three separate municipalities, each managed by 1—2 players. Motives: Provide transportation services for children, who need to reach schools Eliminate “white spots” (places with no or limited access to public transportation) Maximize political support Means: Budget (different amount of money each round and municipality) Political support represented by a number of received votes Network of towns and villages and connections between them. Behaviors: Initialize tenders’ procedures on transport services for school children, and for citizens. That include setting the parameters of the tenders. Request bus companies to provide transportation services to citizens of specific towns/ villages. Acquire political support by serving communities and assuring that citizens have access to affordable transportation system. (2) Self-government—sub-regional level (powiats)Motives: The same as municipal-level, but for the whole sub-region. Means: The same as municipal-level, with the exception that it’s political support is fully depended on municipalities’ political support (it is counted not as a separate number but as a total sum of the political support from all the three municipalities). Behaviors: Coordinates of municipalities and looks for optimal solutions for the whole sub-region Supports for each municipality with some additional budget (3) Bus companies Three types of bus companies (big, medium, small, each with different numbers of buses and operational costs), each managed by 1—2 players. Motives: Deliver services to local government Compete for commercial customers (passengers) Balance operating costs and maximize profits. Means: Specific number of available vehicles (two types of buses: standard and premium) Specific amount of money that can be used to cover all the costs and make additional investments. Behaviors: Take part in tenders and make offers to each municipality Send buses to specific locations Set prices for each connection on the map Rent additional buses Pay for the fuel and other operational costs (4) Passengers There are 48 types of passengers each managed by the game system.Motives: Getting from point A to point B. Means: Each passenger has some amount of money that he/she is willing to pay to get to their destination. A passenger can pay less than that, but never more. Behaviors: Can take a bus if there is available space in the vehicle and the price of a ticket does not exceed his/her available resources. Can give or withdraw political support to self-government of specific municipality
Rules	Physical and social laws and principles that determine actors interactions Rules and responsibilities introduced by tested regulation.	Rules of public tender procedures Logic behind passengers’ choices (selecting specific bus) Rules of financial in/stability of bus companies (e.g. leasing costs, fixed costs) Ways of gaining political support related to public transportation on municipality and sub-region levels.
Action situations	The specific points in time when actors are bound to make decisions. Some of them can have substantial procedural value and may trigger other kinds of actions.	Game consists of two rounds (40–75 minutes each). Each round starts with the initiation of public tender procedures. Self-government representatives decide on how to define the tender’s requirements in terms of area of operation, number and type of buses, mode of school transportation (whether it can be used only by children or other passengers as well), selection criteria, and other elements recognized as important Bus companies analyze tenders and make decisions regarding offers Self-governments choose winners for each tender according to criteria defined earlier Bus companies sign contracts, place their buses in on the areas and receive payment Once the public tender phase is complete, bus companies decide where they will send their remaining buses and what prices they should set Every round ends with a summary of results for each involved party
Mechanisms	Interdependencies and causal connections between elements of the systems—especially actors’ behaviors and rules.	Self-governments: By setting tenders’ parameters they influence behaviors and eventually means of bus companies (e.g. companies might be forced to compete, cooperate, rent new buses, set lower or higher tickets’ prices) By deciding to create or suspend some of the connections between municipalities they influence level of political support of their own and other self-governments By lending money to other municipalities (especially in case of sub-region self-government) they influence their capability to provide transport services for schools Bus companies: By taking part in tenders they ensure that self-governments provide transportation services to schools By setting prices for tenders and for regular tickets they influence financial situation of other bus companies By sending vehicles to particular areas and setting prices they influence how many passengers can use their services. If the passengers have no access or cannot afford to buy tickets they will withdraw their political support to local self-government Passengers: The amount of money they can spend for tickets shapes local prices and influence financial situation of bus companies They increase or decrease level of political support for local self-governments
Feedback	The way in which actors learn about the progress on achieving their objectives. Scoring system is required to indicate to what extent players’ strategies are successful.	Self-governments receive feedback in the form of points representing political support at the end of each round, just after all passengers have been taken from the board. Each passenger who could not use public transport (either because he/she could not afford the ticket or there was no bus in his/her area) decrease political support of local self-government by 1 point. Moreover some specific groups of passengers can increase or decrease political support in more profound way (e.g. −10, −5 or +5 + 10). Bus companies receive feedback in the form of money tokens at the end of tender procedure and throughout the remaining phases of the game by collecting payments from passengers and then paying fixed costs and costs of buses’ operation. At the end of the round they sum up they total amount of money, which is treated as their score.

Source: own development based on (Ostrom 2005; Michie, Atkins, and West 2014).

We propose a four-phase process of game development (Figure 1). Its’ dynamic is shaped by two dimensions. On the one hand, designers balance research insight that allows them to organize reality in a systematic way, with practical actions that allow them to create the tangible game. On the other hand, designers balance complex reality, with its key distilled elements that form the game. It is important to point out that there could be number of iterations between those stages before the final game prototype is created.

Figure 1. The process of game design. Source: adapted from (Kumar 2012).

The value of games as a sound research method has been recognized in social science (De Vaus, 2006), and in public policy analysis literature (Dunn, 2017, p.18) because they allow observing the effects of ‘interventions’ in a highly controlled context. However, in order to provide accurate findings, game as a method needs to capture three types of validity (Peters, Vissers, & Heijne 1998):

• Structural validity—degree of isomorphism between structure (actors, information, rules, relationships) of the game and reference system;

• Psychological validity—degree to which players perceive the game as realistic and receptive to real-life behaviors;

• Process validity—degree of isomorphism of behaviors (actions, decisions, and interactions) between the game and reference system.

For the context of our public policy exercise, we have identified five ways for strengthening the validity of the designed game. They are summarized in Table 2.

Table 2. Ways of strengthening validity of game.

	Validity issue
Ways of addressing validity	Structural validity	Psychological validity	Process validity
(1) Involving thematic experts into the entire game design and application process	+		+
(2) Taking into consideration comments made by game testers—especially real participants of the policy in question	+	+
(3) Running the experimental session with real representatives of the actors of the system under research	+	+	+
(4) Comparing the results of the game rounds devoted to current regulation with reality (the extent to which game behaviors mirrored current status quo)	+		+
(5) Comparing the degree of consistency of dynamics and results of a few different game sessions with practitioners	+	+	+

The practical application of this blueprint is discussed in the next section on the example of rural transport regulation.

3. Case study of rural transport regulation

3.1. Policy problem

The goal of the discussed regulation was to respond to the decline of the Polish rural bus transport market and decreasing accessibility of rural areas. Total mileage of buses decreased from ca. 950 million kilometers in 2004 when Poland entered European Union, to ca. 450 million kilometers in 2016, while the number of passengers decreased from 685 millions in 2004 to 319 millions in 2016.¹ Currently approximately 20% of parishes (Stanny et al. 2014) do not have any public bus services, and many others last coach arrival and departures are available in the afternoon (2–3 pm). Many rural bus companies declared bankruptcies or cut their networks, what further negatively affected local transportation networks, excluding spatially entire communes.

Local authorities, experts, and the government agreed, that the solution should be based on the transformation from the unregulated “on the market competition” between operators with almost no regulatory power for the local authorities, to the “for the market competition” based on public service contracts awarded by the local authorities to bus companies within competitive procedures. The overall logic of the regulation was to expand capabilities of self-governments for controlling the whole transportation system by managing public tenders’ procedures and setting all their parameters for transport contract tenders, and in the same time restrict the ability of bus companies to create their own lines and connections. That, in turn, should better match the real needs of local markets. The latter solution has been working in other CEE countries, such as Hungary, Czechia or Slovakia, nevertheless it has been introduced much earlier, at the other stage of economic transformation.

Despite the general agreement on the need of transition, the legislation was shaped within rather chaotic process, which can be characterized as simultaneous convergence policy change (Dunn 2017) with a “garbage can” model of decision-making (Cohen, March, & Olsen 1972). The problem of rural bus transport redevelopment in Poland has never been neither a subject of a comprehensive diagnosis nor systemic analysis of causal relationships and logic of change. It has been developed on an ad hoc basis, from inconsistent ideas.

During subsequent attempts of drafting regulation, controversies and conflicts started to emerge stemming from two issues. First, it turned out that although general direction can be inspired by other countries, there are no easy copy-paste solutions for operational practices. Attempts of implanting solutions from other CEE countries into Polish system resulted in controversial arrangements, with little evidence indicating their possible effectiveness, but visible risks of negative side effects. Examples include an idea of awarding contracting powers to three different tiers of local authorities (that would result in coordination problems) and obligatory division of the network to the number of different operators (that could decrease the efficiency of local markets).

Second, main stakeholders (local authorities, bus companies, central government, and passengers) have been driven by very different, often conflicting motivations, highly divergent mental models—that means assumptions on how regulation would work and concerns what processes it would trigger. It resulted in pushing for different degrees of regulating the market or keeping status quo. Local authorities were conscious of the social problems, nevertheless hesitant to taking over new duties without additional funding, after the introduction of the new regulation; they were also afraid of the highly formalized procedures.

Bus operators were highly divided. On the one hand, numerous small bus companies, mostly cherry-picking the market, were afraid on taking part in public tenders, and were afraid their businesses will bankrupt in the new, regulated model; they used their political influence to restrain the transformation. On the other hand, bigger bus operators were awaiting changes, because of the constant decline of their businesses in the current system.

The central government was mostly afraid of the influence of the new regulation on the future of local businesses (as they are influential stakeholders) and wanted to have greater public control on the financing system.

Finally, the passengers (mostly high-school students and seniors) where rather passive, switching gradually to private cars or limiting their aspirations and being excluded in a way, that disables to even to protest and express their discontent.

It seemed that an ex ante evaluation of the draft regulation could show the way forward, if effectively addressing three questions:

1. What mechanisms will regulation trigger among stakeholders and their areas of operation?

2. How will the regulation affect key stakeholders, in particular the situation of local bus companies?

3. Will the regulation be effective in improving the accessibility of remote areas through a redevelopment of rural bus transport or would it have a reverse effect?

The game was intended to address these questions and verify the underlying assumptions of the stakeholders.

3.2. Game development

The team of seven experts (three transport experts, two game designers, one consultant from self-governments’ organization, one methodologist) developed the game over the course of four months (four weeks of exploration & synthesis, five weeks of design, six weeks of testing, one week of production).

The summary of the Rural Transport Game is presented in Table 3, while photos of its main board and players’ stations are in Appendix A. The design followed the four-stage process (compare Figure 1) and we explain it in more details in the rest of this section.

Table 3. Rural Transportation Game in a nutshell.

Players handle the transportation system in a sub-region consisting of four local communities. There are three types of roles in a game and seven players: representatives of local self-government (three municipalities), representative of sub-regional self-government (one powiat), and bus operators (three different companies). Passengers’ decisions are simulated since they are predictable based on literature and available predictive modeling.

Representatives of self-governments help children to reach schools, eliminate “white spots” (places with no or limited access to public transport), and maximize political support. Bus companies deliver services to local government, compete for commercial customers (passengers), balance operating costs and maximize profits. Passengers need to reach their desired destinations. Each player can influence the decisions of other players and the whole transportation system.

Each game session consists of two rounds (40–75 minutes each). Each round has three phases. In phase one, self-governments prepare tenders. They decide how to define requirements in terms of area of operation, number and type of buses, mode of school transportation (whether it can be used only by children or other passengers as well), selection criteria, and other elements established as important. These decisions can influence the level of political support that self-government will receive from citizens. Tender are put on a special board that is visible for all other players.

In phase two, bus companies analyze each tender and make their offers. Once offers are made or time for this phase expires, self-governments choose winners for each tender according to criteria defined earlier. Bus companies sign contracts, place their buses in the correct areas and receive payment. It is possible for bus companies to create more complex models of cooperation and, for example, lease additional buses or ask other players to be their subcontractors.

In phase three, bus companies decide where they will send their remaining buses and what prices they should set for each area. Each bus company is competing with other firms, and they need to efficiently attract passengers and also keep costs low. Bus companies need to constantly make decisions whether it is profitable to send a bus to particular location or just hold it in a bus depot.

The round ends with a summary of results for the whole system as well as for all involved parties. On the map players see which connections have services and where are the white spots with no transport. Self-governments check the balances of their budgets and current political support. Bus companies analyze their income, costs and profit/loss (losses can block a company from further operation and decrease the number of buses that can be used in following rounds).

3.2.1. Exploration of the system

The design started with mapping the elements of reality that are affected by the public intervention in question. The challenge was to identify and describe all key elements of the system.

For this purpose three thematic experts were involved with an extensive—both research-based and practical—knowledge about the system that will be recreated in the simulation. They brought their own expertise as well as results of their earlier studies including Taylor and Ciechański (2008; 2017) and Wolański et al (2016). Experts used an analytical framework to structure their inventory (compare: Table 1). On the technical level, designers used visual aid software that allowed creating groups of concepts and their characteristics (mind map).

3.2.2. Synthesis of the system

Games are abstractions of reality. Therefore, the challenge is to decide on key actors and players, resources, and core cause-and-effect relationships we want to portray in the game while removing the extraneous elements and reducing complexity (Duke & Geurts 2004, 202–209; Fullerton 2008, 111–147).

Since we were testing a new policy, we decided to keep every object that is directly influenced by the new regulation. These elements should be included in the game as realistically as possible. A good example is the procedure of a public tender and the ability of bus companies to decide where their areas of operation are. Since we knew that the new regulation will completely change the rules in these aspects, it was necessary to fully recreate their logic.

We also had to include those elements that are supposed to be indicators of the new policy’s success. For example, the policy-makers’ intention was to help passengers living in small villages outside the main routes to have access to decent transportation system (even though these lines would not be financially profitable). Bus owners were not interested in having their vehicles driving in such places, and they became so-called “white spots on the map”. To determine if the new policy would eliminate “white spots”, we needed to create them in the first place in a geographically diverse region.

Some of the elements had to be implemented to give players realistic boundaries, but at the same time, they could be simplified. A good example of this is the case of vehicle (bus) operating costs and a company’s financial liquidity. It is necessary to include some kind of information and restraints concerning these issues (so bus companies can make reasonable decisions), but does not need to be detailed.

3.2.3. Design of the game

Based on synthesis of the system, we started designing the world in which the players will be immersed, including what the players see, what the players do, and what the players receive. At this stage, a number of practical issues were addressed such as the structure of the game session, space and time, players, and materials.

The structure of the game session was designed as 2 days event. The first day was representing current state of affairs, and the second the situation after the introduction of the regulation. The second day introduced a new policy with a whole new set of rules. Before and after each module briefing/debriefing sessions were set for collecting results and asking questions concerning players’ strategies, motives, and understanding.

Space and time are fundamental building blocks of a game universe. We needed to decide in what spaces players will operate and what the time flow within the game will be. In our game, we designed two main action arenas: a game board that recreated geographical areas of municipalities, and public tender board with information about current tender requests. We divided the game into rounds (2 rounds per day) that were further divided into phases. This allowed maintaining order and control of the game flow. Introducing a limited amount of time for each round/phase created psychological pressure that influenced players’ decision-making processes and made it more realistic.

In complex systems, the challenge is not only to select key actors for the game, but also to decide which of the actors will be governed by players and which will be simulated by the game. If a social actor is highly predictable, then it can be simulated by the game. In our case, we decided that passenger groups could be simulated. Each passenger had a specific need (“I want to travel from A to B”), a specific amount of money to spend (“I am willing to pay for a ticket up to X $”), and clear criteria of bus selection (chose lower prices or higher quality). This characteristic made it possible to generate the whole set of different passengers and place them on the map. Other actors like owners of bus companies and local government representatives are far less predictable and these roles were given to players, who can make independent decisions in each situation.

Participants need to easily navigate throughout the game and have a number of accessories at their disposal. These accessories should be helpful for players and useful for researchers in terms of collecting data and comparing results. We designed for example forms to track players’ results, boards that contain valuable information about changes in ticket pricing, tender bids, passengers’ location and active bus lines.

The final challenge was balancing seriousness and fun. The game must be engaging in order to create the state of flow. However, engagement is not the game’s main purpose; it is only a measure for the ultimate goal: immersing players into the experience so they behave as they would do in reality.

3.2.4. Calibration of game prototype

The game is a simplification of reality, but it still needs to maintain realistic proportions in its game mechanics. The challenge is to calibrate each parameter in a way that will be both realistic and not allow players to exploit the game system (it means blocking “power-gaming”—strategies that can lead to victory in a game, but are completely unrealistic in a real life environment).

We calibrated parameters such as number of routes and bus stops on the board, number of passengers, ticket prices, number of available buses for each company, vehicle operating costs, number of rounds, and time for each round. Still the game has been remaining an open system—players were able to come up with solutions that were not strictly defined during the calibration process. Facilitators’ role was to observe and be ready to adapt game flow to the players’ innovations.

3.3. Game application

The game session took place in October 2017, in Koscierskie county, located in the North of Poland, Pomerania region (http://www.powiatkoscierski.pl/). It has been chosen for the game, due to the typical public transport market configuration with no current involvement of the public sector and an ordinary rural structure—a small city with no urban transport as a local center.

Ten people participated in the game. Five people were representatives of the self-government from the local—municipal level (polish: gmina) and a sub-regional level (polish: powiat). The other five were representatives of small, medium and large bus companies. Participants of the session were playing in the game the roles that matched their actual professional situation and expertise. None of the participants had earlier experience with simulation games.

The game session took place over two days. On day one, participants played two rounds according to the rules that mirrored the current regulation in force (status quo day). The goal of this day was threefold: familiarize players with and allow them to immerse themselves into game method, check if their behaviors in a game mirror decisions of current reality (validity of game as a method), and create a benchmark for the second day—after introduction of new regulation. At the end of day one, each player was individually interviewed about their assessment of game accuracy, and strategies they applied in the game.

The second day started with an introduction of the new rules (new regulation) followed by two rounds of the game, those were equal to two years in real life. The goal of the second day was to observe the transition moment from the old to the new law together with longer-term adaptations and effects. The second day ended with a one-hour plenary debriefing. Participants discussed then extent to which they fulfilled their objectives, key factors that impacted their decisions, key changes in their strategies during the transition from old to new law, alternative strategies they would try if they could play again, and ideas for regulation clarification and improvements.

During this two-day session, the research team collected the following data: records of all decisions made during the game session by all players, photos of the game board that indicated changing spatial situations in the region after each round, transcripts of individual interviews with players after the first day, a transcript of the group debriefing after the second day, and short notes from each participant about (a) doubts concerning regulation that appeared during session, and (b) lessons learnt from the game.

3.4. Findings of the game simulation

The analysis of the session proceedings allowed us to formulate five broad observations that address questions about the overall impact of the regulation, underlying mechanisms that led to it, and implication for stakeholders.

3.4.1. Improvement of transport network and accessibility of rural areas

The simulation showed positive impact of the new regulation, which means improvement of the transport network due to the integration of schools and public services. The accessibility of rural areas became better, even without additional funding that had been requested during the earlier discussions among stakeholders.

The game showed that improvement happened mostly due to a systematic change of behavior of those local authorities that used to procure dedicated school buses. In the new context, when confronted with design and procurement of transport services, they started to transport children to school by public buses. In Poland, because of statutory duties, school buses come to most villages, but in the majority Polish communes they are not open to the public, largely because of habits. The new regulation simply forces to rethink the status quo and change the habit. The positive impact of the regulation, confirmed by the behavior of vast majority of players, was quite surprising because, before the game, the regulation was criticized even by experts for not addressing the problem, although the synergies were well known. Thus, the game revealed an unpredicted process that at least partially contributes to solving the policy problem.

3.4.2. Transitory risks related to tenders

A group of practical conclusions was related to the dynamics of the tendering process. The first tenders that appeared at the market after introduction of new regulation were subject to strong competition between operators, as they were searching for cost recovery, feeling a threat of losing the entire market, and being left with fixed costs. Once getting most of their capacity sold and basic incomes provided, operators were searching for profit, not only by price collusion, but also by their independent actions, such as bidding at high prices or avoiding closing contracts, when they placed low bids and were the only bidder.

This forced local authorities either to repeat the tenders or, in some cases, sign a contract far above cost level and market prices. In macro-scale, this leads to the serious risk that some authorities will not close contracts or overpay. Unclosed contracts are particularly dangerous, as according to the new rules it resulted in no public services provided in the given area.

3.4.3. No need of bundling

The game showed that the players focused less on appropriate bundling of lines, although this was very important to the legislator. In addition, bundling had little to do with neither the successful (effective and efficient) contracting out of the network nor with the profits of operators. This confirmed that the act should not influence the way bundles should be created. Minibus operators’ initial assumptions on possible cross-financing between better and worse lines, turned out to be false after including school services into the public transport network. This may justify separate bundling of more and less popular lines in order to fit quality standards to operators’ competences.

3.4.4. Legal and illegal strategies of operators

The game revealed a spectrum of possible cooperation arrangements among bus operators. They started with subcontracting and consortia, and ended with, in some cases, price collusion.

In the case of the lack of buses or drivers, bus operators that won tenders often subcontracted to smaller companies. Subcontractors are in-demand both by local companies, and by operators coming from other regional markets. In the first case, winners usually did not have enough assets to cover the entire network, as earlier they shared it with competitors. In the latter, they faced challenges when establishing a new operation. After (or even before) getting a contract, they preferred to cooperate instead of covering the fixed costs of increasing their capacity.

An additional mechanism was revealed at the later stage of the procurement phase, when the supply was saturated and the competition was limited. At this stage, companies tended to withdraw solo bids or agree on placed bids, what led to enormous price increases in selected cases.

Players explained this phenomenon in the interviews. Companies were initially searching for ways to covering their basic costs (competing strongly with each other), while later for making profits (where illegal cooperation seems to be the most effective strategy).

3.4.5. Redirecting attention from imagined to real threats

Finally, the game identified a strong discrepancy between threats expressed by the stakeholders during the legislation process and threats revealed by the game. These findings call for redirecting the attention of a legislator. Instead of focusing on protecting local companies or bundling the lines, the legislator should better address—currently totally omitted—problems of protecting local governments against price collusions by the operators, and defending all stakeholders against lack of transport in the transitory period if the contract cannot be efficiently and effectively sealed.

4. Conclusions

The presented case of transport regulation in Poland showed that indeed serious games could be a promising method for anticipating complex responses to processes triggered by policy regulation. The case revealed three main advantages.

First, the game was useful in exposing mechanisms of policy addressees’ responses including their bounded rationality. Some participants reflected that, during the game, they were confronted with other players’ decisions that were inefficient. They had to respond and adapt to these surprises. It should be stressed, however, that a single game, even played number of times, can uncover disadvantages of the tested regulation, rather than directly delivering alternative solutions. New solutions may be developed by the experts and stakeholders, using other methods such as working group panels. After developing alternative solutions, the game may be re-designed and re-played to test new ideas.

Second, the game method allowed designers and stakeholders to observe the full course of events triggered by regulation, demonstrating the consequences of participants’ decisions that in real life would occur with a long delay or be spatially displaced. For example, in the discussed case game showed the changes in the dynamics of tendering with successive tenders, the evolution of behavior of local authorities in relation to school transport, and longer-term, systemic situation of rural areas accessibility.

Third, the game was useful in rising awareness and changing mental models among the stakeholders. The session allowed verifying initial assumptions of the policy actors and their concerns related to the new regulation. It showed a discrepancy between the real threats to regulation effectiveness (illegal cooperation between operators), and perceived fear (threat for local companies’ existence) initially fuelled by the lobby of small bus operators wanting to increase their market power rather than contributing to social goals. The game showed that the latter was unjustified. That, in turn, helped to redirect the attention of the legislator from perceived to actual negative mechanisms, for example, raising awareness of the need to protect local governments against price collusions by the operators and all stakeholders against the lack of transport in the transitory period.

The Rural Transport Game also provided us with a number of practical lessons on game design for public policy analysis. Policy games, just like good research studies, should be guided by a set of research questions. That provides a better focus during the exploration of the policy situation, facilitates the synthesis phase, and guides data collection during the game phase and final debriefing.

The blueprint for game application consisted of conceptual framework, design procedure, and techniques for increasing validity proved to be useful. It especially helped in identifying key elements of the system and distinguishing them from information noise. However, there is a need for more advanced methodological discussion about ways of strengthening the validity of games as a research design—both for research audience, and for policy actors who will use game results as an aid to decision-making.

Our experiment indicated that “time” is one of the key factors that have to be carefully translated from reality into game. It includes both time available for actors’ decisions (time pressure that actors are exposed to), and timing of the information and decision moments. The “time” issue was one of the factors responsible for divergent interpretations of rules by actors and their suboptimal decisions.

Designing a game during the ongoing legislation process is a shot at a moving target. The game must evolve as legal solutions change during the process. Ideally, the game session could be used to test a number of drafts within a number of rounds. However, practical limitations are quite straightforward. More than two versions of the regulation played during one session frustrate and confuse players. Furthermore, recruiting professionals to play a game for a session longer than 1.5 days is highly unlikely.

The designed structure of the game session should allow for separation of two processes—players learning the game (and gaming itself) and learning the new legislation. As some participants may not have experience in playing games, they have to learn it at the very beginning and have space to make some mistakes during this process that will not be attributed to the quality of the regulation. In the analyzed case, this effect was obtained by introducing the new legislation on the second day.

The experimental situation of a game creates certain side effects. On the one hand, players pointed out that the game situation encouraged them to make more risky decisions than in real life. On the other hand, researchers observed that in the game, players were behaving always in a legal way, while in real life that could try illegal behaviors. To overcome these limitations, it is good to run games among non-professional but expert players (such as scientists and consultants).

Finally, the debriefing after the game is an integral part of the research and data collection process. However, our game session revealed an additional potential. Animated debriefing triggered the reflection of the participants and facilitated their learning about the new regulation as well as motivations, rationalities and dynamic responses of other actors. Participants of our game session pointed out the opportunity of using game as a method of public consultation.

Looking back at the overall picture, we are convinced that linking policy design and serious gaming creates opportunities for innovative application. The presented example showed that games can provide policy designers with insight into possible response mechanisms of policy addressees, and impact of the tested policy. At the same time, game sessions can educate policy addressees about things to come. Both of these functions can ultimately improve the chances of success of a particular policy initiative.

Acknowledgements

The authors would like to express their deepest gratitude to colleagues who designed and tested the game: Łukasz Kozak, Wiktor Mrozowski and Mateusz Pieróg. The authors would also like to thank Łukasz Dąbrówka who helped to organize the game session, the authorities of Powiat Kościerski who hosted the final game, and all participants of the game experiment. Last, but not least, the authors would like to thank Michael Howlett - the co-editor of the journal and the two anonymous reviewers for their most valuable comments that substantially improved the content of this article.

Disclosure statement

Michał Wolański was a member of the advisory workgroup established by the Polish Ministry of Transport. He was a delegate of the Association of Polish Cities, without remuneration.

Note

Additional information

Funding

This work was supported by Narodowe Centrum Nauki [grant number 2014/13/B/HS5/03610].

Notes

1 The number—collected by Statistics Poland—includes long distance and rural buses, but is only official and complex number available. Other partial data (Taylor and Ciechański, 2017 and Wolański et al., 2016) suggests that the decline of rural bus services is larger than the decline of long-distance bus services.

Appendix A. The rural transport game