Reflections on ten years of using economics games and experiments in teaching

Abstract

In this paper, the author reflects on his 10 years’ experience of using games and experiments and in the process develops a type of practitioner’s guide. The existing quantitative and qualitative evidence on the impact of using games on student learning is reviewed. On balance, a positive effect, on measures of attainment, is found in the literature. Given these findings, it is surprising that there is also evidence in the UK and US that they are not widely used. Some factors are discussed that might deter tutors from employing them. Unsurprisingly, one of these is the additional cost, which might make the use of online games seem more attractive, given the way results can be automatically recorded. However, some relatively low-cost paper-based games were found to have significant advantages. In particular, they appear to facilitate social interaction which has a positive impact on student motivation and learning. One popular and effective paper-based game is discussed in some detail. A number of recommendations are provided on how to implement the game in order to maximise the learning benefits it can provide. Some ideas on how to maximise the learning benefits from using games more generally are also considered.

Keywords:

• active learning
• learning by doing
• economic games and experiments
• online vs. paper-based games
• public good games
• voluntary contribution mechanism
• economic tools for teaching

Public Interest Statement

Given the increase in tuition fees in a number of countries, there is growing public interest in the most effective ways of teaching students in higher education. In particular, many educationalists have argued that tutors should move away from traditional passive methods of teaching to those that promote a more active learning environment. This article reflects on the experience of a tutor in trying to implement one particular method of active learning—the use of short games and experiments.

The paper discusses the existing evidence on the impact of using games and experiments on different measures of student learning as well as some factors that might deter some tutors from using them. The use of one very effective game is discussed in some detail. In particular, a number of recommendations are provided on how to maximise the positive learning outcomes it can produce.

1. Introduction

In many economics classes, the content is delivered using fast-paced lectures with tutors talking for the majority of the time. Presentations are usually structured around text, diagrams and maths on a number of PowerPoint slides. Perhaps the biggest change in the method of instruction used in large group teaching over the last 20 years has been a move from “chalk and tutor talk” to “animated slide and tutor talk”. Smaller group teaching in tutorials and seminars often involves the tutor working through questions on problem sheets that have been made available on a virtual learning environment (VLE) prior to the class. The students are normally expected to have attempted the questions before attending class. These sessions can become mini-lectures with the tutor explaining the “right” answers/solutions on the whiteboard. In this case, “animated slide and tutor talk” in the lecture is then complemented by “whiteboard and tutor talk” in the tutorial, with passive learning dominating in the classroom.

Many theories of learning such as Race’s “Ripples model” (for more details see Race, 2015) stress the importance and effectiveness of active learning and learning by doing. Introducing economic games and experiments into the classroom is one alternative to the more traditional methods of teaching that a tutor can use to promote a more active learning environment. The majority of mainstream economic theory studies and predicts how people respond to different incentive structures and analyses the implications of these responses for the allocation of resources in an economy. Rather than the tutor simply explaining the theoretical predictions to a predominately passive audience, the games enable the students to learn in a more active manner. In particular, by taking part in the games, the students can experience and reflect upon the impact of different incentive structures from their own behaviour and that of their colleagues’.

In this paper, the author reflects on his 10 years’ experience of using games and experiments in the classroom and in the process offers a sort of practitioner’s guide. The focus throughout will be on discussing ways in which the learning benefits can be maximised. In short, it will discuss a number of ideas the author wished he had known about 10 years ago. Section 2 reviews the research literature on the impact of using games and experiments on measures of student learning. Section 3 discusses two commonly cited reasons given by tutors for not introducing active learning methods including games into their teaching. Section 4 compares the relative strengths and weaknesses of using either paper-based or online versions of the games. Section 5 describes one particular game that the author has found to be both popular and effective—the public good game. Different ways in which the game has been adapted over a number of years and some other recommendations on how to implement it into teaching are also discussed. Section 6 discusses how to maximise the learning benefits from using games more generally.

2. The evidence on the impact of using games and experiments

Before implementing a new method of teaching, an academic would ideally like to see evidence that it has a positive impact on student learning. This research question in the economic education literature has predominately been addressed by using educational production function analysis, i.e. the amount students’ learn is a function of a number of variables including their academic ability, how much they study and the mode of delivery employed by the lecturer. Obtaining an objective and reliable measure of student learning remains a major challenge. Most studies use the results from multiple-choice questions used as part of an in-class test or end-of-year exam. It is questionable the extent to which data from any fixed response assessment can capture the multifaceted nature of learning. Using data from open response-style assessments such as essays or vivas may provide a more holistic measurement of student learning, but involves more subjectivity.¹

One of the earliest studies that used the production function methodology in an attempt to measure the impact of using a number of games was Cardell et al. (1996). The study used the difference between post-course and pre-course test scores on multiple-choice questions taken from the Test of Understanding College Economics (TUCE) as the dependent variable. A control group received traditional lectures, while the experimental group participated in four games in their microeconomics classes and some computer-based experiments in their macroeconomics classes. After controlling for academic ability and other student characteristics, the authors found that the method of instruction had an insignificant impact on improvements in the test scores. Papers by Dickie (2006) and Emerson and Taylor (2004) used a very similar approach, but focused on paper-based games in the microeconomics section of the course.² Both studies found that the test scores of the students in the experimental group improved by significantly more than those in the control group. However, whereas Dickie found that the experiments had the biggest positive impact on the most able students, Emerson and Taylor found the biggest improvement amongst the least able students. Rather than a pre- and post-course test design, Ball, Eckel, and Rojas (2006) used results from multiple-choice questions on an end-of-year examination as the dependent variable. Some of these questions were taken from resources that accompanied the recommended textbook for the module, while some were written by the authors themselves. They also played the games online in larger classes using small wireless handheld devices. After controlling for a number of student characteristics, the authors found that the students in the experimental group did significantly better in the end-of-year examination. Durham, Mckinnon, and Schulman (2007) also developed their own specific multiple-choice questions rather than using those from the TUCE. These were written so that the students’ understanding of the specific economic principles associated with each experiment could be tested. The authors found that some experiments proved to be far more effective at improving learning outcomes than others. In particular, games where the students made a series of decisions over a number of rounds were found to be more effective than those where they only had to make one decision. Also, games where all the students participated had a more positive impact than those where the class simply observed the actions of a few volunteers.

Becker (2001) has questioned an important assumption that underpins all of this research—a more effective method of teaching must always result in more learning. If games and experiments enable students to obtain a certain level of understanding of some key economic principles with less hours of study then this is not necessarily the case. The time saved could be put to a whole range of different uses. These include studying subjects other than principles of economics, undertaking more hours of paid work or simply having more leisure time. The optimal allocation of time by the students will depend on the marginal returns from the last hour spent on each of these different possibilities. It is quite possible that the marginal return might be greater from using the time saved to study other subjects they are finding difficult to understand.

Qualitative evidence has also been obtained from surveys specifically designed to obtain responses on students’ perceptions on the impact of games on their learning. For example, Guest (2009) found that 79% of students in an intermediate economics module agreed with the statement that “per hour in the classroom I learned more about economics from games/experiments than I learned in traditional lectures”. Very positive comments came from the free-text questions. Some examples are reported below:

I feel the games have helped me a great deal as they enabled me to see the theory in action and made it more memorable

They make the concepts a lot easier to understand

Sometimes I find certain theories or concepts very difficult to understand properly when being taught in a normal lecture. However the games present me with a much greater sense of understanding and relating them to current issues or giving examples makes it easier for me to learn and remember

Also, in the standard annual questionnaires administered by the author’s own university, the students are asked to “Identify up to three things that they think are good about the module”. The games have been consistently mentioned in responses to this section of the survey by different cohorts of students over a number of years.

3. Factors that deter tutors from introducing games into their teaching

It is difficult to gauge accurately how widely games are being used in economics teaching. Some evidence is available from surveys carried out in both the US and UK. However, in each case, the representativeness of the survey respondents is unknown. The quinquennial survey of US academics found that games were used by only a small share of tutors in introductory modules and almost never in other modules, Watts and Schaur (2011). Evidence from the National Economics Students Survey carried out by the Economics Network suggests that although the use of games by tutors in the UK has increased in recent years, their incidence is still relatively low. The proportion of students who responded that they had “either rarely or never participated in games and experiments” fell from 81.9% in 2006 to 72.8% in the 2012 survey. Given the relatively positive impact on measures of student learning found in the previously discussed literature, it is perhaps surprising that the evidence suggests that they are not utilised more widely. Evidence from the research literature and discussions with lecturers suggest that there are two major issues that reduce the chances of them being used.³ One of these relates to attitudes about the quantity of material that should be covered in class-time. The other is more of a practical concern and focuses on the potential implementation costs. Each of these factors will be discussed in more detail.

Using contact time to play games, discuss results and complete the follow-up work inevitably means that less material can be covered in timetabled classes. One great advantage of more traditional and passive methods of teaching is that they enable lecturers to get through large amounts of material in a relatively short amount of time. Goffe and Kauper (2014) found “coverage of content” was the most common reason given by tutors to explain why they believed that students learn best in lecture.⁴ Perhaps tutors should focus, however, on what students have learned, rather than what they themselves have covered, Hansen, Salemi, and Siegfried (2002). The debate over the optimal coverage of content has been taking place over a surprisingly long time period. For example, Stigler (1963, p. 657) argued that

The watered down encyclopedia which constitutes the present course in beginning college economics does not teach the student how to think on economic questions …. The student will memorise a few facts, diagrams and policy recommendations and 10 years later will be as untutored in economics as the day they entered the class

A similar point was made nearly 45 years later by Frank (2006, p. 58), who stated that:

Several months after having completed an introductory economics course, most students are no better able to answer simple economic questions than students who never took the course. The problem seems to be that principles courses try to teach students far too much, with the result that everything goes by in a blur.

He goes onto argue (p. 59) that tutors should:

Throw out a lot of the detail, then choose six or seven basic principles and repeat them again and again during the term.

One argument against reducing the number of topics on principles or intermediate economics is that these courses are supposed to provide the theoretical foundations for modules in subsequent semesters and years of study. This leads to an interesting trade-off. Is it better to rely on more traditional methods of instruction that enable the tutor to cover a large amount of material or is it better to cover less material using more active methods of learning in the hope that the student will obtain a deeper understanding of the key principles? If the former approach is taken then students will have some familiarity with most of the topics that will be developed in more detail in subsequent semesters/years of study. However, the students may only have a very superficial understanding of the topics. If the latter approach is taken then the students may meet a number of concepts that they have never come across before but may have a far deeper understanding of the ones they have. Will understanding fewer topics in much greater depth or having some familiarity with a far broader range of topics best prepare students for subsequent years of study? This is a really important issue and one that requires further research.

Another potential barrier is the perceived opportunity cost. Goffe and Kauper (2014) found that the additional time and effort costs required for the preparation of alternative methods of teaching was the most common reason given by those tutors who continued to lecture, even though they thought other methods were more effective. The specific incremental costs of replacing some traditional classes with games can be grouped into three broad different areas. Firstly, there are the pre-class costs of finding, adapting and introducing suitable games into the module. Then there are the incremental costs of running the game as opposed to using more traditional methods of instruction. These include the role of the tutor in collecting, calculating, recording and disseminating the results in class. Finally, there are the post-game costs. These include the posting and integration of any results into teaching materials. Assuming that teaching and research are substitutes in the lecturers cost of effort function, the most likely opportunity costs of this incremental effort is research output. Given the incentive structure facing many academics, they may rationally conclude that the negative impact on their career prospects is considerable. In this case, simply recycling an existing set of lecture slides and problem sheets might seem the optimal course of action.

4. Online vs. paper-based games

The implications of covering less material in class-time will remain an issue no matter what medium is used by the tutor to play the games. However, the degree to which they generate incremental costs may be significantly influenced by whether paper-based or online versions are used. This issue will be discussed in more detail as well as some other strengths and weakness of using each different method.

Holt, with various different authors, published a series of papers in the Journal of Economic Perspectives and the Southern Economic Journal in the late 1990s⁵ that outlined a number of different paper-based teaching games that could be introduced into the classroom. They were designed to keep any incremental costs for the tutor to a minimum and are typically simplified versions of more formal experiments that have been developed for research purposes. They generally work in the following way:

•	The teaching sessions begin with pre-prepared instructions and record sheets being handed out to students. Copies of these instructions and records sheets are included in each of the journal articles and formatted in a way that makes them easy to copy. Versions can also be downloaded from various websites such as the Economics Network.
•	The instructions briefly outline the key features of the game including the decisions the students will have to make and the hypothetical pay-offs of these decisions.
•	In many of the games, ordinary playing cards are used as a low-cost method to facilitate decision-making by the participants in a classroom without computers.
•	The games are played over a number of rounds—typically between 5 and 20. The pay-offs are usually changed after a number of rounds to see how students respond to the different incentives.
•	At the end of each round, students are informed of their own pay-offs which they are asked to enter on their record sheets. The aggregate results are recorded by the tutor on a spreadsheet and posted on the VLE after the class has finished. These results can form the basis of follow-up work in subsequent tutorials or lectures.
•	The games normally take about 25–45 min to complete depending on how many rounds the tutor decides to run. This leaves between 5 and 25 min for some initial class discussion about the incentives and the results.
•	They are usually suitable for class sizes of between 10 and 50 students and work most effectively in tutorials or seminars. Some of the games can be adapted and used in much larger lecture-style classes.

Many computerised versions of the games are also available through a number of sources, including VeconLab, EconPort, Aplia, Feele and Moblab. Using these programmes can further reduce some of the incremental costs involved as results are automatically recorded and do not need to be input onto a spreadsheet by the tutor. In some circumstances, they also enable the instructor to run more rounds and different treatments in a given time period (see Balkenborg & Kaplin, 2009 for more detail). These advantages were found to be particularly important when playing both Bertrand and Cournot games in an intermediate microeconomics module. The logistical demands on the tutor of managing multiple hand-run oligopoly markets in the classroom proved to be considerable and caused significant disruption to the class. From experience, the computerised versions enable these games and therefore the classes to run much more smoothly. Another potential advantage with online games is that they can be run with much larger class sizes. Initially, tutors were restricted by the size of the available computer labs but with the majority of the students now owning smart devices and with a good WiFi signal they can be played in large lecture theatres.

However, having run both paper-based and computerised versions of the same games over a number of years, some disadvantages of using online versions have been observed. Most games cannot progress from one round to another until all the students have made their decisions. Therefore, they can only operate as quickly as the slowest decision-making student in the game allows. Experience has shown that rounds are far less likely to be held up in the paper-based versions for this reason. This is because the whole decision-making process is more transparent so both the tutor and the students can quickly observe which people are causing the game to run slowly. This generates peer pressure for the students to focus on the task and make decisions quickly. The use of computers in a lab or smart devices in a classroom make it more difficult to observe which participants are holding up the game. This reduces the peer pressure and students can become more easily distracted and lose focus. Another major disadvantage with the computerised games is that they do not generate the same face-to-face social interaction that occurs in hand-run games. More formal evidence from the research literature supports the experience of the author. Carter and Emerson (2012) compared the impact on students of using six computerised games (the treatment group) as opposed to six paper-based versions of the same games (the control group). They found that the participants in the control group reported significantly higher levels of interaction with their classmates than those in the treatment group. Evidence from the educational literature suggests that this social interaction can have a positive impact on learning. Carter and Emerson (2012) found no significant differences between their impact on two different measures of student learning, but did find more favourable views on learning from those who participated in the paper-based versions.⁶ The comparative impact of online vs. paper-based games is an area that requires further research. Personally, I have found it much easier to express my interest, passion and motivation for the subject when playing the hand-run games as opposed to their computerised equivalents. With the hand-run games, the tutor is much more directly involved with students in the class as opposed to a situation where the participants are simply reading the instructions and making decisions on a screen. For the reasons discussed above and after trialling different mediums, I opted to use and adapt the following five paper-based games onto a 20-credit module.

•	The negotiated price market game (based on Holt, 1996).
•	A monopoly game (paper-based version of a game on the Veconlab website).
•	A market game with asymmetric information (based on Holt & Sherman, 1999).
•	A prisoners’ dilemma game (based on Holt & Capra, 2000).
•	A public good game (based on Holt & Laury, 1997).

The public goods game has proved to be the most popular and effective. The original design first presented by Holt and Laury (1997) (henceforth referred to as H&L) will be outlined in more detail in the following section. The paper will then go on to discuss a number of specific ways it has been adapted by the author in an attempt to maximise its potential learning benefits. There will also be some discussion about how to get the most out of using games and experiments more generally.

5. A paper-based public good game

5.1. A voluntary contribution mechanism

The original design for this game came from H&L who adapted an experiment that has been used in numerous studies to test the public good predictions of economic theory. This is the voluntary contribution mechanism (VCM). Many papers closely follow the format originally used by Isaac, Walker, and Thomas (1984) where the participants are placed in a controlled computer laboratory setting and play a number of rounds in each different treatment of the experiment. They are typically allocated into groups of four and each participant is provided with a number of online tokens in each round of the game. They then have to decide how many of these tokens to allocate to either a private account (x_i) and public account (g_i). Given its public good characteristics, each token allocated to the public account increases the pay-offs of all the participants in that group. At the end of each round, the participants are provided with information on the total number of tokens allocated by members of their group to the public account and their income. The pay-off structure is illustrated below(1) $${\mathit{\pi }}_{it}=\mathit{\delta }{X}_{it}+\mathit{\gamma }\sum _{j=1}^n{g}_{jt}$$(1)

where π_it is the earnings of player i in round t; δ is the marginal return per person from investing in the private account; γ is the marginal return per person from investing in the public account.

The ratio of the marginal return per person from investing in the public account to the marginal return per person from investing in the private account (γ/δ) is often referred to in the literature as the marginal per capita return (MPCR). The experiments are designed so that the private per capita pay-off for a rational and narrowly self-interested participant from investing in the private account (δ) is greater than the private per capita pay-off from investing in the public account/good (γ), hence MPCR < 1. They are also designed so that the social return from investing in the public account/good (nγ) is greater than the social return from investing in the private account (δ).

H&L outlined how the basic design used in a VCM research experiment could be implemented in a normal classroom setting for teaching purposes whilst keeping the demands on the tutor to a minimum. The main preparation costs prior to running the game involve sorting a pack of cards into ascending order (i.e. four aces followed by four “twos” and so on) and photocopying a set of instructions and results sheets (see Appendix A for examples). The class begins with the distribution of the instructions/results sheets to the students and a brief verbal explanation by the tutor of how the game works. The cards are then dealt out to the students who can work individually or in groups. Given the way in which the deck has previously been sorted, each individual or group is dealt four playing cards consecutively with the same number or face value (i.e. four kings, four queens etc.). As there are 52 cards (ignoring the jokers) in a standard pack of playing cards, this means that there can be a maximum of 13 decision-makers in the class. From experience, the game can be run both easily and effectively with class sizes of between 5 and 52. With class sizes of 13 and under, the students can play on an individual basis, whereas with class sizes of between 14 and 52, at least some of them will have to be arranged into groups. A maximum is set at 52 because once group sizes exceed four, experience has shown that the game does not work as effectively. In the following discussion, it will be assumed that the students are playing in small groups.

Once the cards have been dealt, each group has to decide which two of their four cards they wish to contribute face down. Only the colour of the submitted cards (black or red) determines the hypothetical pay-offs. The number or face picture has no impact and it is the decision of whether or not to contribute or keep any of the red cards that ultimately determines the hypothetical rewards. In this way, the red cards (hearts and diamonds) are used as a cheap and effective way of replacing the tokens in the computerised versions of the experiment. Some examples of the exact pay-offs that can be used in the game will be discussed in more detail later in the paper.

To calculate their hypothetical pay-off, the students need to find out how many red cards have been contributed in total, so the tutor will need to:

•	collect all the submitted cards once all the decisions for that round have been made,
•	count the total number of red cards amongst those that have been submitted,
•	make any necessary calculations to work out the return from the public good,
•	record the results (possibly on a whiteboard or flipchart) and announce them to the class, and
•	return the submitted cards back to each group so that another round can be played.

This whole process needs to be done as quickly as possible to prevent the students from becoming bored and distracted. To keep the process as efficient and simple as possible, it is advisable to have the room arranged so that the students are sat in semi-circle. The tutor can then walk around the room in a clockwise direction carefully collecting the submitted cards off each group in turn. The cards should be carefully added to the stack in the order they have been collected, i.e. with the first group’s cards on the bottom of the stack and the last group’s cards at the top of the stack. Once counted and recorded, they can then be quickly returned to the correct groups by the tutor walking back around the room in an anti-clockwise direction dealing the cards face down from the top of the stack.

The decision to contribute a red card to the stack is the equivalent of allocating a token to the public account in the research experiments. In the original design outlined by H&L, a group earns £1 for each red card that they contribute. The decision to play a red card also positively effects the pay-offs of all the other groups in the class as they also receive a £1. Therefore, with 10 decision-making groups, the social pay-off from each red card played is £10. If a group decides to keep a red card, this is the equivalent of allocating a token to the private account in the research experiments. For each red card a group decides to keep, they earn £4, while the other groups earn nothing so the social return is £4. The private per capita return from keeping a red card is greater than the private per capita return of contributing it to the stack (the MPCR = 0.25), while the social return from playing a red card is greater than the social return of keeping it as long as the number of groups in the class is greater than 4 (nγ > δ).The game captures the classic public good problem with the conflict between the incentives to maximise own income and the incentives to maximise societies income.

5.2. Some advantages with the H&L design

Experience from using the design suggested by H&L illustrated a number of advantages. Using the playing cards and instructions provided in the article effectively kept preparation costs to a minimum. Setting the pay-off from contributing a red card equal to £1 also keeps the computational demands on the tutor to a minimum. The pressure of having to carry out mental arithmetic in front of a class full of expectant students might deter some people from using the game. Multiplying the number of red cards in the stack by £1 in order to calculate each groups’ pay-off from the public good is very straightforward. Hopefully, this helps to keep things as simple as possible for even the most nervous or inexperienced tutor.

Another advantage with the game design is the use of the black cards which makes it possible for decisions to remain private knowledge. If each group submits their two cards face down and they are returned by the tutor face down then it is very difficult, if not impossible, for students to find out which of the participants has contributed to the public good. The only public information is the contribution of the class as a whole when the total number of red cards in the stack is announced by the tutor. If only red cards were used, students would quickly be able to tell which groups had contributed and which had not by simply observing the number of cards submitted. Tutors do have the option to make the decisions public knowledge later in the game. Some different ways of doing this will be discussed in the next section.

5.3. Changing the design to improve learning outcomes

From the experience of running this game on numerous occasions over a number of years, the author has found a number of factors that appear to have a positive impact on the levels of student engagement and learning. These are (1) the size of the hypothetical pay-offs; (2) decision-making in teams; (3) the precise way the pay-offs are altered during the game; (4) enabling effective communication between the groups in some rounds; and (5) playing some rounds where decisions are public knowledge. Each of these factors will be discussed in turn.

Although the pay-offs are hypothetical, students seem to be influenced by their size. If they are relatively small as in the design used by H&L, some students appear to find them rather trivial and so do not exert the cognitive effort required to understand the game. This could also be an example of contextual learning where students learn more effectively when they can relate to new information or knowledge by making references to their own experiences from everyday life. Therefore, in order to increase levels of engagement, the hypothetical payments need to be large enough for the students to take them seriously and also similar to the size of any contributions they might make to a public good in their everyday life, e.g. donating to charity. From experience, multiplying the pay-offs used by H&L by 10 seems to meet both of these criteria and works effectively in the classroom.

These new pay-offs maintain the public good incentive structure and seemed to improve student engagement. The £10 figure was also chosen so it remains relatively simple for the tutor to calculate the pay-out from public good contributions in any given round, i.e. £10 multiplied by the number of red cards in the stack.

As previously discussed, students can play on an individual basis if class sizes are 13 and under, but it is always advisable to get them working in groups of between two and four whenever possible. Even with a small group of 10 students, the author has found that the game works more effectively with students working in five pairs as opposed to individually. There seems to be two key advantages of getting the students to make their decisions in groups. Firstly, it enables them to discuss the incentives involved and the implications of the different decisions they could make as the game progresses. Having this discussion take place during the game rather than only after it has finished helps them to obtain a much better understanding of the issues involved. Tutors may also have concerns that some of the students will simply not take the game seriously and occasionally this does happen. However, getting them to make decisions in groups seems to improve overall levels of engagement. When asked about this issue, discussion in a number of focus groups indicated that some students feel a sense of guilt if their lack of effort results in lower hypothetical returns not only for themselves, but for other members of their group. This sense of guilt seems to encourage more students to fully engage with the game in-class. Interestingly, the results from recent research experiments on oligopoly have also found that results are sensitive to whether the decisions are made individually or in groups.

H&L recommended that the MPCR from investing in the public/private good should be changed after a number of rounds to help maintain the students’ interest and engagement with the game. When changing this treatment variable in the classroom as opposed to a research experiment, it is always advisable to leave the pay-off from contributing to the public good unchanged. This means that the method used to calculate the pay-off from investments in the public account remain the same and so keeps the demands on the mental arithmetic skills of the tutor to a minimum. H&L suggest reducing the pay-off from contributing to the private account from £4 to £2, while leaving the pay-off of contributing to the public good unchanged. A similar change could be made to the pay-offs used above by changing the pay-off from keeping a red card from £40 to £20. This increases the MPCR from 0.25 to 0.5 and reduces the opportunity cost of contributing to the public good. It is then interesting to see if the students respond by contributing more red cards. The MPCR could also be reduced by the same magnitude by changing the pay-off from keeping a red card from £40 to £80, whilst keeping the return from contributing a red card unchanged. This increases the opportunity cost of investing in the public good and it is interesting to see if contributions fall following the change.

As well as changing the MPCR in ways which maintain the conflict of incentives but alters the opportunity cost, the author has found other designs that also seem to improve student understanding. In the research literature, treatments have been used to test for confusion as an explanation for positive contributions in a public good game. One method has been to change the pay-offs so that the MPCR > 1 (see Brandts & Schram, 2001; Palfrey & Prisbrey, 1997). This makes it both privately and socially optimal for a rational and narrowly self-interested participant to allocate all their tokens to the public account and so removes the conflict of incentives. Using a similar design in a number of rounds in a teaching experiment (see rounds 1–3 in Table 1) has also proved to be useful. Instead of testing for the existence of participant confusion, it helps to remove student confusion and hence improves their learning. It is also useful to change the incentive structure so that the MPCR < 1 but δ > nγ. In this case, it is both own income maximising and social income maximising to keep all the red cards and not invest in the public account (see rounds 4–6 in Table 1).

Table 1. Suggested format for a public goods game

Rounds	Pay-offs per red card	Incentive structure
1–3	Keep = £5: Contribute = £10	MPCR = 2: nγ > δ
4–6	Keep = £150: Contribute = £10	MPCR = 0.067: nγ < δ
7–11	Keep = £40: Contribute = £10	MPCR = 0.25: nγ > δ
12–15	Keep = £20: Contribute = £10	MPCR = 0.5: nγ > δ

5.4. Enabling effective communication whilst keeping contribution decisions private

It is also very useful to run some rounds where the groups communicate with each other before making their decisions. From experience, this works most effectively if the tutor actually leaves the room for a specified period (e.g. 2 min) when the groups are invited to discuss the game with each other. If the tutor remains in the room, the communication tends to be impeded as some students feel very self-conscious about discussing the game with their peers in front of the lecturer.

After the specified period, the tutor returns to the room and invites the students to make their decisions in the same way they have previously, i.e. with the cards face down. There are always gasps when the total number of contributed red cards is revealed to the class. Feedback from focus groups indicated that students found this inter-group discussion, in the absence of the lecturer, a really useful learning tool.

5.5. Making contribution decisions public knowledge

As previously explained, the submission of the black cards enables the tutor to keep each group’s decision private. However, it is useful to play at least one round where the decisions are made public knowledge. The following procedure is an effective way of introducing symmetric knowledge into the game. It is important to explain to the students that they will have to commit to a decision and cannot change their minds once they see the contributions of others. Each group is then invited to push forward the two cards they wish to play face down on the table and place an object such as a student’s mobile phone on top of these cards. This helps to stress the point that decisions cannot be reversed once they have been made. Once all the groups have completed this task, the tutor then walks around the room and in turn displays the two cards that have been played by each team in turns. The presentation of the cards is often met with loud cheers or boos from the other groups. The public knowledge treatment works particularly well when it is combined with the communication treatment (i.e. the students play a round with communication but their decisions remain private followed by a round where they communicate and then their decision are made public).

The format illustrated in Table 1 has been used on a number of occasions and generates good discussions in the follow-up work. The 15 rounds can easily be played in a 45-min class.

A typical set of results from using this design with a class of 30 students arranged into 10 groups of three is illustrated in Table 2. In the 14th round, the groups were allowed to communicate with one another, but their decisions remained private. In the 15th round, the groups were allowed to communicate with one another, but their decisions were made public knowledge using the process explained previously.

Table 2. Results from a previous game

Round	Total number of red cards contributed
1	10
2	11
3	11
4	5
5	4
6	11
7	10
8	7
9	5
10	5
11	8
12	5
13	4
14	8
15	16

Interestingly, it is very common in the first three rounds for a significant number of the participants to keep their red cards and so not invest in the public account. This has occurred when the game has been played with academics at conferences as well as in the classroom with students. It always leads to some really interesting discussion when the game has finished. For example, is it evidence of simple confusion or does it indicate that a number of the participants have spiteful preferences, i.e. some groups are willing to pay £5 in forgone income in order to reduce the pay-off to the other groups by £10? In rounds 4–6, it is very common for the majority of the group to keep their red cards. However, it is very unusual for the groups to realise that this is also socially efficient. When the typical public goods incentive structure is implemented, the results tend to replicate those reported in the research experiments as illustrated in Table 2—i.e. average contributions decline as the game progresses, but remain positive. Communication (see round 14) tends to have a positive but relatively small impact on contributions, which, given the nature of the conversations that have previously taken place, surprises a number of the students. Finally, making the decisions public knowledge after allowing communication (see round 15) nearly always results in a large increase in contributions. There are usually a few groups who publicly break their promises, much to the disapproval of their colleagues!

6. Maximising the learning benefits from using games more generally

To maximise the potential learning benefits of using any game, it is very important for the tutor to carefully consider and plan what happens after it has finished. As Ramsden (2003, p. 113) argues:

Student activity does not itself imply that learning will take place

Typically tutors will have between 10 and 20 minutes at the end of a one-hour class to discuss the incentives and results. Although this is important, on its own it is not enough to maximise the benefits. Gibbs (1988, p. 9) argued:

It is not enough just to do, and neither is it enough just to think. Nor is it enough simply to do and think. Learning from experience must involve linking the doing and the thinking

Also, the 5th factor in Race’s ripples model of learning stresses the importance of “providing opportunities for students to make sense of what they have learned”. Feedback from students at a number of focus groups supported this argument. One representative participant argued that:

The games are good, but the actual analysis and interpretation of the results is what I learn most from

Unfortunately, there is simply not enough time at the end of a one-hour class for students to fully reflect on the activity. There is also a danger that with limited time, the tutor will do most of the talking with the students listening in a passive manner. Therefore, it is very important that appropriate follow-up work is designed in such a way that it encourages the students to reflect and think more deeply about the game. These exercises should include the results from the games the students played and be made available prior to class via the VLE. An example problem sheet and follow-up work for the public good game is provided in Appendix B.

7. Conclusion

There is some evidence that using games and experiments can have a positive impact on student learning, but they are still not widely employed in the classroom. Tutors may be deterred from introducing them into their teaching because of concerns over (1) the impact on content coverage and (2) additional time/effort costs. Whereas content coverage is about teaching beliefs, and part of a wider debate, concern about the costs are more of a practical issue. A number of paper-based games exist that keep these additional costs to a minimum. Online versions also have the big advantage of automatically recording the results and so can further reduce these incremental costs. As the only quantitative study to compare paper-based vs. online versions of the same games (Carter & Emerson, 2012) found no significant difference in their impact on student achievement, tutors may be tempted to use the online versions.

However, from experience, the author has found other advantages of using paper-based versions. In particular, the increased social interaction that their use facilitates seems to have a significant impact on student motivation and learning. While this evidence is largely subjective (coming mainly from student feedback and differences in the quality of follow-up work), it has been observed over a number of years. It is also consistent with some of the research findings of Carter and Emerson (2012). More quantitative studies on the relative effectiveness of online vs. paper-based games need to be undertaken, but the challenge of obtaining objective, reliable and holistic measures of learning remain. It is also possible that as the technological background of students increases overtime, preferences may change, leading to online versions becoming more effective.

One great advantage with using games is their flexibility—they can be played in a number of different ways. The exact design and procedure used by the tutor can have an impact on their effectiveness in the classroom—an issue not often discussed in the literature. Using 10 years of tutor experience and student feedback, the paper has also explained how one very popular game—a paper-based public goods game—has been adapted in order to maximise the learning benefits it can provide. A specific 15-round configuration with a typical set of results has been included.

There are a number of important practical issues surrounding the use of games that have not been discussed in detail in this paper. For example, what is the optimal number of games to include in a standard 20-credit module? Do they exhibit significant diminishing marginal productivity? Are they more effective at improving students’ understanding of some topics more than others? Do they suit some types of students more than others? These issues also need to be addressed if the tutor is going to use games in the classroom in the most effective manner.

Acknowledgements

The author would like to thank Helen Guest, Robert Riegler and Caroline Elliot for all of their extremely helpful comments. He would also like to recognise all the support he has received over a number of years from the Economics Network.

Additional information

Funding

Funding. The author received no direct funding for this research.

Notes on contributors

Jonathan Guest

Jonathan Guest has recently joined Warwick Business School as a principal teaching fellow having previously spent over 20 years in the Economics Department at Coventry University. He was awarded a National Teaching Fellowship by the Higher Education Academy in 2011 in recognition of his work and is also an associate of the Economic Network. He has worked on developing teaching methods that promote a more active learning environment in the classroom and has carried out a number of funded research projects on the impact of games and experiments on student learning. His other research interests include the impact of innovative methods of assessment and feedback. Recent papers have investigated the factors that influence the self-evaluation skills of economics undergraduates. He is also the co-author of the 7th edition of the textbook, Economics For Business.

Notes

1. Many studies have found inter-marker reliability on essays in higher education to be relatively low (see Meadows and Billington (2005) and Newstead (2004) for more detail).

2. Emerson and Taylor (2004) use a gap closing measure defined as (post-course TUCE—pre-course TUCE/33—pre-course TUCE). There were 33 questions on the test paper.

3. Some other factors mentioned by colleagues include (1) concerns the game would not work (2) fear of losing control in the classroom.

4. Goffe and Kauper (2014) obtained data from 275 respondents who were attending the 2012 Allied Social Science Association meeting in Chicago. Thirty-three per cent of these respondents believed that students learn best from lecture.

5. These include Anderson and Holt (1996), Ball and Holt (1998), Goeree and Holt (1999), Holt (1996) and Holt and Anderson (1996), see references for a complete list.

6. The two different measures of student learning were (1) performance in the final course score (made up of problem sheets, midterm tests and final exam) and (2) the gap closing measure between post- and pre-course scores on TUCE. Students were asked to rate their agreement with following statement: “Experiments contributed to my learning of the subject matter in this course”. Students in paper-based group reported significantly higher agreement levels than those in the treatment group.

Appendix A

Game instructions

Each group will be dealt four playing cards by the tutor. Two of these cards will be red (hearts or diamonds), and two will be black (clubs or spades). They will have the same number or face value i.e. they will all be “fives” or “kings” etc.

The game consists of a number of rounds. When a round begins, the tutor will come to each group in order, and collect the two cards that group has chosen to play. These two cards are placed face down on top of the stack in the tutor’s hand. The group’s earnings in £s are determined by what it does with its red cards.

The pay-off from keeping a red card will be different from the pay-off from placing them on the stack. In each of the first few rounds, for each red card a group keeps it earns a pay-off of £5 for that round. Red cards that are placed on the stack affect all the groups’ earnings in the following manner:

If a group plays a red card it will earn £10 as will all the other groups in the game. The tutor will count up the total number of red cards in the stack at the end of each round, and each group will earn this number multiplied by £10. Thus if there were 10 groups in the game and each of them contributed one of their red cards there would be 10 red cards in the stack. This would result in each group earning a pay-off of £100 plus £5 for the one red card kept. When the cards are counted, the tutor will not reveal which group played their red cards. They are returned face down to so that students only know the choices made by their own group.

Black cards that are kept or placed on the stack have no impact on a group’s earnings. They simply help to keep the decisions made by each group anonymous.

To summarise, each group’s earnings for the first few rounds will be calculated as follows:

Earnings = £5 × the number of red cards kept + £10 × the total number of red cards the tutor collects.

After a few rounds, the tutor will announce a change in the pay-off for each red card that is kept. The pay-off from placing a red card on the stack will never change i.e. each group will earn £10 for every red card that is played. The tutor will also announce other changes as the game progresses.

Record sheet

Table

Round	No. of red cards kept (0, 1, or 2)	Earnings for each red card kept	Earnings for red cards kept	£10 x No. of red cards in the stack	Total earnings this round	Cumulative earnings
1		£5
2		£5
3		£5
4
5
6
7
8
9
10
11
12
13
14
15

Appendix B.

Post-game exercise

(1)	The rounds of the game were split into 4 sets. Each set had the same pay-off structure. • Set 1 refers to rounds 1–3 • Set 2 refers to rounds 4–6 • Set 3 refers to rounds 7–11 • Set 4 refers to rounds 12–15 Using the table of results shown at the bottom of this worksheet, answer all of the following questions for each of the first three sets. (a) What decisions did your group make i.e. how many red cards did you keep and how many did you contribute to the stack per round? (b) Explain the reasoning behind the decisions that you made i.e. why you decided to either keep or play the red cards? (c) What issues were raised in group discussions? Were there any disagreements over which cards to contribute to the stack? (d) Compare your group’s decisions with those of the class as a whole? (e) What was the opportunity cost of contributing a red card to the stack? (f) What was the MPCR? (g) Would it have been socially efficient for each group to keep their cards or contribute them to the stack? Explain your answer.
(2)	Using some real world examples, explain carefully what is meant if a good has the characteristic of being perfectly non-rival in consumption. Does the red card ever have this characteristic during the game?
(3)	Using some real world examples, explain carefully what is meant if a good has the characteristic of being perfectly non-excludable. Does the red card ever have this characteristic during the game?
(4)	Can you think of any problems that a firm might face if it supplied a good that was both perfectly non-rival and non-excludable in consumption?
(5)	What discussions took place between the different groups when the tutor left the room at the beginning of round 14? Did these discussions have any influence on your group’s decision? Were you surprised by the final results in this round? If so, explain why.
(6)	Did your decisions differ in round 15 compared with round 14? If so, explain why? Were you surprised by any of the decisions made by the other groups in this round?

Results

Table

Round	Pay-offs per red card	Total number of red cards contributed
1	Kept = £5:Contributed = £10	10
2	Kept = £5:Contributed = £10	11
3	Kept = £5:Contributed = £10	11
4	Kept = £150:Contributed = £10	5
5	Kept = £150:Contributed = £10	4
6	Kept = £150:Contributed = £10	11
7	Kept = £40:Contributed = £10	10
8	Kept = £40:Contributed = £10	7
9	Kept = £40:Contributed = £10	5
10	Kept = £40:Contributed = £10	5
11	Kept = £40:Contributed = £10	8
12	Kept = £20:Contributed = £10	5
13	Kept = £20:Contributed = £10	4
14	Kept = £20:Contributed = £10	8
15	Kept = £20:Contributed = £10	16