Evaluating student perceptions of using a game-based approach to aid learning: Braincept

Abstract

Gamification in higher education has steadily been gaining traction as a useful addition to the diversity of learning resources available to both teachers and students. We have invented a card-based, role-playing team game called ‘Braincept’ to help aid pharmacology learning for medical students. The aims of the current study are to determine whether the students who played the game perceived any benefit to their pharmacology learning and to gauge any learning gain as a result of playing the game. Here, we present questionnaire data and thematic analysis collected from students who played Braincept along with our data on learning gain associated with play. Our data show that this style of gamified learning has a positive effect on student confidence in handling pharmacological knowledge and that there was measurable learning gain after playing the game.

Keywords:

• Gamification
• pharmacology
• learning and teaching
• andragogy
• pedagogy

Project rationale

Gamification in higher education is a relatively recent application of an approach widely used in further education and high schools. It’s longevity in these circles is largely because it has been used to great effect to support small group learning and has been linked to concepts of andragogy in the pedagogic literature (Nevin et al., 2014). Gamification in a range of subjects, such as mathematics, engineering and natural science has been established in the contemporary educational literature as offering an ‘interesting and challenging learning style’ so long as there is appropriate care taken to ground the learning in educational theory (Hwang & Wu, 2012). By definition, games have rules that place the ‘play’ in context; they are typically thematic and goal-orientated. The terminology surrounding games for educational purposes refer to them as ‘serious games’ and whilst this might be a small misnomer in that we believe gamification of learning should have an element of fun to aid engagement, serious games such as the one we have invented here have layers of strategy depending on the complexity of the design and are ‘safe’, meaning that mistakes and the impact of these can be explored in a controlled and contextual manner. One additional benefit to game-play as an aid to learning is that games have an intrinsic motivation designed into them (Iosup & Epema, 2014). A dimension of their goal-orientated design, that links extrinsically with the player and can, in the case of a well-designed game, work to enhance the motivations of the player to continue playing and in our case, learning. Further, a central finding in the digital gamification literature is that effective games require learning objectives to be integrated into the core mechanics of the game (Hwang & Wu, 2012). It is this idea that convinced us that gamification could be an extremely useful tool when designing a novel learning aid (Iosup & Epema, 2014).

One of the reasons we opted for a game-based intervention was because when deciding on the pedagogic mode we would adopt, our literature searches convinced us of the additional benefits gamification conferred as mentioned the process and experience of game in learning can help to develop skills which would aid employability (Dascalu, Tesila, & Nedelcu, 2016). In addition, peer-assessment-based gamification mechanisms have also been shown to improve a range of critical factors necessary for ‘deep’ learning, such as creativity and motivation (Hwang & Wu, 2012). In this ‘information age’ there are a number of desirable transferable skills that prospective employers value, such as: creativity, problem-solving skills, communication and collaboration skills, decision-making skills and information literacy/handling skills; these are all aspects that influenced our decisions when developing our game (Moffat, Farrell, Gardiner, McCulloch, & Fairlie, 2015). Recent work using a collaborative electronic gamification approach to support student learning also showed improvements in attitudes, motivation and self-efficacy as a result of increased information organisation skill development (Sung & Hwang, 2013).

Student feedback as the impetus for Braincept^TM

Medical student feedback in end-of-year evaluation data consistently highlights that pharmacology knowledge is a self-perceived weakness. Students in the early years of the 5-year medical course at Keele highlighted that their ‘Pharmacology (knowledge) is insufficient’ in comments made during staff-student voice committee meetings. The majority of pharmacology teaching is delivered in Years 2 and 3 of the course and it is expected that by the end of Year 3, students should be relatively confident on the subject. However, when asked if students could explain the actions and side effects of common drugs (a General Medical Council (GMC) requirement) the percentage of Year 3 students who felt able to do this was worryingly low at 56%, thus highlighting the need for a new way to compliment teaching in the subject of Pharmacology.

Having identified this need within our learners, we created Braincept^TM, an educational card-based, team role-playing game as an aid for pharmacology learning and revision by providing a live, interactive way to explore and expand students’ current knowledge of pharmacological principles.

Game design

The first decisions we had to make were how our game would play and whether it would be a digital or a physical game (Wood, Teräs, & Reiners, 2013). We decided to make Braincept a physical role-playing team game to draw on well-established adult learning pedagogy and because encouraging real life interactions and engagement between players helps to promote and develop skills in communication and group interactions through play (Dascalu et al., 2016). Additionally we wanted our game to facilitate flexible thinking in players by allowing for changing player roles within and between games as a way to encourage deeper learning and inter-professional understanding (Knowles, 1984). It is the collaborative learning ethos of a gamified approach to pharmacology that we were most interested in adopting, as recent literature has convincingly showed that role-play and discussion with peers significantly aided learning gain (Sung & Hwang, 2013).

Braincept^TM is designed in such a way that up to 5 students can play the game to complete a series of these scenarios as a team without needing a specialist tutor to fulfil one of the roles, achieved by capitalising on the structured, goal-orientated aspects alluded to above plus the design decisions we made when creating the game. The intention was also for our game to be useful for learning outside of normal teaching activities and by non-specialists thereby making it more accessible to a wider community beyond a tutor-facilitated session.

The pedagogic aim behind playing Braincept^TM was that students would better learn to link drug names with their primary mechanism(s) of action and common side effect(s) and whilst doing so increase their confidence in handling such knowledge. The gameplay format for pharmacology learning is also relevant to students in a range of other health disciplines including; medicinal chemistry, pharmacy and nursing and has scope for developing transferable skills across these disciplines that aid both employability and enhance confidence (Nevin et al., 2014). As well as informing students in basic pharmacology, the game mechanic of Braincept^TM was also intended to encourage the players to take individual responsibility for the team role they are playing to encourage consideration of inter-professional team work to ensure the whole team succeeds in treating the pharmacological issue they are presented with. We hoped that through playing the game it would help our students to become more self-empowered in their strategy for learning alongside developing these communication and collaboration skills. (Iosup & Epema, 2014). One aspect of our game being intended to support development of pharmacology knowledge is similar in intent to the idea of concept mapping (Wang, Huang, & Hwang, 2016). Concept mapping allows learners to organise and structure learning, which is also a useful way to approach pharmacology as our game aims to increase their information handling/sorting skills through play. The addition of a gamified approach to development of these concept maps allows for a scaffold to be provided by the game that encourages incremental development of concept mapping skills via repeated play (Wang et al., 2016).

Methods

As part of refining the game mechanics and as a way to begin to evaluate the learning benefits of our invention, we explored the use and perception of a professionally produced prototype game. This was subjected to a smaller pilot study from which we reviewed and refined the game mechanics (Aynsley & Crawford, 2017). Following this we broadened into the larger scale study which is described here by running a series of practical class sessions for students to play the game as part of their pharmacology learning during a relevant module on the course. Ethical permission was granted from our institutional ethics committee. The students were allowed to work in self-selected groups of four and set the task of completing three patient scenarios randomly selected from a total of six included with the set, all based around the same body system. Our prototype set focused on the respiratory system and included common conditions such as asthma, chronic obstructive pulmonary disorder (COPD) and cystic fibrosis.

Braincept^TM was played with the entire cohort of 125 Year 2 medical students across three different sessions, comprising 43 (Monday), 41 (Tuesday) and 41 (Wednesday) students per session during a single week of the semester. Students were then given a brief explanation of the rules of the game and allowed to play the game for 90 min working in their groups. A standard ‘round’ of the game takes roughly 10–15 min and students were encouraged to rotate both the cases and their roles multiple times throughout the session.

Thematic analysis of free-text comments

As part of our mixed methods approach, we performed thematic analysis on the free-text comments from all participants with no data being omitted. We observed four emergent themes from this data-set: Physical vs. Digital; Gamification of Learning; Game Mechanics and Perceptions with representative quotes being shown below (Figure 2).

Post-gameplay, students were asked to fill out an anonymous evaluation questionnaire tool. The question instrument comprised a 5-point Likert scale with space for free-text comments, intended to collect perceptual data on player confidence before and after playing the game and to collect data on average play-time for each game played. We selected a Likert data collection method because it is a well-established way to measure opinion using a fixed choice format where participants indicate the relative strength of opinion to pre-set statements (Bowling, 1997). Using a Likert scale assumes a linear relationship in strength of feeling and whilst this might not always be the most accurate way to measure this parameter, having a 5-point scale (from strongly agree to strongly disagree), the middle point on each scale indicates a ‘neutral’ opinion that can be useful in gauging relative strength of opinion (Likert, 1932). Appropriate presentation of the Likert data takes the form of a bar chart using the mode. The reason we included a free-text comments section on our question instrument was to support the validity of the Likert data and to allow participants to qualify their opinions if needed, thereby allowing us to examine deeper into the data.

Learning gain was assessed immediately pre- and post-game as well as 3 days post-game using three short pharmacology quizzes. These quizzes consisted of three different questions on each paper but written in the same format: Q1 – For a given drug what is the mechanism of action?; Q2 – For a given mechanism of action, name a drug?; Q3 – What are the main side effects of a given drug?

Each participant was informed that there were no disadvantages, risks or benefits to taking part in this study beyond improvement of their core pharmacology knowledge and given a minimum of 24 hours’ notice of taking part and had the right to withdraw from this study at any time in accordance with our permitted ethical experimental framework.

Results

Perception

In order to ascertain whether students felt that Braincept^TM was beneficial for learning drug mechanisms, they were asked how confident they felt about the subject before and after playing the game. The data in Figure 1 show a dramatic shift in perceived confidence pre and post-gameplay where only 1% of students felt very confident and 5% of students felt confident of their pharmacology knowledge prior to playing the game. Of the remaining students, 36% felt neutral in this regard, which we interpret to mean they were unsure or unable to comment about their confidence level. Interestingly, 43% felt ‘unconfident’ and 15% felt ‘very unconfident’, revealing 58% of the cohort did not feel confident of their pharmacology knowledge prior to playing the game.

Figure 1. Graphical representation of three questions asked using the question instrument, with number of students on the Y axis and degree of confidence or agreement on the X axis; also see Appendix 1.

In marked contrast to this perceived lack of confidence in starting pharmacology knowledge, directly after playing the game for 90 min the same students reported a shift in their perceived confidence levels with 92% of students’ now self-reporting feeling ‘very confident’ or ‘confident’ and the remaining 8% of students being ‘neutral’. Surprisingly, no student selected ‘unconfident’ or ‘very unconfident’ after playing the game.

When asked to indicate whether they considered that playing the game was a good approach to pharmacology learning, 98% of students ‘agreed’ or ‘strongly agreed’ that it was and the remaining 2% were neutral, again with no-one disagreeing.

Learning gain

Learning gain was assessed immediately pre- and post-game as well as 3 days post-game using three short pharmacology quizzes of comparable difficulty. Figure 3 shows the pre-game data (graph on the left, light bars) indicating around 70% of students able to answer the first question (Q1 – For a given drug what is the mechanism of action?) correctly but this drops to only 30% being correct answering the second question (Q2 – For a given mechanism of action, name a drug?) and only 5% of students were able to correctly answer the third question (Q3 – What are the main side effects of a given drug?). Contrasting these findings, a similar quiz was administered immediately after the 90 min session and the results (graph on the left, dark bars) show that 90% of students answered question 1 correctly, 89% of students answered question 2 correctly and 60% answered question 3 correctly. The graph on the right of Figure 3 shows the results of the third pharmacology test administered 3 days later to the entire cohort, where ~70% answered question 1 correctly and ~60% answered questions 2 and 3 correctly.

Discussion

We have structured our discussion section to explore each of the four emerging themes from the data we collected in our study and finish by exploring the learning gain we measured and discuss the potential of our gamification approach to learning pharmacology moving forward.

Student perceptions were positive towards the format and approach used

Digital vs. physical

The feedback we collected from the medical students was extremely positive towards the idea of a game to assist in pharmacology learning (in agreement with our pilot study (Aynsley & Crawford, 2017). Many players used the free-text space to comment on how there was a need for something beyond traditional classroom-based delivery to address the subject. Many also commented on how much they enjoyed the in person team dynamics of the game and felt that it would aid in their understanding of drug mechanisms. In particular we were very encouraged by the student’s positive reaction to Braincept^TM existing as a physical game, provoking comments such as ‘Really useful, like that it’s real life cards’ and ‘Particularly like the fact it is a REAL game with REAL people NOT an iPhone app’. We feel the overwhelming benefits of live interactions and the resultant group learning from collaborative team participation plays firmly to the pedagogy we discussed in our introduction, where physical gameplay supports the acquisition of employer-relevant skills, such as collaboration, communication, information handling and decision-making. To further explore this interesting pedagogy particularly in the change in interactions seen whilst playing Braincept, investigations are now being planned in collaboration with colleagues from our School of Psychology to explore the ‘pedagogy of interaction’ that our live game format offers (Walsh et al., 2014).

Gamification of learning

One factor we had hoped would come out from our approach was the added pedagogic benefits of the team-playing aspect of our game to enhance student confidence in their own knowledge by seeing that their peers were also of a similar knowledge level. In all the games we observed, students were very supportive of each other while working collaboratively and maintained a positive attitude as they worked towards their mutual goals regardless of which role a student was playing ‘Really enjoyable way to practice mechanism and learn new ones’. From the findings in this theme we are more convinced than ever that gamification in medical education has tremendous potential to support student learning, especially in perceived difficult subjects such as pharmacology (Banfield & Wilkerson, 2014). Indeed, a ‘learning by doing’ approach to gamification has recently been convincingly tied to learning gain, especially when coupled with peer-assessment which encourages ‘deep’ thinking about the knowledge (Hwang, Hung, & Chen, 2014). Our game also builds on this central idea as a core mechanic, where players challenge and support each other to reach a mutual agreed goal.

Gameplay

Although perceptions of increased confidence cannot be extrapolated to imply long-term learning and knowledge it does however indicate that students felt more confident in their understanding, and that learning pharmacology in this format has potentially enhanced their awareness of drug mechanisms. We can certainly conclude that playing the game did not result in any of the volunteers losing confidence through play (Figure 1). One participant comment captures this idea very well, where they reflected that they ‘Really enjoyed playing as a team, made more active interesting learning + makes pharmacology more interesting and less daunting.’

With regards to the concept of the game, the majority of students found the rules easy to understand and the gameplay relatively straightforward to learn, commenting they found the game ‘Easy to use and understand- good for revision’ and liked the fact that they ‘could use the game on your own’. The element of fun was also a factor for many students, with comments such as ‘Fantastic game! Good fun as well as educational.’, ‘I really enjoyed it! Very fun way to learn pharmacology’ and ‘This was a real fun, interactive different way of learning a topic I feel is particularly difficult – I loved it!’ strongly coming out in the open comments post-play.

Enjoyment will logically encourage engagement with the learning activity and our data indicates this to be true, based on comments such as ‘Really good idea – using pair work made it more enjoyable and interactive’ and ‘Really enjoyed it and helped make pharmacology revision which I struggle with really enjoying.’ alongside our collected perceptual data (Figures 1 and 2).

Figure 2. Theme analysis of participant quotes – This figure collects the entire amalgamated free-text comments from all student participants which we observed grouped into four themes after playing the game. The four theme emerging from these comments were identified as: Physical vs. digital, Gamification of learning, Game mechanics and General Perceptions of the game-play experience.

Whilst Braincept^TM was envisioned as a tool which could be played by students independently in small group situations, it has not been designed to be restricted to one type of teaching session, which we believe is a strength of good game design when used in higher education (Banfield & Wilkerson, 2014). Our students indicated they would use Braincept^TM for their learning in a wide range of potential situations, specifically mentioning its use as a revision aid commenting ‘Really good game for learning pharmacology. It made revision fun and allowed me to memorise mechanics of action more easily’ and supported in Figure 2. Indeed, our gameplay format for pharmacology learning may be relevant to students in a range of other health disciplines including; medicinal chemistry, pharmacy and nursing and has scope for developing transferable skills across these disciplines that aid both employability and enhance confidence and our future work will explore to what degree this might be true (Nevin et al., 2014).

Learning gains

Learning gain was assessed immediately pre- and post-game as well as 3 days post-game using three short pharmacology quizzes of comparable difficulty. In our experience, medical students tend to learn pharmacology in a linear perspective of ‘this drug works like this’, which may explain the marked difference between the pre-game test scores between Question 1 (70%) and Question 2 (30%), where the success rate dropped quite significantly when the question reversed from this ‘typical’ linear order of knowledge gain (Figure 3). Taking this idea further, we were surprised by their low test scores when asked about common side effects (5%), which we interpret as simply a deficit in their core knowledge, having not synthesised deeper than the rote learning we typically see.

Figure 3. Comparison of test performance at three points in the study: before, immediately after and 3 days after playing the game. The test format was administered in the form of three conserved question styles in each case; Q1 – For a given drug what is the mechanism of action?; Q2 – For a given mechanism of action, name a drug? and Q3 – What are the main side effects of a given drug? Data shows the percentage of correct answers on the Y-axis and the specific question on the X- axis.

Contrasting these pre-game findings, a similar quiz was administered immediately post-game and the results showed the expected short-term increase in pharmacology knowledge and interestingly, a more even performance across the three styles of question (90% for Question 1; 89% for Question 2 and 60% for Question 3 – Figure 3). We believe this shows a short-term knowledge gain and a positive pedagogic change in the manner of learning away from linear rote methods towards a more comprehension-based acquisition of pharmacology knowledge. Students comments support this assertion, where our learners shared ‘Interesting, it really tests you understanding on diseases and what treatments are suitable’ along with ‘I feel this game was a good tool in identifying the gaps in my knowledge’ as well as ‘It allowed me to relate drug names, actions and side effects easily and make it memorable’.

The real test of learning gain and whether or not our game-based approach aids longer-term learning came in the third test, administered 3 days post-game without access to the game during that time. An impressive 70% of the cohort answered Question 1 successfully and 60% could still correctly answer Questions 2 and 3 after 3 days (Figure 3). This could be interpreted one of two ways and we intend to examine this deeper in future work.

•	The single 90 min session with the game was enough to allow synthesis of their pharmacology learning and improve retention of this information, as evidenced in their testable scores sustained over 3 days.
•	An alternative explanation is that by playing the game students were able to self-evaluate their learning gaps and that in those 3 days they engaged in private study and an increase in their basic knowledge reflected in an increased test score.

From a pedagogic standpoint, we do not really mind which of these is actually true and we suspect it is likely a mixture of both hypotheses. However, it would be informative to know whether we were measuring a change in practice or purely an increase in knowledge gain particularly in respect to other skills that we are attempting to develop in our students. If by engaging with the game they are gaining a better understanding of their own limitations and development needs, this way of thinking could be easily transferred to other subjects and skills as they develop into self-sufficient employable graduates. We would also hypothesis that we may also be seeing an effect akin to ‘transformative pedagogy’, where understanding has been enhanced by the social, collaborative aspects of playing the game, which lends itself to discussion-based information handling and self-evaluation of gaps via shared understanding (Dolan et al., 2013). One limitation we feel is worth pointing out is that without a control group being present it is not possible to know whether simply playing the game results in positive perceptual feedback. Whilst we fully acknowledge this might be one interpretation of the perceptual data, the learning gain results would suggest this is not the only factor in effect here.

Interestingly, whilst we were observing gameplay it became noticeable that groups of students approached the game differently, with some being very possessive over their individual roles whilst others worked much more closely as a larger team. Whilst this did not noticeably detract from the learning taking place (supported by the perceptual data in Figure 1), it highlights the inherent flexibility of a game-based approach to support student learning preferences (Walsh et al., 2014) ‘Mechanisms seem to have stayed in my mind’. Although this is beyond the scope of the current study, our future work will explore this interesting observation further.

Final perspectives

We believe our approach and the Braincept^TM game itself has the potential to have a wide reaching benefit to student pharmacology learning by supporting learning of what is traditionally perceived as a rather difficult subject. Our data show that the students like this style of learning and that it has a positive effect on their confidence handling pharmacological knowledge, with comments such as ‘Very interactive useful session. It has increased my knowledge on pharmacology’ and ‘Good way of approaching pharmacology revision’ supporting this.

Our data exploring learning gain both immediately and after 3 days, showed improved retention of knowledge after playing our game and highlights the possibility that Braincept^TM could be useful to support learning at different points in the students’ education (in-class, revision tool, small group work instrument). In addition, we believe that the positive effects shown here are applicable to a wide range of topics (as the subject of the game) and also the development of personal skills by engaging in learning in this manner.

Our future work will test the game with other types of students who also have a need for pharmacology learning support (pharmacists, medicinal chemists, nurses etc.) to allow comparison with medical student experience playing the game and also to better understand the post-game changes in our students that lead to the learning gain shown here (Appendix 1).

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

This work was supported by the UnLtd Social Enterprise and Keele University School of Medicine.

Acknowledgements

We would like to thank UnLtd for their generous development funding of our game though a ‘Try It’ social enterprise award. We would also like to thank Dr. David Harvey for his work on development of the game and Keele University School of Medicine for their continued support of the Braincept project.

Appendix 1

Source of quotes

The evaluation forms contained an area for free-text comments (please see example attached, typical of the feedback from the groups – full data available upon request), the discursive student feedback on the game from just this pilot phase are as follows:

Example evaluation form Page 1 (of 2).