Game-based career learning support for youth: effects of playing the Youth@Work game on career adaptability

ABSTRACT

Choosing a career is one of the most important decisions that youth has to take but many young people find this a hard issue to engage with. Current career counselling practice does not appear very compelling or motivating to young people. Professional games could provide a more engaging and motivating way of acquiring professional awareness and competence for career decision making and learning. We present the design and effects of playing a game that aims to increase career awareness and adaptabilities in youth (13–19 years). In a Randomized Controlled Trial, 93 high school students from Iceland and Romania were asked to carry out career-oriented activities, with half playing an interactive game and the other half performing a paper-and-pencil version of the same activities. The students were compared on career adaptability, career learning and career awareness scores before and after these interventions. Main results show that engaging players in these career-oriented activities has short term effects on outcome scores for career adaptabilities and for perceptions of career learning competences. Students who played the game report significantly faster growth on career adaptabilities that deal with “concern”, “control” and “confidence”. It can therefore be concluded that introducing game-based learning in career decision support for youth is a promising endeavour.

KEYWORDS:

• Game design
• professional games
• career decision making
• career awareness
• career adaptabilities
• career learning
• career competence framework
• vocational interests

1. Introduction

Choosing a career is one of the most important decisions a young person has to make, but is also one that many find difficult to relate to adequately and at the appropriate time. Young people's confidence, career aspirations, awareness of possibilities and attitudes towards suitable professions to a large degree determine later career choice and success (Hodkinson & Sparkes, 1997). It is therefore crucial that young people start thinking about professional careers at an early stage and in a both positive and realistic way. Acquiring sufficient career awareness and career decision management skills (like setting targets, planning and taking ownership) can be expected to positively increase career outcomes and reduce the risk of dropping out of school later (Law, 1996; Roberts, 1997; Watts, 2001). For many young people however, current career counselling practice does not appear to be compelling or motivating (Amundson, 2003; Lovén, 2003). We need more engaging and active ways of supporting youth in career decision making, ways that are more aligned with their real life learning.

Games that aim to foster the acquisition of learning objectives are called “serious” to denote that they are not just fun to play, but also hold potential as cognitive tools for learning (e.g. Michael & Chen, 2006; Connolly, Boyle, Hainey, MacArthur, & Boyle, 2012). Game-based learning can be a valuable way to engage students in learning, as it fits well with their daily computer use. Especially in vocational education, motivation and effective acquisition of professional skills have been problematic and in need of improvement (e.g. Van der Veen, Weijers, Dikkers, Hornstra, & Peetsma, 2014). So called “professional” games are increasingly being used as a more motivating and immersive way to have learners experience work challenges and assess them on practice skills for professional life in context (Hummel, Geerts, Slootmaker, Kuipers, & Westera, 2015; Hummel, Nadolski, Eshuis, & Slootmaker, 2016; Hummel, Nadolski, Joosten-ten Brinke, & Baartman, 2017). Review studies have shown that effective game design is key (“garbage in, garbage out”) for achieving actual motivation and learning effects in education (Boyle, Hainey, et al., 2016; Clark, Tanner-Smith, & Killingsworth, 2016).

The combination of school and gaming has potential to increase learning, especially for lower performing, disengaged students (Shute, Ventura, Bauer, & Zapata-Rivera, 2009). Playing games in education is generally known for its contribution to improving motoric skills or gaining knowledge about certain school topics. Less known is that serious games also foster the acquisition of more complex and more generic skills, like problem solving and professional competence or awareness (Guillén-Nieto & Aleson-Carbonell, 2012; Yang, 2012), like is required in career decision making. We will therefore argue and examine the compelling and innovative idea to develop and use a serious professional game to support youth in their career decision making. The use of TEL in career counselling has been rare and existing career games are very scarce (with My Tycoon probably being the most relevant exception). Boyle, Allan, et al. (2016) did identify a few papers about games for careers, going back as far as 1971, but none of these contained high quality experimental evaluations of resources.

Serious games for acquiring professional competence (like career decision making) have been found to offer learning activity that better stimulates intrinsic motivation when compared to more traditional ways of learning (Garris, Ahlers, & Driskell, 2002; Ryan & Deci, 2000; Tsai, Tsai, & Lin, 2015; Wouters & van Oostendorp, 2013). Professional games stand the challenge of being both authentic (realistic) and playful (engaging) at the same time. This implies a design effort that requires close collaboration between content experts (i.e. in career counselling practice), game developers and instructional designers. To warrant authenticity, learning contexts have to resemble contexts where students apply what has been learned. In that way learning becomes motivating and more likely to transfer to real world situations (Herrington, Oliver, & Reeves, 2003). To warrant playful learning, the gameplay has to integrate personalised feedback, support and scoring mechanisms in an unobtrusive way (i.e. embedded in the game narrative). Current approaches for integrating personalised feedback gather data about learners’ progress and are technically grounded in learner modelling (Khenissi, Essalmi, & Jemni, 2015). Creating such a learner model requires many observations of learner-game interactions and interpreting these in terms of progress towards learning outcomes. As an example, Shute and colleagues used the framework of Evidence Centered Design (ECD) to develop conceptual assessment models, which in turn support the design of valid assessments (Shute & Kee, 2012). Other researchers (Arnab et al., 2015; Hummel et al., 2017) present frameworks describing how to map the desired learning from a game (learning outcomes) to the game mechanics (gaming outcomes). Such models also emphasise the need to clearly specify the desired learning outcomes and translate these into linked game activities.

This paper describes an empirical study we carried out with the Youth@Work game (Boyle et al., 2016a; Hummel, Boyle, et al., 2015) as a potential solution for more experiential and active career learning. We developed this game to better motivate young people in exploring aspects of themselves and their potential occupations which is considered critical for enabling career decision making and development. Promoting a more active engagement is considered to be an important added value of game-based approaches besides traditional career counselling approaches (Amundson, 2003) and the available career information sites and systems (Sampson & Osborn, 2015). The game was developed as the main deliverable of the YOUTHYES project (Erasmus+ KA2, 2015–2017). We expect and hypothesise that playing this game will increase career awareness and adaptability in young people. From players in various countries we have collected both more qualitative and more quantitative data on their learning outcomes and satisfaction by conducting pre- and post-test questionnaires and by computer logging gaming data. This paper will focus on the more quantitative results that were obtained from a Randomized Controlled Trial that compared pre/post questionnaire scores on career awareness and adaptability (as main learning outcomes). The way we managed the challenge of balancing authenticity and playfulness in our game design is explained in the subsequent sections 2 and 3.

Section 2 (theoretical background) describes the most influential career counselling frameworks that were selected as an authentic and theoretical foundation for designing the main game structure (according to so called “zones”). Section 3 (method) contains a description of the Youth@Work game (play) and the assessment instrument for measuring learning outcomes. We explain how career counselling practice tasks have been mapped upon the learning scenario and its activities (into so called “mini-games”), and how feedback and scoring mechanisms have been unobtrusively integrated to ensure playful learning. We also describe our measuring instrument (that includes scales on career adaptability, awareness and learning). Sections 4 and 5 will present and discuss most important results from our controlled trial comparisons (before and after activity, between players and non-players, between countries), together with some recommendations for career counselling practice and future research.

2. Theoretical background

Career learning and development is a long and complex process that has not been described in any coherent and integrated theoretical framework (e.g. Brown & Lent, 2013; Walsh, Savickas, & Hartung, 2013). Life-long guidance studies from Europe have focussed on career management skills and learning required (e.g. Law, 1996; Watts, 2001). We selected the Skills Development Scotland (SDS, 2012) framework to provide a broad overview of skills people need to manage their careers. From US studies that focussed on individual psychological differences and their measurement we know that when people choose an occupation that fits their interests, values and abilities in they feel more satisfaction with and are more successful in their careers (e.g. Savickas, 2005). We decided to use Holland’s (1959, 1997) model of vocational interests because interests are considered especially important for career development (Savickas & Spokane, 1999). This section provides a description of both frameworks that theoretically underpin our game and define the main learning outcome variables: career adaptability, career awareness and perceptions of career learning and development.

2.1. Skills development Scotland competence framework

This first framework provides a consistent definition of career management competences that young people need to be aware of in thinking about their careers, organised around four themes: Self: Competences that enable individuals to develop their sense of self within society; Strengths: Competences that enable individuals to acquire and build on their strengths and to pursue rewarding learning and work opportunities; Horizons: Competences that enable individuals to visualise, plan and achieve their aspirations throughout life; and Networks: Competences that enable individuals to develop relationships and networks of support. (see https://www.skillsdevelopmentscotland.co.uk/what-we-do/our-products/career-management-skills/) The SDS model seemed to provide an intuitive and useful framework for structuring the game into 4 zones, corresponding to these 4 different areas of competence, and also provided some indication of activities that might take place in these different zones. A final zone of the game, which was not part of the SDS competence framework, was required to provide an interesting conclusion to the game where the player finds out which of the careers he/she is most suited for.

2.2. Holland's vocational interest model

The second major career theory that was used in the game design was Holland’s (1959, 1997) model of vocational interests. This theory is grounded in the study and measurement of individual differences, based on the premise that people who are able to choose an occupation that fits their interests will feel greater satisfaction and be more successful (Savickas & Spokane, 1999). Holland argued that vocational interests can be categorised in terms of six main RIASEC interest types: realistic, investigative, artistic, social, enterprising, and conventional.

Additionally, Prediger (1999) suggested that two bipolar dimensions (People-Things and Data-Ideas) underlie Holland's catergorization of vocational interests (see Figure 1). Interest inventories that measure the RIASEC interest types have been developed and used extensively in career guidance and counselling (Savickas & Spokane, 1999; Tracey & Sodano, 2013). The RIASEC categories have also been used to characterise actual careers in terms of the activities and skills required to perform different careers (see for example www.onetcenter.org). In the Youth@Work game, Holland's individual differences are nested in the proposed Self and Strengths Zones of the SDS Career management skills framework. Six representative professions were selected for each of the six RIASEC categories, leading to a collection of thirty-six possible careers that were implemented and elaborated in the matching algorithms of the game, using the gameplay information that was obtained in the first two game zones.

Figure 1. Holland’s (1997) model and Prediger’s (1999) dimensions.

Our game design was also inspired by the research field of person-environment psychology (e.g. Walsh, Craik, & Price, 2000) where categorising and assessing individual difference characteristics are considered especially important for optimising the “fit” between the person and the requirement of a specific occupation. Working on that fit requires a psychosocial activity that has been conceptualised as “career adaptability” by Savickas (2005) and was defined as the attitudes, competences and behaviours that are needed to successfully fit yourself to suitable work. Playing the game is expected to raise the awareness of what choosing a career entails, to increase readiness, to provide learning and ultimately to increase career adaptability. Matching individuals’ interests, abilitites and attainments to possible careers has been a feature of many career interest guides including Kudos (www.cascaid.co.uk/kudos), My World of Work (http://www.myworldofwork.co.uk/ and Futurewise (www.myfuturewise.org.uk).

3. Method

We conducted a Randomized Controlled Trial in which participants were randomly assigned to one of two intervention conditions: half played the game, half executed a paper-and-pencil version (of the same activities in the game). Before and after the intervention questionnaires were provided to measure (perceived) change on career awareness and adaptabilities. The effects of the interventions were compared across condition and time, and analysed using descriptives, paired and independent t-tests and ANOVAs, that were calculated using SPSS version 22. Effects for potentially confounding variables like gender, age and country were controlled for. In the next subsections we now will describe: (1) the participants; (2) the procedure for administering questionnaires and interventions; (3) the interventions and (4) the scales of the questionnaire to measure outcomes and appreciation.

3.1. Participants

The Youth@work game had been initially targeted at a broad range of young people between the ages of 13 and 19 years who are starting to make decisions about which career they should follow. This includes school pupils (13–16 years old), college (16–19 years old) and even university students (18–24 years old) and those in work and NEETS (not in employment, education or training). However, during previous piloting of a Beta-release the game appeared most suitable for 13–16 years of age who are at the stage of making subject choices for their senior years in high school. The game can be played at home, but is probably best played with the support of a teacher or careers guidance advisor in a classroom context. Participants in this study were selected from that preferred context. Ninety-three high school students from Iceland (n = 42, with 18 male and 24 female) and Romania (n = 51, with 33 male and 18 female) were randomly assigned to playing the game (n = 46) or doing a paper-and-pencil version of the same activity (n = 47) in classroom under supervision of their teacher / counsellor. Their age range was 14–18 years (M = 15.405, SD = 1.019). We controlled for the effect of age, country and gender as possibly confounding co-variables (see Results section).

3.2. Procedure

The teachers / counsellors of the high schools provided names and emails of participants to the researchers and reserved about two hours for executing the study in their scheduled classes about career counselling. Informed consent by parents was guaranteed beforehand, researchers anonymized the contact data into unique IDs and accounts, and randomly assigned these IDs to conditions. At the start of the class, the teacher / counsellor then explained the aim of the activity and divided these classes into two conditions, without informing them about the difference in treatment: one half played the activity (consisting of nine tasks) as a game (in their native language), while the other half executed the same activity and tasks (but without interactive game play and feedback, without the automatic calculation of matching scores in the last task) on paper (in their native language). The main differences between the experimental (game) and control (non-game) conditions were twofold: (a) participants in the control condition had to store and calculate their own scores on paper and did not automatically receive feedback, progress or matching scores based on these scores (therefore also could not do the last and ninth activity in the game); and (b) had to manually calculate their RIASEC vocational types (activity 7) and favourite jobs (activity 8) based on what they filled in as outcomes of activities 1–6. Game play outcomes were computer logged, the paper-and-pencil outcomes were collected on standardised worksheets (available in native languages) and stored in pdf format (these qualitative data were left out of scope for this more quantitative paper). It appeared that participants in both conditions on average took about an hour to complete the activity, so we did not have to control for time-on-task. Directly before and after the activity, participants had to fill in an online pre- and post-test questionnaire (both implemented in Google Form, in their native language), respectively, that had been matched to their unique ID/account. Only Romanian participants considered the paper-and-pencil activity as a game and also filled in scale D of the post-test, and could be used as control group (n = 25) for comparison with gamers that scored D items (n = 44).

3.3. Intervention

The professional game we developed can be considered to be an adventure like game in which the player embarks on a (career) journey in search of the holy grail (which in this case is to find some valuable career advice). The game narrative starts with the player arriving by boat at the harbour of Job Fantasyland where the harbour master welcomes the weary traveller and provides instruction on how to travel across the island, with Crown Castle at the top of the island being the final destination (see first screengrab of Figure 2). In order to arrive there the player has to transverse the four zones (Self-Circus Cove, Strengths-Tiny Town, Horizons-Intercity and Networks-Intercity). In each zone the player is confronted with assistants who demand completion of certain tasks before the traveller is allowed to continue, with tasks provided in the form of mini-games. A total of 9 interactive mini-games are thus played in linear order, each with personalised feedback and scripted scoring and monitoring mechanisms in order to calculate progress and matching. The outcomes of the mini-games are stored in a personal journal (profile) and are also used to calculate scores on the RIASEC dimensions that guide the player in exploring specific career categories and then match with (a core set of 36) jobs. Finally, in Crown Castle, the traveller is handed over this personal journal by the King, together with job compatibility scores for the most relevant career options.

Figure 2. Screengrabs from the game.

In the (five) mini-games in the first zones of the game (Circus Cove and Tiny Town), players are asked about themselves and their strengths, including which subjects they like most at school, what they like to do in their leisure-time (the Ferris wheel mini-game, see second screengrab of Figure 2), what they would value in a future career (the Mystic mini-game, see third screengrab of Figure 2), their personal qualities and their skills. The main aim of the Self and Strength zones was to find out about the players in an unobtrusive way. Players then progress to the (sixth) Networks mini-game, where the “Boss” invites them to take part in a workshop where they are asked to evaluate whether advice offered by 4 different people about 6 different career dilemmas is good or bad (see fourth and fifth screengrabs of Figure 2).

The aim of the Networks zone is to help players evaluate the quality of advice they are given about career decision making problems that they might encounter. In the Careers library in the Horizons zone, players take part in a (seventh) book sorting mini-game, where they help the librarian to sort out the muddled career books (see sixth screengrab of Figure 2). The 36 career books are arranged in 6 shelves (rows) according to RIASEC category. The aim of the library book sorting mini-game is to extend the player's knowledge that jobs differ with respect to certain characteristics. Players then have the opportunity to explore a number of careers in the eighth mini-game, which are suggested on the basis of their responses in the first two zones of the game (see seventh screengrab of Figure 2), with the aim of further extending the player's knowledge about careers that might be of interest to them. Meanwhile all outcomes of playing the mini-games are gradually filling up the pages of the personal journal (My Personal Profile, see eighth screengrab of Figure 2 and example in Table 1), in which scores on RIASEC categories and matches with jobs are calculated.

Table 1. Personal profile for an example player.

Your top 2 RIASEC categories were: Investigative; Social

Your top 5 leisure interests were: Solving puzzles, scientific problems or playing quiz games; Hiking, biking or climbing in nature; Direct school, club or community organisation or committee; Play card or board games; Attending parties

Your top 5 subject preferences were: Psychology; Visual arts (drawing/painting); Geography; Natural sciences; Mathematics

Your top 3 career values were: Independence; Recognition; Relationships

Your skills evaluations were as follows: Active Learning (5); Active Listening (4); Critical Thinking (2); Learning Strategies (3); Mathematics (4); Monitoring (1); Reading Comprehension (2); Science (3); Speaking (4); Writing (4); Complex Problem Solving Skills (5); Time and Resource Management Skills (6); Social Skills (3); Systems skills (2); Technical and Computer Skills (4)

Your top 5 personal attributes were: Introspective; Idealistic; Dependable; Persistent; Caring

Crown Castle represents the end of the player's journey where they find out about their compatibility with the eight potentially most relevant jobs that were selected by playing the game. The “job-omatic” calculates matches based on the player's RIASEC score (derived from their leisure interests, subject preferences and personal qualities), as well as on their top 3 career values and skills ratings to the top 3 RIASEC categories, top 3 career values and 6 top skills for each job as described in O*Net. As players see compatibility scores for jobs the game is a kind of matching game, making recommendations about appropriate careers based on players’ personalised responses. Although the game was set up as a linear sequence of tasks, we envisioned that having a narrative set up as adventure, meaningful personal choices feeding into personalisation of available information, interactivity with non-playing characters that guide and advise players, and luck/chance in random advice are game dynamics that should be engaging and motivating for players.

The game was developed as open source in Unity, and is playable from Safari, Firefox and IE web browsers (although with the latter having slightly slower performance). It is available at http://tinyoaks.ictthatworks.org/game/JFL.html and needs an additional “unity web player” plugin to get installed. The web build runs in Chrome and is available at http://tinyoaks.ictthatworks.org/game/JFLWebGL/. Main advantages of using Unity are its potential for high-end GUI, and that it is easy to distribute as it is easily installed and cross platform. It is fully portable to tablets, androids and mobiles and also exportable to a website. A backdrop is its limited potential for more complex scripting of the narrative and feedback. The game dialogues were scripted initially in English, and later translated into (five) other languages (Icelandic, Romanian, Greek, Dutch and German).

3.4. Outcome measurement

Participants were asked to score (their perception of) three types of outcome measures: career adaptability, career awareness and readiness, and career learning and competences. Furthermore, they were asked to score how they had perceived game play itself.

3.4.1. Career adaptability

The Career Adapt-Abilities Inventory (CAAS 2.0) provides a well-researched and validated measurement instrument internationally (Savickas & Porfeli, 2012). The measure is based on a four dimensional conceptualisation of career adaptability: Curiosity, Concern, Control and Confidence (each containing six items; see Appendix 1, section A, items A1 till A24).

3.4.2. Career awareness and readiness

These two concepts were designed to capture career management skills development though the use of career websites by Howieson and Semple (2013), and were considered as important outcomes of playing the game. Items on Awareness and Readiness are presented in two separate sections each containing six items (see Appendix 1, sections E (items E63 till E68) and F (items F69 till F74).

3.4.3. Career learning

Specific career learning was measured on two concepts, a motivational component measuring perceptions on career Learning containing seven questions (see appendix 1, section B, items B25 till B31) and 10 items where the youngsters were asked to evaluate their own career related learning Competences (see Appendix 1, section C, items C32 till C41).

3.4.4. Perceptions of the game

To evaluate the students’ perceptions after playing the game (post-test only), we distinguished three concepts: Five items (D42 till D46) dealt with the possible Impact of the game on career learning and development; eleven items dealt with the Usability of playing the game (items D47 till D57); and the last four items (D58 till D61) dealt with game Features (such as graphics, narrative and characters (see Appendix 1, section D).

4. Results

This section presents our main findings on the effects of the intervention over time and differences between conditions, but first explains how we checked for potentially spurious effects of baseline differences and co-variates.

4.1. Control for baseline differences and co-variates

We confirmed that pre-test scores for both conditions were not already different as baseline. Before running our comparative analyses we ensured that all questionnaire items within the same (sub)scale had the same (positive or negative) direction in order to meaningfully calculate averages. Six items of the questionnaire (B26, B30, D47, D51, D54, D58, see Appendix 1) had to be transposed, with higher averages on scores A-D and lower averages on scores E-F to be interpreted as more positive outcomes. No differences were found for co-variates age, gender and country on the scores A-F of the pre-test. Significant differences on the independent samples t-test were found for gender on the C scores of the post-test (M_male = 4.068, M_female = 4.167, with p < 0.05), and for country on both the E score (M_Iceland = 1.833, M_Romania = 3.281, with p < 0.01) and the F score (M_Iceland = 1.850, M_Romania = 3.408, with p < 0.01) of the post-test. Please note that high scores on A-D scales are “positive” and high scores on E-F scales are “negative”.

4.2. Effects of the intervention over time

A paired samples t-test comparing the pre and post test scores on scales A, B, C, E and F (see Table 2) shows significant (short-term) growth over all participants on the average scores for A with t(89) = −2.816 and p < 0.01, and for C with t(89) = −1.992 and p < 0.05, but shows decreases (however not significant) on B and E. Scores tend to increase on scales A, C and F (the last not being significant), and tend to decrease on scales B and E (both not significant) over time.

Table 2. Pre- and Post-tests score averages on all (sub)scales.

	Pre-test				Post-test
(sub)scale	gamers (n = 46)		non-gamers (n = 47)		gamers (n = 46)		non-gamers (n  = 47)
	M	SD	M	SD	M	SD	M	SD
A Adaptability^a	3.553	0.655	3.754	0.670	4.042*	0.589	3.735*	0.533
B Learning	3.639	0.401	3.665	0.979	3.581	0.503	3.543	0.468
C Competences^b	3.969	0.592	4.036	0.709	4.068	0.578	4.154	0.517
D Perception^c	–	–	–	–	3.761	0.506	3.652	0.506
E Awareness^d	2.800	0.614	2.688	0.886	2.636	1.222	2.652	0.899
F Readiness^d	2.587	0.614	2.645	1.216	2.742	1.255	2.705	0.932
A_Concern	3.605	0.828	3.730	0.769	4.189**	0.761	3.706**	0.665
A_Control	3.641	0.714	3.833	0.773	4.075**	0.638	3.684**	0.657
A_Curiosity	3.507	0.772	3.702	0.818	3.928	0.767	3.744	0.698
A_Confidence	3.460	0.786	3.751	0.760	3.977*	0.678	3.733*	0.636
D_Impact^c	–	–	–	–	3.859	0.677	3.696	0.656
D_Usability^c	–	–	–	–	3.694	0.545	3.745	0.544
D_Features^c	–	–	–	–	3.823**	0.700	3.340**	0.567

Significant differences over time for all participants, with ^ap < 0.01 and ^bp < 0.05.

Significant differences on post-test between conditions, with *p < 0.05 and **p < 0.01.

^cNote that n_gamers = 44 and n_non-gamers = 25 for averages on D.

^dNote that E and F scores are on a 4-point scale (and have an opposite direction, from more positive to more negative) and A-D scores are on a 5-point scale (from more negative to more positive).

4.3. Effects of experimental condition

When looking for an effect of condition (gamers versus non-gamers) on the average post-test scores on the main scales A-F (see Table 2), no significant differences could be found for B, C, E and F. At the level of the scales, only an effect of condition was found for A (Career Adaptability) with F(1,88) = 6.662, p < 0.05,  = 0.071. However, when looking for the effect of condition on the level of subscales, we did find significant and positive developments on three out of the four career adaptability dimensions (subscales A): for “concern” with F(1,88) = 10.321, p < 0.01,  = 0.105), for “control” with F(1,88) = 8.180, p < 0.01,  = 0.085, and for “confidence” with F(1,88) = 2.188, p < 0.05,  = 0.034. An effect of condition was also found on the D subscale “perceptions of game features” with F(1,67) = 2.946, p < 0.01,  = 0.081. These partial-eta-squared values, according to Cohen's guidelines for interpretation (1988), can be considered as medium to large effect sizes.

4.4. Qualitative views on the game

Participants could verbally express their own views on playing the game at the end of the questionnaire. We present some quotes that might illustrate the benefits of playing the game. Comments provided were generally positive and mostly related to the usability of the game (like: “I liked how easy it was to use”) and players’ views about the game features (like “The characters were nice and they made the instructions easy to follow”; “The images were nice and colourful”). Positive views were also expressed about the benefits and impact of the game (like: “Great way to learn about careers”, “Especially liked being taken on a journey”; “It was a very useful and motivating way to learn about myself”; “I very much liked to find out about all the jobs I was compatible fo in this way”; “I think it's a good way to find out about your skills / more about yourself in a fun way”; “I like that it encourages you to think about values, preferences almost indirectly”).

5. Conclusion and discussion

Careful consideration of two career theories was helpful to theoretically ground and validate learning outcomes of the Youth@Work game, with the SDS model of career competences providing guidance about the organisation of the game and the different zones and Holland's model of vocational interests identifying activities for the different zones. Inclusion of dynamic and engaging game mechanics that include an adventure like narrative, personalised choice, interaction, progression and scoring, along with the “matching” idea that required progression on finding out about yourself and careers and then looking at the match between these, were considered to foster career awareness and capability.

As we found the use of game-based learning in career counselling practice to have been scarce, the aim of this study was to research the added value of providing career decision making activity in such a more interactive and experiential way (i.e. gamers versus non-gamers). We stress that we consider gaming as complementary to current career counselling practice, and did not want to compare more classic and more innovative practices (e.g. gaming versus interviews).

Main results of the comparisons over time (pre- and post-test scores) and between condition (gamers versus non-gamers) show that participants mostly report positive change after doing the career activity. Significant positive effects over time were found for career adaptability (scale A) and perception on career learning competences (scale C). Not finding any more substantial change over time is not that surprising considering the short time lapse (about an hour) between pre- and post-testing. On the level of main scales, a significant (additional) effect for gaming was found for career adaptability (scale A). Furthermore, we were able to identify significant differences between conditions (the added value of carrying out the activities in a more dynamic and engaging way) on the A subscale dimensions “concern”, “control” and “confidence” (but not on “curiosity”). This means that players reported significantly better sense of control of and concern and confidence about careers after game play when compared to non-gamers. Finally and not surprisingly, game features (of the activity) were scored better by gamers than by non-gamers.

Besides the small time-frame of the study, limitations in further growth might be caused by other reasons. Savickas and Porfeli (2012) reported big differences in the discriminative power of the CAAS instrument across countries (generally being more discriminative for Asian countries and less so for European countries). Overall, pre-test scores on the A-C scales were relatively high (and socially desirable) so there might be a “ceiling effect” on further growth. Furthermore, scales B, C and D were not validated (nor grounded in theory) so might not measure what we intended.

Finally, we were left with some intriguing questions like: Why are Icelandic students more positive about growth on career awareness and readiness (scales E and F) after the activity? and Why do girls have more positive perceptions about career learning competences (scale C) after the activity? Regarding the first issue, it is possible that Icelandic students consider themselves more aware of their career opportunities and more ready for decision making, due to the character of their educational system (demanding frequent choices) or their labour market (with a high youth employment), in comparison with their Romanian counterparts. Or they might be a bit more overconfident or “yea-saying.” Regarding the last issue about gender differences on the career learning (scale C), our educated guess is that girls overall are more conscientious students than boys and therefore more willing to engage in the presented activities, and more apt to learn in the process.

Further studies should extend these findings by including interventions and measurements in longer time-frames to see if these outcomes change more over time. Although in this study we did encounter only minor differences between two countries, including more countries in the population would be informative (e.g. Howieson & Semple, 2013). Where the main source of data collection in this study was questionnaires, future game design should enable more objective outcome data to be collected by in-game performance measures. Most important scales used in the questionnaire are based on validation in prior research, but not all. Triangulation of data collection methods, like combining questionnaire results with focussed interviews, observations of playing behaviour, computer logging of progress and performance, and others might shed more light on some of the findings we could not explain. Replicating this study in other contexts of use (beyond the classroom, in more informal or non-formal contexts of career construction) would also be interesting and advisable. The finding from this study that game-based learning is appreciated as (at least) equal to more classical instruction with paper-and-pencil testing, is promising in its own right when looking for the benefits of more time- and place-independent career advice support. Additional growth for gamers on their sense of control, concern and confidence about their careers can be considered as a promising result when considering game-based learning for the career counselling practice.

Acknowledgements

The present work was carried out as part of the YOUTHYES project. Authors thank all other project members who have contributed to the game design and development in some way. This paper does not represent the opinion of the European Community, and the European Community is not responsible for any use that might be made of its content.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Hans G.K. Hummel is associate professor at the Open University of the Netherlands and has a research focus on TEL supported workplace based learning.

Liz Boyle is reader in psychology at the University of the West of Scotland and has a research focus on learning and motivation in e-learning.

Sif Einarsdóttir is professor and Arna Petursdóttir is researcher. Both work at the University of Iceland and have a research focus on career counselling and guidance.

Aurel Graur is director at CERD (Centrul European de Recurse pentru Dezvoltare) in Romania and works as school inspector.

ORCID

Arna Pétursdóttir http://orcid.org/0000-0003-4329-4910

Additional information

Funding

This project was supported by Erasmus + Strategic Partnerships addressing more than one field; Project with multiple beneficiaries; KA2 – Cooperation and Innovation for Good Practices, Grant Agreement no: 2014-1-UK01-KA200-001858.

Appendix 1. Pre- and Post-Questionnaire items / scales.

Today you are going to take part in an activity / play a game that will guide you through some issues that could be important in how you choose your career. Before you take part in the activity / play the game it would be very useful if you could answer some questions about your interest in careers, what you know about careers, how you learn about careers and what is important in choosing a career. Please consider the questions carefully and answer them as honestly as you can.

Section A: Career adaptabilities (4 subscales)

Some people use different strengths to build their careers.

Please rate how strongly you have developed each of the following abilities using the scale below: 1 = Not strong, 2 = Somewhat strong, 3 = Strong, 4 = Very Strong, and 5 = Strongest.

Table

item		1	2	3	4	5
	Concern
A1	Thinking about what my future will be like
A2	Realizing that today's choices shape my future
A3	Preparing for the future
A4	Becoming aware of the educational and career choices that I must make
A5	Planning how to achieve my goals
A6	Concerned about my career
	Control
A7	Keeping upbeat
A8	Making decisions by myself
A9	Taking responsibility for my actions
A10	Sticking up for my beliefs
A11	Counting on myself
A12	Doing what's right for me
	Curiosity
A13	Exploring my surroundings
A14	Looking for opportunities to grow as a person
A15	Investigating options before making a choice
A16	Observing different ways of doing things
A17	Probing deeply into questions I have
A18	Becoming curious about new opportunities
	Confidence
A19	Performing tasks efficiently
A20	Taking care to do things well
A21	Learning new skills
A22	Working up to my ability
A23	Overcoming obstacles
A24	Solving problems

Section B: Perceptions of career learning

In this section you are asked about your views on learning about careers.

Please consider each of the statements below and indicate your level of agreement with it on a scale of 1–5, where: 1 means “I really disagree with this statement”; 2 means “I disagree with this statement”; 3 means “I neither agree nor disagree with this statement”; 4 means “I agree with this statement”, and 5 means “I really agree with this statement”

Table

item		1	2	3	4	5
B25	Learning about careers is interesting
B26	Learning about careers is boring
B27	Learning about careers is useful
B28	Learning about careers is enjoyable
B29	Learning about careers is fun
B30	Learning about careers is difficult
B31	Learning about careers is challenging

Section C: Perceptions of career learning competences

The questions in this section are about career related knowledge and understanding.

Please consider each of the statements below and indicate your level of agreement with it on a scale of 1–5, where: 1 means “I really disagree with this statement”, 2 means “I disagree with this statement”, 3 means “I neither agree nor disagree with this statement”, 4 means “I agree with this statement”, and 5 means “I really agree with this statement”.

Table

item		1	2	3	4	5
C32	In finding out about careers it is important for me to get a better understanding of me and what I am like.
C33	I know which leisure activities are of most interest to me.
C34	I know which school subjects are of most interest to me.
C35	I know which career values are important to me in thinking about the kind of job that I would like to do in future.
C36	I can evaluate which job-related skills I am good at and those I am less good at.
C37	I know which career-related personal attributes are most typical of me.
C38	I have a good understanding of the features of jobs that are of interest to me.
C39	I have a good understanding of how jobs differ. with respect to the types of people who do these jobs and the features and skills involved in the job.
C40	I am able to evaluate advice I am given about careers.
C41	It is important to think about how I can find a job that matches my interests.

Section D: Perceptions of the game /activity (Post-game questionnaire only, 3 subscales)

This questionnaire asks you what you thought about the game / activity.

Please consider each statement in turn and rate these 1–5, where: 1 means “I really disagree with this statement”, 2 means “I disagree with this statement”, 3 means “I neither agree nor disagree with this statement”, 4 means “I agree with this statement”, and 5 means “I really agree with this statement”.

Table

item		1	2	3	4	5
Impact of the game on learning about self and careers
D42	I really learned something about careers while playing the game.
D43	I found out about careers that are of interest to me while playing the game.
D44	This is a good way to learn about careers.
D45	The game helped me to learn about myself.
D46	I found out something about myself while playing the game.
Usability of the game
D47	The game was difficult.
D48	The game was easy to use.
D49	I knew what to do next in the game.
D50	I could easily carry out the activities in the game.
D51	I found it difficult to know what to do next in the game.
D52	Playing the game was interesting.
D53	Playing the game was challenging.
D54	Playing the game was boring.
D55	Playing the game was enjoyable.
D56	Playing the game was fun.
D57	I enjoyed playing the game.
Perceptions of game features
D58	The game content is a little too simplistic.
D59	I liked the graphics in the game
D60	I liked the story/narrative in the game.
D61	I liked the characters in the game.

Section E: Awareness of your personal strengths, weaknesses and values

In this section you are asked about your awareness of your personal strengths, weaknesses and values. In answering the following 6 questions, please put a cross in one box on each line. At the moment, how much do you … 

Table

item		A lot	Quite a lot	Not very much	Nothing
E63	Know what you are good at in school?
E64	Know what you are good at out of school?
E65	Know what you are not so good at in school?
E66	Know what you are not so good at out of school?
E67	Know what is important to you in choosing a good job or a course?
E68	Know how you want to live your life in the future

Section F: Career planning readiness

In this section you are asked about your Career planning readiness. In answering the following 6 questions, please put a cross in one box on each line. At the moment, how much do you … 

Table

item		A lot	Quite a lot	Not very much	Nothing
F69	Know what information and advice you need to help you make a decision about jobs or courses?
F70	Feel ready to make a plan for the future?
F71	Feel clear about your next steps in taking your career forward?
F72	Feel confident your career plan will work out?
F73	Know the sorts of jobs or courses that might suit you?
F74	Know the kinds of skills and attitudes employers are looking for?