The effects of gamification on students’ academic achievement: a meta-analysis study

References

• Aguinis, H., Gottfredson, R. K., & Wright, T. A. (2011). Best-practice recommendations for estimating interaction effects using meta-analysis. Journal of Organizational Behavior, 32(8), 1033–1043.
   Web of Science ®Google Scholar
• American Psychological Association. (2010). American Psychological Association. Washington, DC: Google Scholar.
   Google Scholar
• *Ar, N. A. (2016). The effects of gamification on academic achievement and learning strategies usage of vocational high school students (Unpublished master’s thesis). Department of Computer Education and Instructional Technologies, Sakarya University, Turkey.
   Google Scholar
• Bayraktar, Ö. (2015). Bir iletişim modeli olarak oyunlaştırma. İstanbul: Selis Books.
   Google Scholar
• Begg, C. B., & Berlin, J. A. (1988). Publication bias: A problem in interpreting medical data. Journal of the Royal Statistical Society Series A (Statistics in Society), 151(3), 419–463.
   Web of Science ®Google Scholar
• Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.
   PubMed Web of Science ®Google Scholar
• Berlin, J. A., Laird, N. M., Sacks, H. S., & Chalmers, T. C. (1989). A comparison of statistical methods for combining event rates from clinical trials. Statistics in Medicine, 8(2), 141–151.
   PubMed Web of Science ®Google Scholar
• Bidarra, J., Figueiredo, M., & Natálio, C. (2015). Interactive design and gamification of ebooks for mobile and contextual learning. International Journal of Interactive Mobile Technologies (iJIM), 9(3), 24–32.
   Google Scholar
• Bogost, I. (2011). Why gamification is bullshit. In S. P. Walz & S. Deterding (Eds.), The gameful world: Approaches, issues, applications (pp. 65–80). London: The MIT Press.
   Google Scholar
• Borenstein, M. (1994). Effect sizes for continuous data. In H. Cooper, & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 221–235). New York, NY: Russell Sage Foundation.
   Google Scholar
• Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2005). Comprehensive meta analysis, version 2 [computer software]. Englewood, NJ: Biostat.
   Google Scholar
• Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Hoboken, NJ: Wiley.
   Google Scholar
• *Buckley, P., & Doyle, E. (2016). Gamification and student motivation. Interactive Learning Environments, 24(6), 1162–1175.
   Web of Science ®Google Scholar
• Bunchball, I. (2010). Gamification 101: An introduction to the use of game dynamics to influence behavior. White paper.
   Google Scholar
• Castro, K. A. C., Sibo, ÍPH, & Ting, I. H. (2018). Assessing gamification effects on e-learning platforms: An experimental case. In International Workshop on learning technology for education in Cloud (pp. 3–14). Cham: Springer.
   Google Scholar
• Chorney, A. I. (2012). Taking the game out of gamification. Dalhousie Journal of Interdisciplinary Management, 8(1), 1–14.
   Google Scholar
• Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122.
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Edition). Hillsdale, NJ: Lawrence Earlbaum Associates.
   Google Scholar
• Cooper, H., & Hedges, L. V. (1994). The handbook of research synthesis. New York: Russell Sage Foundation.
   Google Scholar
• Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and metaanalysis (2nd edition). New York, NY: Russell Sage Publication.
   Google Scholar
• *De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and social networking on e-learning. Computers & Education, 75, 82–91.
   Web of Science ®Google Scholar
• *De-Marcos, L., Garcia-Lopez, E., & Garcia-Cabot, A. (2016). On the effectiveness of game-like and social approaches in learning: Comparing educational gaming, gamification & social networking. Computers & Education, 95, 99–113.
   Web of Science ®Google Scholar
• Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek ‘11) (pp. 9–15). New York, NY: ACM. doi:https://doi.org/10.1145/2181037.2181040.
   Google Scholar
• Dixon, D. (2012). Gameful design tools. Retrieved from https://www.slideshare.net/digitaldust/ux-bristol-gameful-design
   Google Scholar
• *Domínguez, A., Saenz-De-Navarrete, J., De-Marcos, L., Fernández-Sanz, L., PagéS, C., & Martínez-HerráIz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380–392.
   Web of Science ®Google Scholar
• Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
   PubMed Web of Science ®Google Scholar
• Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.
   PubMed Web of Science ®Google Scholar
• *Faghihi, U., Brautigam, A., Jorgenson, K., Martin, D., Brown, A., Measures, E., & Maldonado-Bouchard, S. (2014). How gamification applies for educational purpose specially with college algebra. Procedia Computer Science, 41, 182–187.
   Google Scholar
• *Fan, K. K., Xiao, P. W., & Su, C. H. (2015). The effects of learning styles and meaningful learning on the learning achievement of gamification health education curriculum. Eurasia Journal of Mathematics, Science & Technology Education, 11(5), 1211–1229.
   Web of Science ®Google Scholar
• *Fotaris, P., Mastoras, T., Leinfellner, R., & Rosunally, Y. (2015). Who wants to be a pythonista? Using gamification to teach computer programming. Retrived from https://goo.gl/OWWZCH
   Google Scholar
• *Frącz, W. (2015). An empirical study inspecting the benefits of gamification applied to university classes. 2015 7th Computer Science and Electronic Engineering Conference (CEEC) (pp. 135–139). IEEE.
   Google Scholar
• Gentry, S., L'Estrade Ehrstrom, B., Gauthier, A., Alvarez, J., Wortley, D., van Rijswijk, J., … Zary, N. (2016). Serious gaming and gamification interventions for health professional education. The Cochrane Library, 1–16.
   Google Scholar
• Glass, G. V. (1976). Primary, secondary and meta-analysis of research. Educational Researcher, 5(10), 3–8.
   Google Scholar
• Grace, M. V., & Hall, J. (2008). Projecting surveillance entertainment. San Diego, CA: ETech.
   Google Scholar
• *Gudoniene, D., Bartkute, R., Rutkauskiene, D., & Blazauskas, T. (2015). Technological aspects of the gamification model for e-learning participant’s engagement. Baltic J. Modern Computing, 4(4), 1008–1015.
   Web of Science ®Google Scholar
• *Halvorsen, M. (2013). The use of gamification in learning applications (Unpublished master thesis). Department of Information Science and Media Studies, University of Bergen, Norway .
   Google Scholar
• Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? A literature review of empirical studies on gamification. 47th Hawaii International Conference on System Sciences (HICSS) (pp. 3025–3034). IEEE.
   Google Scholar
• *Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152–161.
   Web of Science ®Google Scholar
• Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107–128.
   Google Scholar
• Hedges, L. V. (1992). Meta-analysis. Journal of Educational Statistics, 17, 279–296.
   Google Scholar
• Hedges, L. V. (1994). Statistical considerations. In H. Cooper, & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 29–38). New York: Russell Sage Foundation.
   Google Scholar
• Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta- analysis. Psychological Methods, 3, 486–504.
   Web of Science ®Google Scholar
• Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560.
   PubMed Web of Science ®Google Scholar
• Hozo, S. P., Djulbegovic, B., & Hozo, I. (2005). Estimating the mean and variance from the median, range, and the size of a sample. BMC Medical Research Methodology, 5(13), 1–10.
   PubMedGoogle Scholar
• *Hsu, K. C. (2016). Social gamification in multimedia instruction: assessing the effects of animation, reward strategies, and social interactions on learners motivation and academic performance in online settings (Unpublished doctoral dissertation). Graduate degree program in Educational Leadership and Policy Studies, University of Kansas, USA.
   Google Scholar
• Huang, B., Hew, K. F., & Lo, C. K. (2018). Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement. Interactive Learning Environments, 1–21.
   Web of Science ®Google Scholar
• *Huang, B., & Hew, K. F. (2015). Do points, badges and leaderboard increase learning and activity: A quasi-experiment on the effects of gamification. Proc. of the 23rd International Conference on Computers in Education (pp. 275–280).
   Google Scholar
• *Hudiburg, M. L. (2016). Motivation and learning in an online collaborative project using gamifcation (Unpublished doctoral dissertation). Old Dominion University, USA .
   Google Scholar
• Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: SAGE.
   Google Scholar
• *Ibáñez, M. B., Di-Serio, A., & Delgado-Kloos, C. (2014). Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on Learning Technologies, 7(3), 291–301.
   Web of Science ®Google Scholar
• Iyengar, S., & Greenhouse, J. B. (1994). Sensitivity analysis and diagnostics. In H. Cooper, & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 417–433). New York, NY: Russell Sage Foundation.
   Google Scholar
• Jagušt, T., Botički, I., & So, H. J. (2018). Examining competitive, collaborative and adaptive gamification in young learners’ math learning. Computers & Education, 125, 444–457.
   Web of Science ®Google Scholar
• *Jang, J., Park, J. J. Y., & Yi, M. Y. (2015). Gamification of online learning. International Conference on Artificial Intelligence in Education (pp. 646–649). Springer International Publishing .
   Google Scholar
• *Kalkan, A. (2016). Investigation of effect of gamification in 3D virtual worlds on secondary school 5th grade students’ achievement, flow and attitudes (Unpublished master thesis). Department of Educational Sciences, Ataturk University, Turkey .
   Google Scholar
• Kapp, K. M., Blair, L., & Mesch, R. (2014). The gamification of learning and Instruction Fieldbook Ideas into Practice. San Fransisco: Wiley.
   Google Scholar
• *Kayimbasioglu, D., Oktekin, B., & Hacı, H. (2016). Integration of gamification technology in education. Procedia Computer Science, 102, 668–676.
   Google Scholar
• *Kert, S. B., & Erkoc, M. F. (2016). Game-Based approaches, gamification, and programming language training. In R. A. P. de Queiros & M. T. Pinto (Eds.), Gamification-based e-learning strategies for computer programming education (pp. 131–151). Harshey, PA: IGI Global.
   Google Scholar
• *Ketyi, A. (2016). From mobile language learning to gamification: An overlook of research results with business management students over a five-year period. Porta Linguarum: Revista Internacional de Didáctica de las Lenguas Extranjeras, 1, 45–59.
   Google Scholar
• Khaleel, F. L., Ashaari, N. S., & Wook, T. S. M. T. (2019). An empirical study on gamification for learning programming language website. Jurnal Teknologi, 81(2), 151–162.
   Web of Science ®Google Scholar
• Kim, E., Rothrock, L., & Freivalds, A. (2018). An empirical study on the impact of lab gamification on engineering students’ satisfaction and learning. International Journal of Engineering Education, 34(1), 201–216.
   Web of Science ®Google Scholar
• *Kim, E., Rothrock, L., & Freivalds, A. (2016). The effects of gamification on engineering lab activities. In Frontiers in Education Conference (FIE) (pp. 1–6). IEEE .
   Google Scholar
• Landers, R. N. (2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752–768.
   Google Scholar
• *Laubersheimer, J., Ryan, D., & Champaign, J. (2016). Infoskills2go: Using badges and gamification to teach information literacy skills and concepts to college-bound high school students. Journal of Library Administration, 56(8), 924–938.
   Web of Science ®Google Scholar
• *Leaning, M. (2015). A study of the use of games and gamification to enhance student engagement, experience and achievement on a theory-based course of an undergraduate media degree. Journal of Media Practice, 16(2), 155–170.
   Google Scholar
• Legaki, N. Z., Xi, N., Hamari, J., & Assimakopoulos, V. (2019). Gamification of the future: An experiment on gamifying education of forecasting. Proceedings of the 52nd Hawaii International Conference on System Sciences.
   Google Scholar
• Light, R. J., & Pillemer, D. B. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.
   Google Scholar
• *Lin, C. H., & Shih, J. L. (2015). Evaluations to the gamification effectiveness of digital game-based adventure education course–GILT. Journal of e-Learning and Knowledge Society, 11(3), 41–58.
   Web of Science ®Google Scholar
• Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: SAGE.
   Google Scholar
• Lister, M. C. (2015). Gamification: The effect on student motivation and performance at the post-secondary level. Issues and Trends in Educational Technology, 3(2), 1–22.
   Google Scholar
• Lo, C. K., & Hew, K. F. (2018). A comparison of flipped learning with gamification, traditional learning, and online independent study: The effects on students’ mathematics achievement and cognitive engagement. Interactive Learning Environments, 1–18.
   Web of Science ®Google Scholar
• Lundgren, L., & Olofsson, J. (2015). Serious games and gamification. Retrieved from http://www.diva-portal.org/smash/get/diva2:1029777/FULLTEXT02.pdf
   Google Scholar
• *MacKinnon, R. J., Stoeter, R., Doherty, C., Fullwood, C., Cheng, A., Nadkarni, V., … Chang, T. P. (2015). Self-motivated learning with gamification improves infant CPR performance, a randomised controlled trial. BMJ Simulation and Technology Enhanced Learning, 1(3), 71–76.
   Google Scholar
• Majuri, J., Koivisto, J., & Hamari, J. (2018). Gamification of education and learning: A review of empirical literature. Proceedings of the 2nd International GamiFIN Conference, GamiFIN 2018. CEUR-WS.
   Google Scholar
• Marín, B., Frez, J., Cruz-Lemus, J., & Genero, M. (2018). An empirical Investigation on the Benefits of gamification in Programming courses. ACM Transactions on Computing Education (TOCE), 19(1), 1–22.
   Web of Science ®Google Scholar
• McDonald, M., Musson, R., & Smith, R. (2008). Using productivity games to Prevent Defects. In M. McDonald, R. Musson, & R. Smith (Eds.), The Practical Guide to Defect Prevention (pp. 79–95). Redmond: Microsoft Press.
   Google Scholar
• *Mese, C. (2016). Effectiveness of gamification elements in blended learning environments (Unpublished doctoral dissertation). Institute of Educational Sciences, Anadolu University, Turkey .
   Google Scholar
• Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269.
   PubMed Web of Science ®Google Scholar
• Morris, S. B. (2008). Estimating effect sizes from the pretest-posttest-control group designs. Organizational Research Methods, 11, 364–386.
   Web of Science ®Google Scholar
• Nahl, D., & James, L. (2012). Technology Integration and gamification in university courses. SLACTIONS, 2012, 41–48.
   Google Scholar
• Newzoo. (2018). Global 2018 global esports market report. Retrieved from https://newzoo.com/insights/trend-reports/global-esports-market-report-2018-light/
   Google Scholar
• Note: References with asterisk (*) indicate studies included to meta-analysis.
   Google Scholar
• *Pedro, L. Z., Lopes, A. M., Prates, B. G., Vassileva, J., & Isotani, S. (2015). Does gamification work for boys and girls?: An exploratory study with a virtual learning environment. Proceedings of the 30th Annual ACM Symposium on Applied Computing (pp. 214-219). ACM .
   Google Scholar
• *Poole, S. M., Kemp, E., Williams, K. H., & Patterson, L. (2014). Get your head in the game: Using gamification in business education to connect with Generation Y. Journal for Excellence in Business Education, 3(2), 1–9.
   Google Scholar
• *Rachels, J. R. (2016). The effect of gamification on elementary students’ Spanish language achievement and academic self-efficacy (Unpublished doctoral dissertation). Liberty University, USA .
   Google Scholar
• *Rincón-Flores, E., Ramírez-Montoya, M. S., & Mena, J. (2016). Challenge-based gamification and its impact in teaching mathematical modeling. Proceedings of the fourth International Conference on Technological Ecosystems for Enhancing Multiculturality. TEEM .
   Google Scholar
• Robertson, M. (2010). Can't Play, Won't Play. Retrieved from http://kotaku.com/5686393/cant-play-wont-play
   Google Scholar
• Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.
   Web of Science ®Google Scholar
• *Rouse, K. E. (2013). Gamifcation in Science Education: The Relationship of Educational Games to Motivation and Achievement (Unpublished doctoral dissertation). The University of Southern Mississippi, USA.
   Google Scholar
• *Sahin, M. C., & Arslan Namli, N. (2016). Gamification and effects on students’ science lesson achievement. International Journal on New Trends in Education and Their Implications (IJONTE), 7(1), 41–47.
   Google Scholar
• Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. London: MIT press.
   Google Scholar
• *Sanmugam, M., Abdullah, Z., Mohamed, H., Aris, B., Zaid, N. M., & Suhadi, S. M. (2016). The affiliation between student achievement and elements of gamification in learning science. 4th International Conference on Information and Communication Technology (ICoICT) (pp. 1-4). IEEE .
   Google Scholar
• Shadish, W. R., & Haddock, C. K. (1994). Combining estimates of effect size. In H. Cooper, & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 257–277). New York, NY: Russell Sage Foundation.
   Google Scholar
• *Su, C. H., & Cheng, C. H. (2015). A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning, 31(3), 268–286.
   Web of Science ®Google Scholar
• Surendeleg, G., Tudevdagva, U., & Kim, Y. S. (2015). The Contribution of gamification on User Engagement in Fully Online course. In Creativity in Intelligent, technologies and data Science (pp. 710–719). Cham: Springer International Publishing.
   Google Scholar
• *Tan, M., & Hew, K. F. (2016). Incorporating meaningful gamification in a blended learning research methods class: Examining student learning, engagement, and affective outcomes. Australasian Journal of Educational Technology, 32(5), 19–34.
   Web of Science ®Google Scholar
• Tenório, M. M., Lopes, R. P., de Góis, L. A., & dos Santos Junior, G. (2018). Influence of gamification on Khan Academy in Brazilian high school. PUPIL: International Journal of Teaching, Education and Learning, 2(2), 51–65.
   Google Scholar
• Todd, A. (2017). Why gamification is malarkey. The Morning Watch: Educational and Social Analysis, 44(1-2), 1–7.
   Google Scholar
• *Tunga, Y. (2016). An examination of effects of the use of gamification in e-learning environments on learners’ academic performance and engagement (Unpublished master thesis). Department of Computer Education and Instructional Technologies, Ege University, Turkey .
   Google Scholar
• *Turan, Z., Avinc, Z., Kara, K., & Goktas, Y. (2016). Gamification and education: Achievements, cognitive loads, and views of students. International Journal of Emerging Technologies in Learning, 11(7), 64–69.
   Web of Science ®Google Scholar
• *Vaibhav, A., & Gupta, P. (2014). Gamification of MOOCs for increasing user engagement. International Conference on MOOC, Innovation and Technology in Education (MITE) (pp. 290–295). IEEE .
   Google Scholar
• Werbach, K. (2014). Gamification course. Retrieved from https://www.coursera.org/course/gamification
   Google Scholar
• Werbach, K., & Hunter, D. (2012). For the Win. How game thinking can revolutionize your business. Philadelphia, PA: Wharton Digital Press.
   Google Scholar
• Wood, L. C., & Reiners, T. (2012, November 28–30). Gamification in logistics and supply chain education: Extending active learning. IADIS Internet Technologies and Society, Perth, pp. 101–108.
   Google Scholar
• Xu, Y. (2011). Literature review on web application gamication and analytics. CSDL Technical Report 11-05.
   Google Scholar
• *Yildirim, I. (2016). Development, implementation and evaluation of a gamification based curriculum for the lesson of “teaching principles and methods” (Unpublished doctoral dissertation). Department of Educational Sciences, Gaziantep University, Turkey .
   Google Scholar
• Zicherman, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in Web and Mobile Apps (1st ed.). Sebastopol, CA: O'Reilly Media.
   Google Scholar