Towards Game-Based Assessment of Creative Thinking

References

• Argulewicz, E. N., Bingenheimer, L. T., & Anderson, C. C. (1983). Concurrent validity of the PPVT-R for Anglo-American and Mexican-American students. Journal of Psychoeducational Assessment, 1(2), 163–167. doi:10.1177/073428298300100207
   Google Scholar
• Bai, H., Duan, H., Kroesbergen, E. H., Leseman, P. P., & Hu, W. (2020). The benefits of the learn to think program for preschoolers’ creativity: An explorative study. The Journal of Creative Behavior, 54(3), 699–711. doi:10.1002/jocb.404
   Web of Science ®Google Scholar
• Barbot, B. (2018). The dynamics of creative ideation: Introducing a new assessment paradigm. Frontiers in Psychology, 9, 2529.
   PubMed Web of Science ®Google Scholar
• Batey, M., Furnham, A., & Safiullina, X. (2010). Intelligence, general knowledge and personality as predictors of creativity. Learning and Individual Differences, 20(5), 532–535. doi:10.1016/j.lindif.2010.04.008
   Web of Science ®Google Scholar
• Beaty, R. E., & Johnson, D. R. (2021). Automating creativity assessment with SemDis: An open platform for computing semantic distance. Behavior Research Methods, 53(2), 757–780. doi:10.3758/s13428-020-01453-w
   PubMed Web of Science ®Google Scholar
• Beaty, R. E., Johnson, D. R., Zeitlen, D. C., & Forthmann, B. (2022). Semantic distance and the alternate uses task: Recommendations for reliable automated assessment of originality. Creativity Research Journal, 34(3), 1–16. doi:10.1080/10400419.2022.2025720
   Web of Science ®Google Scholar
• Beaty, R. E., Kenett, Y. N., Christensen, A. P., Rosenberg, M. D., Benedek, M., Chen, Q. … Silvia, P. J. (2018). Robust prediction of individual creative ability from brain functional connectivity. Proceedings of the National Academy of Sciences, 115(5), 1087–1092. doi:10.1073/pnas.1713532115
   PubMed Web of Science ®Google Scholar
• Beaty, R. E., & Silvia, P. J. (2012). Why do ideas get more creative across time? An executive interpretation of the serial order effect in divergent thinking tasks. Psychology of Aesthetics, Creativity, and the Arts, 6(4), 309–319. doi:10.1037/a0029171
   Web of Science ®Google Scholar
• Becker, M., & Cabeza, R. (2021). Assessing creativity independently of language: A normed language-independent remote associate task (LI-RAT) (pp. 1–18). Behavior Research Methods.
   Google Scholar
• Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, validity, and factor structure of the creative achievement questionnaire. Creativity Research Journal, 17(1), 37–50. doi:10.1207/s15326934crj1701_4
   Web of Science ®Google Scholar
• Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54(1), 1. doi:10.1037/h0046743
   Web of Science ®Google Scholar
• Chierchia, G., Fuhrmann, D., Knoll, L. J., Pi-Sunyer, B. P., Sakhardande, A. L., & Blakemore, S. J. (2019). The matrix reasoning item bank (MaRs-IB): Novel, open-access abstract reasoning items for adolescents and adults. Royal Society Open Science, 6(10), 190232. doi:10.1098/rsos.190232
   PubMed Web of Science ®Google Scholar
• Cortes, R. A., Weinberger, A. B., Daker, R. J., & Green, A. E. (2019). Re-examining prominent measures of divergent and convergent creativity. Current Opinion in Behavioral Sciences, 27, 90–93. doi:10.1016/j.cobeha.2018.09.017
   Web of Science ®Google Scholar
• Cropley, A. (2006). In praise of convergent thinking. Creativity Research Journal, 18(3), 391–404. doi:10.1207/s15326934crj1803_13
   Web of Science ®Google Scholar
• Csikszentmihalyi, M. (1997). Flow and the psychology of discovery and invention. HarperPerennial, New York, 39.
   Google Scholar
• Design Council UK. (2005). The Design Process. https://www.designcouncil.org.uk/news-opinion/double-diamond-universally-accepted-depiction-design-process
   Google Scholar
• Dollinger, S. J., Urban, K. K., & James, T. A. (2004). Creativity and openness: Further validation of two creative product measures. Creativity Research Journal, 16(1), 35–47. doi:10.1207/s15326934crj1601_4
   Web of Science ®Google Scholar
• Dumas, D., Organisciak, P., & Doherty, M. D. (2020). Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods. Psychology of Aesthetics, Creativity, and the Arts, 15(4), 645–663. doi:10.1037/aca0000319
   Web of Science ®Google Scholar
• Furnham, A., Batey, M., Anand, K., & Manfield, J. (2008). Personality, hypomania, intelligence and creativity. Personality and Individual Differences, 44(5), 1060–1069. doi:10.1016/j.paid.2007.10.035
   Web of Science ®Google Scholar
• Gajdacz, M., Rafner, J., Langsford, S., Hjorth, A., Bergenholtz, C., Biskjaer, M. M., … & Sherson, J. F. (2021). CREA. blender: A GAN Based Casual Creator for Creativity Assessment. In ICCC (pp. 405-409).
   Google Scholar
• Gerwig, A., Miroshnik, K., Forthmann, B., Benedek, M., Karwowski, M., & Holling, H. (2021). The relationship between intelligence and divergent thinking—a meta-analytic update. Journal of Intelligence, 9(2), 23. doi:10.3390/jintelligence9020023
   PubMed Web of Science ®Google Scholar
• Goldschmidt, G. (2016). Linkographic evidence for concurrent divergent and convergent thinking in creative design. Creativity Research Journal, 28(2), 115–122. doi:10.1080/10400419.2016.1162497
   Web of Science ®Google Scholar
• Groth-Marnat, G. (2009). Handbook of psychological assessment. John Wiley & Sons.
   Google Scholar
• Guilford, J. P. (1956). The structure of intellect. Psychological Bulletin, 53(4), 267. doi:10.1037/h0040755
   PubMed Web of Science ®Google Scholar
• Guilford, J. P. (1967). The nature of human intelligence.
   Google Scholar
• Haase, J., Hoff, E. V., Hanel, P. H., & Innes-Ker, Å. (2018). A meta-analysis of the relation between creative self-efficacy and different creativity measurements. Creativity Research Journal, 30(1), 1–16. doi:10.1080/10400419.2018.1411436
   Web of Science ®Google Scholar
• Hart, Y., Goldberg, H., Striem-Amit, E., Mayo, A. E., Noy, L., & Alon, U. (2018). Creative exploration as a scale-invariant search on a meaning landscape. Nature Communications, 9(1), 1–11. doi:10.1038/s41467-018-07715-8
   PubMedGoogle Scholar
• Hart, Y., Kosoy, E., Liquin, E. G., Leonard, J. A., Mackey, A. P., & Gopnik, A. (2022). The development of creative search strategies. Cognition, 225, 105102. doi:10.1016/j.cognition.2022.105102
   PubMed Web of Science ®Google Scholar
• Hart, Y., Mayo, A. E., Mayo, R., Rozenkrantz, L., Tendler, A., Alon, U., & Noy, L. (2017). Creative foraging: An experimental paradigm for studying exploration and discovery. PLoS One, 12(8), e0182133. doi:10.1371/journal.pone.0182133
   PubMed Web of Science ®Google Scholar
• Hofelich Mohr, A., Sell, A., & Lindsay, T. (2016). Thinking inside the box: Visual design of the response box affects creative divergent thinking in an online survey. Social Science Computer Review, 34(3), 347–359. doi:10.1177/0894439315588736
   Web of Science ®Google Scholar
• Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. Proceedings of the AAAI Workshop on Challenges in Game AI, 4(1), July:1722. https://www.aaai.org/Papers/Workshops/2004/WS-04-04/WS04-04-001.pdf
   Google Scholar
• Johnson, D. R., Kaufman, J. C., Baker, B. S., Patterson, J. D., Barbot, B., Green, A. E. , and Beaty, R. E. (2022). Divergent semantic integration (DSI): Extracting creativity from narratives with distributional semantic modeling. Behavior Research Methods, 1–34. doi:10.3758/s13428-022-01986-2
   Web of Science ®Google Scholar
• Karwowski, M. (2011). It doesn’t hurt to ask … but sometimes it hurts to believe: Polish students’ creative self-efficacy and its predictors. Psychology of Aesthetics, Creativity, and the Arts, 5(2), 154. doi:10.1037/a0021427
   Web of Science ®Google Scholar
• Karwowski, M. (2014). Creative mindsets: Measurement, correlates, consequences. Psychology of Aesthetics, Creativity, and the Arts, 8(1), 62–70. doi:10.1037/a0034898
   Web of Science ®Google Scholar
• Kaufman, J. C. (2015). Why creativity isn’t in IQ tests, why it matters, and why it won’t change anytime soon probably. Journal of Intelligence, 3(3), 59–72. doi:10.3390/jintelligence3030059
   Google Scholar
• Markman, A. B., & Gentner, D. (2001). Thinking. Annual Review of Psychology, 52(1), 223–247. doi:10.1146/annurev.psych.52.1.223
   PubMed Web of Science ®Google Scholar
• McCambridge, J., Witton, J., & Elbourne, D. R. (2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67(3), 267–277. doi:10.1016/j.jclinepi.2013.08.015
   PubMed Web of Science ®Google Scholar
• McCrae, R. R., & Costa, P. T., Jr. (1985). Openness to experience. Perspectives in Personality, 1, 145–172.
   Google Scholar
• Mednick, S. A. (1968). The remote associates test. The Journal of Creative Behavior, 2(3), 213–214. doi:10.1002/j.2162-6057.1968.tb00104.x
   Web of Science ®Google Scholar
• Nouri, R., Erez, M., Lee, C., Liang, J., Bannister, B. D., & Chiu, W. (2015). Social context: Key to understanding culture’s effects on creativity. Journal of Organizational Behavior, 36(7), 899–918. doi:10.1002/job.1923
   Web of Science ®Google Scholar
• Noy, L., Hart, Y., Andrew, N., Ramote, O., Mayo, A. E., & Alon, U. (2012). A quantitative study of creative leaps. ICCC (pp. 72–76).
   Google Scholar
• Nussenbaum, K., Scheuplein, M., Phaneuf, C. V., Evans, M. D., & Hartley, C. A. (2020). Moving developmental research online: Comparing in-lab and web-based studies of model-based reinforcement learning. Collabra: Psychology, 6(1). doi:10.1525/collabra.17213
   Google Scholar
• Olteţeanu, A. -M., & Zunjani, F. H. (2020). A visual remote associates test and its validation. Frontiers in Psychology, 11, 26. doi:10.3389/fpsyg.2020.00026
   PubMed Web of Science ®Google Scholar
• Organisciak, P., Acar, S., Dumas, D., & Berthiaume, K. (2022). Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models. doi:10.13140/RG.2.2.32393.31840
   Google Scholar
• Paek, S. H., Alabbasi, A. M. A., Acar, S., & Runco, M. A. (2021). Is more time better for divergent thinking? A meta-analysis of the time-on-task effect on divergent thinking. Thinking Skills and Creativity, 41, 100894. doi:10.1016/j.tsc.2021.100894
   Web of Science ®Google Scholar
• Pedersen, M. K., Díaz, C. M. C., Alba-Marrugo, M. A., Amidi, A., Basaiawmoit, R. V., Bergenholtz, C., &, and Sherson, J. F. (2020). Cognitive abilities in the wild: Population-scale game-based cognitive assessment (Version posted online). arXiv preprint arXiv:2009.05274. Retrieved April 20, 2021.
   Google Scholar
• Plucker, J. A. (2022). The patient is thriving! current issues, recent advances, and future directions in creativity assessment. Creativity Research Journal, 1–13. doi:10.1080/10400419.2022.2110415
   Web of Science ®Google Scholar
• Plucker, J. A., & Runco, M. A. (1998). The death of creativity measurement has been greatly exaggerated: Current issues, recent advances, and future directions in creativity assessment. Roeper Review, 21(1), 36–39. doi:10.1080/02783199809553924
   Google Scholar
• Rafner, J., Hjorth, A., Risi, S., Philipsen, L., Dumas, C., Biskjær, M. M., … & Sherson, J. (2020, November). CREA. Blender: a neural network-based image generation game to assess creativity. In Extended abstracts of the 2020 annual symposium on computer-human interaction in play (pp. 340-344).
   Google Scholar
• Rafner, J. (2021, June). Creativity assessment games and crowdsourcing. In Creativity and Cognition (pp. 1-5)
   Google Scholar
• Rafner, J., Biskjær, M. M., Zana, B., Langsford, S., Bergenholtz, C., Rahimi, S., & Sherson, J. (2022). Digital games for creativity assessment: Strengths, weaknesses and opportunities. Creativity Research Journal, 34(1), 28–54.
   Web of Science ®Google Scholar
• Rafner, J., Gajdacz, M., Kragh, G., Hjorth, A., Gander, A., Palfi, B. & Sherson, J. (2022). Mapping citizen science through the lens of human-centered AI. Human Computation, 9(1), 66–95. doi:10.15346/hc.v9i1.133
   Google Scholar
• Rahimi, S. (2020). Inspire, instruct, or both? Game-based assessment and support of creativity (Dissertation No. 28153346) [Doctoral dissertation, The Florida State University]. ProQuest Dissertations Publishing.
   Google Scholar
• Rahimi, S., & Shute, V. J. (2021). First inspire, then instruct to improve students’ creativity. Computers & Education, 174, 1–27. doi:10.1016/j.compedu.2021.104312
   Web of Science ®Google Scholar
• Raven, J. (2000). The Raven’s progressive matrices: Change and stability over culture and time. Cognitive Psychology, 41(1), 1–48. doi:10.1006/cogp.1999.0735
   PubMed Web of Science ®Google Scholar
• Reiter-Palmon, R., Forthmann, B., & Barbot, B. (2019). Scoring divergent thinking tests: A review and systematic framework. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 144. doi:10.1037/aca0000227
   Web of Science ®Google Scholar
• Rosenberg, G., & Noy, T. (2022). Social interaction modulates collective creativity, Proceedings from the Annual Meeting of the Cognitive Science Society, CA: Toronto
   Google Scholar
• Rozenkrantz, L., Mayo, A. E., Ilan, T., Hart, Y., Noy, L., Alon, U., & Manalo, E. (2017). Placebo can enhance creativity. PLoS One, 12(9), e0182466. doi:10.1371/journal.pone.0182466
   PubMed Web of Science ®Google Scholar
• Runco, M. A. (1986). Flexibility and originality in children’s divergent thinking. The Journal of Psychology, 120(4), 345–352. doi:10.1080/00223980.1986.9712632
   Web of Science ®Google Scholar
• Runco, M. A., Abdulla, A. M., Paek, S. H., Al-Jasim, F. A., & Alsuwaidi, H. N. (2016). Which test of divergent thinking is best? Creativity Theories–research-Applications, 3(1), 4–18. doi:10.1515/ctra-2016-0001
   Google Scholar
• Runco, M. A., & Acar, S. (2012). Divergent thinking as an indicator of creative potential. Creativity Research Journal, 24(1), 66–75. doi:10.1080/10400419.2012.652929
   Web of Science ®Google Scholar
• Runco, M. A., Illies, J. J., & Eisenman, R. (2005). Creativity, originality, and appropriateness: What do explicit instructions tell us about their relationships? The Journal of Creative Behavior, 39(2), 137–148. doi:10.1002/j.2162-6057.2005.tb01255.x
   Web of Science ®Google Scholar
• Saas, A., Guitart, A., & Periáñez, Á. (2016). Discovering playing patterns: Time series clustering of free-to-play game data. IEEE Conference on Computational Intelligence and Games (CIG), 2016, pp. 1–8, doi: 10.1109/CIG.2016.7860442.
   Google Scholar
• Said-Metwaly, S., Fernández-Castilla, B., Kyndt, E., & Van den Noortgate, W. (2020). Testing conditions and creative performance: Meta-analyses of the impact of time limits and instructions. Psychology of Aesthetics, Creativity, and the Arts, 14(1), 15. doi:10.1037/aca0000244
   Web of Science ®Google Scholar
• Said-Metwaly, S., Taylor, C. L., Camarda, A., & Barbot, B. (2022). Divergent thinking and creative achievement—how strong is the link? An updated meta-analysis. Psychology of Aesthetics, Creativity, and the Arts. Advance online publication. 10.1037/aca0000507
   Web of Science ®Google Scholar
• Said-Metwaly, S., Van den Noortgate, W., & Kyndt, E. (2017). Methodological issues in measuring creativity: A systematic literature review. Creativity Theories–research-Applications, 4(2), 276–301. doi:10.1515/ctra-2017-0014
   Google Scholar
• Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., & Manica, M. (2022). Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100.
   Google Scholar
• Schram, A., & Ule, A. (Eds.). (2019). Handbook of research methods and applications in experimental economics.
   Google Scholar
• Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). Charlotte, NC: Information Age Publishers.
   Google Scholar
• Shute, V. J., & Rahimi, S. (2021). Stealth assessment of creativity in a physics video game. Computers in Human Behavior, 116, 106647. doi:10.1016/j.chb.2020.106647
   Web of Science ®Google Scholar
• Silvia, P. J., Winterstein, B. P., Willse, J. T., Barona, C. M., Cram, J. T., Hess, K. I. … Richard, C. A. (2008). Assessing creativity with divergent thinking tasks: Exploring the reliability and validity of new subjective scoring methods. Psychology of Aesthetics, Creativity, and the Arts, 2(2), 68–85. doi:10.1037/1931-3896.2.2.68
   Google Scholar
• Sternberg, R. J., Glăveanu, V., & Kaufman, J. C. (2022). In quest of creativity: Three paths toward an Elusive Grail. Creativity Research Journal, 1–21. doi:10.1080/10400419.2022.2107299
   Web of Science ®Google Scholar
• Torrance, E. P. (1976). Creativity testing in education. Creative Child & Adult Quarterly.
   Google Scholar
• Wang, T., Roberts, A., Hesslow, D., Scao, T. L., Chung, H. W., Beltagy, I. Raffel, C. (2022). What language model architecture and pretraining objective work best for zero-shot generalization? arXiv preprint arXiv:2204.05832.
   Google Scholar
• World Economic Forum. (2020). Future of jobs report. https://www.weforum.org/reports/the-future-of-jobs-report-2020
   Google Scholar
• Xia, T., Kang, M., Chen, M., Ouyang, J., & Hu, F. (2021). Design training and creativity: Students develop stronger divergent but not convergent thinking. Frontiers in Psychology, 12. doi:10.3389/fpsyg.2021.695002
   Web of Science ®Google Scholar