Evaluating Creativity: How Idea Context and Rater Personality Affect Considerations of Novelty and Usefulness

References

• Acar, S., Berthiaume, K., Grajzel, K., Dumas, D., & Flemister, C., “Tedd,” & Organisciak, P. (2021 December)Applying automated originality scoring to the verbal form of Torrance tests of creative thinking. Gifted Child Quarterly, 001698622110618. 10.1177/00169862211061874
   Web of Science ®Google Scholar
• Acar, S., Burnett, C., & Cabra, J. F. (2017). Ingredients of creativity: Originality and more. Creativity Research Journal, 29(2), 133–144. doi:10.1080/10400419.2017.1302776
   Web of Science ®Google Scholar
• Acar, S., Runco, M. A., & Park, H. (2019). What should people be told when they take a divergent thinking test? A meta-analytic review of explicit instructions for divergent thinking. Psychology of Aesthetics, Creativity, and the Arts, 14(1), 39–49. doi:10.1037/aca0000256
   Web of Science ®Google Scholar
• Amabile, T. M. (1982). Social psychology of creativity: A consensual assessment technique. Journal of Personality and Social Psychology, 43(5), 997–1013. doi:10.1037/0022-3514.43.5.997
   Web of Science ®Google Scholar
• Baer, J., & McKool, S. S. (2014). The gold standard for assessing creativity. International Journal of Quality Assurance in Engineering and Technology Education, 3(1), 81–93. doi:10.4018/ijqaete.2014010104
   Google Scholar
• Barbot, B., Hass, R. W., & Reiter-Palmon, R. (2019). Creativity assessment in psychological research: (Re)setting the standards. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 233–240. doi:10.1037/ACA0000233
   Web of Science ®Google Scholar
• Basadur, M. (1995). Optimal ideation-evaluation ratios. Creativity Research Journal, 8(1), 63–75. doi:10.1207/s15326934crj0801
   Web of Science ®Google Scholar
• Batey, M., & Furnham, A. (2006). Creativity, intelligence, and personality: A critical review of the scattered literature. Genetic, Social, and General Psychology Monographs, 132(4), 355–429. doi:10.3200/MONO.132.4.355-430
   PubMed Web of Science ®Google Scholar
• Beaty, R. E., & Johnson, D. R. (2021). Automating creativity assessment with SemDis: An open platform for computing semantic distance. Behavior Research Methods, 53(2), 757–780. doi:10.3758/s13428-020-01453-w
   PubMed Web of Science ®Google Scholar
• Blais, A., & Weber, E. U. (2006). A Domain-Specific Risk-Taking (DOSPERT) scale for adult populations. Judgement and Decision Making, 1(1), 33–47.
   Web of Science ®Google Scholar
• Boden, M. A. (2007). Creativity in a nutshell. Think, 5(15), 83–96. doi:10.1017/s147717560000230x
   Google Scholar
• Bonetto, E., Pichot, N., Pavani, J. B., & Adam-Troïan, J. (2021). Creative individuals are social risk-takers: Relationships between creativity, social risk-taking and fear of negative evaluations. Creativity, 7(2), 309–320. doi:10.2478/ctra-2020-0016
   Google Scholar
• Campbell, D. T. (1960). Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological Review, 67(6), 380–400. doi:10.1037/H0040373
   PubMed Web of Science ®Google Scholar
• Caroff, X., & Besançon, M. (2008). Variability of creativity judgments. Learning and Individual Differences, 18(4), 367–371. doi:10.1016/j.lindif.2008.04.001
   Web of Science ®Google Scholar
• Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, validity, and factor structure of the creative achievement questionnaire. Creativity Research Journal, 17(1), 37–50. doi:10.1207/S15326934CRJ1701_4
   Web of Science ®Google Scholar
• Ceh, S. M., Edelmann, C., Hofer, G., & Benedek, M. (2022). Assessing raters: What factors predict discernment in novice creativity raters? Journal of Creative Behavior, 56(1), 41–54. doi:10.1002/jocb.515
   Web of Science ®Google Scholar
• Corazza, G. E. (2016). Potential originality and effectiveness: The dynamic definition of creativity. Creativity Research Journal, 28(3), 258–267. doi:10.1080/10400419.2016.1195627
   Web of Science ®Google Scholar
• Cropley, D. H., & Marrone, R. L. (2021). Automated scoring of figural creativity using a convolutional neural network. PsyArXiv, (Online pre-print). doi:10.31234/OSF.IO/8QE7Y
   Google Scholar
• Cseh, G. M., & Jeffries, K. K. (2019). A scattered CAT: A critical evaluation of the consensual assessment technique for creativity research. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 159–166. doi:10.1037/ACA0000220
   Web of Science ®Google Scholar
• Dewett, T. (2007). Linking intrinsic motivation, risk taking, and employee creativity in an R&D environment. R and D Management, 37(3), 197–208. doi:10.1111/j.1467-9310.2007.00469.x
   Web of Science ®Google Scholar
• DeYoung, C. G., Peterson, J. B., & Higgins, D. M. (2005). Sources of openness/intellect: Cognitive and neuropsychological correlates of the fifth factor of personality. Journal of Personality, 73(4), 825–858. doi:10.1111/J.1467-6494.2005.00330.X
   PubMed Web of Science ®Google Scholar
• DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the big five. Journal of Personality and Social Psychology, 93(5), 880–896. doi:10.1037/0022-3514.93.5.880
   PubMed Web of Science ®Google Scholar
• Diedrich, J., Benedek, M., Jauk, E., & Neubauer, A. C. (2015). Are creative ideas novel and useful? Psychology of Aesthetics, Creativity, and the Arts, 9(1), 35–40. doi:10.1037/a0038688
   Web of Science ®Google Scholar
• Diedrich, J., Jauk, E., Silvia, P. J., Gredlein, J. M., Neubauer, A. C., & Benedek, M. (2018). Assessment of real-life creativity: The inventory of creative activities and achievements (ICAA). Psychology of Aesthetics, Creativity, and the Arts, 12(3), 304–316. doi:10.1037/ACA0000137
   Web of Science ®Google Scholar
• Edwards, K. M., Peng, A., Miller, S. R., & Ahmed, F. (2021). If a picture is worth 1000 words, is a word worth 1000 features for design metric estimation? Proceedings of the ASME Design Engineering Technical Conference, 6. doi:10.1115/DETC2021-70158
   Google Scholar
• Efron, B., & Morris, C. (1977). Stein’s paradox in statistics. Scientific American, 236(5), 119–127. Retrieved from http://www.jstor.org/stable/24954030
   Web of Science ®Google Scholar
• Erbas, A. K., & Bas, S. (2015). The contribution of personality traits, motivation, academic risk-taking and metacognition to the creative ability in mathematics. Creativity Research Journal, 27(4), 299–307. doi:10.1080/10400419.2015.1087235
   Web of Science ®Google Scholar
• Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. doi:10.3758/BF03193146
   PubMed Web of Science ®Google Scholar
• Feist, G. J. (1998). A meta-analysis of personality in scientific and artistic creativity. Personality and Social Psychology Review, 2(4), 290–309. doi:10.1207/s15327957pspr0204_5
   PubMedGoogle Scholar
• Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory, research, and applications. Cambridge, MA: MIT Press.
   Google Scholar
• Gibson, C., & Mumford, M. D. (2013). Evaluation, criticism, and creativity: Criticism content and effects on creative problem solving. Psychology of Aesthetics, Creativity, and the Arts, 7(4), 314–331. doi:10.1037/A0032616
   Web of Science ®Google Scholar
• Gilhooly, K. J., Fioratou, E., Anthony, S. H., & Wynn, V. (2007). Divergent thinking: Strategies and executive involvement in generating novel uses for familiar objects. British Journal of Psychology, 98(4), 611–625. doi:10.1111/J.2044-8295.2007.TB00467.X
   PubMed Web of Science ®Google Scholar
• Glover, J. A., & Sautter, F. (1977). Relation of four components of creativity to risk-taking preferences. Psychological Reports, 41(1), 227–230. doi:10.2466/pr0.1977.41.1.227
   PubMed Web of Science ®Google Scholar
• Guilford, J. P. (1967). The Nature of Human Intelligence. New York, NY: Mc-Graw Hill.
   Google Scholar
• Han, J., Forbes, H., & Schaefer, D. (2021). An exploration of how creativity, functionality, and aesthetics are related in design. Research in Engineering Design, 32(3), 289–307. doi:10.1007/s00163-021-00366-9
   Web of Science ®Google Scholar
• Hennessey, B. A., & Amabile, T. M. (2010). Creativity. Annual Review of Psychology, 61, 569–598. doi:10.1146/annurev.psych.093008.100416
   PubMed Web of Science ®Google Scholar
• Herman, A., & Reiter-Palmon, R. (2011). The effect of regulatory focus on idea generation and idea evaluation. Psychology of Aesthetics, Creativity, and the Arts, 5(1), 13–20. doi:10.1037/a0018587
   Web of Science ®Google Scholar
• Ivancovsky, T., Shamay-Tsoory, S., Lee, J., Morio, H., & Kurman, J. (2019). A dual process model of generation and evaluation: A theoretical framework to examine cross-cultural differences in the creative process. Personality and Individual Differences, 139, 60–68. July 2018. doi:10.1016/j.paid.2018.11.012
   Web of Science ®Google Scholar
• Karwowski, M., Czerwonka, M., & Kaufman, J. C. (2020). Does intelligence strengthen creative metacognition? Psychology of Aesthetics, Creativity, and the Arts, 14(3), 353–360. doi:10.1037/ACA0000208
   Web of Science ®Google Scholar
• Katahira, K. (2016). How hierarchical models improve point estimates of model parameters at the individual level. Journal of Mathematical Psychology, 73, 37–58. doi:10.1016/j.jmp.2016.03.007
   Web of Science ®Google Scholar
• Kaufman, J. C. (2019). Self-assessments of creativity: Not ideal, but better than you think. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 187–192. doi:10.1037/ACA0000217
   Web of Science ®Google Scholar
• Kaufman, J. C., Lee, J., Baer, J., & Lee, S. (2007). Captions, consistency, creativity, and the consensual assessment technique: New evidence of reliability. Thinking Skills and Creativity, 2(2), 96–106. doi:10.1016/J.TSC.2007.04.002
   Web of Science ®Google Scholar
• Kaufman, S. B., Quilty, L. C., Grazioplene, R. G., Hirsh, J. B., Gray, J. R., Peterson, J. B., … DeYoung, C. G. (2016). Openness to experience and intellect differentially predict creative achievement in the arts and sciences. Journal of Personality, 84(2), 248–258. doi:10.1111/jopy.12156
   PubMed Web of Science ®Google Scholar
• Kharkhurin, A. V. (2014). Creativity.4in1: Four-criterion construct of creativity. Creativity Research Journal, 26(3), 338–352. doi:10.1080/10400419.2014.929424
   Web of Science ®Google Scholar
• Kleinmintz, O. M., Goldstein, P., Mayseless, N., Abecasis, D., Shamay-Tsoory, S. G., & Antonietti, A. (2014). Expertise in musical improvisation and creativity: The mediation of idea evaluation. PLoS ONE, 9(7), e101568. doi:10.1371/journal.pone.0101568
   PubMed Web of Science ®Google Scholar
• Kleinmintz, O. M., Ivancovsky, T., & Shamay-Tsoory, S. G. (2019). The twofold model of creativity: The neural underpinnings of the generation and evaluation of creative ideas. Current Opinion in Behavioral Sciences, 27, 131–138. doi:10.1016/j.cobeha.2018.11.004
   Web of Science ®Google Scholar
• Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. doi:10.3389/fpsyg.2013.00863
   PubMed Web of Science ®Google Scholar
• Lee, Y. S., Chang, J. Y., & Choi, J. N. (2017). Why reject creative ideas? Fear as a driver of implicit bias against creativity. Creativity Research Journal, 29(3), 225–235. doi:10.1080/10400419.2017.1360061
   Web of Science ®Google Scholar
• Long, H. (2014). More than appropriateness and novelty: Judges’ criteria of assessing creative products in science tasks. Thinking Skills and Creativity, 13, 183–194. doi:10.1016/j.tsc.2014.05.002
   Web of Science ®Google Scholar
• Lubart, T. I. (2001). Models of the creative process: Past, present and future. Creativity Research Journal, 13(3–4), 295–308. doi:10.1207/S15326934CRJ1334
   Web of Science ®Google Scholar
• Luft, C. D. B., Zioga, I., Thompson, N. M., Banissy, M. J., & Bhattacharya, J. (2018). Right temporal alpha oscillations as a neural mechanism for inhibiting obvious associations. Proceedings of the National Academy of Sciences of the United States of America, 115(52), E12144–E12152. doi:10.1073/pnas.1811465115
   PubMed Web of Science ®Google Scholar
• Martin, L., & Wilson, N. (2017). Defining creativity with discovery. Creativity Research Journal, 29(4), 417–425. doi:10.1080/10400419.2017.1376543
   Web of Science ®Google Scholar
• Mastria, S., Agnoli, S., Corazza, G. E., & Eisenbarth, H. (2019). How does emotion influence the creativity evaluation of exogenous alternative ideas? PLOS ONE, 14(7), e0219298. doi:10.1371/JOURNAL.PONE.0219298
   PubMed Web of Science ®Google Scholar
• McCarthy, M., Chen, C. C., & McNamee, R. C. (2018). Novelty and usefulness trade-off: Cultural cognitive differences and creative idea evaluation. Journal of Cross-Cultural Psychology, 49(2), 171–198. doi:10.1177/0022022116680479
   Web of Science ®Google Scholar
• McCrae, R. R., & Ingraham, L. J. (1987). Creativity, divergent thinking, and openness to experience. Journal of Personality and Social Psychology, 52(6), 1258–1265. doi:10.1037/0022-3514.52.6.1258
   Web of Science ®Google Scholar
• McIntosh, T., Mulhearn, T. J., & Mumford, M. D. (2021). Taking the good with the bad: The impact of forecasting timing and valence on idea evaluation and creativity. Psychology of Aesthetics, Creativity, and the Arts, 15(1), 111–124. doi:10.1037/ACA0000237
   Web of Science ®Google Scholar
• McNeish, D., & Kelley, K. (2019). Fixed effects models versus mixed effects models for clustered data: Reviewing the approaches, disentangling the differences, and making recommendations. Psychological Methods, 24(1), 20–35. doi:10.1037/MET0000182
   PubMed Web of Science ®Google Scholar
• Mueller, J. S., Melwani, S., & Goncalo, J. A. (2012). The bias against creativity: Why people desire but reject creative ideas. Psychological Science, 23(1), 13–17. doi:10.1177/0956797611421018
   PubMed Web of Science ®Google Scholar
• Myszkowski, N., & Storme, M. (2019). Judge response theory? A call to upgrade our psychometrical account of creativity judgments. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 167–175. doi:10.1037/aca0000225
   Web of Science ®Google Scholar
• Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixedeffects models. Methods in Ecology and Evolution, 4(2), 133–142. doi:10.1111/J.2041-210X.2012.00261.X
   Web of Science ®Google Scholar
• Oleynick, V. C., DeYoung, C. G., Hyde, E., Barry Kaufman, S., Beaty, R. E., & Silvia, P. J. (2017). Openness/Intellect: The core of the creative personality. In G. J. Feist, R. Reiter-Palmon, & J. C. Kaufman (Eds.), Cambridge handbook of creativity and personality research (pp. 9–27). New York: Cambridge University Press.
   Google Scholar
• Pétervári, J. (2018). The evaluation of creative ideas – Analysing the differences between expert and novice judges. (Doctoral Thesis): Queen Mary University of London, London. Retrieved from http://qmro.qmul.ac.uk/xmlui/handle/123456789/42526.
   Google Scholar
• Pinho, A. L., de Manzano, Ö., Fransson, P., Eriksson, H., & Ullén, F. (2014). Connecting to create: Expertise in musical improvisation is associated with increased functional connectivity between premotor and prefrontal areas. Journal of Neuroscience, 34(18), 6156–6163. doi:10.1523/JNEUROSCI.4769-13.2014
   PubMed Web of Science ®Google Scholar
• Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004). Why isn’t creativity more important to educational psychologists? Potentials, pitfalls, and future directions in creativity research. Educational Psychologist, 39(2), 83–96. doi:10.1207/s15326985ep3902_1
   Web of Science ®Google Scholar
• Plucker, J., Qian, M., & Wang, S. (2011). Is originality in the eye of the beholder? Comparison of scoring techniques in the assessment of divergent thinking. Journal of Creative Behavior, 45(1), 1–22. doi:10.1002/j.2162-6057.2011.tb01081.x
   Web of Science ®Google Scholar
• Prabhakaran, R., Green, A. E., & Gray, J. R. (2014). Thin slices of creativity: Using single-word utterances to assess creative cognition. Behavior Research Methods, 46(3), 641. doi:10.3758/S13428-013-0401-7
   PubMed Web of Science ®Google Scholar
• Puryear, J. S., & Lamb, K. N. (2020). Defining creativity: How far have we come since Plucker, Beghetto, and Dow? Creativity Research Journal, 32(3), 206–214. doi:10.1080/10400419.2020.1821552
   Web of Science ®Google Scholar
• Rodriguez, W. A., Cheban, Y., Shah, S., & Watts, L. L. (2020). The general factor of personality and creativity: Diverging effects on intrapersonal and interpersonal idea evaluation. Personality and Individual Differences, 167, 110229. doi:10.1016/j.paid.2020.110229
   Web of Science ®Google Scholar
• Rominger, C., Papousek, I., Perchtold, C. M., Weber, B., Weiss, E. M., & Fink, A. (2018). The creative brain in the figural domain: Distinct patterns of EEG alpha power during idea generation and idea elaboration. Neuropsychologia, 118, 13–19. doi:10.1016/J.NEUROPSYCHOLOGIA.2018.02.013
   PubMed Web of Science ®Google Scholar
• Runco, M. A., & Charles, R. E. (1993). Judgments of originality and appropriateness as predictors of creativity. Personality and Individual Differences, 15(5), 537–546. doi:10.1016/0191-8869(93)90337-3
   Web of Science ®Google Scholar
• Runco, M. R., Illies, J. J., & Eisenman, R. (2005). Creativity, originality, and appropriateness: What do explicit instructions tell us about their relationships? Journal of Creative Behavior, 39(2), 137–148. doi:10.1002/j.2162-6057.2005.tb01255.x
   Web of Science ®Google Scholar
• Runco, M. A., & Jaeger, G. J. (2012). The standard definition of creativity. Creativity Research Journal, 24(1), 92–96. doi:10.1080/10400419.2012.650092
   Web of Science ®Google Scholar
• Runco, M. A., & Smith, W. R. (1992). Interpersonal and intrapersonal evaluations of creative ideas. Personality and Individual Differences, 13(3), 295–302. doi:10.1016/0191-8869(92)90105-X
   Web of Science ®Google Scholar
• Shen, W., Hommel, B., Yuan, Y., Chang, L., & Zhang, W. (2018). Risk-taking and creativity: Convergent, but not divergent thinking is better in low-risk takers. Creativity Research Journal, 30(2), 224–231. doi:10.1080/10400419.2018.1446852
   Web of Science ®Google Scholar
• Silvia, P. J. (2008). Discernment and creativity: How well can people identify their most creative ideas? Psychology of Aesthetics, Creativity, and the Arts, 2(3), 139–146. doi:10.1037/1931-3896.2.3.139
   Google Scholar
• Simonton. (1999). Origins of genius. Darwinian perspectives on creativity. Oxford: NY: Oxford University Press.
   Google Scholar
• Simonton, D. K. (2013). Creative thought as blind variation and selective retention: Why creativity is inversely related to sightedness. Journal of Theoretical and Philosophical Psychology, 33(4), 253–266. doi:10.1037/a0030705
   Google Scholar
• Simonton, D. K. (2018). Defining creativity: Don’t we also need to define what is not creative? Journal of Creative Behavior, 52(1), 80–90. doi:10.1002/jocb.137
   Web of Science ®Google Scholar
• Singmann, H., & Kellen, D. (2019). An introduction to mixed models for experimental psychology. In New methods in cognitive psychology (pp. 4–31). Taylor and Francis. doi:10.4324/9780429318405-2
   Google Scholar
• Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712. doi:10.1177/1745691616658637
   Web of Science ®Google Scholar
• Stein, M. I. (1953). Creativity and culture. The Journal of Psychology, 36(2), 311–322. doi:10.1080/00223980.1953.9712897
   Web of Science ®Google Scholar
• Sternberg, R. J. (2018). A triangular theory of creativity. Psychology of Aesthetics, Creativity, and the Arts, 12(1), 50–67. doi:10.1037/aca0000095
   Web of Science ®Google Scholar
• Storme, M., & Lubart, T. (2012). Conceptions of creativity and relations with judges’ intelligence and personality. Journal of Creative Behavior, 46(2), 138–149. doi:10.1002/jocb.10
   Web of Science ®Google Scholar
• Taylor, C. W. (1988). Various approaches to and definitions of creativity. In R. J. Sternberg (Ed.), The nature of creativity: Contemporary psychological perspectives (pp. 99–121). Cambridge University Press.
   Google Scholar
• Treffinger, D. (1992). Stimulating creativity: Issues and future directions. In S. G. Isaksen, M. C. Murdock, R. L. Firestein, & D. J. Treffinger (Eds.), Nurturing and developing creativity: The emergence of a discipline (pp. 8–27). Norwood, NJ: Ablex.
   Google Scholar
• Tyagi, V., Hanoch, Y., Hall, S. D., Runco, M., & Denham, S. L. (2017). The risky side of creativity: Domain specific risk taking in creative individuals. Frontiers in Psychology, 8. doi:10.3389/fpsyg.2017.00145
   PubMed Web of Science ®Google Scholar
• Watts, L. L., Steele, L. M., Medeiros, K. E., & Mumford, M. D. (2019). Minding the gap between generation and implementation: Effects of idea source, goals, and climate on selecting and refining creative ideas. Psychology of Aesthetics, Creativity, and the Arts, 13(1), 2–14. doi:10.1037/ACA0000157
   Web of Science ®Google Scholar
• Wilken, A., Forthmann, B., & Holling, H. (2020). Instructions moderate the relationship between creative performance in figural divergent thinking and reasoning capacity. Journal of Creative Behavior, 54(3), 582–597. doi:10.1002/jocb.392
   Web of Science ®Google Scholar
• Wilson, R. C., Guilford, J. P., & Christensen, P. R. (1953). The measurement of individual differences in originality. Psychological Bulletin, 50(5), 362–370. doi:10.1037/H0060857
   PubMed Web of Science ®Google Scholar