Is What You See What You Really Get? Comparison of Scoring Techniques in the Assessment of Real-World Divergent Thinking

REFERENCES

• Amabile , T. M. ( 1982 ). Social psychology of creativity: A consensual assessment technique . Journal of Personality and Social Psychology , 43 , 997 – 1013 .
   Web of Science ®Google Scholar
• Amabile , T. M. ( 1996 ). Creativity in context: Update to “the social psychology of creativity.” Boulder, CO: Westview Press.
   Google Scholar
• Caroff , X. , & Besançon , M. ( 2008 ). Variability of creativity judgments . Learning and Individual Differences , 18 , 367 – 371 .
   Web of Science ®Google Scholar
• Chand , I. , & Runco , M. A. (1992). Problem finding skills as components in the creative process. Personality and Individual Differences , 14, 155–162.
   Web of Science ®Google Scholar
• Clark , P. M. , & Mirels , H. L. ( 1970 ). Fluency as a pervasive element in the measurement of creativity . Journal of Educational Measurement , 7 , 83 – 86 .
   Web of Science ®Google Scholar
• Davis , G. A. ( 1989 ). Testing for creative potential . Contemporary Educational Psychology , 14 , 257 – 274 .
   Web of Science ®Google Scholar
• Gough , H. G. ( 1979 ). A creative personality scale for the Adjective Check List . Journal of Personality and Social Psychology , 37 , 1398 – 1405 .
   Web of Science ®Google Scholar
• Hocevar , D. ( 1979a ). A comparison of statistical infrequency and subjective judgment as criteria in the measurement of originality . Journal of Personality Assessment , 43 , 297 – 299 .
   PubMed Web of Science ®Google Scholar
• Hocevar , D. ( 1979b ). Ideational fluency as a confounding factor in the measurement of originality . Journal of Educational Psychology , 71 , 191 – 196 .
   Web of Science ®Google Scholar
• Hocevar , D. ( 1979c ). The unidimensional nature of creative thinking in fifth grade children . Child Study Journal , 9 , 273 – 278 .
   Google Scholar
• Hocevar , D. ( 1981 ). Measurement of creativity: Review and critique . Journal of Personality Assessment , 45 , 450 – 464 .
   PubMed Web of Science ®Google Scholar
• Hocevar , D. , & Bachelor , P. ( 1989 ). A taxonomy and critique of measurements used in the study of creativity . In J. A. Glover , R. R. Ronning & C. R. Reynolds (Eds.), Handbook of creativity (pp. 53 – 75 ). New York , NY : Plenum Press .
   Google Scholar
• Hocevar , D. , & Michael , W. B. ( 1979 ). The effects of scoring formulas on the discriminant validity of tests of divergent thinking . Educational and Psychological Measurement , 39 , 917 – 921 .
   Web of Science ®Google Scholar
• Hong , E. , Milgram , R. M. , & Gorsky , H. ( 1995 ). Original thinking as a predictor of creative performance in young children . Roeper Review , 18 , 147 – 149 .
   Google Scholar
• Kane , M. T. ( 2008 ). Terminology, emphasis, and utility in validation . Educational Researcher , 37 , 76 – 82 .
   Web of Science ®Google Scholar
• Kaufman , J. C. , Baer , J. , Cole , J. C. , & Sexton , J. D. ( 2008 ). A comparison of expert and nonexpert raters using the consensual assessment technique . Creativity Research Journal , 20 , 171 – 178 .
   Web of Science ®Google Scholar
• Kaufman , J. C. , Plucker , J. A. , & Baer , J. ( 2008 ). Essentials of creativity assessment . New York , NY : Wiley .
   Google Scholar
• Messick , S. ( 1989 ). Meaning and values in test validation: The science and ethics of assessment . Educational Researcher , 18 , 5 – 11 .
   Google Scholar
• Messick , S. ( 1995 ). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning . American Psychologist , 50 , 741 – 749 .
   Web of Science ®Google Scholar
• Milgram , R. M. , & Hong , E. ( 1993 ). Creative thinking and creative performance in adolescents as predictors of creative attainments in adults: A follow-up study after 18 years . Roeper Review , 15 , 135 – 139 .
   Google Scholar
• Milgram , R. M. , & Milgram , N. A. ( 1976 ). Creative thinking and creative performance in Israeli students . Journal of Educational Psychology , 68 , 255 – 259 .
   PubMed Web of Science ®Google Scholar
• Milgram , R. M. , & Milgram , N. A. ( 1978 ). Quality and quantity of creative thinking in children and adolescents . Child Development , 49 , 385 – 388 .
   Web of Science ®Google Scholar
• Okuda , S. M. , Runco , M. A. , & Berger , D. E. ( 1991 ). Creativity and the finding and solving or real-world problems . Journal of Psychoeducational Assessment , 9 , 45 – 53 .
   Web of Science ®Google Scholar
• Plucker , J. A. ( 1999a ). Is the proof in the pudding? Reanalyses of Torrance's (1958 to Present) longitudinal data . Creativity Research Journal , 12 , 103 – 114 .
   Web of Science ®Google Scholar
• Plucker , J. A. ( 1999b ). Reanalyses of student responses to creativity checklists: Evidence of content generality . Journal of Creative Behavior , 33 , 126 – 137 .
   Web of Science ®Google Scholar
• Plucker , J. , Beghetto , R. A. , & Dow , G. ( 2004 ). Why isn't creativity more important to educational psychologists? Potential, pitfalls, and future directions in creativity research . Educational Psychologist , 39 , 83 – 96 .
   Web of Science ®Google Scholar
• Plucker , J. , Kaufman , J. C. , Temple , J. S. , & Qian , M. ( 2009 ). Do experts and novices evaluate movies the same way? Psychology & Marketing , 26 , 470 – 478 .
   Web of Science ®Google Scholar
• Plucker , J. , & Makel , M. ( 2010 ). Assessment of creativity . In R. J. Sternberg & J. C. Kaufman (Eds.), The Cambridge handbook of creativity (pp. 48 – 73 ). New York , NY : Cambridge .
   Google Scholar
• Plucker , J. , Qian , M. , & Wang , S. ( 2011 ). Is originality in the eye of the beholder? Comparison of scoring techniques in the assessment of divergent thinking . Journal of Creative Behavior , 45 , 1 – 22 .
   Web of Science ®Google Scholar
• Plucker , J. A. , & Runco , M. ( 1998 ). The death of creativity measurement has been greatly exaggerated: Current issues, recent advances, and future directions in creativity assessment . Roeper Review , 21 , 36 – 39 .
   Google Scholar
• Runco , M. A. ( 1984 ). Teachers' judgments of creativity and social validation of divergent thinking tests . Perceptual and Motor Skills , 59 , 711 – 717 .
   Web of Science ®Google Scholar
• Runco , M. A. ( 1985 ). Reliability and convergent validity of ideational flexibility as a function of academic achievement . Perceptual and Motor Skills , 61 , 1075 – 1081 .
   Web of Science ®Google Scholar
• Runco , M. A. ( 1987 ). Interrater agreement on a socially valid measure of students' creativity . Psychological Reports , 61 , 1009 – 1010 .
   Web of Science ®Google Scholar
• Runco , M. A. ( 1991 ). Divergent thinking . Norwood , NJ : Ablex .
   Google Scholar
• Runco , M. A. ( 1992 ). Children's divergent thinking and creative ideation . Developmental Review , 12 , 233 – 264 .
   Web of Science ®Google Scholar
• Runco , M. A. ( 1999 ). Divergent and creative thinking . Cresskill , NJ : Hampton Press .
   Google Scholar
• Runco , M. A. ( 2008 ). Commentary: Divergent thinking is not synonymous with creativity . Psychology of Aesthetics, Creativity, and the Arts , 2 , 93 – 96 .
   Google Scholar
• Runco , M. A. , & Albert , R. S. ( 1985 ). The reliability and validity of ideational originality in the divergent thinking of academically gifted and nongifted children . Educational and Psychological Measurement , 45 , 483 – 501 .
   Web of Science ®Google Scholar
• Runco , M. A. , & Charles , R. E. ( 1993 ). Judgments of originality and appropriateness as predictors of creativity . Personality and Individual Differences , 15 , 537 – 546 .
   Web of Science ®Google Scholar
• Runco , M. A. , Illies , J. J. , & Eisenman , R. ( 2005 ). Creativity, originality, and appropriateness: What do explicit instructions tell us about their relationships? Journal of Creative Behavior , 39 , 137 – 148 .
   Web of Science ®Google Scholar
• Runco , M. A. , & Mraz , W. ( 1992 ). Scoring divergent thinking tests using total ideational output and a creativity index . Educational and Psychological Measurement , 52 , 213 – 221 .
   Web of Science ®Google Scholar
• Runco , M. A. , & Okuda , S. M. ( 1988 ). Problem-discovery, divergent thinking, and the creative process . Journal of Youth and Adolescence , 17 , 211 – 220 .
   PubMed Web of Science ®Google Scholar
• Runco , M. A. , Okuda , S. M. , & Thurston , B. J. ( 1987 ). The psychometric properties of four systems for scoring divergent thinking tests . Journal of Psychoeducational Assessment , 5 , 149 – 156 .
   Google Scholar
• Runco , M. A. , & Smith , W. R. ( 1992 ). Interpersonal and intrapersonal evaluations of creative ideas . Personality and Individual Differences , 13 , 295 – 302 .
   Web of Science ®Google Scholar
• Sawyers , J. K. , & Canestaro , N. C. ( 1989 ). Creativity and achievement in design coursework . Creativity Research Journal , 2 , 126 – 133 .
   Google Scholar
• Silvia , P. J. ( 2008 ). Creativity and intelligence revisited: A latent variable analysis of Wallach and Kogan (1965) . Creativity Research Journal , 20 , 34 – 39 .
   Web of Science ®Google Scholar
• Silvia , P. J. ( 2011 ). Subjective scoring of divergent thinking: Examining the reliability of unusual uses, instances, and consequences tasks . Thinking Skills and Creativity , 6 , 24 – 30 .
   Web of Science ®Google Scholar
• Silvia , P. J. , Martin , C. , & Nusbaum , E. C. ( 2009 ). A snapshot of creativity: Evaluating a quick and simple method for assessing divergent thinking . Thinking Skills and Creativity , 4 , 79 – 85 .
   Web of Science ®Google Scholar
• Silvia , P. J. , Winterstein , B. P. , Willse , J. T. , Barona , C. M. , Cram , J. T. , Hess , K. I. , … Richard , C. A. ( 2008 ). Assessing creativity with divergent thinking tasks: Exploring the reliability and validity of new subjective scoring methods . Psychology of Aesthetics, Creativity, and the Arts , 2 , 68 – 85 .
   Google Scholar
• Torrance , E. P. (1988). The nature of creativity as manifest in its testing. In R. J. Sternberg (Ed.), The nature of creativity: Contemporary psychological perspectives (pp. 43–75). New York , NY : Cambridge University Press.
   Google Scholar
• Wallach , M. A. , & Kogan , N. ( 1965 ). Modes of thinking in young children: A study of the creativity-intelligence distinction . New York , NY : Holt, Rinehart and Winston .
   Google Scholar
• Wilson , R. C. , Guilford , J. P. , & Christensen , P. R. ( 1953 ). The measurement of individual differences in originality . Psychological Bulletin , 50 , 362 – 370 .
   PubMed Web of Science ®Google Scholar
• Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/hcrj.
   Google Scholar