Is What You See What You Really Get? Comparison of Scoring Techniques in the Assessment of Real-World Divergent Thinking

Abstract

In recent years, the social sciences have seen a resurgence in the study of divergent thinking (DT) measures. However, many of these recent advances have focused on abstract, decontextualized DT tasks (e.g., list as many things as you can think of that have wheels). This study provides a new perspective by exploring the reliability and validity evidence for several methods for scoring real-world DT tasks (i.e., tasks situated within a real-world problem or situation). The results suggest a combination of objective and subjective scoring methods may be optimal for originality scoring for contextualized DT tasks, which stands in contrast to recent research suggesting the objective, percentage scoring technique may be optimal for scoring originality on abstract tasks.

Acknowledgments

The research reported in this article was supported, in part, by a grant to the first author from the Kempf Assessment Fund at the Indiana University School of Education. We appreciate the input and encouragement provided by James Kaufman and Paul Silvia throughout this line of research. A version of this article was presented at the 2011 conference of the National Association for Gifted Children in New Orleans, LA, and at a special symposium in 2011 at East China Normal University in Shanghai, China.

Notes

¹See Runco (1984, 1985) for exceptions.

Note. ICC: intraclass correlation coefficient.

Note. r =Pearson product-moment correlations. tau =Kendall rank order correlations.

*p < .05. **p < .01.

Note. r =Pearson product-moment correlations. tau =Kendall rank order correlations.

*p < .05. **p < .01.

Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/hcrj.