An Analytic Creativity Assessment Scale for Digital Game Story Design: Construct Validity, Internal Consistency and Interrater Reliability

Abstract

Mobile technology has rapidly made digital games a popular entertainment to this digital generation, and thus digital game design received considerable attention in both the game industry and design education. Digital game design involves diverse dimensions in which digital game story design (DGSD) particularly attracts our interest, as the literature needs more information, especially the creativity assessment of DGSD. Existing measuring tools do not adequately address the characteristics of game-story duality. Thus, an analytic creativity assessment scale of DGSD (CAS-DGSD) based on literature and original ideas was developed in our previous work. This study aims to statistically examine its construct validity, internal consistency reliability, and interrater reliability to verify its effectiveness. Three commercial games of 3 different game genres (action, puzzle, and role-play) were rated by 32 student raters and 4 expert raters. Statistical results show acceptable construct validity, internal consistency reliability, and interrater reliability of the CAS-DGSD. The CAS-DGSD not only helps evaluators like teachers identify which aspects of DGSD are short of creativity, but also serves as a guideline for digital game story designers like design students and product developers to tailor creative and entertaining game stories.