Diagnosing Competency Mastery in Science: An Application of GDM to TIMSS 2011 Data

ABSTRACT

Numerous diagnostic studies have been conducted on large-scale assessments to illustrate the students’ mastery profile in the areas of math and reading; however, for science a limited number of investigations are reported. This study investigated Iranian eighth graders’ competency mastery of science and examined the utility of the General Diagnostic Model (GDM) to produce diagnostic information using TIMSS 2011 data. Eight diagnostic attributes were extracted, using documentary analysis of the major large-scale assessment frameworks, including basic science knowledge, using models, reasoning, using science, representing data, explaining of phenomena, predicting, and scientific inquiry. These attributes were then assigned to each item in order to construct the Q matrix, through focus group discussions, survey of head science teachers, think-aloud verbal protocol, and analysis of written answers. Results show the utility of GDM to generate rich diagnostic information for a national large-scale assessment. Moreover, the findings indicated that students performed relatively well in using science, but performed weakly in reasoning, explaining of phenomena, and scientific inquiry, which all required complex skills and higher-order thinking abilities.