Retrofitting Non-cognitive-diagnostic Reading Assessment Under the Generalized DINA Model Framework

ABSTRACT

Cognitive diagnosis models (CDMs) are psychometric models developed mainly to assess examinees’ specific strengths and weaknesses in a set of skills or attributes within a domain. By adopting the Generalized-DINA model framework, the recently developed general modeling framework, we attempted to retrofit the PISA reading assessments, a non-cognitive-diagnostic assessment, in three steps: (a) Q-matrices construction based on item content analysis; (b) Q-matrices validation and refinement based on a saturated CDM; and (c) reduced CDMs evaluations based on acceptable Q-matrix. Our research showed that (a) the practice of retrofitting non-cognitive-diagnostic assessment for cognitive diagnostic purpose was feasible; (b) Q-matrix construction and CDM selection were especially important when retrofitting approach was adopted; (c) compensatory and saturated CDMs could fit the PISA reading assessment better than other CDMs; and (d) the application of CDMs to reading assessments may promote research on reading construct.

Funding

This research was supported by the Ministry of Education in China under the Humanities and Social Sciences Research Grant No. 14YJA880005.

Additional information

Funding

This research was supported by the Ministry of Education in China under the Humanities and Social Sciences Research Grant No. 14YJA880005.