Relationship validation among items and attributes

Abstract

In the typical application of a cognitive diagnosis assessment, the Q matrix, which illustrates the relationship between skills and the items, is assumed to be known. However, the Q matrix is usually determined by subject matter experts and test developers, and so there may be misspecification of its elements. This paper proposes a data-driven approach to jointly estimate the Q matrix, model parameters, and the examinees’ skill profiles. The key component is the likelihood ratio statistic that relates the Q matrix, responses, and the cognitive diagnosis model. Simulation studies show that the Q matrix validation based on the likelihood ratio statistic has a promising performance.

KEYWORDS:

• Formative assessment
• cognitive diagnosis
• Q matrix
• likelihood ratio
• DINA
• EM algorithm

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by The Social Science Planning Program of Jiangxi Province: [grant number 16JY11]; the Youth Foundation of Social Science and Humanity, China Ministry of Education: [grant number 17YJC190029]; the Humanities and Social Sciences Program of Jiangxi Provincial Department of Education: [grant number XL1509]; Research Project for Teaching Reform in Colleges and Universities in Jiangxi: [grant number JXJG-19-2-13,JXJG-19-23-2]; the Education Science and Technology Research Program of Jiangxi Provincial Department of Education: [grant number GJJ160309,GJJ191128,GJJ191691].