Using Process Data to Improve Classification Accuracy of Cognitive Diagnosis Model

Abstract

With the advance of computer-based assessments, many process data, such as response times (RTs), action sequences, Eye-tracking data, the log data for collaborative problem-solving (CPS) and mouse click/drag becomes readily available. Findings from previous studies (e.g., Peng et al., Multivariate Behavioral Research, 1–20, 2021; Xu, The British Journal of Mathematical and Statistical Psychology, 73(3), 474–505, 2020; He & von Davier, Handbook of research on technology tools for real-world skill development (pp. 750–777). IGI Global, 2016; Man & Harring, Educational and Psychological Measurement, 81(3), 441–465, 2021) suggest a substantial relationship between this human–computer interactive process information and proficiency, which means these process data were potentially useful variables for psychological and educational measurement. To make full use of the process data, this paper aims to combine two useful and easily available types of process data, including the mouse click/drag traces and the response times, to the conventional cognitive diagnostic model (CDM) to better understand individual’s response behavior and improve the classification accuracy of existing CDM. Then the full Bayesian analysis using Markov chain Monte Carlo (MCMC) was employed to estimate the proposed model parameters. The viability of the proposed model was investigated by an empirical data and two simulation studies. Results indicated the proposed model combing both types of process data could not only improve the attribute classification reliability in real data analysis, but also provide an improvement on item parameters recovery and person classification accuracy.

Keywords:

• Computer-based assessments
• process data
• cognitive diagnosis model
• mouse click/drag