Applications of Diagnostic Classification Models: A Literature Review and Critical Commentary

References

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: AERA.
   Google Scholar
• Bandalos, D. L., & Finney, S. J. (2010). Factor analysis: Exploratory and confirmatory. In G. R. Hancock, & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 93–114). New York, NY: Routledge.
   Google Scholar
• Boomsma, A. (2000). Reporting analyses of covariance structures. Structural Equation Modeling: A Multidisciplinary Journal, 7, 461–483. doi:10.1207/S15328007SEM0703_6
   Web of Science ®Google Scholar
• Borsboom, D., & Mellenberg, G. J. (2007). Test validity in cognitive assessment. In J. P. Leighton, & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 85–118). New York, NY: Cambridge University Press.
   Google Scholar
• Bradshaw, L., Izsak, A., Templin, J., & Jacobson, E. (2014). Diagnosing teachers’ understandings of rational numbers: Building a multidimensional test within the diagnostic classification framework. Educational Measurement: Issues and Practice, 33, 2–14. doi:10.1111/emip.12020
   Web of Science ®Google Scholar
• Buck, G., & Tatsuoka, K. (1998). Application of the rule space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15, 119–157. doi:10.1177/026553229801500201
   Google Scholar
• Chen, H, & Chen, J. (2016). Exploring reading comprehension skill relationships through the g-dina model. Educational Psychology, 36, 1049–1064.
   Web of Science ®Google Scholar
• Chen, J., & de la Torre, J. (2014). A procedure for diagnostically modeling extant large-scale assessment data: The case of the programme for international student assessment in reading. Psychology, 5, 1967–1978. doi:10.4236/psych.2014.518200
   Google Scholar
• Chen, Y.-H. (2012). Cognitive diagnosis of mathematics performance between rural and urban students in Taiwan. Assessment in Education: Principles, Policy, & Practice, 19, 193–209. doi:10.1080/0969594X.2011.560562
   Google Scholar
• Chen, Y.-H., Ferron, J. M., Thompson, M. S., Gorin, J. S., & Tatsuoka, K. K. (2010). Group comparisons of mathematics performance from a cognitive diagnostic perspective. Educational Research and Evaluation, 16, 325–343. doi:10.1080/13803611.2010.527760
   Google Scholar
• Chiu, C., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225–250. doi:10.1007/s00357-013-9132-9
   Web of Science ®Google Scholar
• Choi, K. M., Lee, Y.-S., & Park, Y. S. (2015). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics. Eurasia Journal of Mathematics, Science, & Technology Education, 11, 1563–1577.
   Web of Science ®Google Scholar
• Cudeck, R., & MacCallum, R. C. (Eds.). (2007). Factor analysis at 100: Historical developments and future directions. Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• de la Torre, J. (2008). An empirically based method of q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362. doi:10.1111/jedm.2008.45.issue-4
   Web of Science ®Google Scholar
• de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130. doi:10.3102/1076998607309474
   Web of Science ®Google Scholar
• de la Torre, J., van der Ark, L. A., & Rossi, G. (in press). Analysis of clinical data from cognitive diagnosis modeling framework. Measurement and Evaluation in Counseling and Development.
   Google Scholar
• DiBello, L. V., Henson, R. A., & Stout, W. F. (2015). A family of generalized diagnostic classification models for multiple choice option-based scoring. Applied Psychological Measurement, 39, 62–79. doi:10.1177/0146621614561315
   Web of Science ®Google Scholar
• Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 272, 299.
   Google Scholar
• Feng, Y., Habing, B. T., & Huebner, A. (2014). Parameter estimation of the reduced RUM using the EM algorithm. Applied Psychological Measurement, 38, 137–150. doi:10.1177/0146621613502704
   Web of Science ®Google Scholar
• Garcia, P. E., Olea, J., & de la Torre, J. (2014). Application of cognitive diagnosis models to competency-based situational judgment tests. Psicothema, 26, 372–377.
   PubMed Web of Science ®Google Scholar
• George, A. C., & Robitzsch, A. (2014). Multiple group cognitive diagnosis models, with an emphasis on differential item functioning. Psychological Test and Assessment Modeling, 56, 405–432.
   Google Scholar
• Gierl, M. J. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44, 325–340. doi:10.1111/jedm.2007.44.issue-4
   Web of Science ®Google Scholar
• Gierl, M. J., Alves, C., & Majeau, R. T. (2010). Using the attribute hierarchy method to make diagnostic inferences about examinees’ knowledge and skills in mathematics: An operational implementation of cognitive diagnostic assessment. International Journal of Testing, 10, 318–341. doi:10.1080/15305058.2010.509554
   Google Scholar
• Gierl, M. J., & Cui, Y. (2008). Defining characteristics of diagnostic classification models and the problem of retrofitting in cognitive diagnostic assessment. Measurement: Interdisciplinary Research and Perspectives, 6, 263–268.
   Google Scholar
• Gierl, M. J., Cui, Y., & Zhou, J. (2009). Reliability and attribute-based scoring in cognitive diagnostic assessment. Journal of Educational Measurement, 46, 293–313. doi:10.1111/jedm.2009.46.issue-3
   Web of Science ®Google Scholar
• Gierl, M. J., Leighton, J. P., & Hunka, S. M. (2007). Using the attribute hierarchy method to make diagnostic inferences about examinees’ cognitive skills. In J. P. Leighton, & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 242–274). New York, NY: Cambridge University Press.
   Google Scholar
• Gierl, M. J., & Leighton, J. P. (2007). Directions for future research in cognitive diagnostic assessment. In J. P. Leighton, & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 341–351). New York, NY: Cambridge University Press.
   Google Scholar
• Gorin, J. S. (2009). Diagnostic classification models: Are they necessary? Measurement: Interdisciplinary Research and Perspectives, 7, 30–33.
   Google Scholar
• Grob, J., Robitzsch, A., & George, A. C. (2016). Cognitive diagnosis model for baseline testing of educational standards in math. Journal of Applied Statistics, 43, 229–243. doi:10.1080/02664763.2014.1000841
   Web of Science ®Google Scholar
• Haberman, S. J., & Von Davier, M. (2007). Some notes on models for cognitively based skills diagnosis. In C. R. Rao, & S. Sinharay (Eds.), Handbook of statistics 26 (pp. 1031–1038). Oxford, UK: Elsevier.
   Google Scholar
• Henson, R. A. (2009). Diagnostic classification models: Thoughts and future directions. Measurement: Interdisciplinary Research and Perspectives, 7, 34–36.
   Google Scholar
• Henson, R. A., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210. doi:10.1007/s11336-008-9089-5
   Web of Science ®Google Scholar
• Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66, 393–416. doi:10.1177/0013164405282485
   Web of Science ®Google Scholar
• Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. doi:10.1080/10705519909540118
   Web of Science ®Google Scholar
• Huff, K., & Goodman, D. P. (2007). The demand for cognitive diagnostic assessment. In J. P. Leighton, & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 19–60). New York, NY: Cambridge University Press.
   Google Scholar
• Im, S., & Park, H. J. (2010). A comparison of US and Korean students’ mathematics skills using a cognitive diagnostic testing method: Linkage to instruction. Educational Research and Evaluation, 16, 287–301. doi:10.1080/13803611.2010.523294
   Google Scholar
• Im, S., & Yin, Y. (2009). Diagnosing skills of statistical hypothesis testing using the rule space method. Studies in Educational Evaluation, 35, 193–209. doi:10.1016/j.stueduc.2009.12.004
   Google Scholar
• Jang, E. E. (2009). Demystifying a q-matrix for making diagnostic inferences about L2 reading skills. Language Assessment Quarterly, 6, 210–238. doi:10.1080/15434300903071817
   Web of Science ®Google Scholar
• Jang, E. E., Dunlop, M., Park, G., & van der Boom, E. H. (2015). How do young students with different profiles of reading skill mastery, perceived ability, and goal orientation respond to holistic diagnostic feedback? Language Testing, 32, 359–383. doi:10.1177/0265532215570924
   Web of Science ®Google Scholar
• Jang, E. E., Dunlop, M., Wagner, M., Kim, Y.-H., & Gu, Z. (2013). Elementary school ELLs’ reading skill profiles using cognitive diagnosis modeling: Roles of length of residence and home language environment. Language Learning, 63, 400–436. doi:10.1111/lang.2013.63.issue-3
   Web of Science ®Google Scholar
• Jurich, D. P., & Bradshaw, L. P. (2014). An illustration of diagnostic classification modeling in student learning outcomes assessment. International Journal of Testing, 14, 49–72. doi:10.1080/15305058.2013.835728
   Google Scholar
• Kim, A.-H. (2015). Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability. Language Testing, 32, 227–258. doi:10.1177/0265532214558457
   Web of Science ®Google Scholar
• Kim, S. H., & Kim, S. (2013). Incorporating diagnostic aspects to mathematical affects inventory development. International Journal of Evaluation and Research in Education, 2, 163–174.
   Google Scholar
• Kunina-Habenicht, O. (2010). Theoretical and practical considerations for implementing diagnostic classification models: Insights from simulation-based and applied research. Unpublished doctoral dissertation, Humboldt-Universitat zu Berlin, Germany.
   Google Scholar
• Kunina-Habenicht, O., Rupp, A. A., & Wilhelm, O. (2009). A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models. Studies in Educational Evaluation, 35, 64–70. doi:10.1016/j.stueduc.2009.10.003
   Google Scholar
• Lai, H., Gierl, M. J., & Babenko, O. (2015). Application of conditional means for diagnostic scoring. International Journal of Learning, Teaching, and Educational Research, 12, 61–79.
   Google Scholar
• Lee, Y.-S., Park, Y. S., & Taylan, D. (2011). A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the U. S. national sample using the TIMSS 2007. International Journal of Testing, 11, 144–177. doi:10.1080/15305058.2010.534571
   Google Scholar
• Lee, Y.-W., & Sawaki, Y. (2009). Application of three cognitive diagnosis models to ESL reading and listening assessments. Language Assessment Quarterly, 6, 239–263. doi:10.1080/15434300903079562
   Web of Science ®Google Scholar
• Li, F., Cohen, A., Bottge, B., & Templin, J. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76, 181–204.
   PubMed Web of Science ®Google Scholar
• Li, H., Hunter, C. V., & Lei, P.-W. (2016). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33, 391–409.
   Web of Science ®Google Scholar
• Li, H., & Suen, H. K. (2013a). Constructing and validating a q-matrix for cognitive diagnostic analyses of reading test. Educational Assessment, 18, 1–25. doi:10.1080/10627197.2013.761522
   Google Scholar
• Li, H., & Suen, H. K. (2013b). Detecting native language group differences at the subskills level of reading: A differential skills functioning approach. Language Testing, 30, 273–298. doi:10.1177/0265532212459031
   Web of Science ®Google Scholar
• Liu, H.-Y., You, X.-F., Wang, W.-Y., Ding, S.-L., & Chang, -H.-H. (2013). The development of computerized adaptive testing with cognitive diagnosis for an English achievement test in China. Journal of Classification, 30, 152–172. doi:10.1007/s00357-013-9128-5
   Web of Science ®Google Scholar
• McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64–82. doi:10.1037/1082-989X.7.1.64
   PubMed Web of Science ®Google Scholar
• Ravand, H. (2016). Application of a cognitive diagnostic model to a high-stakes reading comprehension test. Journal of Psychoeducational Assessment, 34, 782–799.
   Web of Science ®Google Scholar
• Ravand, H., Barati, H., & Widhiarso, W. (2013). Exploring diagnostic capacity of a high stakes reading comprehension test: A pedagogical demonstration. Iranian Journal of Language Testing, 3, 11–37.
   Google Scholar
• Roberts, M. R., & Gierl, M. J. (2010). Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29(3), 25–38. doi:10.1111/emip.2010.29.issue-3
   Google Scholar
• Romero, S. J., & Ordonez, X. G. (2014). Validation of the cognitive structure of an arithmetic test with the least squares distance model (LSDM). Universitas Psychologica, 13, 333–344.
   Web of Science ®Google Scholar
• Roussos, L. A., DiBello, L. V., Stout, W., Hartz, S. M., Henson, R. A., & Templin, J. L. (2007). The fusion models skills diagnosis system. In J. P. Leighton, & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 276–318). New York, NY: Cambridge University Press.
   Google Scholar
• Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research and Perspectives, 6, 219–262.
   Google Scholar
• Rupp, A. A., & Templin, J. L. (2009). The (un)usual suspects? A measurement community in search of its identity. Measurement: Interdisciplinary Research and Perspectives, 7, 115–121.
   Google Scholar
• Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guildford Press.
   Google Scholar
• Sen, S., & Arican, M. (2015). A diagnostic comparison of Turkish and Korean students’ mathematics performances on the TIMSS 2011 assessment. Journal of Measurement and Evaluation in Education and Psychology, 6, 238–253.
   Google Scholar
• Sinharay, S., & Haberman, S. J. (2009). How much can we reliably know about what examinees know? Measurement: Interdisciplinary Research and Perspectives, 7, 46–49.
   Google Scholar
• Sun, Y., & Suzuki, M. (2013). Diagnostic assessment for improving teacher practice. International Journal of Information and Education Technology, 3, 607–610. doi:10.7763/IJIET.2013.V3.345
   Google Scholar
• Svetina, D., Gorin, J. S., & Tatsuoka, K. K. (2011). Defining and comparing the reading comprehension construct: A cognitive psychometric modeling approach. International Journal of Testing, 11, 1–23. doi:10.1080/15305058.2010.518261
   Google Scholar
• Tatsuoka, K. K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 453–488). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251–275. doi:10.1007/s00357-013-9129-4
   Web of Science ®Google Scholar
• Templin, J., & Hoffman, L. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37–50. doi:10.1111/emip.2013.32.issue-2
   Web of Science ®Google Scholar
• Templin, J. L, & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287-305.
   PubMed Web of Science ®Google Scholar
• Wang, C., & Gierl, M. J. (2011). Using the attribute hierarchy method to make diagnostic inferences about examinees’ cognitive skills in critical reading. Journal of Educational Measurement, 48, 165–187. doi:10.1111/jedm.2011.48.issue-2
   Web of Science ®Google Scholar
• Yang, X., & Embretson, S. E. (2007). Construct validity and cognitive diagnostic assessment. In J. P. Leighton, & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 119–145). New York, NY: Cambridge University Press.
   Google Scholar
• Zhang, T., Torney-Purta, J., & Barber, C. (2012). Students’ conceptual knowledge and process skills in civic education: Identifying cognitive profiles and classroom correlates. Theory & Research in Social Education, 40, 1–34.
   Google Scholar