Cognitive diagnosis models and automated test assembly: an approach incorporating response times

References

• Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229. https://doi.org/10.1177/014662169602000303
   Web of Science ®Google Scholar
• Chen, H., & Chen, J. (2016). Exploring reading comprehension skill relationships through the G-DINA model. Educational Psychology, 36, 1049–1064. https://doi.org/10.1080/01443410.2015.1076764
   Web of Science ®Google Scholar
• Chen, J., & de la Torre, J. (2014). A procedure for diagnostically modeling extant large-scale assessment data: The case of the Programme for International Student Assessment in Reading. Psychology, 5, 1967–1978. https://doi.org/10.4236/psych.2014.518200
   Google Scholar
• Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619–632. https://doi.org/10.1007/s11336-009-9123-2
   Web of Science ®Google Scholar
• Chiu, C.-Y., & Köhn, H.-F. (2016). The reduced RUM as a logit model: Parameterization and constraints. Psychometrika, 81, 350–370. https://doi.org/10.1007/s11336-015-9460-2
   PubMed Web of Science ®Google Scholar
• Choi, K. M., Lee, Y.-S., & Park, Y. S. (2015). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics. Eurasia Journal of Mathematics, Science & Technology Education, 11, 1563–1577. https://doi.org/10.12973/eurasia.2015.1421a
   Web of Science ®Google Scholar
• Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. John Wiley.
   Google Scholar
• de la Torre, J. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362. https://doi.org/10.1111/j.1745-3984.2008.00069.x
   Web of Science ®Google Scholar
• de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199. https://doi.org/10.1007/s11336-011-9207-7
   Web of Science ®Google Scholar
• de la Torre, J., & Douglas, J. A. (2004). Higher order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353. https://doi.org/10.1007/BF02295640
   Web of Science ®Google Scholar
• de la Torre, J., & Sun, Y. (2018, October). The accordion procedure: A method for accommodating a large number of attributes in cognitive diagnosis modeling. Poster presentation at the annual Educational Technology and Computational Psychometrics Symposium, Iowa City, IA.
   Google Scholar
• de la Torre, J., van der Ark, L. A., & Rossi, G. (2018). Analysis of clinical data from a cognitive diagnosis modeling framework. Measurement and Evaluation in Counseling and Development, 51, 281–296. https://doi.org/10.1080/07481756.2017.1327286
   Web of Science ®Google Scholar
• Fan, Z., Wang, C., Chang, H.-H., & Douglas, J. (2012). Utilizing response time distributions for item selection in CAT. Journal of Educational and Behavioral Statistics, 37, 655–670. https://doi.org/10.3102/1076998611422912
   Web of Science ®Google Scholar
• Fenton, L. (1960). The sum of log-normal probability distributions in scatter transmission systems. IRE Transactions on Communication Systems, 8, 57–67. https://doi.org/10.1109/TCOM.1960.1097606
   Google Scholar
• Finkelman, M., Kim, W., & Roussos, L. A. (2009). Automated test assembly for cognitive diagnosis models using a genetic algorithm. Journal of Educational Measurement, 46, 273–292. https://doi.org/10.1111/j.1745-3984.2009.00081.x
   Web of Science ®Google Scholar
• Finkelman, M. D., Kim, W., Roussos, L. A., & Verschoor, A. (2010). A binary programming approach to automated test assembly for cognitive diagnosis models. Applied Psychological Measurement, 34, 310–326. https://doi.org/10.1177/0146621609344846
   Web of Science ®Google Scholar
• Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29, 262–277. https://doi.org/10.1177/0146621604272623
   Web of Science ®Google Scholar
• Henson, R., Roussos, L., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32, 275–288. https://doi.org/10.1177/0146621607302478
   Web of Science ®Google Scholar
• Henson, R., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210. https://doi.org/10.1007/s11336-008-9089-5
   Web of Science ®Google Scholar
• Hu, B., Qin, L., Sullivan, M., & Templin, J. (2017). Contemporary approaches to psychometrics: Item response theory and diagnostic classification models. Cultura y Educación, 29, 461–491. https://doi.org/10.1080/11356405.2017.1367171
   Web of Science ®Google Scholar
• IBM (2013). 2013 IBM ILOG CPLEX 12.6 User Manual. IBM Corp.
   Google Scholar
• Jang, E. E. (2005). A validity narrative: Effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL. Unpublished doctoral dissertation. University of Illinois, Champaign, IL.
   Google Scholar
• Jang, E. E. (2006, April). Pedagogical implications of cognitive skills diagnostic assessment for teaching and learning. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
   Google Scholar
• Jang, E. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing, 26, 31–73. https://doi.org/10.1177/0265532208097336
   Web of Science ®Google Scholar
• Jang, E. E., Dunlop, M., Park, G., & van der Boom, E. H. (2015). How do young students with different profiles of reading skill mastery, perceived ability, and goal orientation respond to holistic diagnostic feedback? Language Testing, 32, 359–383. https://doi.org/10.1177/0265532215570924
   Web of Science ®Google Scholar
• Jang, E. E., Dunlop, M., Wagner, M., Kim, Y.-H., & Gu, Z. (2013). Elementary school ELLs’ reading skill profiles using cognitive diagnosis modeling: roles of length of residence and home language environment. Language Learning, 63, 400–436. https://doi.org/10.1111/lang.12016
   Web of Science ®Google Scholar
• Kuo, B.-C., Pai, H.-S., & de la Torre, J. (2016). Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction. Applied Psychological Measurement, 40, 315–330. https://doi.org/10.1177/0146621616638643
   PubMed Web of Science ®Google Scholar
• Lin, Y., Gong, Y.-J., & Zhang, J. (2017). An adaptive ant colony optimization algorithm for constructing cognitive diagnosis tests. Applied Soft Computing Journal, 52, 1–13. https://doi.org/10.1016/j.asoc.2016.11.042
   Web of Science ®Google Scholar
• Liu, H.-Y., You, X.-F., Wang, W.-Y., Ding, S.-L., & Chang, H.-H. (2013). The development of computerized adaptive testing with cognitive diagnosis for an English achievement test in China. Journal of Classification, 30, 152–172. https://doi.org/10.1007/s00357-013-9128-5
   Web of Science ®Google Scholar
• Ma, W., Iaconangelo, C., & de la Torre, J. (2016). Model similarity, model selection, and attribute classification. Applied Psychological Measurement, 40, 200–217. https://doi.org/10.1177/0146621615621717
   PubMed Web of Science ®Google Scholar
• R Core Team (2013). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. http://www.R-project.org/. Accessed May 21, 2018.
   Google Scholar
• Ravand, H. (2016). Application of a cognitive diagnostic model to a high-stakes reading comprehension test. Journal of Psychoeducational Assessment, 34, 782–799. https://doi.org/10.1177/0734282915623053
   Web of Science ®Google Scholar
• Roussos, L. A., DiBello, L. V., Stout, W., Hartz, S. M., Henson, R. A., & Templin, J. L. (2007). The fusion model skills diagnosis system. In J. P. Leighton & M. J. Gierl (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 275–318). Cambridge University Press.
   Google Scholar
• Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research and Perspectives, 6, 219–262. https://doi.org/10.1080/15366360802490866
   Google Scholar
• Schnipke, D. L., & Scrams, D. J. (1997). Representing response time information in item banks (LSAC Computerized Testing Report No. 97-09). Law School Admission Council.
   Google Scholar
• Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354. https://doi.org/10.1111/j.1745-3984.1983.tb00212.x
   Web of Science ®Google Scholar
• Tatsuoka, K. K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 453–488). Erlbaum.
   Google Scholar
• Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305. https://doi.org/10.1037/1082-989X.11.3.287
   PubMed Web of Science ®Google Scholar
• Theunissen, T. J. J. M. (1985). Binary programming and test design. Psychometrika, 50, 411–420. https://doi.org/10.1007/BF02296260
   Web of Science ®Google Scholar
• Thissen, D. (1983). Timed testing: An approach using item response theory. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179–203). Academic Press.
   Google Scholar
• Tu, D., Zheng, C., Cai, Y., Gao, X., & Wang, D. (2018). A polytomous model of cognitive diagnostic assessment for graded data. International Journal of Testing, 18, 231–252. https://doi.org/10.1080/15305058.2017.1396465
   Web of Science ®Google Scholar
• van der Linden, W. J. (2005). Linear models for optimal test design. Springer.
   Google Scholar
• van der Linden, W. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204. https://doi.org/10.3102/10769986031002181
   Web of Science ®Google Scholar
• van der Linden, W. J. (2009). Predictive control of speededness in adaptive testing. Applied Psychological Measurement, 33, 25–41. https://doi.org/10.1177/0146621607314042
   Web of Science ®Google Scholar
• van der Linden, W. J. (2011a). Test design and speededness. Journal of Educational Measurement, 48, 44–60. https://doi.org/10.1111/j.1745-3984.2010.00130.x
   Web of Science ®Google Scholar
• van der Linden, W. J. (2011b). Setting time limits on tests. Applied Psychological Measurement, 35, 183–199. https://doi.org/10.1177/0146621610391648
   Web of Science ®Google Scholar
• van der Linden, W. J., Ariel, A., & Veldkamp, B. P. (2006). Assembling a computerized adaptive testing item pool as a set of linear tests. Journal of Educational and Behavioral Statistics, 31, 81–99. https://doi.org/10.3102/10769986031001081
   Web of Science ®Google Scholar
• van der Linden, W. J., & Boekkooi-Timminga, E. (1989). A maximin model for IRT-based test design with practical constraints. Psychometrika, 54, 237–247. https://doi.org/10.1007/BF02294518
   Web of Science ®Google Scholar
• van der Linden, W. J., Scrams, D. J., & Schnipke, D. L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195–210. https://doi.org/10.1177/01466219922031329
   Web of Science ®Google Scholar
• van der Linden, W. J., & Xiong, X. (2013). Speededness and adaptive testing. Journal of Educational and Behavioral Statistics, 38, 418–438. https://doi.org/10.3102/1076998612466143
   Web of Science ®Google Scholar
• Veldkamp, B. P., Matteucci, M., & de Jong, M. G. (2013). Uncertainties in the item parameter estimates and robust automated test assembly. Applied Psychological Measurement, 37, 123–139. https://doi.org/10.1177/0146621612469825
   Web of Science ®Google Scholar
• Wang, S., Zhang, S., Douglas, J., & Culpepper, S. (2018). Using response times to assess learning progress: A joint model for responses and response times. Measurement: Interdisciplinary Research and Perspectives, 16, 45–58. https://doi.org/10.1080/15366367.2018.1435105
   Web of Science ®Google Scholar
• Yu, X., & Cheng, Y. (2019). Data-driven Q-matrix validation using a residual-based statistic in cognitive diagnostic assessment. British Journal of Mathematical and Statistical Psychology. Advance online publication. https://doi.org/10.1111/bmsp.12191
   Google Scholar
• Yu, X., Cheng, Y., & Chang, H.-H. (2019). Recent developments in cognitive diagnostic computerized adaptive testing (CD-CAT): A comprehensive review. In M. von Davier & Y.-S. Lee (Eds.), Handbook of diagnostic classification models: Models and model extensions, applications, software packages (pp. 307–331). Springer.
   Google Scholar
• Zhan, P., Jiao, H., & Liao, D. (2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71, 262–286. https://doi.org/10.1111/bmsp.12114
   PubMed Web of Science ®Google Scholar
• Zhan, P., Liao, M., & Bian, Y. (2018). Joint testlet cognitive diagnosis modeling for paired local item dependence in response times and response accuracy. Frontiers in Psychology, 9, 607. https://doi.org/10.3389/fpsyg.2018.00607
   PubMed Web of Science ®Google Scholar