Dimensionality in Compensatory MIRT When Complex Structure Exists: Evaluation of DETECT and NOHARM

REFERENCES

• Ackerman, T.A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255–278.
   Google Scholar
• Adams, R.J., Wilson, M., & Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23.
   Web of Science ®Google Scholar
• Allen, M., & Yen, W. (2002). Introduction to measurement theory. Long Grove, : Waveland Press. (Original work published 1979)
   Google Scholar
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
   Google Scholar
• Cai, L., Yang, J.S., & Hansen, M. (2011). Generalized full-information item bifactor analysis. Psychological Methods, 16(3), 221–248. doi: doi:10.1037/a0023350
   PubMed Web of Science ®Google Scholar
• Cattell, R.B. (1978). The scientific use of factor analysis in behavioral and life sciences. New York, NY: Platinum Press.
   Google Scholar
• Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.
   Web of Science ®Google Scholar
• Childs, R.A. & Oppler, S.H. (2000). Implications of test dimensionality for unidimensional IRT scoring: An investigation of a high-stakes testing program. Educational and Psychological Measurement, 60(6), 939–955.
   Web of Science ®Google Scholar
• De Champlain, A., & Gessaroli, M.E. (1998). Assessing the dimensionality of item response matrices with small sample sizes and short test lengths. Applied Measurement in Education, 11(3), 231–253.
   Web of Science ®Google Scholar
• Donoghue, J.R., & Allen, N.L. (1993). Thin versus thick matching in the Mantel–Haenszel procedure for detecting DIF. Journal of Educational Statistics, 18, 131–154.
   Google Scholar
• Finch, H., & Habing, B. (2003, April). Performance of DIMTEST and NOHARM based statistics for testing unidimensionality. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.
   Google Scholar
• Finch, H., & Habing, B. (2005). Comparison of NOHARM and DETECT in item cluster recovery: Counting dimensions and allocating items. Journal of Educational Measurement, 42, 149–169.
   Web of Science ®Google Scholar
• Finch, H., & Habing, B. (2007). Performance of DIMTEST- and NOHARM-based statistics for testing unidimensionality. Applied Psychological Measurement, 31, 292–307.
   Web of Science ®Google Scholar
• Fraser, C., & McDonald, R.P. (1988). NOHARM: Least squares item factor analysis. Multivariate Behavioral Research, 23, 267–269.
   PubMed Web of Science ®Google Scholar
• Froelich, A.G., & Habing, B. (2008). Conditional covariance-based subtest selection for DIMTEST. Applied Psychological Measurement, 32, 138–155.
   Web of Science ®Google Scholar
• Gierl, M.J., Leighton, J.P., & Tan, X. (2006). Evaluating DETECT classification accuracy and consistency when data display complex structure. Journal of Educational Measurement, 43, 265–289.
   Web of Science ®Google Scholar
• Gessaroli, M.E., & De Champlain, A.F. (1996). Using an approximate chi-square statistic to test the number of dimensions underlying the responses to a set of items. Journal of Educational Measurement, 33, 157–192.
   Web of Science ®Google Scholar
• Gorsuch, R.L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Hattie, J., Krakowski, K., Rogers, H.J., & Swaminathan, H. (1996). An assessment of Stout's index of essential unidimensionality. Applied Psychological Measurement, 20, 1–14.
   Web of Science ®Google Scholar
• Jang, E., & Roussos, L. (2007). An investigation into the dimensionality of TOEFL using conditional covariance-based nonparametric approach. Journal of Educational Measurement, 44, 1–22.
   Web of Science ®Google Scholar
• Kim, H.R. (1994). New techniques for the dimensionality assessment of standardized test data (Doctoral dissertation, University of Illinois at Urbana–Champaign). Dissertation Abstracts International, 55-12B, 5598.
   Google Scholar
• Leighton, J.P., Gokiert, R.J., & Cui, Y. (2007). Using exploratory and confirmatory methods to identify the cognitive dimension in a large-scale science assessment. International Journal of Testing, 7, 141–189.
   Google Scholar
• Levy, R., Mislevy, R.J., & Sinharay, S. (2009). Posterior predictive model checking for multidimensionality in item response theory. Applied Psychological Measurement, 33, 519–537.
   Web of Science ®Google Scholar
• Lord, F.M. (1968). An Analysis of the verbal scholastic aptitude test using Birnbaum's three-parameter logistic model. Educational and Psychological Measurement, 28, 989–1020.
   Web of Science ®Google Scholar
• Maydeu-Olivares, A. (2001). Multidimensional item response theory modeling of binary data: Large sample properties of NOHARM estimates. Journal of Educational and Behavioral Statistics, 26, 51–71.
   Google Scholar
• McDonald, R.P. (1997). Normal-ogive multidimensional model. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory (pp. 257–269). New York, NY: Springer-Verlag.
   Google Scholar
• McDonald, R.P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Mislevy, R.J., Almond, R.G., & Lukas, J.F. (2003). A brief introduction to evidence centered design. (Technical report RR-03-16). Princeton, NJ: Educational Testing Service.
   Google Scholar
• Mroch, A.A., & Bolt, D.M. (2006). A Simulation comparison of parametric and nonparametric dimensionality detection procedures. Applied Measurement in Education, 19(1), 67–91. doi: doi:10.1207/s15324818ame1901_4
   Web of Science ®Google Scholar
• Mokken, R.J. (1971). A theory and procedure of scale analysis. Berlin, Germany: De Gruyter.
   Google Scholar
• Muthén, L.K., & Muthén, B.O. (1998–2010). Mplus user's guide (6th ed.). Los Angeles, CA: Muthén & Muthén.
   Google Scholar
• Nandakumar, R. (1991). Traditional dimensionality versus essential dimensionality. Journal of Educational Measurement, 28, 99–117.
   Web of Science ®Google Scholar
• Nandakumar, R., & Stout, W. (1993). Refinement of Stout's procedure for assessing latent trait unidimensionality. Journal of Educational Statistics, 18(1), 41–68.
   Google Scholar
• Paek, I. & Han, K.T. (2013). IRTPRO 2.1 for Windows (Item Response Theory for Patient-Reported Outcomes). Applied Psychological Measurement, 37, 242–252.
   Web of Science ®Google Scholar
• R Development Core Team. (2010). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org
   Google Scholar
• Reckase, M.D. (1985). Models for multidimensional tests and hierarchically structured training materials (Research report no. ONR85-1). Retrieved from http://files.eric.ed.gov/fulltext/ED262103.pdf
   Google Scholar
• Reckase, M.D. (1997). A linear logistic multidimensional model for dichotomous item response data. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271–286). New York, NY: Springer.
   Google Scholar
• Reckase, M.D. (2009). Multidimensional item response theory. New York, NY: Springer.
   Google Scholar
• Reckase, M.D., Thompson, T., & Nering, M. (1997, June). Identifying similar item content clusters on multiple test forms. In T. Miller (Chair), High-dimensional simulation of item response data for CAT research. Symposium conducted at the annual meeting of the Psychometric Society, Gatlingburg, TN.
   Google Scholar
• Roussos, L., Stout, W., & Marden, J. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 1–30.
   Web of Science ®Google Scholar
• Seraphine, A.E. (2000). The performance of DIMTEST when latent trait and item difficulty distributions differ. Applied Psychological Measurement, 24, 82–94.
   Web of Science ®Google Scholar
• Spray, J.A., Davey, T.C., Reckase, M.D., Ackerman, T.A., & Carlson, J.E. (1990). Comparison of two logistic multidimensional item response theory models (Research report series no. ONR90-8). Retrieved from http://act.org/research/researchers/reports/pdf/ACT_RR90-08.pdf
   Google Scholar
• Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589–617.
   Web of Science ®Google Scholar
• Stout, W., Habing, B., Douglas, J., Kim, H.R., Roussos, L., & Zhang, J. (1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331–354.
   Web of Science ®Google Scholar
• Svetina, D. (2013). Assessing dimensionality in noncompensatory MIRT with complex structure. Educational and Psychological Measurement, 73, 312–338.
   Web of Science ®Google Scholar
• Svetina, D., & Levy, R. (2014). A framework for dimensionality assessment for multidimensional item response models: A methodological review. Educational Assessment, 19, 35–57. doi: doi:10.1080/10627197.2014.869450
   Google Scholar
• Sympson, J.B. (1978). A model for testing with multidimensional items. In D.J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82–89). Minneapolis, MN: University of Minneapolis, Department of Psychology, Psychometric Methods Program.
   Google Scholar
• Tate, R. (2002). Test dimensionality. In J. Tindal & T.M. Haladyna (Eds.), Large scale assessment programs for all students: Development, implementation, and analysis (pp. 181–211). Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Tate, R. (2003). A comparison of selected empirical methods for assessing the structure of responses to test items. Applied Psychological Measurement, 27, 159–203.
   Web of Science ®Google Scholar
• Thurstone, L.L. (1947). Multiple factor analysis. Chicago, IL: University of Chicago Press.
   Google Scholar
• van Abswoude, A.A., van der Ark, L., & Sijtsma, K. (2004). A comparative study of test data dimensionality assessment procedures under nonparametric IRT models. Applied Psychological Measurement, 28, 3–24.
   Web of Science ®Google Scholar
• Walker, C.M., & Beretvas, S.N. (2003). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement, 40, 255–275.
   Web of Science ®Google Scholar
• Way, W.D., Ansley, T.N., & Forsyth, R.A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimates. Applied Psychological Measurement, 12, 239–252.
   Web of Science ®Google Scholar
• Whitely, S.E. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45, 479–494.
   Web of Science ®Google Scholar
• Yen, W.M. (1985). Increasing item complexity: A possible cause of scale shrinkage for unidimensional item response theory. Psychometrika, 50, 399–410.
   Web of Science ®Google Scholar
• Zhang, J. (2007). Conditional covariance theory and detect for polytomous items. Psychometrika, 72, 69–91.
   Web of Science ®Google Scholar
• Zhang, J., & Stout, W. (1999a). Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika, 64, 129–152.
   Web of Science ®Google Scholar
• Zhang, J., & Stout, W. (1999b). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213–249.
   Web of Science ®Google Scholar