Using Multidimensional Scaling to Assess the Dimensionality of Dichotomous Item Data

Abstract

In this study, we investigated the utility of multidimensional scaling (MDS) for assessing the dimensionality of dichotomous test data. Two MDS proximity measures were studied: one based on the PC statistic proposed by Chen and Davison (1996), the other based on inter-item Euclidean distances. Stout's (1987) test of essential unidimensionality (DIMTEST) was also used as a standard for comparison. Twenty different conditions of unidimensional and multidimensional data were simulated, varying the number of test items, correlations among dimensions, and type of data generation model (Rasch or two-parameter IRT model). DIMTEST performed best overall, but had some trouble detecting multidimensionality when the number of test items was small. The PC statistic correctly identified the dimensionality of the unidimensional data, whereas the use of Euclidean distances suggested the two-parameter unidimensional data were multidimensional. Both MDS procedures correctly identified multidimensionality under the low correlation conditions, but were generally unable to detect multidimensionality when the dimensions were highly correlated. Analysis of Euclidean distances were best for determining the precise dimensionality of the multidimensional data under the low correlation condition. Implications of the findings are discussed, and suggestions for future research are provided.