When Are Multidimensional Data Unidimensional Enough for Structural Equation Modeling? An Evaluation of the DETECT Multidimensionality Index

Abstract

In structural equation modeling (SEM), researchers need to evaluate whether item response data, which are often multidimensional, can be modeled with a unidimensional measurement model without seriously biasing the parameter estimates. This issue is commonly addressed through testing the fit of a unidimensional model specification, a strategy previously determined to be problematic. As an alternative to the use of fit indexes, we considered the utility of a statistical tool that was expressly designed to assess the degree of departure from unidimensionality in a data set. Specifically, we evaluated the ability of the DETECT “essential unidimensionality” index to predict the bias in parameter estimates that results from misspecifying a unidimensional model when the data are multidimensional. We generated multidimensional data from bifactor structures that varied in general factor strength, number of group factors, and items per group factor; a unidimensional measurement model was then fit and parameter bias recorded. Although DETECT index values were generally predictive of parameter bias, in many cases, the degree of bias was small even though DETECT indicated significant multidimensionality. Thus we do not recommend the stand-alone use of DETECT benchmark values to either accept or reject a unidimensional measurement model. However, when DETECT was used in combination with additional indexes of general factor strength and group factor structure, parameter bias was highly predictable. Recommendations for judging the severity of potential model misspecifications in practice are provided.

Keywords:

• dimensionality assessment
• structural equation modeling
• bifactor model

Notes

¹ Moreover, the common practice of “hiding” multidimensionality through the formation of parcels has been subject to much criticism of late (Bandalos, 2002; Meade & Kroustalis, 2006; Sterba & MacCallum, 2010).

² Note that this structure is completely consistent with DETECT’s assumption of approximate simple structure and previous research because any bifactor structure with one general and 1 – P group factors can be transformed into a simple structure correlated factors model with 1 – P factors.

³ Because of the balanced design, when multidimensional data are forced into a unidimensional model, estimated loadings in the unidimensional model will always be higher than the true loadings on the general factor in the bifactor model. The degree of bias is influenced by other factors, as described later.

⁴ Note that we did not vary the sample size because this is not a study of estimation error, but rather a study of DETECT’s ability to predict bias in parameter estimates, and the factors that influence its predictive power.

⁵ Again, if a bifactor structure has one general and three group factors, DETECT should recognize this as three groups of homogeneous items, and base the computation of the maximum DETECT value on this correct partitioning.

⁶ One problem occurred in data structure 9 (24 items, 8 group factors of 3 items) in the .6 general and .3 group loading condition where DETECT identified six dimensions rather than eight. The remaining eight inaccuracies were from data structure 15 (36 items, 12 group factors of 3 items) in the conditions where the group factor loadings were small (.3 and .4). In those conditions, DETECT slightly underestimated the number of latent dimensions as being between 8 and 11, rather than the correct 12. We comment further on these findings in the discussion.