Implications of Parcel-Allocation Variability for Comparing Fit of Item-Solutions and Parcel-Solutions

Abstract

This article relates a still-popular motivation for using parceling to an unrecognized cost. The still-popular motivation is improvement in fit with respect to the item-solution. The cost is uncertainty in fit due to the selection of one out of many possible item-to-parcel allocations. A theoretical framework establishes the reason for this relationship: The same mechanisms that cause larger item- versus parcel-solution differences in the minimized discrepancy function also cause larger allocation to allocation variability in the parcel-solution's minimized discrepancy function. Study 1 illustrates that these shared causal mechanisms lead to a strong positive association between average item–parcel differences in minimized discrepancy function values and parcel-allocation variability in those values. Study 2 extends these results from discrepancy function values to fit indexes, showing that the association remains positive, but varies in magnitude depending on what quantities other than the discrepancy function are involved in computing the fit index. The important implication for practice is that when item–parcel fit differences are large enough to alter conclusions about model adequacy, parcel-allocation variability tends to be large enough for parcel-solution model adequacy to depend on the particular allocation chosen.

Keywords:

• parceling
• parcel-allocation variability
• model fit
• confirmatory factor analysis

Notes

¹ Sterba and MacCallum's (2010) paper was solely focused on introducing the concept of parcel-allocation variability and documenting its occurrence in the context of alternative parcel-solutions. Their simulation did not consider item-solutions, nor did they relate the concept of parcel-allocation variability to differences between item- and parcel-solutions, as done here.

²The points raised in this article should apply to situations where observed and model-implied means are also included in the discrepancy function, but this situation is not specifically considered here.

³One popular discrepancy function is the maximum likelihood discrepancy function. I use it in the simulation described later, but this framework does not apply to just one particular discrepancy function.

⁴These authors used terms that were proxies or related to discrepancy function values in their articles, but it is necessary for the purposes of this presentation to rephrase their comments in terms of discrepancy function values.

⁵This framework is presented in the context of confirmatory factor analysis (CFA), but concepts are generalizable to structural equation models more generally.

⁶ Bandalos (2002) and Sass and Smith (2006) also allowed that there would be a different ( _i – _p ) for different allocations, but did not state the implications of this.

⁷Others have considered the generating model to be at the parcel-level, and conceive of it as being mis- or properly specified depending on the sample allocation chosen (e.g., Kim & Hagtvet, 2003), but I believe most researchers conceptualize their generating model at the item-level.

⁸Additional higher order interaction and power terms involving the predictors were tried in the Table 1 regression model as well as Study 2 regression models, but because they did not explain sizable amounts of variance they were not included for parsimony.

⁹Cutoff values are often used to distinguish good fit from poor fit in SEM, but the particular choice of cutoff value for a given index is ultimately arbitrary. Other choices for cutoff values could instead be used to make the same point (e.g., RMSEA < .05 and SRMR < .06 as designating good fit).

^aFor RMSEA, this is an absolute difference for reasons discussed subsequently in the article.

¹⁰Mplus software employed in this simulation uses N rather than N – 1.

¹¹For comparison purposes, a subset of cells (24) were rerun with a purposive parceling algorithm—the correlational algorithm from Rogers and Schmitt (2004). This correlational algorithm was designed to outperform random allocations in terms of parcel-solution fit due to its allocation of items that highly correlate into the same parcel (thus reducing the chance of high unaccounted for correlated uniquenesses). Using this algorithm, 23% of item-solutions had better RMSEA fit than parcel-solutions, but 10% of these instances were inevitable due to the item solution having an RMSEA of 0.

¹²Mplus software employed in this simulation uses N rather than N – 1.