The reliability of multidimensional neuropsychological measures: from alpha to omega

Abstract

Objective: To demonstrate that Coefficient omega, a model-based estimate, is more a more appropriate index of reliability than coefficient alpha for the multidimensional scales that are commonly employed by neuropsychologists. Method: As an illustration, a structural model of an overarching general factor and four first-order factors for the WAIS-IV based on the standardization sample of 2200 participants was identified and omega coefficients were subsequently computed for WAIS-IV composite scores. Results: Alpha coefficients were ≥ .90 and omega coefficients ranged from .75 to .88 for WAIS-IV factor index scores, indicating that the blend of general and group factor variance in each index score created a reliable multidimensional composite. However, the amalgam of variance from general and group factors did not allow the precision of Full Scale IQ (FSIQ) and factor index scores to be disentangled. In contrast, omega hierarchical coefficients were low for all four factor index scores (.10–.41), indicating that most of the reliable variance of each factor index score was due to the general intelligence factor. In contrast, the omega hierarchical coefficient for the FSIQ score was .84. Conclusions: Meaningful interpretation of WAIS-IV factor index scores as unambiguous indicators of group factors is imprecise, thereby fostering unreliable identification of neurocognitive strengths and weaknesses, whereas the WAIS-IV FSIQ score can be interpreted as a reliable measure of general intelligence. It was concluded that neuropsychologists should base their clinical decisions on reliable scores as indexed by coefficient omega.

Keywords:

• Intelligence
• reliability
• omega
• alpha
• WAIS-IV

Notes

1. Formulas for omega have been presented by, among others, Brunner, Nagy, and Wilhelm (2012), Gignac (2014), McDonald (1999), Reise (2012), and Rodriguez, Reise, and Haviland (2016a, 2016b). See those publications for technical details.

2. Wechsler (2008b, p. 42) reported that ‘reliability coefficients were obtained utilizing the split-half and the Cronbach’s coefficient alpha methods … calculated with the formula recommended by Guilford (1954) and Nunnally and Bernstein (1994).’ Gignac (2014) has suggested that inter-subtest standard alpha might be more appropriate given the multidimensional nature of the WAIS-IV. Standard alpha coefficients are more consistent with ω and ω_s coefficients in this case, but they remain dependent upon statistical assumptions, including essential tau-equivalence, whereas coefficient omega does not.

3. Omega may also be computed via bifactor confirmatory analysis and exploratory factor analysis models with orthogonalization or target bifactor rotation (Brunner et al., 2012; Reise et al., 2013; Zinbarg et al., 2005). The bifactor confirmatory method is preferred by many measurement specialists (Chen et al., 2012; Green & Yang, 2009; Reise et al., 2013; Rodriguez et al., 2016a, 2016b) but exploratory models might be useful in the absence of clear theoretical or empirical support (Zinbarg et al., 2006). In the current case, results from bifactor confirmatory analysis and exploratory factor analysis models with orthogonalization were almost identical (±.02) to those reported in Table 1. Proportionality constraints might cause some variation in results from exploratory and confirmatory models with other data (Brunner et al., 2012; Reise, 2012).