Factorial Invariance and Orthogonal Rotation

Abstract

A large number of methods of factor rotation are available in the literature, but the development of formal criteria by which to compare them is an understudied field of research. One possible criterion is the Thurstonian concept of “factorial invariance”, which was applied by Kaiser to the varimax rotation method in 1958 and has been subsequently neglected. In the present study, we propose two conditions for establishing whether a method satisfies factorial invariance, and we apply them to 11 orthogonal rotation methods. The results show that 3 methods do not exhibit factorial invariance under either condition, 3 are invariant under one but not the other, and 5 are invariant under both. Varimax rotation is one of the 5 methods that satisfy factorial invariance under both conditions and is the only method that satisfies the invariance condition originally advocated by Kaiser in 1958. From this perspective, it appears that varimax rotation is the method that best ensures factorial invariance.

Keywords:

• Factorial invariance
• orthogonal rotation
• varimax
• Crawford–Ferguson family
• entropy methods

Notes

1 The most notable example of a parametric objective function is the Crawford-Ferguson function, which unifies various classical objective functions such as the quartimax, varimax, and equamax functions (Crawford & Ferguson, 1970). In some sense, geomin rotation (Yates, 1987; Browne, 2001) is also a family of methods that depend on a single parameter (ε). There are, however, other examples with more parameters (Katz & Rohlf, 1974; Rozeboom, 1991). For a review of parametric objective functions, see Jennrich (2004).

2 Hereafter, we shall refer to “tests” rather than “items” or “responses” when discussing the manifest variables of factor analysis, following Thurstone.

3 The "domain", or the "universe" of variables, refers to the entire ensemble of tests that is logically conceivable in a given “behavior domain” or psychological field (personality, capacity, or attitude). According to Mulaik, the first formulation of the problem of psychometric inference, namely, sampling from a test universe, dates back to Hotelling (1933). See Mulaik (2010, p. 266).

4 Thurstone also considered situations with identical populations in which the two batteries are perfectly disconnected (with a complete lack of common tests). In this case, factorial invariance loses every syntactic connotation and becomes purely semantic. Hence, the analyst must limit himself to checking whether the meanings of the factors that emerge are the same because he is unable to control the stability of the loadings (since there are no common tests). See Thurstone (1947, p. 366).

5 It cannot be dismissed that such an approach, which seeks to decompose every object into its elementary constituents (e.g., genes of chromosomes, phonemes of a language, elements of chemistry, subatomic particles of atoms), was influenced by the zeitgeist of the 1930s and 1940s, a time strongly dominated by neopositivism and structuralism. On this point, see Mulaik’s interesting considerations about factorial analysis as one of the many examples of “structural theories” (Mulaik, 2010, pp. 1-2).

6 When the number of factors is 2, the varimax method is indistinguishable from two other rotation methods, namely, equamax rotation (Saunders, 1962) and parsimax rotation (Crawford & Ferguson, 1970), which were introduced after the varimax method (Kaiser, 1955).

7 The most prominent methods of this type are nearest-neighbor and farthest-neighbor methods (Fisher & Van Ness, 1971).

8 For example, it is possible, for a relatively broad class of matrices, to calculate the degree of deviation between the factorial compositions of common tests from two matrices of loadings and to compare rotation methods using this approach.

9 As Ledermann (1937) demonstrated, the minimum rank of a P x P matrix that can be obtained by choosing the values placed on the main diagonal (i.e., the commonalities) is given by M ≤ [2*P+1−(8*P+1)^1/2]/2. Thus, the greatest M should respect the Ledermann limit.

10 This is the standardization procedure proposed by Kaiser (1958).

11 Bernaards and Jennrich (2003); Jennrich (2004).

12 According to Thurstone, the complexity of a test is defined as the number of factors on which it depends, i.e., the number of nonzero loadings (Thurstone, 1935, 1947). Naturally, the complexity may vary with rotation because the profile of the test changes with rotation.

13 In factor analysis, the loading matrices that can be built (before and after rotation) are unique up to the number of permutations and reflections of the columns, which produce 2^M M! equivalent matrices (Ichikawa & Konishi, 1995). This form of uncertainty is often referred to in the literature as the "alignment problem" (Clarkson, 1979; Jennrich, 2007; Myers, Ahn, Lu, Celimli, & Zopluoglu, 2017).

14 This index is calculated as half of the sum of the absolute differences between the relative frequency vectors of the two loading matrices (Pedersen, 1979). Its value ranges from 0 to 1.

15 However, the same results (not presented here) were obtained when the Mix test was performed on matrices that differed in both length and mix.

16 For all methods, gradient projection algorithms were used (Bernaards & Jennrich, 2005).

17 Various scholars agree that the value of the ε parameter should be chosen based on the number of factors (Browne, 2001; Asparouhov & Muthén, 2009; Browne, Cudeck, Tateneni, & Mels, 2008; Hattori et al., 2017). However, there are several opinions regarding how ε should be modulated. Some have suggested ε = 0.01 for M = 3, 4 and slightly higher values for M > 4 (Browne, 2001; Browne et al., 2008; Hattori et al., 2017), while others have suggested ε = 0.0001 for M = 2, ε = 0.001 for M = 3 and ε = 0.01 for M ≥ 4 (Muthén & Muthén, 1998-2017).

18 The 0.40 threshold was empirically determined based on the fact that values very close to 1 are not plausible.

19 The R code of these functions is available upon request.

20 The same result is arrived at using different statistics such as the percentage of absolute residuals exceeding .10, .05 or .01. In particular, class I methods shown an average of 28.3% of absolute residuals > .10 (whereas the average percentage was 34.4% for the other methods), an average of 47.8% of absolute residuals > .05 (64.4% for the other methods) and an average of 86.1% of absolute residuals > .01 (89.8% for the other methods).