On Common Factor and Principal Component Representations of Data: Implications for Theory and for Confirmatory Replications

Abstract

Common factor analysis (FA) and principal component analysis (PCA) are commonly used to obtain lower-dimensional representations of matrices of correlations among manifest variables. Whereas some experts argue that differences in results from use of FA and PCA are small and relatively unimportant in empirical studies, the fundamental rationales for the two methods are very different. Here, FA and PCA are contrasted on four key issues: the range of possible dimensional loadings, the range of potential correlations among dimensions, the structure of residual covariances and correlations, and the relation between population parameters and the correlational structures with which they are associated. For decades, experts have emphasized indeterminacies of the FA model, particularly indeterminacy of common factor scores. Determinate in most respects, a heretofore unacknowledged, pernicious indeterminacy of PCA is demonstrated: the indeterminacy between PCA structural representations and the correlational structures from which they are derived. Researchers are often advised to use either FA or PCA in exploratory rounds of data analysis to understand and refine the dimensional structure of a domain before moving to Structural Equation Modeling in later theory-testing, confirmatory, replication studies. Results from the current study suggest that PCA is an unreliable method to use for such purposes and may lead to serious misrepresentation of the structure of a domain. Hence, PCA should never be used if the goal is to understand and represent the latent structure of a domain; only FA techniques should be used for this purpose, as only FA provides reliable structural representations as the basis for confirmatory tests in future studies.

Keywords:

• Common factor analysis
• factor loadings
• oblique factors
• principal component analysis
• structural equation modeling

Notes

¹ As noted, the quoted material is from Thurstone (1947, p. 361). The asterisk (*) in the quote referred readers to Thurstone (1935, p. 55) and the dagger (†) to Thurstone (1935, p. 120) where the italicized parts of the quote, with slight modifications, were first presented.