Constructing Common Factors from Continuous and Categorical Data

Abstract

The method of principal components is widely used to estimate common factors in large panels of continuous data. This article first reviews alternative methods that obtain the common factors by solving a Procrustes problem. While these matrix decomposition methods do not specify the probabilistic structure of the data and hence do not permit statistical evaluations of the estimates, they can be extended to analyze categorical data. This involves the additional step of quantifying the ordinal and nominal variables. The article then reviews and explores the numerical properties of these methods. An interesting finding is that the factor space can be quite precisely estimated directly from categorical data without quantification. This may require using a larger number of estimated factors to compensate for the information loss in categorical variables. Separate treatment of categorical and continuous variables may not be necessary if structural interpretation of the factors is not required, such as in forecasting exercises.

Keywords:

• Alternating least squares
• Factor models
• Ordinal data
• Principal components

JEL Classification:

• C5
• C6
• C25
• C35

ACKNOWLEDGMENTS

I thank Aman Ullah for teaching me econometrics and especially grateful for his guidance and support over the years. Comments from two anonymous referees are greatly appreciated. I also thank Nickolay Trendafilov for helpful comments and discussions.

Notes

¹A 1983 issue of Journal of Econometrics (de Leeuw and Wansbeek editors) was devoted to these methods.

²The focus is rather different from the structural factor analysis considered in Cunha and Heckman (2008) and Almund et al. (2011).

³It is important that the lower bound is attainable and does not depend on x. If the lower bound was 1 instead of 2, no meaning could be attached to x = 3 because the lower bound of 1 is not attainable.

⁴A matrix is sub-orthonormal if it can be made orthonormal by appending rows or columns.

⁵The orthogonal Procrustes problem was solved in Schonmenn (1966). See Gower and Dijksterhuis (2004) for a review for subsequent work.

⁶Suppose that two continuous variables X ₁ and X ₂ are jointly normal with correlation coefficient ρ. The probability that (X ₁ > τ₁, X ₂ > τ₂) is given by

The tetrachoric correlation proposed by Pearson (1900) is the ρ such that p ₁₂(ρ) equals the sample proportion . Polychoric correlations are then generalizations of tetrachoric correlations from two dichotomous indicators to multiple ordered class.

⁷The initial procedure proposed by Takane et al. (1979) and refined by Nevels (1989) both have shortcomings. FACTALS fixes those bugs. Special thanks to H. Kiers for sharing the MATLAB code.

⁸The method has been discovered and rediscovered under different names, including as quantification, multiple correspondence analysis, dual or optimal scaling, and homogeneity analysis. See Tenenhaus and Young (1985) for a synthesis of these procedures. However, none of these methods are familiar to economists.

⁹Olsson et al. (1982) show that ρ _Yz is downward biased for ρ _YZ if Y and Z are jointly normal. The greatest attenuation occurs when there are few categories and the data are opposite skewed. In the special case when consecutive integers are assigned to categories of Y, it can be shown that and φ(·) is the standard normal density and q is the categorization attenuation factor.

For y = X (latent continuous data), x (categorical data), and G (adjacency matrix of indicators), IC _y denotes the number of factors selected by the Bai and Ng (2002) criterion with penalty when the principal components are constructed from data y. AO _y denotes factors determined using the criterion of Onatski (2010). The columns denote the average R ² when each of the factors estimated from y are regressed on the true factors.

¹⁰In an earlier version of the article when x and G were not demeaned, PCA estimated one more factor in both x and G.

For y = X, x, G, Z where Z denotes quantified data, denotes the number of factors estimated by the IC criterion of Bai and Ng (2002) with penalty g ₂. is the average R ² when each of the factors estimated from y is regerssed on all the true factors.

¹¹The FACTALS has convergence problems when N is 100 and the dimension of G is large.

¹²Given weights w ₁,…, w _T and real numbers x ₁,…, x _T , the monotone (isotonic) regression problem finds to minimize subject to the monotonicity condition t ⪯ k implies y _t  ≤ y _k where ⪯ is a partial ordering on the index set [1,…T]. An up-and-down-block algorithm is given in Kruskal (1964). See also de Leeuw (2005).