Quantifying Model Error in P-technique Factor Analysis

Abstract

P-technique factor analysis is an exploratory factor model for multivariate time series data. Assessing model fit of P-technique factor models is non-trivial because time series data are correlated at nearby time points. We present a test statistic that is appropriate for P-technique factor analysis. In addition, the test statistic allows researchers to quantify the amount of model error. We explore the statistical properties of the test statistic with simulated data and we illustrate its use with an empirical study of personality states. Results of the simulation study include (1) the empirical distributions of the test statistic approximately followed their respective theoretical chi-square distributions, (2) the empirical Type I error rates of the test of perfect fit are close to the nominal level and the empirical Type I error rates of the test of close fit are slightly lower than the nominal level, and (3) the empirical power rates of the test of perfect fit are satisfactory but the empirical power rates of the test of close fit are only satisfactory for small models.

Keywords:

• factor analysis
• P-technique
• time series
• intensive longitudinal data

Article Information

Conflict of Interest Disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical Principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This study was not supported.

Role of the Funders/Sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgements: The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institution is not intended and should not be inferred.

Notes

1 If variances of manifest variables (the original measurement units) are of interest, researchers can consider a model for covariance matrices rather than correlation matrices. These covariance matrices do not have a Wishart distribution because time series data are dependent (Hannan, 1970, Equation (3.3)). Because these statistics are affected by their original measurement units, many researchers prefer to examine correlations and lagged correlations. Therefore, the usual maximum likelihood estimates are no longer optimal.

2 Computing lagged covariance matrices at different time lags involves different number of terms. For example, the lag zero covariance matrix involves T terms, the lag one covariance matrix involves T-1 terms, and lag L covariance matrix involves T-L terms. We divide these lagged covariance matrices using the same number T regardless of lags. This is a typical practice in time series analysis literature (Brockwell & Davis, 1991, Equation (7.2.4)), which guarantees that the covariance matrices of multivariate time series are positive definite.

3 We can arrange the manifest variable correlation matrix and lagged correlation matrices into a block Toeplitz matrix to reflect the concurrent and lagged relations of the manifest variable time series. A reviewer points out that one can construct a time series that has the same concurrent and lagged relations by linearly transforming the time series of random numbers. This linear transformation is a special case of the linear process described in Brockwell and Davis, (1991, p. 404, Definition 11.1.12). Although the weight matrices of the linear transformation can reveal properties of the manifest variable time series, it does not aid our understanding of relations between manifest variables and factors.

4 Because the columns of the unrotated factor loading matrix $\mathit{A}$ are orthogonal to each other, $\frac{n_f(n_f-1)}{2}$ constraints are implicitly imposed on this $n_v*n_f$ matrix. Therefore, the effective number of elements in A is $q=n_v*n_f-\frac{n_f(n_f-1)}{2}.$

5 Readers interested in understanding more about the relation of the Jacobian to the test statistic can refer to the multivariate delta method (Agresti, 2014; Greene, 2018).

6 We choose τ = 20 in the simulation study and in the empirical illustration. The largest difference between the estimates with τ = 20 and those with τ = 21 is $1.07e^{-06}.$ Although the time series models in the simulation studies and empirical illustration involve vector autoregressive lag one time series, we expect that τ = 20 is sufficient for most stationary time series processes because correlations at higher lags move to zero in stationary time series (Brockwell & Davis, 1991, p. 94). Additionally, τ = 20 is computationally feasible and it cost 8.6 seconds for 12-variate time series on a 2.9 GHz Intel Core i5 processor with 16 GB of memory.

7 Some applications may require factor loading patterns that are more general than the Thurstone simple structure. Maydeu-Olivares and Coffman, (2006) suggested a random intercept factor model to accommodate participants’ heterogeneity in response patterns. Molenaar and Nesselroade, (2001) proposed a rotation procedure whose primary goal is to seek a clearer interpretation of factor time series rather than a simple structure of factor loadings.

8 A complex factor loading pattern is not defined for 4-1 and 6-2 models, because complex factor loading matrices in 4-1 models have only one column and a model with only two factors will not have cross-loadings following the procedure outlined in Tucker et al., (1969).

9 We also generated data from the high communalities. Higher communalities lead to more accurate Type I and power rates. We include the corresponding results in an online appendix.

10 We include figures comparing the empirical distribution and the theoretical distributions of test statistics under the other conditions in the online appendix. The general findings from the other conditions are similar to those of Figure 1.

11 Heywood cases occur in some conditions. In particular for the uncorrelated unique factor condition, the 12-4 model with a complex factor loading matrix has 435, 360, 279, and 181 Heywood cases at T = 100, T = 200, T = 500, and T = 1000, respectively for the perfect population models. For the correlated unique factor condition, the 12-4 model with a complex factor loading matrix has 434, 379, 282, and 183 Heywood cases at T = 100, T = 200, T = 500, and T = 1000, respectively for the perfect population models. The Heywood cases do not seem to affect the comparisons between the empirical distributions and the theoretical distributions. The plots with Heywood cases are almost identical to the plots without Heywood cases. We decide to report the plots with Heywood cases so the results can be generalized to such cases as well. We include the plots without Heywood cases in an online support file for comparison purpose.