Ridge Regression Under Dense Factor Augmented Models

References

• Abadie, A., Diamond, A., and Hainmueller, J. (2010). “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program,” Journal of the American Statistical Association, 105, 493–505. DOI: 10.1198/jasa.2009.ap08746.
   Web of Science ®Google Scholar
• Agarwal, A., Shah, D., Shen, D., and Song, D. (2021) “On Robustness of Principal Component Regression,” Journal of the American Statistical Association, 116, 1731–1745. DOI: 10.1080/01621459.2021.1928513.
   Web of Science ®Google Scholar
• Alter, O., Brown, P. O., and Botstein, D. (2000). “Singular Value Decomposition for Genome-wide Expression Data Processing and Modeling,” Proceedings of the National Academy of Sciences, 97, 10101–10106. DOI: 10.1073/pnas.97.18.10101.
   PubMed Web of Science ®Google Scholar
• Amjad, M., Shah, D., and Shen, D. (2018). “Robust Synthetic Control,” Journal of Machine Learning Research, 19, 802–852.
   Web of Science ®Google Scholar
• Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., and Wager, S. (2021). “Synthetic Difference-In-Differences,” American Economic Review, 111, 4088–4118. DOI: 10.1257/aer.20190159.
   Web of Science ®Google Scholar
• Bai, J. (2003), “Inferential Theory for Factor Models of Large Dimensions,” Econometrica, 71, 135–171. DOI: 10.1111/1468-0262.00392.
   Web of Science ®Google Scholar
• Bai, J., and Ng, S. (2002), “Determining the Number of Factors in Approximate Factor Models,” Econometrica, 70, 191–221. DOI: 10.1111/1468-0262.00273.
   Web of Science ®Google Scholar
• Bai, J., and Ng, S. (2021), “Approximate Factor Models with Weaker Loadings,” ArXiv Working Paper arXiv:2109.03773.
   Google Scholar
• Bai, Z. D., and Silverstein, J. W. (1998). “No Eigenvalues Outside the Support of the Limiting Spectral Distribution of Large-Dimensional Sample Covariance Matrices,” The Annals of Probability, 26, 316–345. DOI: 10.1214/aop/1022855421.
   Web of Science ®Google Scholar
• Bai, Z. D., and Silverstein, J. W. (2010), Spectral Analysis of Large Dimensional Random Matrices, New York: Springer.
   Google Scholar
• Bergmeir, C., Hyndman, R. J., and Koo, B. (2018), “A Note on the Validity of Cross-Validation for Evaluating Autoregressive Time Series Prediction,” Computational Statistics & Data Analysis, 120, 70–83. DOI: 10.1016/j.csda.2017.11.003.
   Web of Science ®Google Scholar
• Blum, J., and Eisenberg, B. (1975), “The Law of Large Numbers for Subsequences of a Stationary Process,” The Annals of Probability, 3, 281–288. DOI: 10.1214/aop/1176996398.
   Web of Science ®Google Scholar
• Blum, J., and Hanson, D.(1960), “On the Mean Ergodic Theorem for Subsequences,” Bulletin of the American Mathematical Society, 66, 308–311. DOI: 10.1090/S0002-9904-1960-10481-8.
   Google Scholar
• Bradley, R. C. (1986), “Basic Properties of Strong Mixing Conditions,” in Dependence in Probability and Statistics: A Survey of Recent Results, eds. E. Eberlein and M. S. Taqqu, pp. 165–192, Boston: Birkhäuser.
   Google Scholar
• Bradley, R. C. (2005), “Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions,” Probability Surveys, 2, 107–144. DOI: 10.1214/154957805100000104.
   Google Scholar
• Cai, T. T., Han, X., and Pan, G. (2020), “Limiting Laws for Divergent Spiked Eigenvalues and Largest Nonspiked Eigenvalue of Sample Covariance Matrices,” The Annals of Statistics, 48, 1255–1280. DOI: 10.1214/18-AOS1798.
   Web of Science ®Google Scholar
• Carrasco, M., and Rossi, B. (2016), “In-Sample Inference and Forecasting in Misspecified Factor Models,” Journal of Business & Economic Statistics, 34, 313–338. DOI: 10.1080/07350015.2016.1186029.
   Web of Science ®Google Scholar
• Castle, J. L., Clements, M. P., and Hendry, D. F. (2013), “Forecasting by Factors, by Variables, by Both or Neither?,” Journal of Econometrics, 177, 305–319. DOI: 10.1016/j.jeconom.2013.04.015.
   Web of Science ®Google Scholar
• Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J. (1995). “Active Shape Models-Their Training and Application,” Computer Vision and Image Understanding, 61, 38–59. DOI: 10.1006/cviu.1995.1004.
   Web of Science ®Google Scholar
• Davis, R. A., and Mikosch, T. (2009). “Probabilistic Properties of Stochastic Volatility Models,” in Handbook of Financial Time Series, eds. T. G. Andersen, R. A. Davis, J. P. Kreiß, and T. V. Mikosch, pp. 255–267, Berlin: Springer.
   Google Scholar
• De Mol, C., Giannone, D., and Reichlin, L. (2008), “Forecasting Using a Large Number of Predictors: Is Bayesian Shrinkage a Valid Alternative to Principal Components?,” Journal of Econometrics, 146, 318–328. DOI: 10.1016/j.jeconom.2008.08.011.
   Web of Science ®Google Scholar
• Dicker, L. H. (2016). “Ridge Regression and Asymptotic Minimax Estimation Over Spheres of Growing Dimension,” Bernoulli, 22, 1–37. DOI: 10.3150/14-BEJ609.
   Web of Science ®Google Scholar
• Dobriban, E., and Wager, S. (2018), “High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification,” The Annals of Statistics, 46, 247–279. DOI: 10.1214/17-AOS1549.
   Web of Science ®Google Scholar
• Fan, J., Ke, Y., and Wang, K. (2020), “Factor-Adjusted Regularized Model Selection,” Journal of Econometrics, 216, 71–85. DOI: 10.1016/j.jeconom.2020.01.006.
   PubMed Web of Science ®Google Scholar
• Giannone, D., Lenza, M., and Primiceri, G. E. (2021), “Economic Predictions with Big Data: The Illusion of Sparsity,” Econometrica, 89, 2409–2437. DOI: 10.3982/ECTA17842.
   Web of Science ®Google Scholar
• Hastie, T., Tibshirani, R., and Friedman, J. H. (2009), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, New York: Springer.
   Google Scholar
• Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. J. (2022), “Surprises in High-Dimensional Ridgeless Least Squares Interpolation,” The Annals of Statistics, 50, 949–986. DOI: 10.1214/21-aos2133.
   PubMed Web of Science ®Google Scholar
• Hayashi, F. (2000), Econometrics Princeton: Princeton University Press.
   Google Scholar
• Hallin, M., and Liska, R. (2007). “The Generalized Dynamic Factor Model: Determining the Number of Factors,” Journal of the American Statistical Association, 102, 603–617. DOI: 10.1198/016214506000001275.
   Web of Science ®Google Scholar
• Hoerl, A. E., and Kennard, R. W. (1970), “Ridge Regression: Biased Estimation for Nonorthogonal Problems,” Technometrics, 12, 55–67. DOI: 10.1080/00401706.1970.10488634.
   Web of Science ®Google Scholar
• James, G., Witten, D., Hastie, T., and Tibshirani, R. (2021), An Introduction to Statistical Learning: with Applications in R (2nd ed), New York: Springer.
   Google Scholar
• Johnstone, I. M. (2001). “On the Distribution of the Largest Eigenvalue in Principal Components Analysis,” The Annals of Statistics, 29, 295–327. DOI: 10.1214/aos/1009210544.
   Web of Science ®Google Scholar
• Johnstone, I. M., and Lu, A. Y. (2009). “On Consistency and Sparsity for Principal Components Analysis in High Dimensions,” Journal of the American Statistical Association, 104, 682–693. DOI: 10.1198/jasa.2009.0121.
   PubMed Web of Science ®Google Scholar
• Johnstone, I. M., and Paul, D. (2018). “PCA in High Dimensions: An Orientation,” Proceedings of the IEEE, 106, 1277–1292. DOI: 10.1109/JPROC.2018.2846730.
   PubMed Web of Science ®Google Scholar
• Jolliffe, I. T. (1982). “A Note on the Use of Principal Components in Regression,” Journal of the Royal Statistical Society, Series C, 31, 300–303. DOI: 10.2307/2348005.
   Web of Science ®Google Scholar
• Jung, S., and Marron, J. S. (2009). “PCA Consistency in High Dimension, Low Sample Size Context,” The Annals of Statistics, 37, 4104–4130. DOI: 10.1214/09-AOS709.
   Web of Science ®Google Scholar
• Kneip, A., and Sarda, P. (2011), “Factor Models and Variable Selection in High-Dimensional Regression Analysis,” The Annals of Statistics, 39, 2410–2447. DOI: 10.1214/11-AOS905.
   Web of Science ®Google Scholar
• Liu, S., and Dobriban, E. (2020), “Ridge Regression: Structure, Cross-Validation, and Sketching,” in International Conference on Learning Representations.
   Google Scholar
• Marčenko, V. A., and Pastur, L. A. (1967). “Distribution of Eigenvalues for Some Sets of Random Matrices,” Mathematics of the USSR-Sbornik, 1, 457–483. DOI: 10.1070/SM1967v001n04ABEH001994.
   Google Scholar
• McCracken, M. W., and Ng, S. (2016), “FRED-MD: A Monthly Database for Macroeconomic Research,” Journal of Business & Economic Statistics, 34, 574–589. DOI: 10.1080/07350015.2015.1086655.
   Web of Science ®Google Scholar
• Mitchell, T. J., and Beauchamp, J. J. (1988), “Bayesian Variable Selection in Linear Regression,” Journal of the American Statistical Association, 83, 1023–1032. DOI: 10.1080/01621459.1988.10478694.
   Web of Science ®Google Scholar
• Ng, S. (2013), “Variable Selection in Predictive Regressions,” In Handbook of Economic Forecasting (Vol. 2), eds. G. Elliott and A. Timmermann, pp. 752–789, Amsterdam: Elsevier.
   Google Scholar
• Onatski, A. (2009). “Testing Hypotheses About the Number of Factors in Large Factor Models,” Econometrica, 77, 1447–1479.
   Web of Science ®Google Scholar
• Onatski, A. (2012). “Asymptotics of the Principal Components Estimator of Large Factor Models With Weakly Influential Factors,” Journal of Econometrics, 168, 244–258.
   Web of Science ®Google Scholar
• Onatski, A., and Wang, C. (2021), “Spurious Factor Analysis,” Econometrica, 89, 591–614. DOI: 10.3982/ECTA16703.
   Web of Science ®Google Scholar
• Paul, D. (2007). “Asymptotics of Sample Eigenstructure for a Large Dimensional Spiked Covariance Model,” Statistica Sinica, 17, 1617–1642.
   Web of Science ®Google Scholar
• Paul, D., and Silverstein, J. W. (2009). “No Eigenvalues Outside the Support of the Limiting Empirical Spectral Distribution of a Separable Covariance Matrix,” Journal of Multivariate Analysis, 100, 37–57. DOI: 10.1016/j.jmva.2008.03.010.
   Web of Science ®Google Scholar
• Preisendorfer, R. W. (1988). Principal Component Analysis in Meteorology and Oceanography, Amsterdam: Elsevier.
   Google Scholar
• Stock, J. H., and Watson, M. W. (2002), “Forecasting Using Principal Components from a Large Number of Predictors,” Journal of the American Statistical Association, 97, 1167–1179. DOI: 10.1198/016214502388618960.
   Web of Science ®Google Scholar
• Stock, J. H., and Watson, M. W. (2012), “Generalized Shrinkage Methods for Forecasting Using Many Predictors,” Journal of Business & Economic Statistics, 30, 481–493.
   Web of Science ®Google Scholar
• Tikhonov, A. N. (1963a), “Regularization of Incorrectly Posed Problems,” Soviet Mathematics Doklady, 4, 1624–1627 (English translation).
   Google Scholar
• Tikhonov, A. N. (1963b), “Solution of Incorrectly Formulated Problems and the Regularization Method,” Soviet Mathematics Doklady, 4, 1035–1038 (English translation).
   Google Scholar