A new data adaptive elastic net predictive model using hybridized smoothed covariance estimators with information complexity

References

• Zhou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Ser B. 2005;67(2):301–320.
   Google Scholar
• Hastie T, Tibshirani R, Wainright M. Statistical learning with sparsity: the lasso and generalizations. Boca Raton: Chapman and Hall, CRC; 2015.
   Google Scholar
• Tibshirani R. Regression shrinkage and selection via lasso. J R S Soc Ser B. 1996;58:267–288.
   Google Scholar
• Bellman R. Adaptive control processes: a guide tour. Princeton (NJ): Princeton University Press; 1961.
   Google Scholar
• Miguel AC. Continuous latent variable models for dimensionality reduction and sequential data reconstruction [dissertation]. UK: University of Sheffield; 2001.
   Google Scholar
• Murphy K. Machine learning: a probabilistic perspective. Cambridge, Massachusetts: The MIT Press; 2012.
   Google Scholar
• Gosh S. Adaptive elastic net: an improvement of elastic net to achieve oracle properties. 2007; Available from: www.math.iupui.edu.
   Google Scholar
• Efron B, Hastie T, Johnstone I, et al. Least angle regression. Ann Stat. 2004;32(2):407–499.
   Web of Science ®Google Scholar
• Masák T. Sparse principal component analysis; 2017. Available from: www.karlin.mff.cuni.cz/masak/prezentace.pdf.
   Google Scholar
• Sjöstrand K, Clemmensen L, Einarsson GLR, et al. Spasm: a MATLAB toolbox for sparse statistical modeling. J Stat Softw. 2018;84:1–37.
   PubMed Web of Science ®Google Scholar
• Ahmed SBY. Big data analytics: integrating penalty strategies. Int J Manag Sci Eng Manag. 2016;11(2):105–115.
   Google Scholar
• Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc. 2001;96(456):1348–2001.
   Web of Science ®Google Scholar
• Zou H. The adaptive lasso and its oracle properties. J Am Stat Assoc. 2006;101(476):1418–1429.
   Web of Science ®Google Scholar
• Zhang CH. Nearly unbiased variable selection under minimax concave penalty. Ann Stat. 2010;38(2):894–942.
   Web of Science ®Google Scholar
• Yuzbasi B, Arashi M. Double shrunken selection operator. Commun Stat Simul Comput. 2010:1–9. Available from doi:10.1080/03610918.2017.1395040.
   Google Scholar
• Ahmed S, Yuzbasi B. Big and complex data analysis: methodologies and applications. New York: Springer; 2017.
   Google Scholar
• James W, Stein C. Estimation with quadratic loss. In: Willems JC, editor. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics. University of California Press; 1961. p. 361–379.
   Google Scholar
• Haff LR. Empirical Bayes estimation of the multivariate normal covariance matrix. Ann Stat. 1980;8(3):586–597.
   Web of Science ®Google Scholar
• Theil H, Laitinen K. 1980. Singular moment matrices in applied econometrics. In Krishnaiah P. R., editor. Multivariate Analysis-V, Amsterdam: North-Holland; 1980. p. 629–649.
   Google Scholar
• Feibig DG. On the maximum-entropy approach to undersized samples. Appl Math Comput. 1984;14(3):301–312.
   Web of Science ®Google Scholar
• Theil H, Fiebig DG. Exploiting continuity: maximum entropy estimation of continuous distributions. Cambridge (MA): Ballinger Publishing Company; 1984.
   Google Scholar
• Shurygin A. The linear combination of the simplest discriminator and Fisher's one. In: Nauka, editor. Applied statistics. Moscow: Nauka; 1983.
   Google Scholar
• Press S. J. Estimation of a normal covariance matrix. Santa Monica (CA): RAND Corporation; 1975; https://www.rand.org/pubs/papers/P5436.html.
   Google Scholar
• Chen MCF. Estimation of covariance matrices under a quadratic loss function. Research Report S-46, Department of Mathematics, SUNY at Albany (Island of Capri, Italy). 1976. p. 1–33.
   Google Scholar
• Bozdogan H. Shrinkage covariance estimators; 2010. Unpublished Lecture Notes.
   Google Scholar
• Chen Y, Wiesel A, Eldar YC, et al. Shrinkage algorithms for mmse covariance estimation. IEEE Trans Signal Process. 2010;58(10):5016–5029.
   Web of Science ®Google Scholar
• Ledoit O, Wolf M. A well-conditioned estimator for large dimensional covariance matrices. J Multivar Anal. 2004;88(2):365–411.
   Web of Science ®Google Scholar
• Ollila E. Optimal high-dimensional shrinkage covariance estimation for elliptical distributions. 2017 25th European Signal Processing Conference (EUSIPCO). 2017; p. 1689–1693.
   Google Scholar
• Thomaz C. Maximum entropy covariance estimate for statistical pattern recognition [dissertation]. Imperial College, University of London, London, UK; 2004.
   Google Scholar
• Pamukcu E, Bozdogan H, Calik S. A novel hybrid dimension reduction technique for undersized high dimensional gene expression data sets using information complexity criterion for cancer classification. Comput Math Methods Med. 2015;2015:Article ID 370640,14 p.
   Google Scholar
• Bozdogan H. Akaike's information criterion and recent developments in information complexity. J Math Psychol. 2000;44:62–91.
   PubMed Web of Science ®Google Scholar
• Bozdogan H. A new class of information complexity (icomp) criteria with an application to customer profiling and segmentation. Istanbul Univ J Sch Bus Adm. 2010;39(2):370–398.
   Google Scholar
• Bozdogan H, Ueno M. A unified approach to information-theoretic and Bayesian model selection criteria; 2016. Paper presented at the JSM-2016, Bayesian Model Selection Session, Chicago, IL, July 30–August 4, 2016.
   Google Scholar
• Akaike H. Information theory and an extension of the maximum likelihood principle. In: Petrov B, Csaki F, editors. Second International Symposium on Information Theory. Budapest: Academiai Kiado; 1973. p. 267–281.
   Google Scholar
• Van Emden H. An analysis of complexity. 1971.
   Google Scholar
• Takeuchi K. Distribution of information statistics and a criterion of model fitting. Suri-Kagaku Math Sci. 1976;153:12–18.
   Google Scholar
• Hosking JRM. Lagrange-multiplier tests of time series models. J R Stat Soc Ser B. 1980;42:170–181.
   Google Scholar
• Shibata R. Statistical aspects of model selection. In: Willems JC, editor. From the data to modeling. Springer-Verlag; 1989. p. 216–240.
   Google Scholar
• Rissanen J. Modeling by shortest data description. Automatica. 1978;14:465–471.
   Web of Science ®Google Scholar
• Schwarz G. Estimating the dimension of a model. Ann Stat. 1978;6:461–464.
   Web of Science ®Google Scholar
• Bozdogan H. Model selection and Akaike's information criteria (AIC): the general theory and its analytical extensions. Psychometrica. 1987;52:317–332.
   Web of Science ®Google Scholar
• Frieden B. Physics from Fisher information. Cambridge (UK): Cambridge University Press; 1998.
   Google Scholar
• Cramér H. Mathematical methods of statistics. Princeton (NJ): Princeton University Press; 1946.
   Google Scholar
• Rao C. Information and accuracy attainable in the estimation of statistical parameters. Bull Calcutta Math Soc. 37;1945:81.
   Google Scholar
• Rao C. Minimum variance and the estimation of several parameters. Proc Cambridge Philos Soc. 43;1947:280.
   Google Scholar
• Rao C. Sufficient statistics and minimum variance estimates. Proc Cambridge Philos Soc. 45;1948:213.
   Google Scholar
• Bozdogan H. Statistical data mining and knowledge discovery. Boca Raton: Chapman and Hall, CRC; 2004.
   Google Scholar
• Bozdogan H, Pamukcu E. Novel dimension reduction techniques for high dimensional data using information complexity. In: Gupta A, Capponi A, editors. Optimization Challenges in Complex, Networked, and Risky Systems. INFORMS; 2016. p. 140–170.
   Google Scholar
• Bozdogan H, Haughton DM. Information complexity criteria for regression models. Comput Stat Data Anal. 1998;28:51–76.
   Web of Science ®Google Scholar
• Roozbeh M. Generalized ridge regression estimator in high dimensional sparse regression models. Stat Optim Inf Comput. 2018;6:415–426.
   Google Scholar
• McDonald G, Galarneau DI. A Monte Carlo evaluation of some ridge-type estimators. J Am Stat Assoc. 1975;70(350):407–416.
   Web of Science ®Google Scholar
• Li Q, Lin N. The Bayesian elastic net. Bayesian Anal. 2010;5(1):151–170.
   Web of Science ®Google Scholar
• Bühlmann P, Kalisch MLM. High-dimensional statistics with view toward applications in biology. Annu Rev Stat Appl. 2014;1:255–278.
   Web of Science ®Google Scholar
• Hilafu H, Yin X. Sufficient dimension reduction and variable selection for large-p-small-n data with highly correlated predictors. J Comput Graph Stat. 2016;26(1):26–34.
   Web of Science ®Google Scholar
• Osborne MR, Presnell B, Turlach BA. On the lasso and its dual. J Comput Graph Stat. 2000;9(2):319–337.
   Web of Science ®Google Scholar
• Wasserman L, Roeder K. High-dimensional variable selection. Ann Stat. 2009;37(5A):2178–2201. Available from: https://doi.org/10.1214/08-AOS646.
   PubMed Web of Science ®Google Scholar