Forward Stepwise Deep Autoencoder-Based Monotone Nonlinear Dimensionality Reduction Methods

References

• Baldi, P., and Hornik, K. (1989), “Neural Networks and Principal Component Analysis: Learning From Examples Without Local Minima,” Neural Networks, 2, 53–58. DOI: https://doi.org/10.1016/0893-6080(89)90014-2.
   Web of Science ®Google Scholar
• Barlow, R. E., Bartholomew, D. J., Bremner, J. M., and Brunk, H. D. (1972), Statistical Inference Under Order Restrictions: The Theory and Application of Isotonic Regression, New York: Wiley.
   Google Scholar
• Belkin, M., and Niyogi, P. (2002), “Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering,” in Advances in Neural Information Processing Systems, pp. 585–591.
   Google Scholar
• Bishop, C., Svensen, M., and Williams, C. (1998), “GTM: The Generative Topographic Mapping,” Neural Computation, 10, 215–234. DOI: https://doi.org/10.1162/089976698300017953.
   Web of Science ®Google Scholar
• Bourlard, H., and Kamp, Y. (1988), “Auto-Association by Multilayer Perceptrons and Singular Value Decomposition,” Biological Cybernetics, 59, 291–294. DOI: https://doi.org/10.1007/BF00332918.
   PubMed Web of Science ®Google Scholar
• Brunk, H. (1958), “On the Estimation of Parameters Restricted by Inequalities,” The Annals of Mathematical Statistics, 29, 437–454. DOI: https://doi.org/10.1214/aoms/1177706621.
   Google Scholar
• Cybenko, G. (1989), “Approximation by Superpositions of a Sigmoidal Function,” Mathematics of Control, Signals and Systems, 2, 303–314. DOI: https://doi.org/10.1007/BF02551274.
   Google Scholar
• Donoho, D. L., and Grimes, C. (2003), “Hessian Eigenmaps: Locally Linear Embedding Techniques for High-Dimensional Data,” Proceedings of the National Academy of Sciences of the United States of America, 100, 5591–5596. DOI: https://doi.org/10.1073/pnas.1031596100.
   PubMed Web of Science ®Google Scholar
• Duchamp, T., and Stuetzle, W. (1996), “Extremal Properties of Principal Curves in the Plane,” The Annals of Statistics, 24, 1511–1520. DOI: https://doi.org/10.1214/aos/1032298280.
   Web of Science ®Google Scholar
• Feng, T., Li, S. Z., Shum, H. Y., and Zhang, H. (2002), “Local Non-Negative Matrix Factorization as a Visual Representation,” in Proceedings 2nd International Conference on Development and Learning. ICDL 2002, IEEE, pp. 178–183. DOI: https://doi.org/10.1109/DEVLRN.2002.1011835.
   Google Scholar
• Fong, Y., Shen, X., Ashley, V. C., Deal, A., Seaton, K. E., Yu, C., Grant, S. P., Ferrari, G., deCamp, A. C., Bailer, R. T., and Koup, R. A. (2018), “Vaccine-Induced Antibody Responses Modify the Association Between T-Cell Immune Responses and HIV-1 Infection Risk in HVTN 505,” The Journal of Infectious Diseases, 217, 1280–1288. DOI: https://doi.org/10.1093/infdis/jiy008.
   PubMed Web of Science ®Google Scholar
• Fukunaga, K., and Olsen, D. R. (1971), “An Algorithm for Finding Intrinsic Dimensionality of Data,” IEEE Transactions on Computers, 100, 176–183. DOI: https://doi.org/10.1109/T-C.1971.223208.
   Web of Science ®Google Scholar
• Gijbels, I. (2005), “Monotone Regression,” in The Encyclopedia of Statistical Sciences, Hoboken, NJ: Wiley.
   Google Scholar
• Goodfellow, I., Bengio, Y., and Courville, A. (2016), Deep Learning, Adaptive Computation and Machine Learning, Cambridge, MA: MIT Press.
   Google Scholar
• Grassberger, P., and Procaccia, I. (1983), “Measuring the Strangeness of Strange Attractors,” Physica D: Nonlinear Phenomena, 9, 189–208. DOI: https://doi.org/10.1016/0167-2789(83)90298-1.
   Web of Science ®Google Scholar
• Hall, P., and Huang, L. S. (2001), “Nonparametric Kernel Regression Subject to Monotonicity Constraints,” The Annals of Statistics, 29, 624–647.
   Web of Science ®Google Scholar
• Ham, J., Lee, D. D., Mika, S., and Schölkopf, B. (2004), “A Kernel View of the Dimensionality Reduction of Manifolds,” in Proceedings of the Twenty-First International Conference on Machine Learning, p. 47.
   Google Scholar
• Hammer, S. M., Sobieszczyk, M. E., Janes, H., Karuna, S. T., Mulligan, M. J., Grove, D., Koblin, B. A., Buchbinder, S. P., Keefer, M. C., Tomaras, G. D., and Frahm, N. (2013), “Efficacy Trial of a DNA/rAd5 HIV-1 Preventive Vaccine,” New England Journal of Medicine, 369, 2083–2092. DOI: https://doi.org/10.1056/NEJMoa1310566.
   PubMed Web of Science ®Google Scholar
• Hastie, T., and Stuetzle, W. (1989), “Principal Curves,” Journal of the American Statistical Association, 84, 502–516. DOI: https://doi.org/10.1080/01621459.1989.10478797.
   Web of Science ®Google Scholar
• He, K., Zhang, X., Ren, S., and Sun, J. (2015), “Delving Deep Into Rectifiers: Surpassing Human-Level Performance on Imagenet Classification,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034.
   Google Scholar
• Hinton, G. E., and Salakhutdinov, R. R. (2006), “Reducing the Dimensionality of Data With Neural Networks,” Science, 313, 504–507. DOI: https://doi.org/10.1126/science.1127647.
   PubMed Web of Science ®Google Scholar
• Hoyer, P. O. (2004), “Non-Negative Matrix Factorization With Sparseness Constraints,” Journal of Machine Learning Research, 5, 1457–1469.
   Web of Science ®Google Scholar
• Janes, H. E., Cohen, K. W., Frahm, N., De Rosa, S. C., Sanchez, B., Hural, J., Magaret, C.A., Karuna, S., Bentley, C., Gottardo, R., and Finak, G. (2017), “Higher T-Cell Responses Induced by DNA/rAd5 HIV-1 Preventive Vaccine Are Associated With Lower HIV-1 Infection Risk in an Efficacy Trial,” The Journal of Infectious Diseases, 215, 1376–1385. DOI: https://doi.org/10.1093/infdis/jix086.
   PubMed Web of Science ®Google Scholar
• Jolliffe, I. T., and Cadima, J. (2016), “Principal Component Analysis: A Review and Recent Developments,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374, 20150202. DOI: https://doi.org/10.1098/rsta.2015.0202.
   PubMed Web of Science ®Google Scholar
• Kambhatla, N., and Leen, T. K. (1997), “Dimension Reduction by Local Principal Component Analysis,” Neural Computation, 9, 1493–1516. DOI: https://doi.org/10.1162/neco.1997.9.7.1493.
   Web of Science ®Google Scholar
• Kingma, D. P., and Ba, J. (2014), “Adam: A Method for Stochastic Optimization,” in Proceedings of the 3rd International Conference on Learning Representations (ICLR).
   Google Scholar
• Kramer, M. A. (1991), “Nonlinear Principal Component Analysis Using Autoassociative Neural Networks,” AIChE Journal, 37, 233–243. DOI: https://doi.org/10.1002/aic.690370209.
   Web of Science ®Google Scholar
• Kruskal, J. B., and Wish, M. (1978), Multidimensional Scaling, Beverly Hills, CA: SAGE.
   Google Scholar
• LeCun, Y. (1987), “Modeles connexionnistes de l’apprentissage,” Ph.D. thesis.
   Google Scholar
• Lee, D. D., and Seung, H. S. (1999), “Learning the Parts of Objects by Non-Negative Matrix Factorization,” Nature, 401, 788–791. DOI: https://doi.org/10.1038/44565.
   PubMed Web of Science ®Google Scholar
• Lee, J. A., and Verleysen, M. (2007), Nonlinear Dimensionality Reduction, New York: Springer.
   Google Scholar
• Mammen, E., Marron, J., Turlach, B., and Wand, M. (2001), “A General Projection Framework for Constrained Smoothing,” Statistical Science, 16, 232–248. DOI: https://doi.org/10.1214/ss/1009213727.
   Web of Science ®Google Scholar
• Meredith, W., and Millsap, R. E. (1985), “On Component Analyses,” Psychometrika, 50, 495–507. DOI: https://doi.org/10.1007/BF02296266.
   Web of Science ®Google Scholar
• Mika, S., Ratsch, G., Weston, J., Scholkopf, B., and Mullers, K. (1999), “Fisher Discriminant Analysis With Kernels,” in Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop, IEEE, pp. 41–48.
   Google Scholar
• Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013), “Efficient Estimation of Word Representations in Vector Space,” International Conference on Learning Representations.
   Google Scholar
• Neidich, S. D., Fong, Y., Li, S. S., Geraghty, D. E., Williamson, B. D., Young, W. C., Goodman, D., Seaton, K. E., Shen, X., Sawant, S., and Zhang, L. (2019), “Antibody Fc Effector Functions and IgG3 Associate With Decreased HIV-1 Risk,” Journal of Clinical Investigation, 129, 4838–4849. DOI: https://doi.org/10.1172/JCI126391.
   PubMed Web of Science ®Google Scholar
• Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L. and Lerer, A. (2017), “Automatic Differentiation in PyTorch,” in NIPS-W.
   Google Scholar
• Permar, S. R., Fong, Y., Vandergrift, N., Fouda, G. G., Gilbert, P., Parks, R., Jaeger, F. H., Pollara, J., Martelli, A., Liebl, B. E., and Lloyd, K. (2015), “Maternal HIV-1 Envelope–Specific Antibody Responses and Reduced Risk of Perinatal Transmission,” Journal of Clinical Investigation, 125, 2702–2706. DOI: https://doi.org/10.1172/JCI81593.
   PubMed Web of Science ®Google Scholar
• Plaut, E. (2018), “From Principal Subspaces to Principal Components With Linear Autoencoders,” arXiv no. 1804.10253.
   Google Scholar
• Ramsay, J. O. (1988), “Monotone Regression Splines in Action,” Statistical Science, 3, 425–441. DOI: https://doi.org/10.1214/ss/1177012761.
   Google Scholar
• Rich, K. C., Fowler, M. G., Mofenson, L. M., Abboud, R., Pitt, J., Diaz, C., Hanson, I. C., Cooper, E., Mendez, H., and Women and Infants Transmission Study Group (2000), “Maternal and Infant Factors Predicting Disease Progression in Human Immunodeficiency Virus Type 1-Infected Infants,” Pediatrics, 105, e8. DOI: https://doi.org/10.1542/peds.105.1.e8.
   PubMed Web of Science ®Google Scholar
• Roweis, S. T., and Saul, L. K. (2000), “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, 290, 2323–2326. DOI: https://doi.org/10.1126/science.290.5500.2323.
   PubMed Web of Science ®Google Scholar
• Rumelhart, D., Hinton, G., and Williams, R. (1986), “Learning Internal Representations by Error Propagation” (Chapter 8), in Parallel Distributed Processing, Cambridge, MA: MIT Press.
   Google Scholar
• Sammon, J. W. (1969), “A Nonlinear Mapping for Data Structure Analysis,” IEEE Transactions on Computers, 100, 401–409. DOI: https://doi.org/10.1109/T-C.1969.222678.
   Web of Science ®Google Scholar
• Schölkopf, B., Smola, A., and Müller, K. (1997), “Kernel Principal Component Analysis,” in Lecture Notes in Computer Science, Springer, pp. 583–588.
   Google Scholar
• Scholz, M., and Vigário, R. (2002), “Nonlinear PCA: A New Hierarchical Approach,” in Proceedings ESANN, pp. 439–444.
   Google Scholar
• Silverman, B. W., and Green, P. (1993), Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach, Boca Raton, FL: Chapman and Hall/CRC.
   Google Scholar
• Smola, A. J., Williamson, R. C., Mika, S., and Schölkopf, B. (1999), “Regularized Principal Manifolds,” in European Conference on Computational Learning Theory, Springer, pp. 214–229.
   Google Scholar
• Sorzano, C. O. S., Vargas, J., and Montano, A. P. (2014), “A Survey of Dimensionality Reduction Techniques,” arXiv no. 1403.2877.
   Google Scholar
• Tenenbaum, J. B., De Silva, V., and Langford, J. C. (2000), “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, 290, 2319–2323. DOI: https://doi.org/10.1126/science.290.5500.2319.
   PubMed Web of Science ®Google Scholar
• Ting, D., and Jordan, M. I. (2018), “On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding,” arXiv no. 1803.02432.
   Google Scholar
• Van Der Maaten, L., Postma, E., and Van den Herik, J. (2009), “Dimensionality Reduction: A Comparative,” Journal of Machine Learning Research, 10, 13.
   Google Scholar
• Van der Waerden, B. (1952), “Order Tests for the Two-Sample Problem and Their Power,” in Indagationes Mathematicae (Proceedings) (Vol. 55), Elsevier, pp. 453–458. DOI: https://doi.org/10.1016/S1385-7258(52)50063-5.
   Google Scholar
• Weinberger, K. Q., and Saul, L. K. (2006), “Unsupervised Learning of Image Manifolds by Semidefinite Programming,” International Journal of Computer Vision, 70, 77–90. DOI: https://doi.org/10.1007/s11263-005-4939-z.
   Web of Science ®Google Scholar
• Westfall, P. H., Arias, A. L., and Fulton, L. V. (2017), “Teaching Principal Components Using Correlations,” Multivariate Behavioral Research, 52, 648–660. DOI: https://doi.org/10.1080/00273171.2017.1340824.
   PubMed Web of Science ®Google Scholar
• Williams, C. K. (2002), “On a Connection Between Kernel PCA and Metric Multidimensional Scaling,” Machine Learning, 46, 11–19.
   Web of Science ®Google Scholar
• Zhang, Z., and Zha, H. (2004), “Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment,” SIAM Journal on Scientific Computing, 26, 313–338. DOI: https://doi.org/10.1137/S1064827502419154.
   Web of Science ®Google Scholar