General Nonlinear Function-on-Function Regression via Functional Universal Approximation

References

• Acar, E., Bro, R., and Schmidt, B. (2008), “New Exploratory Clustering Tool,” Journal of Chemometrics: A Journal of the Chemometrics Society, 22, 91–100. DOI: 10.1002/cem.1106.
   Web of Science ®Google Scholar
• Chen, D., Hall, P., and Müller, H.-G. (2011), “Single and Multiple Index Functional Regression Models with Nonparametric Link,” The Annals of Statistics, 39, 1720–1747. DOI: 10.1214/11-AOS882.
   Web of Science ®Google Scholar
• Chiou, J.-M., Yang, Y.-F., and Chen, Y.-T. (2016), “Multivariate Functional Linear Regression and Prediction,” Journal of Multivariate Analysis, 146, 301–312. DOI: 10.1016/j.jmva.2015.10.003.
   Web of Science ®Google Scholar
• Cybenko, G. (1989), “Approximation by Superpositions of a Sigmoidal Function,” Mathematics of Control, Signals and Systems, 2, 303–314. DOI: 10.1007/BF02551274.
   Google Scholar
• Fan, J., and Zhang, J.-T. (2000), “Two-Step Estimation of Functional Linear Models with Applications to Longitudinal Data,” Journal of the Royal Statistical Society, Series B, 62, 303–322. DOI: 10.1111/1467-9868.00233.
   Google Scholar
• Hanin, B. (2019), “Universal Function Approximation by Deep Neural Nets with Bounded Width and Relu Activations,” Mathematics, 7, 992. DOI: 10.3390/math7100992.
   Web of Science ®Google Scholar
• Hastie, T., and Tibshirani, R. (1993), “Varying-Coefficient Models,” Journal of the Royal Statistical Society, Series B, 55, 757–779. DOI: 10.1111/j.2517-6161.1993.tb01939.x.
   Google Scholar
• Hornik, K. (1991), “Approximation Capabilities of Multilayer Feedforward Networks,” Neural Networks, 4, 251–257. DOI: 10.1016/0893-6080(91)90009-T.
   Web of Science ®Google Scholar
• Ivanescu, A. E., Staicu, A.-M., Scheipl, F., and Greven, S. (2015), “Penalized Function-on-Function Regression,” Computational Statistics, 30, 539–568. DOI: 10.1007/s00180-014-0548-4.
   Web of Science ®Google Scholar
• James, G. M., and Silverman, B. W. (2005), “Functional Adaptive Model Estimation,” Journal of the American Statistical Association, 100, 565–576. DOI: 10.1198/016214504000001556.
   Web of Science ®Google Scholar
• Kidger, P., and Lyons, T. (2020), “Universal Approximation with Deep Narrow Networks,” in Conference on Learning Theory, pp. 2306–2327, PMLR.
   Google Scholar
• Kim, J. S., Staicu, A.-M., Maity, A., Carroll, R. J., and Ruppert, D. (2018), “Additive Function-on-Function Regression,” Journal of Computational and Graphical Statistics, 27, 234–244. DOI: 10.1080/10618600.2017.1356730.
   PubMed Web of Science ®Google Scholar
• Leshno, M., Lin, V. Y., Pinkus, A., and Schocken, S. (1993), “Multilayer Feedforward Networks with a Nonpolynomial Activation Function Can Approximate any Function,” Neural Networks, 6, 861–867. DOI: 10.1016/S0893-6080(05)80131-5.
   Web of Science ®Google Scholar
• Liang, H., Wu, H., and Carroll, R. J. (2003), “The Relationship between Virologic and Immunologic Responses in Aids Clinical Research Using Mixed-Effects Varying-Coefficient Models with Measurement Error,” Biostatistics, 4, 297–312. DOI: 10.1093/biostatistics/4.2.297.
   PubMed Web of Science ®Google Scholar
• Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. (2021), “Learning Nonlinear Operators via Deeponet Based on the Universal Approximation Theorem of Operators,” Nature Machine Intelligence, 3, 218–229.
   Google Scholar
• Lu, Z., Pu, H., Wang, F., Hu, Z., and Wang, L. (2017), “The Expressive Power of Neural Networks: A View from the Width,” in Advances in Neural Information Processing Systems (Vol. 30).
   Google Scholar
• Luo, R., and Qi, X. (2017), “Function-on-Function Linear Regression by Signal Compression,” Journal of the American Statistical Association, 112, 690–705. DOI: 10.1080/01621459.2016.1164053.
   Web of Science ®Google Scholar
• Luo, R., and Qi, X. (2021), “Modeling Spiky Functional Data with Derivatives of Smooth Functions in Function-on-Function Regression,” Statistica Sinica (Accepted), available at http://www3.stat.sinica.edu.tw/ss/_newpaper/SS-2020-0294/_na.pdf.
   Google Scholar
• Luo, R., Qi, X., and Wang, Y. (2016), “Functional Wavelet Regression for Linear Function-on-Function Models,” Electronic Journal of Statistics, 10, 3179–3216. DOI: 10.1214/16-EJS1204.
   Web of Science ®Google Scholar
• Pinkus, A. (1999), “Approximation Theory of the mlp Model in Neural Networks,” Acta Numerica, 8, 143–195. DOI: 10.1017/S0962492900002919.
   Google Scholar
• Qi, X., and Luo, R. (2019), “Nonlinear Function-on-Function Additive Model with Multiple Predictor Curves,” Statistica Sinica, 29, 719–739. DOI: 10.5705/ss.202017.0249.
   Web of Science ®Google Scholar
• Radchenko, P., Qiao, X., and James, G. M. (2015), “Index Models for Sparsely Sampled Functional Data,” Journal of the American Statistical Association, 110, 824–836. DOI: 10.1080/01621459.2014.931859.
   Web of Science ®Google Scholar
• Ramsay, J. O., and Silverman, B. W. (2005), Functional Data Analysis (2nd ed.), New York: Springer.
   Google Scholar
• Reimherr, M., Sriperumbudur, B., and Taoufik, B. (2018), “Optimal Prediction for Additive Function-on-Function Regression,” Electronic Journal of Statistics, 12, 4571–4601. DOI: 10.1214/18-EJS1505.
   Web of Science ®Google Scholar
• Rossi, F., Conan-Guez, B., and Fleuret, F. (2002), “Functional Data Analysis with Multi Layer Perceptrons,” in Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN’02 (Cat. No. 02CH37290) (Vol. 3), pp. 2843–2848, IEEE.
   Google Scholar
• Scheipl, F., Staicu, A.-M., and Greven, S. (2015), “Functional Additive Mixed Models,” Journal of Computational and Graphical Statistics, 24, 477–501. DOI: 10.1080/10618600.2014.901914.
   PubMed Web of Science ®Google Scholar
• Schmidt-Hieber, J. (2020), “Nonparametric Regression Using Deep Neural Networks with Relu Activation Function,” Annals of Statistics, 48, 1875–1897.
   Web of Science ®Google Scholar
• Wang, Q., Wang, H., Gupta, C., Rao, A. R., and Khorasgani, H. (2020), “A Non-linear Function-on-Function Model for Regression with Time Series Data,” in 2020 IEEE International Conference on Big Data (Big Data), pp. 232–239, IEEE. DOI: 10.1109/BigData50022.2020.9378087.
   Google Scholar
• Wu, C. O., Chiang, C.-T., and Hoover, D. R. (1998), “Asymptotic Confidence Regions for Kernel Smoothing of a Varying-Coefficient Model with Longitudinal Data,” Journal of the American statistical Association, 93, 1388–1402. DOI: 10.1080/01621459.1998.10473800.
   Web of Science ®Google Scholar
• Yao, F., Müller, H.-G., and Wang, J.-L. (2005), “Functional Linear Regression Analysis for Longitudinal Data,” The Annals of Statistics, 33, 2873–2903. DOI: 10.1214/009053605000000660.
   Web of Science ®Google Scholar
• Yarotsky, D. (2017), “Error Bounds for Approximations with Deep Relu Networks,” Neural Networks, 94, 103–114. DOI: 10.1016/j.neunet.2017.07.002.
   PubMed Web of Science ®Google Scholar