Ultra-Fast Approximate Inference Using Variational Functional Mixed Models

References

• Blei, D. M. (2006), “Variational Inference for Dirichlet Process,” Bayesian Analysis, 1, 121–144. DOI: 10.1214/06-BA104.
   Web of Science ®Google Scholar
• Blei, D. M., Kucukelbir, A., and McAuliffe, D. (2017), “Variational Inference: A Review for Statisticians,” Journal of the American Statistical Association, 112, 859–877. DOI: 10.1080/01621459.2017.1285773.
   Web of Science ®Google Scholar
• Cardot, H., Ferraty, F., and Sarda, P. (1999), “Functional Linear Model,” Statistics & Probability Letters, 45, 11–22.
   Web of Science ®Google Scholar
• Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010, April), “The Horseshoe Estimator for Sparse Signals,” Biometrika, 97, 465–480. DOI: 10.1093/biomet/asq017.
   Web of Science ®Google Scholar
• Cuevas, A., Febrero, M., and Fraiman, R. (2002), “Linear Functional Regression: The Case of Fixed Design and Functional Response,” Canadian Journal of Statistics, 30, 285–300. DOI: 10.2307/3315952.
   Web of Science ®Google Scholar
• Earls, C., and Hooker, G. (2017a), “Combining Functional Data Registration and Factor Analysis,” Journal of Computational and Graphical Statistics, 26, 296–305. DOI: 10.1080/10618600.2016.1164709.
   Web of Science ®Google Scholar
• Earls, C., and Hooker, G. (2017b), “Variational Bayes for Functional Data Registration, Smoothing, and Prediction,” Bayesian Analysis, 12, 557–582.
   Web of Science ®Google Scholar
• Faes, C., Ormerod, J. T., and Wand, M. P. (2011), “Variational Bayesian Inference for Parametric and Nonparametric Regression with Missing Data,” Journal of the American Statistical Association, 106, 959–971. DOI: 10.1198/jasa.2011.tm10301.
   Web of Science ®Google Scholar
• Fan, Y. (2003), “On the Approximate Decorrelation Property of the Discrete Wavelet Transform for Fractionally Differenced Processes,” IEEE Transactions on Information Theory, 49, 516–521.
   Web of Science ®Google Scholar
• Fan, Y., James, G. M., and Radchenko, P. (2015, October), “Functional Additive Regression,” The Annals of Statistics, 43, 2296–2325. DOI: 10.1214/15-AOS1346.
   Web of Science ®Google Scholar
• Ferraty, F., and Romain, Y., eds. (2018, August), The Oxford Handbook of Functional Data Analysis, Oxford: Oxford University Press.
   Google Scholar
• Frackowiak, R., Friston, K., Frith, C., Dolan, R., Price, C., Zeki, S., Ashburner, J., and Penny, W. (2003), Human Brain Function (2nd ed.), London: Academic Press.
   Google Scholar
• Giordano, R. J., Broderick, T., and Jordan, M. I. (2015), “Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes,” in Neural Information Processing Systems.
   Google Scholar
• Goldsmith, J., Wand, M. P., and Crainiceanu, C. (2011), “Functional Regression via Variational Bayes,” Electronic Journal of Statistics, 5, 572–602. DOI: 10.1214/11-ejs619.
   PubMed Web of Science ®Google Scholar
• Griffin, J. E., and Brown, P. J. (2005, May), “Alternative Prior Distributions for Variable Selection with Very Many More Variables than Observations,” Technical Report, University of Warwick, Centre for Research in Statistical Methodology.
   Google Scholar
• Guo, W. (2002), “Functional Mixed Effects Models,” Biometrics, 58, 121–128. DOI: 10.1111/j.0006-341x.2002.00121.x.
   PubMed Web of Science ®Google Scholar
• Hall, P., and Horowitz, J. L. (2007), “Methodology and Convergence Rates for Functional Linear Regression,” The Annals of Statistics, 35, 70–91. DOI: 10.1214/009053606000000957.
   Web of Science ®Google Scholar
• Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J. (2013), “Stochastic Variational Inference,” Journal of Machine Learning Research, 14, 1303–1347.
   Web of Science ®Google Scholar
• Hui, F. K. C., You, C., Shang, H. L., and Müller, S. (2019), “Semiparametric Regression using Variational Approximations,” Journal of the American Statistical Association, 114, 1765–1777. DOI: 10.1080/01621459.2018.1518235.
   Web of Science ®Google Scholar
• Ishwaran, H., and Rao, J. S. (2005), “Spike and Slab Variable Selection: Frequentist and Bayesian Strategies,” The Annals of Statistics, 33, 730–773. DOI: 10.1214/009053604000001147.
   Web of Science ®Google Scholar
• James, G. M. (2002), “Generalized Linear Models with Functional Predictors,” Journal of the Royal Statistical Society, Series B, 64, 411–432. DOI: 10.1111/1467-9868.00342.
   Google Scholar
• Johnson, V. E., and Rossell, D. (2010), “On the Use of Non-local Prior Densities in Bayesian Hypothesis Tests,” Journal of the Royal Statistical Society, Series B, 72, 143–170. DOI: 10.1111/j.1467-9868.2009.00730.x.
   Google Scholar
• Lancia, L., Rausch, P., and Morris, J. S. (2015), “Automated Quantitative Analysis of Ultrasound Tongue Contours via Wavelet-based Functional Mixed Models,” Journal of the Acoustical Society of America, 137, EL178–EL183. DOI: 10.1121/1.4905881.
   PubMed Web of Science ®Google Scholar
• Lee, W., Miranda, M. F., Rausch, P., Baladandayuthapani, V., Fazio, M., Downs, J. C., and Morris, J. S. (2019), “Bayesian Semiparametric Functional Mixed Models for Serially Correlated Functional Data, with Application to Glaucoma Data,” Journal of the American Statistical Association, 114, 495–513. DOI: 10.1080/01621459.2018.1476242.
   PubMed Web of Science ®Google Scholar
• Luts, J., Broderick, T., and Wand, M. P. (2014), “Real-Time Semiparametric Regression,” Journal of Computational and Graphical Statistics, 23, 589–615. DOI: 10.1080/10618600.2013.810150.
   Web of Science ®Google Scholar
• Luts, J., and Wand, M. P. (2015), “Variational Inference for Count Response Semiparametric Regression,” Bayesian Analysis, 10, 991–1023. DOI: 10.1214/14-BA932.
   Web of Science ®Google Scholar
• Martinez, J. G., Bohn, K. M., Carroll, R. J., and Morris, J. S. (2013), “A Study of Mexican Free-Tailed Bat Chirp Syllables: Bayesian Functional Mixed Models for Nonstationary Acoustic Time Series,” Journal of the American Statistical Association, 108, 514–526. DOI: 10.1080/01621459.2013.793118.
   PubMed Web of Science ®Google Scholar
• Mason, J. C., Rodriguez, G., and Seatzu, S. (1993), “Orthogonal Splines based on B-splines—with Applications to Least Squares, Smoothing and Regularisation Problems,” Numerical Algorithms, 5, 25–40. DOI: 10.1007/BF02109281.
   Google Scholar
• McLean, M. W., Hooker, G., Staicu, A.-M., Scheipl, F., and Ruppert, D. (2014), “Functional Generalized Additive Models,” Journal of Computational and Graphical Statistics, 23, 249–269. DOI: 10.1080/10618600.2012.729985.
   PubMed Web of Science ®Google Scholar
• McLean, M. W., Scheipl, F., Hooker, G., Greven, S., and Ruppert, D. (2017), “Bayesian Functional Generalized Additive Models with Sparsely Observed Covariates.”
   Google Scholar
• Meyer, M. J., Coull, B. A., Versace, F., Cinciripini, P., and Morris, J. S. (2015), “Bayesian Function-on-Function Regression for Multilevel Functional Data,” Biometrics, 71, 563–74. DOI: 10.1111/biom.12299.
   PubMed Web of Science ®Google Scholar
• Minka, T. P. (2001), “Expectation Propagation for Approximate Bayesian Inference,” in Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, UAI’01, San Francisco, CA, USA, pp. 362–369. Morgan Kaufmann Publishers Inc.
   Google Scholar
• Morris, J. S. (2015), “Functional Regression,” Annual Review of Statistics and Its Application, 2, 321–359. DOI: 10.1146/annurev-statistics-010814-020413.
   Web of Science ®Google Scholar
• Morris, J. S. (2017), “Comparison and Contrast of Two General Functional Regression Modeling Frameworks,” Statistical Modeling, 17, 59–85.
   PubMed Web of Science ®Google Scholar
• Morris, J. S., Arroyo, C., Coull, B. A., Louise, M. R., Herrick, R., and Gortmaker, S. (2006), “Using Wavelet-based Functional Mixed Models to Characterize Population Heterogeneity in Accelerometer Profiles: A Case Study,” Journal of the American Statistical Association, 101, 1352–1364. DOI: 10.1198/016214506000000465.
   PubMed Web of Science ®Google Scholar
• Morris, J. S., Baladandayuthapani, V., Herrick, R. C., Sanna, P., and Gutstein, H. (2011), “Automated Analysis of Quantitative Image Data using Isomorphic Functional Mixed Models, with Application to Proteomics Data,” Annals of Applied Statistics, 5, 894–923.
   PubMed Web of Science ®Google Scholar
• Morris, J. S., Brown, P. J., Herrick, R. C., Baggerly, K. A., and Coombes, K. R. (2008), “Bayesian Analysis of Mass Spectrometry Proteomics Data using Wavelet based Functional Mixed Models,” Biometrics, 64, 479–489. DOI: 10.1111/j.1541-0420.2007.00895.x.
   PubMed Web of Science ®Google Scholar
• Morris, J. S., and Carroll, R. J. (2006), “Wavelet-based Functional Mixed Models,” Journal of the Royal Statistical Society, Series B, 68, 179–199. DOI: 10.1111/j.1467-9868.2006.00539.x.
   Google Scholar
• Morris, J. S., Coombes, K. R., Koomen, J., Baggerly, K. A., and Kobayashi, R. (2005), “Feature Extraction and Quantification for Mass Spectrometry in Biomedical Applications using the Mean Spectrum,” Bioinformatics, 21, 1764–1775. DOI: 10.1093/bioinformatics/bti254.
   PubMed Web of Science ®Google Scholar
• Müller, H.-G., and Stadtmüller, U. (2005, April), “Generalized Functional Linear Models,” The Annals of Statistics, 33, 774–805. DOI: 10.1214/009053604000001156.
   Web of Science ®Google Scholar
• Ormerod, J. T., and Wand, M. P. (2012), “Gaussian Variational Approximate Inference for Generalized Linear Mixed Models,” Journal of Computational and Graphical Statistics, 21, 2–17. DOI: 10.1198/jcgs.2011.09118.
   Web of Science ®Google Scholar
• Ormerod, J. T., You, C., and Müller, S. (2017), “A Variational Bayes Approach to Variable Selection,” Electronic Journal of Statistics, 11, 3549–3594. DOI: 10.1214/17-EJS1332.
   Web of Science ®Google Scholar
• Park, T., and Casella, G. (2008), “The Bayesian Lasso,” Journal of the American Statistical Association, 103, 681–686. DOI: 10.1198/016214508000000337.
   Web of Science ®Google Scholar
• Ramsay, J. O., and Silverman, B. W. (2005), Functional Data Analysis (Section ed.), New York: Springer.
   Google Scholar
• Ray, K., and Szabó, B. (2021), “Variational Bayes for High-Dimensional Linear Regression with Sparse Priors,” Journal of the American Statistical Association, 1–12.
   PubMed Web of Science ®Google Scholar
• Ruli, E., Sartori, N., and Ventura, L. (2016), “Improved Laplace Approximation for Marginal Likelihoods,” Electronic Journal of Statistics, 10, 3986–4009. DOI: 10.1214/16-EJS1218.
   Web of Science ®Google Scholar
• Sarlos, T. (2006, October), “Improved Approximation Algorithms for Large Matrices via Random Projections,” in 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS’06), pp. 143–152.
   Google Scholar
• Searle, S. R., Casella, G., and McCulloch, C. E. (1992), Variance Components, New York: Wiley.
   Google Scholar
• Serra, J. G., Mateos, J., Molina, R., and Katsaggelos, A. K. (2019), “Variational em Method for Blur Estimation using the Spike-and-Slab Image Prior,” Digital Signal Processing, 88, 116–129. DOI: 10.1016/j.dsp.2019.01.004.
   Web of Science ®Google Scholar
• Stein, J., Hua, X., Lee, S., Ho, A., Leow, A., Toga, A., Saykin, A., Shen, L., Foroud, T., Pankratz, N., Huentelman, M., Craig, D., Gerber, J., Allen, A., Corneveaux, J., Dechairo, B., Potkin, S., Weiner, M., and Thompson, P. (2010), “Voxelwise Genome-Wide Association Study (vgwas),” NeuroImage, 53, 1160–1174. DOI: 10.1016/j.neuroimage.2010.02.032.
   PubMed Web of Science ®Google Scholar
• Sun, S. (2013), “A Review of Deterministic Approximate Inference Techniques for Bayesian Machine Learning,” Neural Computing and Applications, 23, 2039–2050. DOI: 10.1007/s00521-013-1445-4.
   Web of Science ®Google Scholar
• Wang, C., Chen, M.-H., Schifano, E., Wu, J., and Yan, J. (2016), “Statistical Methods and Computing for Big Data,” Stat Interface, 9, 399–414. DOI: 10.4310/SII.2016.v9.n4.a1.
   PubMed Web of Science ®Google Scholar
• Wang, J.-L., Chiou, J.-M., and Müller, H.-G. (2016), “Functional Data Analysis,” Annual Review of Statistics and Its Application, 3, 257–295. DOI: 10.1146/annurev-statistics-041715-033624.
   Web of Science ®Google Scholar
• Wikipedia contributors. (2019), “Embarrassingly parallel — Wikipedia, the free encyclopedia,” Accessed October 17, 2019.
   Google Scholar
• Yang, H., Baladandayuthapani, V., Rao, A. U. K., and Morris, J. S. (2020), “Quantile Function on Scalar Regression Analysis for Distributional Data,” Journal of the American Statistical Association, 115, 90–106. DOI: 10.1080/01621459.2019.1609969.
   PubMed Web of Science ®Google Scholar
• Yao, F., Müller, H.-G., and Wang, J.-L. (2005), “Functional Linear Regression Analysis for Longitudinal Data,” The Annals of Statistics, 33, 2873–2903. DOI: 10.1214/009053605000000660.
   Web of Science ®Google Scholar
• Yuan, M., and Cai, T. T. (2010, 12), “A Reproducing Kernel Hilbert Space Approach to Functional Linear Regression,” The Annals of Statistics, 38, 3412–3444. DOI: 10.1214/09-AOS772.
   Web of Science ®Google Scholar
• Zhang, C.-X., Xu, S., and Zhang, J.-S. (2019), “A Novel Variational Bayesian Method for Variable Selection in Logistic Regression Models,” Computational Statistics & Data Analysis, 133, 1–19.
   Web of Science ®Google Scholar
• Zhang, L., Baladandayuthapani, V., Zhu, H., Baggerly, K. A., Majewski, T., Czerniak, B. A., and Morris, J. S. (2016), “Functional Car Models for Large Spatially Correlated Functional Datasets,” Journal of the American Statistical Association, 111, 772–786. DOI: 10.1080/01621459.2015.1042581.
   PubMed Web of Science ®Google Scholar
• Zhou, L., Huang, J. Z., Martinez, J. G., Maity, A., Baladandayuthapani, V., and Carroll, R. J. (2010), “Reduced Rank Mixed Effects Models for Spatially Correlated Hierarchical Functional Data,” Journal of the American Statistical Association, 105, 390–400. DOI: 10.1198/jasa.2010.tm08737.
   PubMed Web of Science ®Google Scholar
• Zhu, H., Brown, P. J., and Morris, J. S. (2011), “Robust, Adaptive Functional Regression in Functional Mixed Model Framework,” Journal of the American Statistical Association, 495, 1167–1179. DOI: 10.1198/jasa.2011.tm10370.
   Web of Science ®Google Scholar
• Zhu, H., Brown, P. J., and Morris, J. S. (2012), “Robust Classification of Functional and Quantitative Image Data using Functional Mixed Models,” Biometrics, 68, 1260–1268.
   PubMed Web of Science ®Google Scholar
• Zhu, H., and Cox, D. D. (2009), “A Functional Generalized Linear Model with Curve Selection in Cervical Pre-cancer Diagnosis using Fluorescence Spectroscopy,” in Optimality: The Third Erich L. Lehmann Symposium, volume 57 of Lecture Notes Monograph Series, pp. 173–189, Beachwood, OH: Institute of Mathematical Statistics.
   Google Scholar
• Zhu, H., Morris, J. S., Wei, F., and Cox, D. D. (2017), “Multivariate Functional Response Regression, with Application to Fluorescence Spectroscopy in a Cervical Pre-Cancer Study,” Computational Statistics and Data Analysis, 111, 88–101. DOI: 10.1016/j.csda.2017.02.004.
   PubMed Web of Science ®Google Scholar
• Zhu, H., Versaceb, F., Cinciripini, P. M., Rausch, P., and Morris, J. S. (2018), “Robust and Gaussian Spatial Functional Regression Models for Analysis of Event-Related Potentials,” NeuroImage, 181, 501–512. DOI: 10.1016/j.neuroimage.2018.07.006.
   PubMed Web of Science ®Google Scholar