A Scalable Method to Exploit Screening in Gaussian Process Models with Noise

References

• Ambikasaran, S., Foreman-Mackey, D., Greengard, L., Hogg, D., and O’Neil, M. (2016), “Fast Direct Methods for Gaussian Processes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 252–265. DOI: 10.1109/TPAMI.2015.2448083.
   PubMed Web of Science ®Google Scholar
• Anitescu, M., Chen, J., and Wang, L. (2012), “A Matrix-Free Approach for Solving the Parametric Gaussian Process Maximum Likelihood Problem,” SIAM Journal on Scientific Computing, 34, A240–A262. DOI: 10.1137/110831143.
   Web of Science ®Google Scholar
• Avron, H., and Toledo, S. (2011), “Randomized Algorithms for Estimating the Trace of an Implicit Symmetric Positive Semi-Definite Matrix,” Journal of the ACM, 58, 8:1–8:34. DOI: 10.1145/1944345.1944349.
   Web of Science ®Google Scholar
• Beckman, P. G., Geoga, C. J., Stein, M. L., and Anitescu, M. (2022), “Scalable Computations for Nonstationary Gaussian Processes,” Technical Report arXiv:2206.05220, arXiv. arXiv:2206.05220 [stat] type: article.
   Google Scholar
• Chen, J., and Stein, M. L. (2021), “Linear-Cost Covariance Functions for Gaussian Random Fields,” Journal of the American Statistical Association, 118, 147–164. DOI: 10.1080/01621459.2021.1919122.
   Web of Science ®Google Scholar
• Chen, T., and Hallman, E. (2022), “Krylov-Aware Stochastic Trace Estimation,” Technical Report arXiv:2205.01736, arXiv. arXiv:2205.01736 [cs, math] type: article.
   Google Scholar
• Cressie, N., and Johannesson, G. (2008), “Fixed Rank Kriging for Very Large Spatial Data Sets,” Journal of the Royal Statistical Society, Series B, 70, 209–226. DOI: 10.1111/j.1467-9868.2007.00633.x.
   Google Scholar
• Datta, A., Banerjee, S., Finley, A. O., and Gelfand, A. E. (2016), “Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets,” Journal of the American Statistical Association, 111, 800–812. DOI: 10.1080/01621459.2015.1044091.
   PubMed Web of Science ®Google Scholar
• Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), “Maximum Likelihood from Incomplete Data via the EM Algorithm,” Journal of the Royal Statistical Society, Series B, 39, 1–22. DOI: 10.1111/j.2517-6161.1977.tb01600.x.
   Google Scholar
• Finley, A. O., Datta, A., Cook, B. D., Morton, D. C., Andersen, H. E., and Banerjee, S. (2019), “Efficient Algorithms for Bayesian Nearest Neighbor Gaussian Processes,” Journal of Computational and Graphical Statistics, 28, 401–414. DOI: 10.1080/10618600.2018.1537924.
   PubMed Web of Science ®Google Scholar
• Geoga, C. J., Anitescu, M., and Stein, M. L. (2020), “Scalable Gaussian Process Computations Using Hierarchical Matrices,” Journal of Computational and Graphical Statistics, 29, 227–237. DOI: 10.1080/10618600.2019.1652616.
   Web of Science ®Google Scholar
• Geoga, C. J., Anitescu, M., and Stein, M. L. (2021), “Flexible Nonstationary Spatiotemporal Modeling of High-Frequency Monitoring Data,” Environmetrics, 32, e2670.
   Web of Science ®Google Scholar
• Geoga, C. J., Marin, O., Schanen, M., and Stein, M. L. (2023), “Fitting Matérn Smoothness Parameters Using Automatic Differentiation,” Statistics and Computing, 33, 48. DOI: 10.1007/s11222-022-10127-w.
   Web of Science ®Google Scholar
• Girolami, M., Febrianto, E., Yin, G., and Cirak, F. (2021), “The Statistical Finite Element Method (Statfem) for Coherent Synthesis of Observation Data and Model Pedictions,” Computer Methods in Applied Mechanics and Engineering, 375, 113533. DOI: 10.1016/j.cma.2020.113533.
   Web of Science ®Google Scholar
• Gu, M., Wang, X., and Berger, J. O. (2018), “Robust Gaussian Stochastic Process Emulation,” The Annals of Statistics, 46, 3038–3066. DOI: 10.1214/17-AOS1648.
   Web of Science ®Google Scholar
• Guinness, J. (2018), “Permutation and Grouping Methods for Sharpening Gaussian Process Approximations,” Technometrics, 60, 415–429. DOI: 10.1080/00401706.2018.1437476.
   PubMed Web of Science ®Google Scholar
• Guinness, J. (2021), “Gaussian Process Learning via Fisher Scoring of Vecchia’s Approximation,” Statistics and Computing, 31, 1–8.
   Web of Science ®Google Scholar
• Hutchinson, M. F. (1990), “A Stochastic Estimator of the Trace of the Influence Matrix for Laplacian Smoothing Splines,” Communications in Statistics – Simulation and Computation, 19, 433–450. DOI: 10.1080/03610919008812866.
   Web of Science ®Google Scholar
• Katzfuss, M., and Cressie, N. (2012), “Bayesian Hierarchical Spatio-Temporal Smoothing for Very Large Datasets,” Environmetrics, 23, 94–107. DOI: 10.1002/env.1147.
   Web of Science ®Google Scholar
• Katzfuss, M., and Guinness, J. (2021), “A General Framework for Vecchia Approximations of Gaussian Processes,” Statistical Science, 36, 124–141. DOI: 10.1214/19-STS755.
   Web of Science ®Google Scholar
• Kaufman, C. G., Schervish, M. J., and Nychka, D. W. (2008), “Covariance Tapering for Likelihood-based Estimation in Large Spatial Data Sets,” Journal of the American Statistical Association, 103, 1545–1555. DOI: 10.1198/016214508000000959.
   Web of Science ®Google Scholar
• Lin, L., Lu, J., and Ying, L. (2011), “Fast Construction of Hierarchical Matrix Representation from Matrix-Vector Multiplication,” Journal of Computational Physics, 230, 4071–4087. DOI: 10.1016/j.jcp.2011.02.033.
   Web of Science ®Google Scholar
• Lindgren, F., Rue, H., and Lindström, J. (2011), “An Explicit Link Between Gaussian Fields and Gaussian Markov Random Fields: The Stochastic Partial Differential Equation Approach,” Journal of the Royal Statistical Society, Series B, 73, 423–498. DOI: 10.1111/j.1467-9868.2011.00777.x.
   Google Scholar
• Litvinenko, A., Sun, Y., Genton, M., and Keyes, D. (2019), “Likelihood Approximation with Hierarchical Matrices for Large Spatial Datasets,” Computational Statistics & Data Analysis, 137, 115–132. DOI: 10.1016/j.csda.2019.02.002.
   Web of Science ®Google Scholar
• Liu, C., and Rubin, D. (1994), “The ECME Algorithm: A Simple Extension of EM and ECM with Faster Monotone Convergence,” Biometrika, 81, 633–648. DOI: 10.1093/biomet/81.4.633.
   Web of Science ®Google Scholar
• Meyer, R. A., Musco, C., Musco, C., and Woodruff, D. P. (2021), “Hutch++: Optimal Stochastic Trace Estimation,” in Proceedings of Symposium on Simplicity in Algorithms (SOSA), pp. 142–155, Society for Industrial and Applied Mathematics.
   Google Scholar
• Minden, V., Damle, A., Ho, K., and Ying, L. (2017), “Fast Spatial Gaussian Process Maximum Likelihood Estimation via Skeletonization Factorizations,” Multiscale Modeling & Simulation, 15, 1584–1611. DOI: 10.1137/17M1116477.
   Google Scholar
• Mockus, J. (2012), Bayesian Approach to Global Optimization: Theory and Applications, Dordrecht: Springer.
   Google Scholar
• Muradyan, P., and Coulter, R. L. (2020), Radar Wind Profiler (RWP) and Radio Acoustic Sounding System (RASS) Instrument Handbook (DOE/SC-ARM-TR-044). DOE Office of Science Atmospheric Radiation Measurement (ARM) Program, United States.
   Google Scholar
• Murray, I. (2016), “Differentiation of the Cholesky Decomposition,” arXiv preprint arXiv:1602.07527.
   Google Scholar
• Neal, R. M., and Hinton, G. E. (1998), “A View of the EM Algorithm that Justifies Incremental, Sparse, and other Variants,” in Learning in Graphical Models, pp. 355–368, Springer.
   Google Scholar
• Newsom, R. K. (2012), “Doppler lidar (Dl) Handbook,” Technical Report DOE/SC-ARM/TR-101, DOE Office of Science Atmospheric Radiation Measurement (ARM) Program (United States).
   Google Scholar
• Persson, D., Cortinovis, A., and Kressner, D. (2022), “Improved Variants of the Hutch++ Algorithm for Trace Estimation,” SIAM Journal on Matrix Analysis and Applications, 43, 1162–1185. DOI: 10.1137/21M1447623.
   Web of Science ®Google Scholar
• Revels, J., Lubin, M., and Papamarkou, T. (2016), “Forward-Mode Automatic Differentiation in Julia,” arXiv:1607.07892 [cs.MS].
   Google Scholar
• Roosta-Khorasani, F., and Ascher, U. (2015), “Improved Bounds on Sample Size for Implicit Matrix Trace Estimators,” Foundations of Computational Mathematics, 15, 1187–1212. DOI: 10.1007/s10208-014-9220-1.
   Web of Science ®Google Scholar
• Rue, H., and Held, L. (2005), Gaussian Markov Random Fields: Theory and Applications, New York: Chapman and Hall/CRC.
   Google Scholar
• Saad, Y. (2003), Iterative Methods for Sparse Linear Systems, Philadelphia, PA: SIAM.
   Google Scholar
• Saha, A., and Datta, A. (2018), “Brisc: Bootstrap for Rapid Inference on Spatial Covariances,” Stat, 7, e184. DOI: 10.1002/sta4.184.
   Web of Science ®Google Scholar
• Schäfer, F., Katzfuss, M., and Owhadi, H. (2021), “Sparse Cholesky Factorization by Kullback–Leibler Minimization,” SIAM Journal on Scientific Computing, 43, A2019–A2046. DOI: 10.1137/20M1336254.
   Web of Science ®Google Scholar
• Stein, M. L. (1999), Interpolation of Spatial Data: Some Theory for Kriging, New York: Springer.
   Google Scholar
• Stein, M. L. (2002), “The Screening Effect in Kriging,” The Annals of Statistics, 30, 298–323.
   Web of Science ®Google Scholar
• Stein, M. L. (2011), “2010 Rietz Lecture: When Does the Screening Effect Hold?” The Annals of Statistics, 39, 2795–2819.
   PubMed Web of Science ®Google Scholar
• Stein, M. L. (2022), “Local Variogram Models with Negative Inverse “Range” Parameters,” Spatial Statistics, 48, 100633.
   Web of Science ®Google Scholar
• Stein, M. L., Chen, J., and Anitescu, M. (2013), “Stochastic Approximation of Score Functions for Gaussian Processes,” The Annals of Applied Statistics, 7, 1162–1191. DOI: 10.1214/13-AOAS627.
   Web of Science ®Google Scholar
• Stein, M. L., Chi, Z., and Welty, L. J. (2004), “Approximating Likelihoods for Large Spatial Data Sets,” Journal of the Royal Statistical Society, Series B, 66, 275–296. DOI: 10.1046/j.1369-7412.2003.05512.x.
   Google Scholar
• Stokes, G. M., and Schwartz, S. E. (1994), “The Atmospheric Radiation Measurement (ARM) Program: Programmatic Background and Design of the Cloud and Radiation Test Bed,” Bulletin of the American Meteorological Society, 75, 1201–1222. DOI: 10.1175/1520-0477(1994)075<1201:TARMPP>2.0.CO;2.
   Web of Science ®Google Scholar
• Stroud, J. R., Müller, P., and Sansó, B. (2001), “Dynamic Models for Spatiotemporal Data,” Journal of the Royal Statistical Society, Series B, 63, 673–689. DOI: 10.1111/1467-9868.00305.
   Google Scholar
• Sun, Y., and Stein, M. L. (2016), “Statistically and Computationally Efficient Estimating Equations for Large Spatial Datasets,” Journal of Computational and Graphical Statistics, 25, 187–208. DOI: 10.1080/10618600.2014.975230.
   Web of Science ®Google Scholar
• Ubaru, S., Chen, J., and Saad, Y. (2017), “Fast Estimation of tr(f(A)) via Stochastic Lanczos Quadrature,” SIAM Journal on Matrix Analysis and Applications, 38, 1075–1099.
   Web of Science ®Google Scholar
• Vecchia, A. V. (1988), “Estimation and Model Identification for Continuous Spatial Processes,” Journal of the Royal Statistical Society, Series B, 50, 297–312. DOI: 10.1111/j.2517-6161.1988.tb01729.x.
   Google Scholar
• Wikle, C. K., and Cressie, N. (1999), “A Dimension-Reduced Approach to Space-Time Kalman Filtering,” Biometrika, 86, 815–829. DOI: 10.1093/biomet/86.4.815.
   Web of Science ®Google Scholar
• Wächter, A., and Biegler, L. T. (2006), “On the Implementation of an Interior-Point Filter Line-Search Algorithm for Large-Scale Nonlinear Programming,” Mathematical Programming, 106, 25–57. DOI: 10.1007/s10107-004-0559-y.
   Web of Science ®Google Scholar