Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations

References

• Andrieu, C., and Roberts, G. O. (2009), “The Pseudo-Marginal Approach for Efficient Monte Carlo Computations,” The Annals of Statistics, 37, 697–725.
   Web of Science ®Google Scholar
• Atkeson, C. G., Moore, A. W., and Schaal, S. (1997), “Locally Weighted Learning,” Artificial Intelligence Review, 11, 11–73.
   Web of Science ®Google Scholar
• Bal, G., Langmore, I., and Marzouk, Y. M. (2013), “Bayesian Inverse Problems With Monte Carlo Forward Models,” Inverse Problems and Imaging, 7, 81–105.
   Web of Science ®Google Scholar
• Bliznyuk, N., Ruppert, D., and Shoemaker, C. A. (2012), “Local Derivative-Free Approximation of Computationally Expensive Posterior Densities,” Journal of Computational and Graphical Statistics, 21, 476–495.
   Web of Science ®Google Scholar
• Bliznyuk, N., Ruppert, D., Shoemaker, C., Regis, R., Wild, S., and Mugunthan, P. (2008), “Bayesian Calibration and Uncertainty Analysis for Computationally Expensive Models Using Optimization and Radial Basis Function Approximation,” Journal of Computational and Graphical Statistics, 17, 270–294.
   Web of Science ®Google Scholar
• Chakraborty, A., Mallick, B. K., Mcclarren, R. G., Kuranz, C. C., Bingham, D., Grosskopf, M. J., Rutter, E. M., Stripling, H. F., and Drake, R. P. (2013), “Spline-Based Emulators for Radiative Shock Experiments With Measurement Error,” Journal of the American Statistical Association, 108, 411–428.
   Web of Science ®Google Scholar
• Christen, J. A., and Fox, C. (2005), “Markov Chain Monte Carlo Using an Approximation,” Journal of Computational and Graphical Statistics, 14, 795–810.
   Web of Science ®Google Scholar
• Cleveland, W. S. (1979), “Robust Locally Weighted Regression and Smoothing Scatterplots,” Journal of the American Statistical Association, 74, 829–836.
   Web of Science ®Google Scholar
• Cleveland, W. S., and Loader, C. (1996), “Smoothing by Local Regression: Principles and Methods,” in Statistical Theory and Computational Aspects of Smoothing (Vol. 1049), eds. W. Haerdle and M. G. Schimek, New York: Springer, pp. 10–49.
   Google Scholar
• Conn, A. R., Gould, N. I. M., and Toint, P. L. (2000), Trust Region Methods, Philadelphia, PA: SIAM.
   Google Scholar
• Conn, A. R., Scheinberg, K., and Vicente, L. N. (2009), Introduction to Derivative-Free Optimization, Philadelphia, PA: SIAM.
   Google Scholar
• Conrad, P. R. (2014), “Accelerating Bayesian Inference in Computationally Expensive Computer Models Using Local and Global Approximations,” Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, MA.
   Google Scholar
• Conrad, P. R., and Marzouk, Y. M. (2013), “Adaptive Smolyak Pseudospectral Approximations,” SIAM Journal of Scientific Computing, 35, A2643–A2670.
   Web of Science ®Google Scholar
• Constantine, P. G., Eldred, M. S., and Phipps, E. T. (2012), “Sparse Pseudospectral Approximation Method,” Computer Methods in Applied Mechanics and Engineering, 229–232, 1–30.
   Web of Science ®Google Scholar
• Cotter, S. L., Dashti, M., and Stuart, A. M. (2010), “Approximation of Bayesian Inverse Problems,” SIAM Journal of Numerical Analysis, 48, 322–345.
   Web of Science ®Google Scholar
• Cotter, S. L., Roberts, G. O., Stuart, A. M., and White, D. (2013), “MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster,” Statistical Science, 28, 424–446.
   Web of Science ®Google Scholar
• Cressie, N. (1991), Statistics for Spatial Data (revised ed.), New York: Wiley.
   Google Scholar
• Cui, T., Fox, C., and O’Sullivan, M. J. (2011), “Bayesian Calibration of a Large-scale Geothermal Reservoir Model by a New Adaptive Delayed Acceptance Metropolis Hastings Algorithm,” Water Resources Research, 47, W10521. Available at http://onlinelibrary.wiley.com/doi/10.1029/2010WR010352/full
   Web of Science ®Google Scholar
• Cui, T., Marzouk, Y. M., and Willcox, K. E. (2014), “Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems,” International Journal for Numerical Methods in Engineering, 102, 966–990.
   Web of Science ®Google Scholar
• Dashti, M., and Stuart, A. (2011), “Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem,” SIAM Journal of Numerical Analysis, 49, 2524–2542.
   Web of Science ®Google Scholar
• Efendiev, Y., Hou, T., and Luo, W. (2006), “Preconditioning Markov Chain Monte Carlo Simulations Using Coarse-Scale Models,” SIAM Journal on Scientific Computing, 28, 776–803.
   Web of Science ®Google Scholar
• Fielding, M., Nott, D. J., and Liong, S.-Y. (2011), “Efficient MCMC Schemes for Computationally Expensive Posterior Distributions,” Technometrics, 53, 16–28.
   Web of Science ®Google Scholar
• Fort, G., Moulines, E., and Priouret, P. (2012), “Convergence of Adaptive and Interacting Markov Chain Monte Carlo Algorithms,” Annals of Statistics, 39, 3262–3289.
   Web of Science ®Google Scholar
• Frangos, M., Marzouk, Y., Willcox, K., and van Bloemen Waanders, B. (2010), Surrogate and Reduced-Order Modeling: A Comparison of Approaches for Large-Scale Statistical Inverse Problems, New York: Wiley.
   Google Scholar
• Friedman, J. (1991), “Multivariate Adaptive Regression Splines,” The Annals of Statistics, 19, 1–141.
   Web of Science ®Google Scholar
• Gardner, T. S., Cantor, C. R., and Collins, J. J. (2000), “Construction of a Genetic Toggle Switch in Escherichia coli,” Nature, 403, 339–342.
   PubMed Web of Science ®Google Scholar
• Gramacy, R. B., and Apley, D. W. (2015), “Local Gaussian Process Approximation for Large Computer Experiments,” Journal of Computational and Graphical Statistics, 24, 561–578.
   Web of Science ®Google Scholar
• Haario, H., Saksman, E., and Tamminen, J. (2001), “An Adaptive Metropolis Algorithm,” Bernoulli, 7, 223–242.
   Web of Science ®Google Scholar
• Higdon, D., Lee, H., and Holloman, C. (2003), “Markov Chain Monte Carlo-based Approaches for Inference in Computationally Intensive Inverse Problems,” in Bayesian Statistics 7, eds. J. M. Bernardo, M. J. Bayarri, A. P. Dawid, J. O. Berger, D. Heckerman, A. F. M. Smith, and M. West, Oxford, UK: Oxford University Press, pp. 181–197.
   Google Scholar
• Joseph, V. R. (2012), “Bayesian Computation Using Design of Experiments-Based Interpolation Technique,” Technometrics, 54, 209–225.
   Web of Science ®Google Scholar
• Kaipio, J., and Somersalo, E. (2007), “Statistical Inverse Problems: Discretization, Model Reduction and Inverse Crimes,” Journal of Computational and Applied Mathematics, 198, 493–504.
   Web of Science ®Google Scholar
• Kennedy, M., and O’Hagan, A. (2001), “Bayesian Calibration of Computer Models,” Journal of the Royal Statistical Society, Series B, 63, 425–464.
   Google Scholar
• Korattikara, A., Chen, Y., and Welling, M. (2013), “Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget,” Proceedings of The 31st International Conference on Machine Learning, 32, 1–13.
   Google Scholar
• Law, K., Stuart, A., and Zygalakis, K. (2015), Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics, New York: Springer International Publishing.
   Google Scholar
• Li, J., and Marzouk, Y. M. (2014), “Adaptive Construction of Surrogates for the Bayesian Solution of Inverse Problems,” SIAM Journal on Scientific Computing, 36, A1163–A1186.
   Web of Science ®Google Scholar
• Lieberman, C., Willcox, K., and Ghattas, O. (2010), “Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems,” SIAM Journal on Scientific Computing, 32, 2523–2542.
   Web of Science ®Google Scholar
• Marin, J.-M., Pudlo, P., Robert, C. P., and Ryder, R. J. (2011), “Approximate Bayesian Computational Methods,” Statistics and Computing, 22, 1167–1180.
   Web of Science ®Google Scholar
• Marzouk, Y., and Xiu, D. (2009), “A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems,” Communications in Computational Physics, 6, 826–847.
   Web of Science ®Google Scholar
• Marzouk, Y. M., Najm, H. N., and Rahn, L. A. (2007, June), “Stochastic Spectral Methods for Efficient Bayesian Solution of Inverse Problems,” Journal of Computational Physics, 224, 560–586.
   Web of Science ®Google Scholar
• Muja, M., and Lowe, D. G. (2009), “Fast Approximate Nearest Neighbors With Automatic Algorithm Configuration,” VISAPP, 1, 331–340.
   Google Scholar
• Nobile, F., Tempone, R., and Webster, C. G. (2007), “A Sparse Grid Stochastic Collocation Method for Partial Differential Equations With Random Input Data,” SIAM Journal on Numerical Analysis, 46, 2309–2345.
   Web of Science ®Google Scholar
• Rasmussen, C. E. (2003), “Gaussian Processes to Speed up Hybrid Monte Carlo for Expensive Bayesian Integrals,” in Bayesian Statistics 7, eds. J. M. Bernardo, M. J. Bayarri, A. P. Dawid, J. O. Berger, D. Heckerman, A. F. M. Smith, and M. West, Oxford, UK: Oxford University Press, pp. 651–659.
   Google Scholar
• Roberts, G. O., and Rosenthal, J. S. (2004), “General State Space Markov Chains and MCMC Algorithms,” Probability Surveys, 1, 20–71.
   Google Scholar
• Roberts, G. O., and Tweedie, R. L. (1996), “Geometric Convergence and Central Limit Theorems for Multidimensional Hastings and Metropolis Algorithms,” Biometrika, 83, 95–110.
   Web of Science ®Google Scholar
• Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989), “Design and Analysis of Computer Experiments,” Statistical Science, 4, 409–423.
   Google Scholar
• Santner, T. J., Williams, B. J., and Notz, W. I. (2003), The Design and Analysis of Computer Experiments, New York: Springer.
   Google Scholar
• Snelson, E., and Ghahramani, Z. (2007), “Local and Global Sparse Gaussian Process Approximations,” in Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS-07), pp. 524–531.
   Google Scholar
• Stein, M. L., Chi, Z., and Welty, L. J. (2004), “Approximating Likelihoods for Large Spatial Data Sets,” Journal of the Royal Statistical Society, Series B, 66, 275–296.
   Google Scholar
• Vecchia, A. V. (1988), “Estimation and Model Identification for Continuous Spatial Processes,” Journal of the Royal Statistical Society, Series B, 50, 297–312.
   Google Scholar
• Xiu, D., and Hesthaven, J. S. (2005), “High-Order Collocation Methods for Differential Equations With Random Inputs,” SIAM Journal on Scientific Computing, 27, 1118–1139.
   Web of Science ®Google Scholar