Vecchia-Approximated Deep Gaussian Processes for Computer Experiments

Abstract

Deep Gaussian processes (DGPs) upgrade ordinary GPs through functional composition, in which intermediate GP layers warp the original inputs, providing flexibility to model nonstationary dynamics. Two DGP regimes have emerged in recent literature. A “big data” regime, prevalent in machine learning, favors approximate, optimization-based inference for fast, high-fidelity prediction. A “small data” regime, preferred for computer surrogate modeling, deploys posterior integration for enhanced uncertainty quantification (UQ). We aim to bridge this gap by expanding the capabilities of Bayesian DGP posterior inference through the incorporation of the Vecchia approximation, allowing linear computational scaling without compromising accuracy or UQ. We are motivated by surrogate modeling of simulation campaigns with upwards of 100,000 runs—a size too large for previous fully-Bayesian implementations—and demonstrate prediction and UQ superior to that of “big data” competitors. All methods are implemented in the deepgp package on CRAN. Supplementary materials for this article are available online.

Keywords:

• Nearest neighbor
• Nonstationary
• Sparse matrix
• Surrogate
• Uncertainty quantification

Disclosure Statement

The authors report there are no competing interests to declare.

Supplementary Materials

Equations for simulated functions (Schaffer, G-function) and evaluation metrics (RMSE, CRPS). Derivation of Vecchia-approximated posterior predictive moments (12). Investigation of varying conditioning set size (m). Additional simulations with noise and higher dimension. Computation times for all exercises.

Additional information

Funding

This work was supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research and Office of High Energy Physics, Scientific Discovery through Advanced Computing (SciDAC) program under Award Number 0000231018.