Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression

Abstract

Maximum likelihood estimation of hyperparameters in Gaussian processes (GPs) as well as other spatial regression models usually requires the evaluation of the logarithm of the matrix determinant, in short, log det. When using matrix decomposition techniques, the exact implementation of log det is of O(N ³) operations, where N is the matrix dimension. In this paper, a power-series expansion-based framework is presented for approximating the log det of general positive-definite matrices. Three novel compensation schemes are proposed to further improve the approximation accuracy and computational efficiency. The proposed log det approximation requires only 50N ² operations. The theoretical analysis is substantiated by a large number of numerical experiments, including tests on randomly generated positive-definite matrices, randomly generated covariance matrices, and sequences of covariance matrices generated online in two GP regression examples. The average approximation error is ∼9%.

Keywords:

• Gaussian process
• Logarithm of matrix determinant
• Power-series expansion
• Compensation

Acknowledgements

This work was supported by Science Foundation Ireland grant, 00/PI.1/C067, and by the EPSRC, GR/M76379/01. Moreover, one of the authors, Y. Zhang, would like to thank D. A. Griffith and R. J. Martin for sending us the invaluable preprints of their papers and research reports. He would also like to thank D. J. C. MacKay, C. K. I. Williams, and M. Seeger for their inspiring emails and comments on approximate implementation of GP.