On Recursive Bayesian Predictive Distributions

ABSTRACT

A Bayesian framework is attractive in the context of prediction, but a fast recursive update of the predictive distribution has apparently been out of reach, in part because Monte Carlo methods are generally used to compute the predictive. This article shows that online Bayesian prediction is possible by characterizing the Bayesian predictive update in terms of a bivariate copula, making it unnecessary to pass through the posterior to update the predictive. In standard models, the Bayesian predictive update corresponds to familiar choices of copula but, in nonparametric problems, the appropriate copula may not have a closed-form expression. In such cases, our new perspective suggests a fast recursive approximation to the predictive density, in the spirit of Newton’s predictive recursion algorithm, but without requiring evaluation of normalizing constants. Consistency of the new algorithm is shown, and numerical examples demonstrate its quality performance in finite-samples compared to fully Bayesian and kernel methods. Supplementary materials for this article are available online.

KEYWORDS:

• Copula
• Density estimation
• Nonparametric Bayes
• Prediction
• Recursive estimation

Supplementary Materials

The online supplementary material contains an extension of the proposed recursive algorithm to the bivariate case, a generalization of the result derived in Example 1, and the proofs of Lemmas 1 and 2.

Acknowledgments

The authors thank the Editor, Associate Editor, and referees for their helpful comments on the previous version of this article.

Additional information

Funding

This work is partially supported by the U. S. National Science Foundation, grants DMS -1507073 and DMS -1506879, and by the U. S. Army Research Offices, Award #W911NF-15-1-0154.