Comparing and Weighting Imperfect Models Using D-Probabilities

Abstract

We propose a new approach for assigning weights to models using a divergence-based method (D-probabilities), relying on evaluating parametric models relative to a nonparametric Bayesian reference using Kullback–Leibler divergence. D-probabilities are useful in goodness-of-fit assessments, in comparing imperfect models, and in providing model weights to be used in model aggregation. D-probabilities avoid some of the disadvantages of Bayesian model probabilities, such as large sensitivity to prior choice, and tend to place higher weight on a greater diversity of models. In an application to linear model selection against a Gaussian process reference, we provide simple analytic forms for routine implementation and show that D-probabilities automatically penalize model complexity. Some asymptotic properties are described, and we provide interesting probabilistic interpretations of the proposed model weights. The framework is illustrated through simulation examples and an ozone data application. Supplementary materials for this aricle are available online.

Keywords:

• Gaussian process
• Gibbs posterior
• Kullback–Leibler divergence
• Model aggregation
• Model selection
• M-open

Additional information

Funding

This research was partially supported by grant 1R24MH117529 of the BRAIN Initiative and grant R01ES027498 of the National Institute of Environmental Health Science of theUnited States National Institutes of Health.