On Sampling Strategies in Bayesian Variable Selection Problems With Large Model Spaces

Abstract

One important aspect of Bayesian model selection is how to deal with huge model spaces, since the exhaustive enumeration of all the models entertained is not feasible and inferences have to be based on the very small proportion of models visited. This is the case for the variable selection problem with a moderately large number of possible explanatory variables considered in this article. We review some of the strategies proposed in the literature, from a theoretical point of view using arguments of sampling theory and in practical terms using several examples with a known answer. All our results seem to indicate that sampling methods with frequency-based estimators outperform searching methods with renormalized estimators. Supplementary materials for this article are available online.

Keywords:

• Bayesian model selection
• g-priors
• Searching strategies

Acknowledgments

We thank Susie Bayarri and Jim Berger for very fruitful discussions about the problem of estimation in large model spaces. We also thank Rafael Espinosa at the Supercomputing Center in the Institute for Research in Information Technology at the Universidad de Castilla-La Mancha for providing us with technical support. We also thank the suggestions of two anonymous reviewers as they have led to a much improved version of the article. This study has been partially funded by a project granted by the Spanish Ministry of Science and Education coded MTM2010-19528.