Tractable Bayesian Variable Selection: Beyond Normality

ABSTRACT

Bayesian variable selection often assumes normality, but the effects of model misspecification are not sufficiently understood. There are sound reasons behind this assumption, particularly for large p: ease of interpretation, analytical, and computational convenience. More flexible frameworks exist, including semi- or nonparametric models, often at the cost of some tractability. We propose a simple extension that allows for skewness and thicker-than-normal tails but preserves tractability. It leads to easy interpretation and a log-concave likelihood that facilitates optimization and integration. We characterize asymptotically parameter estimation and Bayes factor rates, under certain model misspecification. Under suitable conditions, misspecified Bayes factors induce sparsity at the same rates than under the correct model. However, the rates to detect signal change by an exponential factor, often reducing sensitivity. These deficiencies can be ameliorated by inferring the error distribution, a simple strategy that can improve inference substantially. Our work focuses on the likelihood and can be combined with any likelihood penalty or prior, but here we focus on nonlocal priors to induce extra sparsity and ameliorate finite-sample effects caused by misspecification. We show the importance of considering the likelihood rather than solely the prior, for Bayesian variable selection. The methodology is in R package ‘mombf.’  Supplementary materials for this article are available online.

KEYWORDS:

• Bayes factors
• Model misspecification
• Robust regression
• Two-piece errors
• Variable selection

Supplementary Materials

The supplement contains proofs of Propositions 1–5, Corollaries 1–2, details on the Newton-Raphson and Coordinate Descent optimization algorithms, log-likelihood derivatives, and additional results for the simulation studies and the DLD data.

Additional information

Funding

DR was partially funded by the NIH grant R01 CA158113-01 and the Ramón y Cajal Fellowship RYC-2015-18544.