Nonlocal Priors for High-Dimensional Estimation

ABSTRACT

Jointly achieving parsimony and good predictive power in high dimensions is a main challenge in statistics. Nonlocal priors (NLPs) possess appealing properties for model choice, but their use for estimation has not been studied in detail. We show that for regular models NLP-based Bayesian model averaging (BMA) shrink spurious parameters either at fast polynomial or quasi-exponential rates as the sample size n increases, while nonspurious parameter estimates are not shrunk. We extend some results to linear models with dimension p growing with n. Coupled with our theoretical investigations, we outline the constructive representation of NLPs as mixtures of truncated distributions that enables simple posterior sampling and extending NLPs beyond previous proposals. Our results show notable high-dimensional estimation for linear models with p > >n at low computational cost. NLPs provided lower estimation error than benchmark and hyper-g priors, SCAD and LASSO in simulations, and in gene expression data achieved higher cross-validated R² with less predictors. Remarkably, these results were obtained without prescreening variables. Our findings contribute to the debate of whether different priors should be used for estimation and model selection, showing that selection priors may actually be desirable for high-dimensional estimation. Supplementary materials for this article are available online.

KEYWORDS:

• Bayesian model averaging
• MCMC
• Model selection
• Nonlocal priors
• Shrinkage

Funding

Both authors were partially funded by the NIH grant R01 CA158113-01.