Bayesian Variable Selection on Model Spaces Constrained by Heredity Conditions

Abstract

This article investigates Bayesian variable selection when there is a hierarchical dependence structure on the inclusion of predictors in the model. In particular, we study the type of dependence found in polynomial response surfaces of orders two and higher, whose model spaces are required to satisfy weak or strong heredity conditions. These conditions restrict the inclusion of higher-order terms depending upon the inclusion of lower-order parent terms. We develop classes of priors on the model space, investigate their theoretical and finite sample properties, and provide a Metropolis–Hastings algorithm for searching the space of models. The tools proposed allow fast and thorough exploration of model spaces that account for hierarchical polynomial structure in the predictors and provide control of the inclusion of false positives in high posterior probability models.

Key Words:

• Intrinsic prior
• Markov chain Monte Carlo
• Model priors
• Multiple testing
• Multiplicity penalization
• Strong heredity
• Weak heredity
• Well-formulated models

ACKNOWLEDGMENTS

The authors were supported by the National Science Foundation grant DMS-1105127. Taylor–Rodriguez was additionally supported by the National Science Foundation under grant DMS-1127914 to the Statistical and Applied Mathematical Sciences Institute. Bliznyuk was additionally supported by the National Institutes of Health grants U01GM070749 and U54GM111274. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or the National Institutes of Health.

Additional information

Notes on contributors

Daniel Taylor-Rodriguez

Daniel Taylor-Rodriguez is first co-author, Postdoctoral Research Associate, SAMSI / Duke University, Research Triangle Park, NC 27709-4006 (E-mail: [email protected]). Andrew Womack is first co-author, Assistant Professor, Department of Statistics, Indiana University Bloomington, IN 47404 (E-mail: [email protected]). Nikolay Bliznyuk is corresponding author, Assistant Professor of Statistics, Department of Agricultural & Biological Engineering, University of Florida, Gainesville, FL 32611 (E-mail: [email protected]).

Andrew Womack

Daniel Taylor-Rodriguez is first co-author, Postdoctoral Research Associate, SAMSI / Duke University, Research Triangle Park, NC 27709-4006 (E-mail: [email protected]). Andrew Womack is first co-author, Assistant Professor, Department of Statistics, Indiana University Bloomington, IN 47404 (E-mail: [email protected]). Nikolay Bliznyuk is corresponding author, Assistant Professor of Statistics, Department of Agricultural & Biological Engineering, University of Florida, Gainesville, FL 32611 (E-mail: [email protected]).

Nikolay Bliznyuk

Daniel Taylor-Rodriguez is first co-author, Postdoctoral Research Associate, SAMSI / Duke University, Research Triangle Park, NC 27709-4006 (E-mail: [email protected]). Andrew Womack is first co-author, Assistant Professor, Department of Statistics, Indiana University Bloomington, IN 47404 (E-mail: [email protected]). Nikolay Bliznyuk is corresponding author, Assistant Professor of Statistics, Department of Agricultural & Biological Engineering, University of Florida, Gainesville, FL 32611 (E-mail: [email protected]).