Confidence Intervals for Nonparametric Empirical Bayes Analysis

Abstract

In an empirical Bayes analysis, we use data from repeated sampling to imitate inferences made by an oracle Bayesian with extensive knowledge of the data-generating distribution. Existing results provide a comprehensive characterization of when and why empirical Bayes point estimates accurately recover oracle Bayes behavior. In this paper, we develop flexible and practical confidence intervals that provide asymptotic frequentist coverage of empirical Bayes estimands, such as the posterior mean or the local false sign rate. The coverage statements hold even when the estimands are only partially identified or when empirical Bayes point estimates converge very slowly. Supplementary materials for this article are available online.

Keywords:

• Bias-aware inference
• Empirical Bayes
• Local false sign rate
• Mixture models
• Partial identification

Acknowledgments

This paper was first presented on May 24th, 2018 at a workshop in honor of Bradley Efron’s 80th birthday. We are grateful to Timothy Armstrong, Bradley Efron, Jiaying Gu, Guido Imbens, Panagiotis Lolas, Michail Savvas, Paris Syminelakis, Han Wu, and seminar participants at several venues for helpful feedback and discussions. We thank Jiaying Gu for suggesting the Anderson-Rubin construction, and the anonymous referees and associate editor for helpful comments and suggestions. Some of the computing for this project was performed on the Sherlock cluster. We would like to thank Stanford University and the Stanford Research Computing Center for providing computational resources and support that contributed to these research results.

Data Availability Statement

All numerical results in this paper can be reproduced with the code available on the Github repository https://github.com/nignatiadis/empirical-bayes-confidence-intervals-paper. There we provide an implementation of the proposed methods as a package in the Julia programming language (Bezanson et al. 2017) that depends, among others, on JuMP.jl (Dunning, Huchette, and Lubin 2017) and Distributions.jl (Besançon et al. 2021).

Additional information

Funding

We acknowledge support from a Ric Weiland Graduate Fellowship, a gift from Google, and National Science Foundation grant DMS-1916163.