SURE Estimates for a Heteroscedastic Hierarchical Model

Abstract

Hierarchical models are extensively studied and widely used in statistics and many other scientific areas. They provide an effective tool for combining information from similar resources and achieving partial pooling of inference. Since the seminal work by James and Stein (1961) and Stein (1962), shrinkage estimation has become one major focus for hierarchical models. For the homoscedastic normal model, it is well known that shrinkage estimators, especially the James-Stein estimator, have good risk properties. The heteroscedastic model, though more appropriate for practical applications, is less well studied, and it is unclear what types of shrinkage estimators are superior in terms of the risk. We propose in this article a class of shrinkage estimators based on Stein’s unbiased estimate of risk (SURE). We study asymptotic properties of various common estimators as the number of means to be estimated grows (p → ∞). We establish the asymptotic optimality property for the SURE estimators. We then extend our construction to create a class of semiparametric shrinkage estimators and establish corresponding asymptotic optimality results. We emphasize that though the form of our SURE estimators is partially obtained through a normal model at the sampling level, their optimality properties do not heavily depend on such distributional assumptions. We apply the methods to two real datasets and obtain encouraging results.

Keywords:

• Asymptotic optimality
• Heteroscedasticity
• Shrinkage estimator
• Stein’s unbiased risk estimate (SURE)

Acknowledgments

S. C. Kous’ research is supported in part by NIH/NIGMS grant R01GM090202 and NSF grant DMS-0449204. L. Brown’s research is supported in part by NSF grant DMS-1007657. The authors thank Professor Philippe Rigollet at Princeton University for helpful discussion.