Selection Adjusted Confidence Intervals With More Power to Determine the Sign

Abstract

In many current large-scale problems, confidence intervals (CIs) are constructed only for the parameters that are large, as indicated by their estimators, ignoring the smaller parameters. Such selective inference poses a problem to the usual marginal CIs that no longer offer the right level of coverage, not even on the average over the selected parameters. We address this problem by developing three methods to construct short and valid CIs for the location parameter of a symmetric unimodal distribution, while conditioning on its estimator being larger than some constant threshold. In two of these methods, the CI is further required to offer early sign determination, that is, to avoid including parameters of both signs for relatively small values of the estimator. One of the two, the Conditional Quasi-Conventional CI, offers a good balance between length and sign determination while protecting from the effect of selection. The CI is not symmetric, extending more toward 0 than away from it, nor is it of constant shape. However, when the estimator is far away from the threshold, the proposed CI tends to the usual marginal one. In spite of its complexity, it is specified by closed form expressions, up to a small set of constants that are each the solution of a single variable equation.

When multiple testing procedures are used to control the false discovery rate or other error rates, the resulting threshold for selecting may be data dependent. We show that conditioning the above CIs on the data-dependent threshold still offers false coverage-statement rate (FCR) for many widely used testing procedures. For these reasons, the conditional CIs for the parameters selected this way are an attractive alternative to the available general FCR adjusted intervals. We demonstrate the use of the method in the analysis of some 14,000 correlations between hormone change and brain activity change in response to the subjects being exposed to stressful movie clips. Supplementary materials for this article are available online.

Keywords:

• Conditional confidence intervals
• False coverage rate
• Selective inference

Acknowledgments

We are thankful to Professor Larry Brown who, following the presentation of the FCR controlling intervals at the 2005 IMS/Bernoulli meeting in Barcelona, suggested constructing conditional confidence intervals as an alternative, and to Dr. Dani Yekutieli for the enlightening discussions on selective inference. We are thankful to Professor Talma Hendler from Tel Aviv University who lead the imaging research and with whom the first author is cooperating, and to Sharon Vaisvaser and to Yonatan Weinetraub who participated in that research. Y.B. thanks Stanford University for a Sabbatical stay during which the cooperation with W.F. started. The research of Y.B. was partially supported by grants from the Israeli Science Foundation and the European Research Council Advanced Researcher grant (PSARPS). W.F. was supported by the VIGRE DMS-0502385 grant from the National Science Foundation. The data analysis of the example was supported by a grant from the U.S. Department of Defense.