Stoppong rules and observed significance levels

Abstract

It is well known how to combine the significance levels observed in a number of independent experiments. When this number is a random variable determined by a stopping rule, the observed significance level can still be calculated if there is an acceptable ordering of the points in the extended sample space. But what can be said if the stopping time is ill defined This paper obtains explicit lower bounds on the level of significance by considering orderings based on a family of alternative hypotheses. These bounds give some measure of the effect of failing to specify the stopping rule in advance.

Keywords:

• hypothesis testing
• likelihood ratios
• Neyman-Pearson theory
• sequential decisions