The Median Significance Level and other Small Sample Measures of Test Efficacy

Abstract

The concepts of the “median significance level” (MSL) and the “significance level of the average” (SLA) are introduced and some relationships among these measures and the recently introduced “expected significance level” (ESL), “average critical value” (ACV), and “median critical value” (MCV) are considered. The median significance level is defined as the median of the distribution of the observed significance level for a given alternative and is shown to be equivalent to the significance level attained by the median of the test statistic for one-sided tests. The “significance level of the average” is analogously defined to be the significance level attained by the average (expectation) of the test statistic and the MSL and SLA are shown to be inverse functions of Geary's MCV and ACV. Some relations between these small sample measures of test efficacy, and Pitmans' and Bahadur's asymptotic measures are described. The MCV is shown to be formally related to Hamaker's “indifference quality” method of classifying acceptance sampling plans. Several simple examples are given illustrating some relationships among the several criteria.