Adjusted power estimates in monte carlo experiments

Abstract

Critical values and powers of competing tests are often evaluated through Monte Carlo simulations. Since the true levels of those tests are often very different, comparisons of the power estimates are not valid. We suggest that Monte Carlo estimates of critical values be used to create adjusted power estimates which are then comparable. The main contribution is to analyze the variability of the adjusted estimates and to point out implications for the planning of Monte Carlo power studies.

Keywords:

• Critical value
• empirical level
• McNemar's test
• true level