Methodological commentary: Effect of measurement error on tests of statistical significance

Abstract

Using the domain-sampling model from classical test theory, the effects of measurement error on statistical tests for the difference between an obtained mean and a hypothesized mean, and the difference between two means, are demonstrated. The results indicate that lowering the reliability (i. e., increasing measurement error) of dependent variable data increases the chance of obtaining a nonsignificant result when a significant result is the correct outcome. Lowering the reliability also produces reduced estimates of strength of association.