Adjusting Published Estimates for Exploratory Biases Using the Truncated Normal Distribution

Abstract

Abstract–Publication bias can occur for many reasons, including the perceived need to present statistically significant results. We propose and compare methods for adjusting a single published estimate for possible publication bias using a truncated normal distribution. We attempt to estimate the mean of the underlying normal sampling distribution using only summary data readily available in most published work, making the results practical for use by a consumer of research. The adjustment methods are investigated via simulation and their results compared in terms of bias, mean squared error, and confidence interval coverage. The methods are also applied to eleven previously published studies. We find the proposed methods improve but do not eliminate biases from the statistical significance filter.

Keywords:

• Hypothesis tests
• Publication bias
• Replication
• Statistical significance filter

Supplementary Materials

Trunc-norm-supplement Detailed description of inverse estimator and tablular description of results presented in Figure 3. (.pdf file)

Simulations R files to recreate simulation and figures presented in article. (directory)

Application Material (data and code) to produce the applied example presented in article. (directory)