Statistical significance and scientific misconduct: improving the style of the published research paper

Abstract

A science, business, or law that is basing its validity on the level of p-values, t statistics and other tests of statistical significance is looking less and less relevant and more and more unethical. Today’s economist uses a lot of wit putting a clever index of opportunity cost into his models; but then, like the amnesiac, he fails to see opportunity cost in statistical estimates he makes of those same models. Medicine, psychology, pharmacology and other fields are similarly damaged by this fundamental error of science, keeping bad treatments on the market and good ones out. A few small changes to the style of the published research paper using statistical methods can bring large beneficial effects to more than academic research papers. It is suggested that misuse of statistical significance be added to the definition of scientific misconduct currently enforced by the NIH, NSF, Office of Research Integrity and others.

Keywords:

• Research ethics
• econometrics
• Bayes factors
• human values
• statistical reporting

Acknowledgments

The article began as part of a keynote address on the use and misuse of randomization and statistical significance in economics and medicine, presented by the author at the International Workshop on Scientific Misconduct and Research Ethics in Economics, Izmir, Turkey, 2014. A previous version was drafted as a discussion paper for the American Statistical Association Ad Hoc Committee on P Values and Statistical Significance. The author wishes to thank John Mullahy and Workshop and Committee participants, especially Naomi Altman, Brad Carlin, Erwin Dekker, Wilfred Dolfsma, Val Johnson, Michael Lavine, Michael Lew, Ioana Negru, Ron Wasserstein, James Wible and Altug Yalcintas for helpful comments and suggestions. Any errors are my own.

Notes

¹ The journals are: American Journal of Health Economics, European Journal of Health Economics, Forum for Health Economics & Policy, Health Economics Policy and Law, Health Economics Review, Health Economics, International Journal of Health Economics and Management, and Journal of Health Economics.

² Lavine and Schervich (1999) caution that Bayes factors can sometimes lead to incoherence in the technical statistical sense of that term.