Moving college health research forward: Reconsidering our reliance on statistical significance testing

References

• Nissen S, Magidson T, Gross K, Bergstrom C. Publication bias and the canonization of false facts. eLife. 2016;5:e21451.
   PubMed Web of Science ®Google Scholar
• Trafimow D, Rice S. A test of the null hypothesis significance testing procedure correlation argument. J Gen Psych. 2009;136(3):261–269. doi:10.3200/GENP.136.3.261-270.
   Google Scholar
• Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015;13(3):e1002106. doi:10.1371/journal.pbio.1002106.
   PubMed Web of Science ®Google Scholar
• Nuzzo R. Statistical errors. Nature 2014;506(7487):150–152. doi:10.1038/506150a.
   PubMed Web of Science ®Google Scholar
• Fisher RA. The mathematical distributions used in the common tests of significance. Econometrica, J Econ Soc. 1935;3(4):353–365.
   Google Scholar
• Nuzzo R. How scientists fool themselves – and how they can stop. Fooling Ourselves. Nature. 2015;526(7572):182–185. doi:10.1038/526182a.
   Google Scholar
• Thompson B. Foundations of Behavioral Statistics: An Insight-Based Approach. New York: Guilford Press; 2006.
   Google Scholar
• Hubbard R. Corrupt Research: The Case for Recognizing Empirical Management and Social Science. Los Angeles, CA: Sage; 2006.
   Google Scholar
• Thompson B. The “significance” crisis in psychology and education. J Soc-Econ. 2004;33(5):607–613. doi:10.1016/j.socec.2004.09.034.
   Google Scholar
• Cortina J, Dunlap W. On the logic and purpose of significance tests. Psych Methods. 1997;2(2):161–172. doi:10.1037/1082-989X.2.2.161.
   Web of Science ®Google Scholar
• Schervish M. P Values: what are they and what are they not. Am Stat 1996;50(3):203–206.
   Web of Science ®Google Scholar
• Nickerson SR. Null hypothesis significance testing: A review of an old and continuing controversy. Psych Method. 2000;5(2):241–301. doi:10.1037/1082-989X.5.2.241.
   PubMed Web of Science ®Google Scholar
• Goodman SN. Aligning statistical and scientific reasoning. Science 2016;352(6290):1180–1181. doi:10.1126/science.aaf5406.
   PubMed Web of Science ®Google Scholar
• Goodman S. A dirty dozen: Twelve p-value misconceptions. Sems Hem. 2008;45(3):135–140. doi:10.1053/j.seminhematol.2008.04.003.
   PubMed Web of Science ®Google Scholar
• Thompson B. The use of statistical significance in research: bootstrap and other alternatives. J Exp Educ. 1993;61(4):361–377. doi:10.1080/00220973.1993.10806596.
   Web of Science ®Google Scholar
• Johnson DH. The insignificance of statistical significance testing. J Wild Manage. 1999;63(3):763–772. doi:10.2307/3802789.
   Web of Science ®Google Scholar
• Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129–133. doi:10.1080/00031305.2016.1154108.
   Web of Science ®Google Scholar
• Thompson B. Why “encouraging” effect size reporting is not working: the etiology of researcher resistance to changing practices. J Psych. 1999;133(2):133–140. doi:10.1080/00223989909599728.
   Web of Science ®Google Scholar
• Hubbard R, Armstrong JS. Replications and extensions in marketing: Rarely published but quite contrary. Int J Res in Mark. 1994;11(3):233–248. doi:10.1016/0167-8116(94)90003-5.
   Google Scholar
• Barry AE, Szucs LE, Reyes JV, Ji Q, Wilson KL, Thompson B. Failure to report effect sizes: the handling of quantitative results in published. Health Educ Behav. 2016;43(5)1–10, :518–527.
   PubMed Web of Science ®Google Scholar
• Plucker JA. Debunking the myth of the ‘highly significant’ result: effect sizes in gifted education research. Roeper Rev. 1997;20(2):122–126. doi:10.1080/02783199709553873.
   Google Scholar
• Thompson B. Improving research clarity and usefulness with effect size indices as supplements to statistical significance tests. Excep Child. 1999;65(3):329–337. doi:10.1177/001440299906500304.
   Web of Science ®Google Scholar
• Field A. Discovering Statistics Using SPSS. London: Sage publications; 2006.
   Google Scholar
• Sullivan G, Feinn R. Using effect size— or why p value is not enough. J Grad Med Educ. 2012;4(3):279–282. doi:10.4300/JGME-D-12-00156.1.
   PubMedGoogle Scholar
• SIUC/Core Institute. 2011 annual reference group: Core alcohol and drug survey long form – Form 194. Executive Summary. Available at: http://core.siu.edu/_common/documents/report11.pdf; 2013.
   Google Scholar
• Neighbors C, Atkins DC, Lewis MA, et al. Event-specific drinking among college students. Psych Add Behav. 2011;25(4):702. doi:10.1037/a0024051.
   PubMed Web of Science ®Google Scholar
• Neighbors C, Walters ST, Lee CM, et al. Event-specific prevention: Addressing college student drinking during known windows of risk. Add. Beh 2007;32(11):2667–2680. doi:10.1016/j.addbeh.2007.05.010.
   Google Scholar
• Finch S, Cumming G, Thomason N. Reporting of statistical inference in the journal of applied psychology: Little evidence of reform. Educ Psych Measur. 2001;61(2):181–210.
   Web of Science ®Google Scholar
• Kirk RE. Promoting good statistical practices: some suggestions. Educ Psych Meas. 2001;61(2):213–218. doi:10.1177/00131640121971185.
   Web of Science ®Google Scholar
• Thompson B. If statistical significance tests are broken/misused, what practices should supplement or replace them. Theory & Psych. 1999;9(2):165–181. doi:10.1177/095935439992006
   Web of Science ®Google Scholar
• Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Front Psychol. 2013;4:863. doi:10.3389/fpsyg.2013.00863.
   PubMed Web of Science ®Google Scholar
• Cumming G. The new statistics: Why and how. Psychol Sci. 2014;25(1):7–29. doi:10.1177/0956797613504966.
   PubMed Web of Science ®Google Scholar
• Wilkinson L. Task Force on Statistical Inference Statistical methods in psychology journals: guidelines and explanations. Am Psych. 1999;54(8):594–604. doi:10.1037/0003-066X.54.8.594.
   Web of Science ®Google Scholar
• Sterling TD. Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. J Am Stat Assoc. 1959;54(285):30–34.
   Web of Science ®Google Scholar
• Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337(8746):867–872. doi:10.1016/0140-6736(91)90201-Y.
   PubMed Web of Science ®Google Scholar
• Colliver AJ. Call for greater emphasis on effect-size measures in published articles in teaching and learning in medicine. Teach Learn Med. 2002;14(4):206–210. doi:10.1207/S15328015TLM1404_1.
   PubMed Web of Science ®Google Scholar
• Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi:10.1371/journal.pmed.0020124.
   PubMed Web of Science ®Google Scholar
• Siegfried T. P value ban: Small step for a journal, giant leap for science. Available at: https://www.sciencenews.org/blog/context/p-value-ban-small-step-journal-giant-leap-science. Published 2015. Accessed April 27, 2016.
   Google Scholar
• Ioannidis JP. Why most discovered true associations are inflated. Epidemiology. 2008;19(5):640–648.
   PubMed Web of Science ®Google Scholar
• Lambdin C. Significance tests as sorcery: science is empirical—significance tests are not. Theory & Psych. 2012;22(1):67–90.
   Web of Science ®Google Scholar
• Hoekstra R, Finch S, Kiers HA, Johnson A. Probability as certainty: dichotomous thinking and the misuse of p values. Psychon Bull Rev. 2006;13(6):1033–1037. doi:10.3758/BF03213921.
   PubMed Web of Science ®Google Scholar
• Shrout PE. Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychol Sci. 1997;8(1):1–2. doi:10.1111/j.1467-9280.1997.tb00533.x.
   Web of Science ®Google Scholar