Moving Towards the Post p < 0.05 Era via the Analysis of Credibility

References

• Al-Lamee, R., Thompson, D., Dehbi, H. M., Sen, S., Tang, K., Davies, J., Keeble, T., Mielewczik, M., Kaprielian, R., Malik, I. S., and Nijjer, S. S. (2018), “Percutaneous Coronary Intervention in Stable Angina (ORBITA): A Double-Blind, Randomised Controlled Trial,” The Lancet, 391, 31–40. DOI: 10.1016/S0140-6736(17)32714-9.
   PubMed Web of Science ®Google Scholar
• Altman, D. G., and Bland, M. J. (2004), “Confidence Intervals Illuminate Absence of Evidence,” BMJ, 328, 1016–1017.
   PubMed Web of Science ®Google Scholar
• Austin, P. C., Mamdani, M. M., Juurlink, D. N., and Hux, J. E. (2006), “Testing Multiple Statistical Hypotheses Resulted in Spurious Associations: A Study of Astrological Signs and Health,” Journal of Clinical Epidemiology, 59, 964–969. DOI: 10.1016/j.jclinepi.2006.01.012.
   PubMed Web of Science ®Google Scholar
• Baker, M. (2016), “1,500 Scientists Lift the Lid on Reproducibility,” Nature, 533, 452–454. DOI: 10.1038/533452a.
   PubMed Web of Science ®Google Scholar
• Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E. J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., and Cesarini, D. (2018), “Redefine Statistical Significance,” Nature Human Behaviour, 2, 6–10. DOI: 10.1038/s41562-017-0189-z.
   PubMed Web of Science ®Google Scholar
• Bennett, C. M., Miller, M. B., and Wolford, G. L. (2009), “Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument for Multiple Comparisons Correction,” Neuroimage, 47, S125. DOI: 10.1016/S1053-8119(09)71202-9.
   Google Scholar
• Bracken, M. B. (2009), “Why Are So Many Epidemiology Associations Inflated or Wrong? Does Poorly Conducted Animal Research Suggest Implausible Hypotheses?,” Annals of Epidemiology, 19, 220–224. DOI: 10.1016/j.annepidem.2008.11.006.
   PubMed Web of Science ®Google Scholar
• Brown, D. L., and Redberg, R. F. (2017), “Last Nail in the Coffin for PCI in Stable Angina?,” The Lancet, 391, 3–4. DOI: 10.1016/S0140-6736(17)32757-5.
   PubMed Web of Science ®Google Scholar
• Carlin, B. P., and Louis, T. A. (1996), “Identifying Prior Distributions That Produce Specific Decisions, With Application to Monitoring Clinical Trials,” in Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner, eds. D. Berry, K. Chaloner, and J. Geweke, New York: Wiley, pp. 493–503.
   Google Scholar
• Colquhoun, D. (2018), “The False Positive Risk: A Proposal Concerning What to Do About p-Values,” arXiv no. 1802.04888.
   Google Scholar
• Cumming, G. (2008), “Replication and p Intervals: p Values Predict the Future Only Vaguely, But Confidence Intervals Do Much Better,” Perspectives on Psychological Science, 3, 286–300. DOI: 10.1111/j.1745-6924.2008.00079.x.
   Web of Science ®Google Scholar
• Cumming, G., Williams, J., and Fidler, F. (2004), “Replication and Researchers’ Understanding of Confidence Intervals and Standard Error Bars,” Understanding Statistics, 3, 299–311. DOI: 10.1207/s15328031us0304_5.
   Google Scholar
• Devlin, H. (2017), “Exaggerations Threaten Public Trust in Science, Says Leading Statistician,” Guardian.com, available at www.tinyurl.com/Spiegelhalter-interview.
   Google Scholar
• Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., Kerekes, M.F., Gerken, M. and Wieseler, B. (2010), “Reboxetine for Acute Treatment of Major Depression: Systematic Review and Meta-Analysis of Published and Unpublished Placebo and Selective Serotonin Ruptake Inhibitor Controlled Trials.” BMJ, 341, c4737. DOI: 10.1136/bmj.c4737.
   PubMed Web of Science ®Google Scholar
• Fisher, R. A. (1929) “The Statistical Method in Psychical Research,” Proceedings of the Society for Psychical Research, 39, 189–192.
   Google Scholar
• Gaist, D., Hallas, J., Friis, S., Hansen, S., and Sørensen, H. T. (2014), “Statin Use and Survival Following Glioblastoma Multiforme,” Cancer Epidemiology, 38, 722–727. DOI: 10.1016/j.canep.2014.09.010.
   PubMed Web of Science ®Google Scholar
• Gardner, M. J., and Altman, D. G. (1986), “Confidence Intervals Rather Than P Values: Estimation Rather Than Hypothesis Testing,” British Medical Journal, 292, 746–750. DOI: 10.1136/bmj.292.6522.746.
   PubMed Web of Science ®Google Scholar
• Good, I. J. (1950), Probability and the Weighing of Evidence, London, UK: Griffin, pp. 35–36.
   Google Scholar
• Goodman, S. N. (1992), “A Comment on Replication, P-Values and Evidence,” Statistics in Medicine, 11, 875–879. DOI: 10.1002/sim.4780110705.
   PubMed Web of Science ®Google Scholar
• Goodman, S. N. (2008), “A Dirty Dozen: Twelve P-Value Misconceptions,” Seminars in Hematology, 45, 135–140. DOI: 10.1053/j.seminhematol.2008.04.003.
   PubMed Web of Science ®Google Scholar
• Goodman, S. N. (2016a), “The Next Questions: Who, What, When, Where, and Why?,” Online Commentary to Wasserstein and Lazar (2016).
   Google Scholar
• Goodman, S. N. (2016b), “Aligning Statistical and Scientific Reasoning,” Science, 352, 1180–1181. DOI: 10.1126/science.aaf5406.
   PubMed Web of Science ®Google Scholar
• Goodman, S. N., and Berlin, J. A. (1994), “The Use of Predicted Confidence Intervals When Planning Experiments and the Misuse of Power When Interpreting Results,” Annals of Internal Medicine, 121, 200–206. DOI: 10.7326/0003-4819-121-3-199408010-00008.
   PubMed Web of Science ®Google Scholar
• GREAT Group (1992), “Feasibility, Safety, and Efficacy of Domiciliary Thrombolysis by General Practitioners: Grampian Region Early Anistreplase Trial,” British Medical Journal, 305, 548–553.
   PubMed Web of Science ®Google Scholar
• Greenland, S. (2011), “Null Misinterpretation in Statistical Testing and Its Impact on Health Risk Assessment,” Preventative Medicine, 53, 225–228. DOI: 10.1016/j.ypmed.2011.08.010.
   PubMed Web of Science ®Google Scholar
• Greenland, S. (2017), “A Serious Misinterpretation of a Consistent Inverse Association of Statin Use With Glioma Across 3 Case-Control Studies,” European Journal of Epidemiology, 32, 87–88. DOI: 10.1007/s10654-016-0205-z.
   PubMed Web of Science ®Google Scholar
• Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., and Altman, D. G. (2016), “Statistical Tests, P Values, Confidence Intervals, and Power: A Guide to Misinterpretations,” European Journal of Epidemiology, 31, 337–350. DOI: 10.1007/s10654-016-0149-3.
   PubMed Web of Science ®Google Scholar
• Held, L. (2013), “Reverse-Bayes Analysis of Two Common Misinterpretations of Significance Tests,” Clinical Trials, 10, 236–242. DOI: 10.1177/1740774512468807.
   PubMed Web of Science ®Google Scholar
• Held, L. (2018), “A New Argument for p < 0.005,” arXiv no. 1803.10052.
   Google Scholar
• Hines, T. M. (1998), “Comprehensive Review of Biorhythm Theory,” Psychological Reports, 83, 19–64. DOI: 10.2466/pr0.1998.83.1.19.
   PubMed Web of Science ®Google Scholar
• Hoekstra, R., Morey, R. D., Rouder, J. N., and Wagenmakers, E. J. (2014), “Robust Misinterpretation of Confidence Intervals,” Psychonomic Bulletin & Review, 21, 1157–1164. DOI: 10.3758/s13423-013-0572-3.
   PubMed Web of Science ®Google Scholar
• Hoenig, J. M., and Heisey, D. M. (2001), “The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis,” The American Statistician, 55, 19–24. DOI: 10.1198/000313001300339897.
   Web of Science ®Google Scholar
• Horton, R. (2015), “Offline: What Is Medicine’s 5 Sigma?,” Lancet, 385, 1380. DOI: 10.1016/S0140-6736(15)60696-1.
   Web of Science ®Google Scholar
• Hubbard, R. (2016a), Corrupt Research: The Case for Reconceptualizing Empirical Management and Social Science. Thousand Oaks, CA: Sage, pp. 209–213.
   Google Scholar
• Hubbard, R. (2016b), Corrupt Research: The Case for Reconceptualizing Empirical Management and Social Science. Thousand Oaks, CA: Sage, pp. 232–234.
   Google Scholar
• Ioannidis, J. P. A. (2013), “Implausible Results in Human Nutrition Research,” BMJ, 347, I6698
   PubMed Web of Science ®Google Scholar
• Matthews, R. A. J. (2001a), “Why Should Clinicians Care about Bayesian Methods?,” Journal of Statistical Inference and Planning, 94, 43–58.
   Google Scholar
• Matthews, R. A. J. (2001b), “Methods for Assessing the Credibility of Clinical Trial Outcomes,” Drug Information Journal, 35, 1469–1478. DOI: 10.1177/009286150103500442.
   Google Scholar
• Matthews, R. A. J. (2018) “Beyond ‘Significance’: Principles and Practice of the Analysis of Credibility,” Royal Society Open Science, 5, 171047. DOI: 10.1098/rsos.171047.
   PubMed Web of Science ®Google Scholar
• Matthews, R. A. J., Wasserstein, R., and Spiegelhalter, D. (2017), “The ASA’s P-Value Statement, One Year On,” Significance, 14, 38–41. DOI: 10.1111/j.1740-9713.2017.01021.x.
   Google Scholar
• Morrison, L. J., Verbeek, P. R., McDonald, A. C., Sawadsky, B. V., and Cook, D. J. (2000), “Mortality and Prehospital Thrombolysis for Acute Myocardial Infarction: A Meta-Analysis,” Journal of the American Medical Association, 283, 2686–2692. DOI: 10.1001/jama.283.20.2686.
   PubMed Web of Science ®Google Scholar
• Nuzzo, R. (2014), “Scientific Method: Statistical Errors,” Nature, 506, 150–152. DOI: 10.1038/506150a.
   PubMed Web of Science ®Google Scholar
• Pocock, S. J., and Spiegelhalter, D. J. (1992), “Domiciliary Thrombolysis by General Practitioners,” British Medical Journal, 305, 1015. DOI: 10.1136/bmj.305.6860.1015.
   PubMed Web of Science ®Google Scholar
• Rothman, K. J. (1978), “A Show of Confidence,” New England Journal of Medicine, 299, 1362–1363. DOI: 10.1056/NEJM197812142992410.
   PubMed Web of Science ®Google Scholar
• Rothman, K. J., Greenland, S., and Lash, T. L. (Eds.) (2008), Modern Epidemiology, Philadelphia: Lippincott Williams & Wilkins, chapter 18.
   Google Scholar
• Sagan, C. (1980), Broca’s Brain: Reflections on the Romance of Science, New York: Random House, p. 73, chapter 5.
   Google Scholar
• Schwab, A., Abrahamson, E., Starbuck, W. H., and Fidler, F. (2011), “Perspective: Researchers Should Make Thoughtful Assessments Instead of Null-Hypothesis Significance Tests,” Organization Science, 22, 1105–1120. DOI: 10.1287/orsc.1100.0557.
   Web of Science ®Google Scholar
• Seliger, C., Meier, C. R., Becker, C., Jick, S. S., Bogdahn, U., Hau, P., and Leitzmann, M. F. (2016), “Statin Use and Risk of Glioma: Population-Based Case–Control Analysis,” European Journal of Epidemiology, 31, 947–952. DOI: 10.1007/s10654-016-0145-7.
   PubMed Web of Science ®Google Scholar
• Smaldino, P. E., and McElreath, R. (2016), “The Natural Selection of Bad Science,” Royal Society Open Science, 3, 160384. DOI: 10.1098/rsos.160384.
   PubMed Web of Science ®Google Scholar
• Spiegelhalter, D. J. (2004), “Incorporating Bayesian Ideas Into Health-Care Evaluation,” Statistical Science, 19, 156–174. DOI: 10.1214/088342304000000080.
   Web of Science ®Google Scholar
• Spiegelhalter, D. J., Abrams, R., and Myles, J. P. (2004) Bayesian Approaches to Clinical Trials and Health-Care Evaluation, Chichester: Wiley & Sons, chapter 3.
   Google Scholar
• Wagenmakers, E.-J., Verhagen, J., Ly, A., Bakker, M., Lee, M. D., Matzke, D., Rouder, J. N., and Morey, R. D. (2015), “A Power Fallacy,” Behavior Research Methods, 47, 913–917. DOI: 10.3758/s13428-014-0517-4.
   PubMed Web of Science ®Google Scholar
• Wasserstein, R. L., and Lazar, N. A. (2016), “The ASA’s Statement on P-Values: Context, Process, and Purpose,” The American Statistician, 70, 129–133. DOI: 10.1080/00031305.2016.1154108.
   Web of Science ®Google Scholar