The False Positive Risk: A Proposal Concerning What to Do About p-Values

References

• Bartlett, M. S. (1957), “A Comment on D.V. Lindley’s Statistical Paradox,” Biometrika, 44, 533–534. DOI: 10.1093/biomet/44.3-4.533.
   Google Scholar
• Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R. Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017), “Redefine Statistical Significance,” in PsyRxiv, available https://psyarxiv.com/mky9j/
   Google Scholar
• Benjamini, Y. A., and Hochberg, Y. (1995), “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,” Journal of the Royal Statistical Society, Series B, 57, 289–300, available at http://www.math.tau.ac.il/∼ybenja/MyPapers/benjamini_hochberg1995.pdf DOI: 10.1111/j.2517-6161.1995.tb02031.x.
   Web of Science ®Google Scholar
• Berger, J. O., and Delampady, M. (1987), “Testing Precise Hypotheses,” Statistical Science, 2, 317–352. DOI: 10.1214/ss/1177013238.
   Google Scholar
• Berger, J. O., and Sellke, T. (1987), “Testing A Point Null Hypothesis – the Irreconcilability of P-Values and Evidence,” Journal of the American Statistical Association, 82, 112–122. DOI: 10.2307/2289131.
   Web of Science ®Google Scholar
• Carlin, B. P., and Louis, T. A. (1996), “Identifying Prior Distributions that Produce Specific Decisions, with Application to Monitoring Clinical Trials,” in Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner, eds. D. Berry, Chaloner and J. Geweke, New York: Wiley, pp. 493–503.
   Google Scholar
• Colquhoun, D. (1971) Lectures on Biostatistics, London, UK: Oxford University Press, available at http://www.dcscience.net/Lectures\_on\_biostatistics-ocr4.pdf.
   Google Scholar
• Colquhoun, D. (2014), “An Investigation of the False Discovery Rate and the Misinterpretation of p-values,” Royal Society Open Science, 1, 140216, available http://rsos.royalsocietypublishing.org/content/1/3/140216
   PubMed Web of Science ®Google Scholar
• Colquhoun, D. (2015), “The Perils of p-values,” Chalkdust Magazine, available at http://chalkdustmagazine.com/features/the-perils-of-p-values/
   Google Scholar
• Colquhoun, D. (2016), “The Problem with p-values,” Aeon Magazine, available at https://aeon.co/essays/it-s-time-for-science-to-abandon-the-term-statistically-significant
   Google Scholar
• Colquhoun, D. (2017), “The Reproducibility of Research and the Misinterpretation of P Values,” Royal Society Open Science, 4, 171085, available at http://rsos.royalsocietypublishing.org/content/4/12/171085.
   PubMed Web of Science ®Google Scholar
• Colquhoun, D. (2018), “The False Positive Risk: A Proposal Concerning What to Do About p-values (version 2),” available at https://www.youtube.com/watch?v=jZWgijUnIxI
   Google Scholar
• Colquhoun, D., and Longstaff, C. (2017), “False Positive Risk Calculator”, available http://fpr-calc.ucl.ac.uk/
   Google Scholar
• Curtis, M. J., Bond, R. A., Spina, D., Ahluwalia, A., Alexander, S. P. A., Giembycz, M. A., Gilchrist, A., Hoyer, D., Insel, P. A., Izzo, A. A., et al. (2014), “Experimental Design and Analysis and their Reporting: New Guidance for Publication in BJP,” British Pharmacological Society, available at http://onlinelibrary.wiley.com/doi/10.1111/bph.12856/full
   Google Scholar
• Cushny, A. R., & Peebles, A. R. (1905), “The Action of Optical Isomers: II. Hyoscines,” The Journal of Physiology, 32, 501–510, available at https://physoc.onlinelibrary.wiley.com/doi/abs/10.1113/jphysiol.1905.sp001097 DOI: 10.1113/jphysiol.1905.sp001097.
   PubMedGoogle Scholar
• Gigerenzer, G., Krauss, S., & Vitouch, O. (2004), “The Null Ritual. What You Always Wanted to Know About Significance Testing but Were Afraid to Ask,” in ed. D. Kaplan, The Sage Handbook of Quantitative Methodology for the Social Sciences, Thousand Oaks, CA: Sage Publishing, pp. 391–408.
   Google Scholar
• Good, I. J. (1950), Probability and the Weighing of Evidence, London: Charles Griffin and Co. Ltd.
   Google Scholar
• Goodman, S. N. (1993), “p Values, Hypothesis Tests, and Likelihood: Implications for Epidemiology of a Neglected Historical Debate,” American Journal of Epidemiology, 137, 485–496. DOI: 10.1093/oxfordjournals.aje.a116700.
   PubMed Web of Science ®Google Scholar
• Goodman, S. N. (1999a), “Toward Evidence-based Medical Statistics. 1: The P value Fallacy,” Annals of Internal Medicine, 130, 995–1004.
   PubMed Web of Science ®Google Scholar
• Goodman, S. N. (1999b), “Toward Evidence-based Medical Statistics. 2: The Bayes Factor,” Annals of Internal Medicine, 130, 1005–1013.
   PubMed Web of Science ®Google Scholar
• Lennox, K. (2016), “All About that Bayes: Probability, Statistics, and the Quest to Quantify Uncertainty,” available at https://www.youtube.com/watch?v=eDMGDhyDxuY
   Google Scholar
• Lindley, D. V. (1957), “A Statistical Paradox,” Biometrika, 44, 187–192. DOI: 10.1093/biomet/44.1-2.187.
   Web of Science ®Google Scholar
• Held, L. (2013), “Reverse-Bayes Analysis of Two Common Misinterpretations of Significance Tests,” Clinical Trials (London, England), 10, 236–242. DOI: 10.1177/1740774512468807.
   PubMed Web of Science ®Google Scholar
• Held, L., and Ott, M. (2018), “On p-Values and Bayes Factors,” Annual Review of Statistics and Its Application, 5, 393–419 DOI: 10.1146/annurev-statistics-031017-100307.
   Web of Science ®Google Scholar
• Johnson, V. E. (2013a), “Revised Standards for Statistical Evidence,” Proceedings of the National Academy of Sciences of the United States of America, 110, 19313–19317. DOI: 10.1073/pnas.1313476110.
   PubMed Web of Science ®Google Scholar
• Johnson, V. E. (2013b), “Uniformly Most Powerful Bayesian Tests,” The Annals of Statistics, 41, 1716–1741. [2]
   PubMed Web of Science ®Google Scholar
• Johnson, V. E. (2018), “Beyond ‘Significance’: Principles and Practice of the Analysis of Credibility,” Royal Society Open Science, 5, 171047, available http://rsos.royalsocietypublishing.org/content/5/1/171047
   PubMed Web of Science ®Google Scholar
• Matthews, R. A. J. (2001), “Why Should Clinicians Care About Bayesian Methods?,” Journal of Statistical Planning and Inference, 94, 43–58.
   Google Scholar
• Sellke, T., Bayarri, M. J., and Berger, J. O. (2001), “Calibration of p Values for Testing Precise Null Hypotheses,” The American Statistician, 55, 62–71. DOI: 10.1198/000313001300339950.
   Web of Science ®Google Scholar
• Senn, S. J. (2015), “Double Jeopardy?: Judge Jeffreys Upholds the Law (Sequel to the Pathetic P-value),” in Error Statistics Philosophy, available at https://errorstatistics.com/2015/05/09/stephen-senn-double-jeopardy-judge-jeffreys-upholds-the-law-guest-post/2017
   Google Scholar
• Student (1908), “The Probable Error of a Mean,” Biometrika, 6, 1–25.
   Google Scholar
• Wacholder, S., Chanock, S., Garcia-Closas, M., El, G. L., and Rothman, N. (2004), “Assessing the Probability that a Positive Report is False: An Approach for Molecular Epidemiology Studies,” Journal of the National Cancer Institute, 96, 434–442. DOI: 10.1093/jnci/djh075.
   PubMed Web of Science ®Google Scholar
• Wang, J. X., Rogers, L. M., Gross, E. Z., Ryals, A. J., Dokucu, M. E., Brandstatt, K. L., Hermiller, M. S., and Voss, J. L. (2014), “Targeted Enhancement of Cortical-hippocampal Brain Networks and Associative Memory,” Science, 345, 1054–1057. DOI: 10.1126/science.1252900.
   PubMed Web of Science ®Google Scholar
• Wasserstein, R. L., and Lazar, N. A. (2016), “The ASA’s Statement on p-Values: Context, Process, and Purpose,” The American Statistician, 70, 129–133. DOI: 10.1080/00031305.2016.1154108.
   Web of Science ®Google Scholar