Advanced search
Publication Cover
The American Statistician Volume 73, 2019 - Issue sup1: Statistical Inference in the 21st Century: A World Beyond p < 0.05
Submit an article Journal homepage
19,133
Views
85
CrossRef citations to date
0
Altmetric
Supplementing or Replacing p

The False Positive Risk: A Proposal Concerning What to Do About p-Values

David ColquhounUniversity College London, London, UKCorrespondence[email protected]
Pages 192-201 | Received 12 Feb 2018, Accepted 12 Jul 2018, Published online: 20 Mar 2019

Citations (85)

Keep up to date with the latest research on this topic with citation updates for this article.

Subscribe to citation updates

Read on this site (7)

David R. Bickel. (2023) Fiducialize statistical significance: transforming p-values into conservative posterior probabilities and Bayes factors. Statistics 57:4, pages 941-959.
Read now
Nathaniel T. Stevens & Luke Hagar. (2022) Comparative Probability Metrics: Using Posterior Probabilities to Account for Practical Equivalence in A/B tests. The American Statistician 76:3, pages 224-237.
Read now
Joshua Habiger & Ye Liang. (2022) Publication Policies for Replicable Research and the Community-Wide False Discovery Rate. The American Statistician 76:2, pages 131-141.
Read now
Piero Quatto, Enrico Ripamonti & Donata Marasini. (2022) Beyond p < .05: a critical review of new Bayesian proposals for assessing the p-value. Journal of Biopharmaceutical Statistics 32:2, pages 308-329.
Read now
David R. Bickel. (2021) Null Hypothesis Significance Testing Defended and Calibrated by Bayesian Model Checking. The American Statistician 75:3, pages 249-255.
Read now
David R. Bickel. (2021) Null Hypothesis Significance Testing Interpreted and Calibrated by Estimating Probabilities of Sign Errors: A Bayes-Frequentist Continuum. The American Statistician 75:1, pages 104-112.
Read now
Ronald L. Wasserstein, Allen L. Schirm & Nicole A. Lazar. (2019) Moving to a World Beyond “p < 0.05”. The American Statistician 73:sup1, pages 1-19.
Read now

Articles from other publishers (78)

Roger Giner-Sorolla, Amanda K. Montoya, Alan Reifman, Tom Carpenter, Neil A. LewisJrJr, Christopher L. Aberson, Dries H. Bostyn, Beverly G. Conrique, Brandon W. Ng, Alexander M. Schoemann & Courtney Soderberg. (2024) Power to Detect What? Considerations for Planning and Evaluating Sample Size. Personality and Social Psychology Review 28:3, pages 276-301.
Crossref
Ahtisham Younas. (2024) Beyond ‘statistical significance’: A nontechnical primer of Bayesian statistics and Bayes factors for health researchers. Journal of Evaluation in Clinical Practice.
Crossref
Lari M. Koponen, Miles Martinez, Eleanor Wood, David L. K. Murphy, Stefan M. Goetz, Lawrence G. Appelbaum & Angel V. Peterchev. (2024) Transcranial magnetic stimulation input–output curve slope differences suggest variation in recruitment across muscle representations in primary motor cortex. Frontiers in Human Neuroscience 18.
Crossref
David Sidebotham, Felicity Dominick, Carolyn Deng, Jake Barlow & Philip M. Jones. (2024) Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia. British Journal of Anaesthesia 132:1, pages 116-123.
Crossref
Margaret MacDougall. 2024. A Medical Educator's Guide to Thinking Critically about Randomised Controlled Trials: Deconstructing the "Gold Standard". A Medical Educator's Guide to Thinking Critically about Randomised Controlled Trials: Deconstructing the "Gold Standard" 1 32 .
David Sidebotham, C. Jake Barlow, Janet Martin & Philip M. Jones. (2023) Interpreting frequentist hypothesis tests: insights from Bayesian inferenceInterprétation des tests d’hypothèses fréquentistes : des enseignements tirés de l’inférence bayésienne. Canadian Journal of Anesthesia/Journal canadien d'anesthésie 70:10, pages 1560-1575.
Crossref
Noé Valtierra, Lloyd A. Courtenay, Albert Fabregat‐Sanjuan & Lucía López‐Polín. (2023) Cleaning archaeological bones: Influence of water, ethanol, and acetone on microhardness. International Journal of Osteoarchaeology 33:5, pages 967-972.
Crossref
Jong-Seol Kang, Dongha Kim, Joonwoo Rhee, Ji-Yun Seo, Inkuk Park, Ji-Hoon Kim, Daewon Lee, WonUk Lee, Ye Lynne Kim, Kyusang Yoo, Sunghwan Bae, Jongkyeong Chung, Rho Hyun Seong & Young-Yun Kong. (2023) Baf155 regulates skeletal muscle metabolism via HIF-1a signaling. PLOS Biology 21:7, pages e3002192.
Crossref
Alejandra Ospina-Bohórquez, Susana Del Pozo, Lloyd A. Courtenay & Diego González-Aguilera. (2023) Handheld stereo photogrammetry applied to crime scene analysis. Measurement 216, pages 112861.
Crossref
Lloyd A. Courtenay, Julia Aramendi & Diego González‐Aguilera. (2023) Recruiting a skeleton crew—Methods for simulating and augmenting paleoanthropological data using Monte Carlo based algorithms. American Journal of Biological Anthropology 181:3, pages 454-473.
Crossref
Duygu Uygun Tunç, Mehmet Necip Tunç & Daniël Lakens. (2023) The epistemic and pragmatic function of dichotomous claims based on statistical hypothesis tests. Theory & Psychology 33:3, pages 403-423.
Crossref
Lloyd A. Courtenay, José Yravedra, Darío Herranz-Rodrigo, Alexia Serrano-Ramos, Verónica Estaca-Gómez, Diego González-Aguilera, José Antonio Solano & Juan Manuel Jiménez Arenas. (2023) Is Barranco León (1.46 Ma, Orce, Granada, Spain) much more complicated than we think? A reply to Palmqvist et al. (2023). Quaternary Science Reviews 310, pages 108119.
Crossref
Zachary Chuang, Janet Martin, Jordan Shapiro, Derek Nguyen, Penelope Neocleous & Philip M. Jones. (2023) Minimum false-positive risk of primary outcomes and impact of reducing nominal P-value threshold from 0.05 to 0.005 in anaesthesiology randomised clinical trials: a cross-sectional study. British Journal of Anaesthesia 130:4, pages 412-420.
Crossref
Paweł Sosnowski, Piotr Sass, Anna Stanisławska-Sachadyn, Michał Krzemiński & Paweł Sachadyn. (2023) Between therapy effect and false-positive result in animal experimentation. Biomedicine & Pharmacotherapy 160, pages 114317.
Crossref
Lloyd A. Courtenay. (2022) Can we Restore Balance to Geometric Morphometrics? A Theoretical Evaluation of how Sample Imbalance Conditions Ordination and Classification. Evolutionary Biology 50:1, pages 90-110.
Crossref
Lloyd A. Courtenay, José Yravedra, Darío Herranz-Rodrigo, Juan José Rodríguez-Alba, Alexia Serrano-Ramos, Verónica Estaca-Gómez, Diego González-Aguilera, José Antonio Solano & Juan Manuel Jiménez-Arenas. (2023) Deciphering carnivoran competition for animal resources at the 1.46 Ma early Pleistocene site of Barranco León (Orce, Granada, Spain). Quaternary Science Reviews 300, pages 107912.
Crossref
N. Christy Evangeline, S. Srinivasan & E. Suresh. (2023) Application of non-contact thermography as a screening modality for Diabetic Foot Syndrome – A real time cross sectional research outcome. Biomedical Signal Processing and Control 79, pages 104054.
Crossref
Eric-Jan Wagenmakers & Alexander Ly. (2022) History and nature of the Jeffreys–Lindley paradox. Archive for History of Exact Sciences 77:1, pages 25-72.
Crossref
David Sidebotham. (2022) Fooled by Significance Testing: An Analysis of the LOVIT Vitamin C Trial. The Journal of ExtraCorporeal Technology 54:4, pages 324-329.
Crossref
Rocío Mora, Julia Aramendi, Lloyd A. Courtenay, Diego González-Aguilera, José Yravedra, Miguel Ángel Maté-González, Diego Prieto-Herráez, José Mª Vázquez-Rodríguez & Isabel Barja. (2022) Ikhnos: A Novel Software to Register and Analyze Bone Surface Modifications Based on Three-Dimensional Documentation. Animals 12:20, pages 2861.
Crossref
Eleftherios Gkioulekas, Peter A. McCullough & Vladimir Zelenko. (2022) Statistical Analysis Methods Applied to Early Outpatient COVID-19 Treatment Case Series Data. COVID 2:8, pages 1139-1182.
Crossref
José Yravedra, Lloyd Austin Courtenay, Darío Herranz-Rodrigo, Gonzalo Linares-Matás, Juan José Rodríguez-Alba, Verónica Estaca-Gómez, Carmen Luzón, Alexia Serrano-Ramos, Miguel Ángel Maté-González, José Antonio Solano, Diego González-Aguilera & Juan Manuel Jiménez-Arenas. (2022) Taphonomic characterisation of tooth marks of extinct Eurasian carnivores through geometric morphometrics. Science Bulletin 67:16, pages 1644-1648.
Crossref
Lloyd A. Courtenay, Inés Barbero-García, Julia Aramendi, Diego González-Aguilera, Manuel Rodríguez-Martín, Pablo Rodríguez-Gonzalvez, Javier Cañueto & Concepción Román-Curto. (2022) A Novel Approach for the Shape Characterisation of Non-Melanoma Skin Lesions Using Elliptic Fourier Analyses and Clinical Images. Journal of Clinical Medicine 11:15, pages 4392.
Crossref
Sara Garofalo, Sara Giovagnoli, Matteo Orsoni, Francesca Starita & Mariagrazia Benassi. (2022) Interaction effect: Are you doing the right thing?. PLOS ONE 17:7, pages e0271668.
Crossref
Sarah Nostedt & Ari R. Joffe. (2021) Reverse Bayesian Implications of p-Values Reported in Critical Care Randomized Trials. Journal of Intensive Care Medicine 37:7, pages 954-964.
Crossref
Margaret Slota, Maureen McLaughlin, Sarah Vittone & Nancy Crowell. (2022) Visual intelligence education using an art-based intervention: Outcomes evaluation with nursing graduate students. Journal of Professional Nursing 41, pages 1-7.
Crossref
Leonhard Held, Robert Matthews, Manuela Ott & Samuel Pawel. (2021) Reverse‐Bayes methods for evidence assessment and research synthesis . Research Synthesis Methods 13:3, pages 295-314.
Crossref
Maximilian Maier & Daniël Lakens. (2022) Justify Your Alpha: A Primer on Two Practical Approaches. Advances in Methods and Practices in Psychological Science 5:2, pages 251524592210803.
Crossref
Igor Khorozyan. (2021) Defining practical and robust study designs for interventions targeted at terrestrial mammalian predators. Conservation Biology 36:2.
Crossref
Alain Bultez, Christian Derbaix & Jean-Luc Herrmann. (2021) Statistically significant? Let us recognize that estimates of tested effects are uncertain. Recherche et Applications en Marketing (English Edition) 37:1, pages 82-105.
Crossref
David Sidebotham & C. Jake Barlow. (2022) False-positive and false-negative risks for individual multicentre trials in critical care. BJA Open 1, pages 100003.
Crossref
Alain Bultez, Christian Derbaix & Jean-Luc Herrmann. (2021) « Statistiquement significatif » ? Respectons l’incertitude de l’effet testé. Recherche et Applications en Marketing (French Edition) 37:1, pages 87-112.
Crossref
Masood M. Khan & Jordan Vice. (2022) Toward Accountable and Explainable Artificial Intelligence Part One: Theory and Examples. IEEE Access 10, pages 99686-99701.
Crossref
Jennifer A. Cooper, David Jenkinson, Chris Stinton, Matthew G. Wallis, Sue Hudson & Sian Taylor-Phillips. (2021) Optimising breast cancer screening reading: blinding the second reader to the first reader’s decisions. European Radiology 32:1, pages 602-612.
Crossref
Gulsah Gurkan, Yoav Benjamini & Henry Braun. (2021) Defensible inferences from a nested sequence of logistic regressions: a guide for the perplexed. Large-scale Assessments in Education 9:1.
Crossref
David Robert Grimes & James Heathers. (2021) The new normal? Redaction bias in biomedical science. Royal Society Open Science 8:12.
Crossref
José Yravedra, Darío Herranz-Rodrigo, Cecilia Mendoza, Pablo Aragón-Poza & Lloyd A. Courtenay. (2021) The use of tooth marks for new research into identifying and understanding the first domestic dogs in Palaeolithic populations. Journal of Archaeological Science: Reports 40, pages 103252.
Crossref
Isaac Kwesi Abuley & Jens Grønbech Hansen. (2021) An epidemiological analysis of the dilemma of plant age and late blight (Phytophthora infestans) susceptibility in potatoes. European Journal of Plant Pathology 161:3, pages 645-663.
Crossref
Asmita Sengupta, Kanthi Arum Widayati, Yamato Tsuji, Risma Yanti, Muhammad Fadli Rahman, Nagarathna Balakrishna & Sindhu Radhakrishna. (2021) Why do people visit primate tourism sites? Investigating macaque tourism in Japan and Indonesia. Primates 62:6, pages 981-993.
Crossref
Morteza Khodaee, Bjørn Irion, Jack Spittler, Anahita Saeedi & Martin D. Hoffman. (2021) Characteristics of runners meeting acute kidney injury criteria following a 161‐km ultramarathon. TRANSLATIONAL SPORTS MEDICINE 4:6, pages 733-740.
Crossref
Chenlu Di, Jesus Murga Moreno, Diego F Salazar-Tortosa, M Elise Lauterbur & David Enard. (2021) Decreased recent adaptation at human mendelian disease genes as a possible consequence of interference between advantageous and deleterious variants. eLife 10.
Crossref
Petr Soukup. (2021) Může Bayesův faktor nahradit P hodnotu?. Sociální studia / Social Studies 18:1, pages 73-99.
Crossref
Tomas L. Bothe, Andreas Patzak, Rudolf Schubert & Niklas Pilz. (2021) Getting it right matters! Covid‐19 pandemic analogies to everyday life in medical sciences. Acta Physiologica 233:1.
Crossref
Chris M. Bleakley, Mark Matthews & James M. Smoliga. (2021) Most ankle sprain research is either false or clinically unimportant: A 30-year audit of randomized controlled trials. Journal of Sport and Health Science 10:5, pages 523-529.
Crossref
Mikko Ketokivi & Saku Mantere. (2021) What warrants our claims? A methodological evaluation of argument structure. Journal of Operations Management 67:6, pages 755-776.
Crossref
Darío Herranz-Rodrigo, Silvia J. Tardáguila-Giacomozzi, Lloyd A. Courtenay, Juan-José Rodríguez-Alba, Antonio Garrucho, Jesús Recuero & José Yravedra. (2021) New Geometric Morphometric Insights in Digital Taphonomy: Analyses into the Sexual Dimorphism of Felids through Their Tooth Pits. Applied Sciences 11:17, pages 7848.
Crossref
Igor Khorozyan. (2021) Setting Statistical Thresholds Is Useful to Define Truly Effective Conservation Interventions. Frontiers in Ecology and Evolution 9.
Crossref
Lloyd A. Courtenay, Darío Herranz-Rodrigo, José Yravedra, José Mª Vázquez-Rodríguez, Rosa Huguet, Isabel Barja, Miguel Ángel Maté-González, Maximiliano Fernández Fernández, Ángel-Luis Muñoz-Nieto & Diego González-Aguilera. (2021) 3D Insights into the Effects of Captivity on Wolf Mastication and Their Tooth Marks; Implications in Ecological Studies of Both the Past and Present. Animals 11:8, pages 2323.
Crossref
Igor Khorozyan. (2021) Dealing with false positive risk as an indicator of misperceived effectiveness of conservation interventions. PLOS ONE 16:8, pages e0255784.
Crossref
Lloyd A. Courtenay, Diego González-Aguilera, Susana Lagüela, Susana del Pozo, Camilo Ruiz-Mendez, Inés Barbero-García, Concepción Román-Curto, Javier Cañueto, Carlos Santos-Durán, María Esther Cardeñoso-Álvarez, Mónica Roncero-Riesco, David Hernandez-Lopez, Diego Guerrero-Sevilla & Pablo Rodríguez-Gonzalvez. (2021) Hyperspectral imaging and robust statistics in non-melanoma skin cancer analysis. Biomedical Optics Express 12:8, pages 5107.
Crossref
Nancy A. Crowell, Alan Hanson, Louisa Boudreau, Robyn Robbins & Rosemary K. Sokas. (2021) Union Efforts to Reduce COVID-19 Infections Among Grocery Store Workers. NEW SOLUTIONS: A Journal of Environmental and Occupational Health Policy 31:2, pages 170-177.
Crossref
Alvin Gavel, Pieter Gruyters, Ulrike Heiter, Andreas J. Korn, Thomas Nordlander, Kilian H. Scheutwinkel & Olivier A. Richard. (2021) Atomic diffusion and mixing in old stars. Astronomy & Astrophysics 652, pages A75.
Crossref
Yassine Souilmi, M. Elise Lauterbur, Ray Tobler, Christian D. Huber, Angad S. Johar, Shayli Varasteh Moradi, Wayne A. Johnston, Nevan J. Krogan, Kirill Alexandrov & David Enard. (2021) An ancient viral epidemic involving host coronavirus interacting genes more than 20,000 years ago in East Asia. Current Biology 31:16, pages 3504-3514.e9.
Crossref
Salem Alawbathani, Mehreen Batool, Jan Fleckhaus, Sarkawt Hamad, Floyd Hassenrück, Yanhong Hou, Xia Li, Jon Salmanton-García, Sami Ullah, Frederique Wieters & Martin C. Michel. (2021) A teaching tool about the fickle p value and other statistical principles based on real-life data. Naunyn-Schmiedeberg's Archives of Pharmacology 394:6, pages 1315-1319.
Crossref
Lloyd A. Courtenay, Darío Herranz-Rodrigo, Diego González-Aguilera & José Yravedra. (2021) Developments in data science solutions for carnivore tooth pit classification. Scientific Reports 11:1.
Crossref
Mark Matthews, William Johnston, Chris M. Bleakley, Richard J. Davies, Alan T. Rankin, Michael Webb, Brian C. Caulfield & H.A.P. Archbold. (2021) Concussion History and Balance Performance in Adolescent Rugby Union Players. The American Journal of Sports Medicine 49:5, pages 1348-1354.
Crossref
María José Díaz, Christian Buschbaum, Paul E. Renaud & Markus Molis. (2021) Effects of Detached Seaweeds on Structure and Function of Arctic Intertidal Soft-Bottom Communities. Frontiers in Marine Science 8.
Crossref
Lucas R Philipp, Caio M Matias, Sara Thalheimer, Shyle H Mehta, Ashwini Sharan & Chengyuan Wu. (2021) Robot-Assisted Stereotaxy Reduces Target Error: A Meta-Analysis and Meta-Regression of 6056 Trajectories. Neurosurgery 88:2, pages 222-233.
Crossref
Zad Rafi & Sander Greenland. (2020) Semantic and cognitive tools to aid statistical science: replace confidence and significance by compatibility and surprise. BMC Medical Research Methodology 20:1.
Crossref
Jian Gao. (2020) P-values – a chronic conundrum. BMC Medical Research Methodology 20:1.
Crossref
Fredi Alexander Diaz-Quijano, Fernando Morelli Calixto & José Mário Nunes da Silva. (2020) How feasible is it to abandon statistical significance? A reflection based on a short survey. BMC Medical Research Methodology 20:1.
Crossref
Chris Bleakley & James M Smoliga. (2020) Validating new discoveries in sports medicine: we need FAIR play beyond p values. British Journal of Sports Medicine 54:21, pages 1239-1240.
Crossref
Peter M. B. Cahusac. 2020. Evidence‐Based Statistics. Evidence‐Based Statistics 197 218 .
D. Sidebotham. (2020) Are most randomised trials in anaesthesia and critical care wrong? An analysis using Bayes’ theorem. Anaesthesia 75:10, pages 1386-1393.
Crossref
Monise Helen Masuchi Buscato, Franziska Müller, Walter Vetter, Jochen Weiss & Hanna Salminen. (2020) Furan fatty acids in enriched ω-3 fish oil: Oxidation kinetics with and without added monomethyl furan fatty acid as potential natural antioxidant. Food Chemistry 327, pages 127087.
Crossref
A. Richter & A. Zink. (2020) Gehört die statistische Signifikanz aufs Altenteil?Should statistical significance be retired?. Zeitschrift für Rheumatologie 79:7, pages 692-695.
Crossref
NELSON LERNER BARTH & CARLOS EDUARDO LOURENÇO. (2020) O P AINDA TEM VALOR?. Revista de Administração de Empresas 60:3, pages 235-241.
Crossref
M.P.K. Webb & D. Sidebotham. (2020) Bayes' formula: a powerful but counterintuitive tool for medical decision-making. BJA Education 20:6, pages 208-213.
Crossref
James B. Schreiber. (2020) New paradigms for considering statistical significance: A way forward for health services research journals, their authors, and their readership. Research in Social and Administrative Pharmacy 16:4, pages 591-594.
Crossref
Chris Bleakley, Jonathan Reijgers & James M. Smoliga. (2020) Many High-Quality Randomized Controlled Trials in Sports Physical Therapy Are Making False-Positive Claims of Treatment Effect: A Systematic Survey. Journal of Orthopaedic & Sports Physical Therapy 50:2, pages 104-109.
Crossref
Sarah Farrow & Ari R. Joffe. (2020) Association Between Maternal Fluoride Exposure and Child IQ. JAMA Pediatrics 174:2, pages 214.
Crossref
Martin C. Michel, T.J. Murphy & Harvey J. Motulsky. (2019) New Author Guidelines for Displaying Data and Reporting Data Analysis and Statistical Methods in Experimental Biology. Molecular Pharmacology 97:1, pages 49-60.
Crossref
Martin C. Michel, T.J. Murphy & Harvey J. Motulsky. (2019) New Author Guidelines for Displaying Data and Reporting Data Analysis and Statistical Methods in Experimental Biology. Journal of Pharmacology and Experimental Therapeutics 372:1, pages 136-147.
Crossref
Martin C. Michel, T.J. Murphy & Harvey J. Motulsky. (2019) New Author Guidelines for Displaying Data and Reporting Data Analysis and Statistical Methods in Experimental Biology. Drug Metabolism and Disposition 48:1, pages 64-74.
Crossref
Gordon Drummond. (2019) A world beyond P : policies, strategies, tactics and advice . Experimental Physiology 105:1, pages 13-16.
Crossref
David Colquhoun. (2019) A response to critiques of ‘The reproducibility of research and the misinterpretation of p -values’ . Royal Society Open Science 6:11, pages 190819.
Crossref
Ulrich Gunter, Irem Önder & Egon Smeral. (2019) Scientific value of econometric tourism demand studies. Annals of Tourism Research 78, pages 102738.
Crossref
Harriet Pike. (2019) Statistical significance should be abandoned, say scientists. BMJ, pages l1374.
Crossref

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.