What Have We (Not) Learnt from Millions of Scientific Papers with P Values?

References

• Benjamin, D. J., et al. (2018), “Redefine Statistical Significance,” Nature Human Behaviour, 2, 6–10.
   PubMed Web of Science ®Google Scholar
• Bermon, S., and Garnier, P. Y. (2018), “Serum Androgen Levels and their Relation to Performance in Track and Field: Mass Spectrometry Results from 2127 Observations in Male and Female Elite Athletes,” British Journal of Sports Medicine, 51, 1309–1314.
   Web of Science ®Google Scholar
• Boutron, I., Altman, D. G., Hopewell, S., Vera-Badillo, F., Tannock, I., and Ravaud, P. (2014), “Impact of Spin in the Abstracts of Articles Reporting Results of Randomized Controlled Trials in the Field of Cancer: the SPIIN Randomized Controlled Trial,” Journal of Clinical Oncology, 32, 4120–4126.
   PubMed Web of Science ®Google Scholar
• Bruns, S. B., and Ioannidis, J.P (2016), “p-Curve and p-Hacking in Observational Research,” PLoS One, 11, e0149144.
   PubMed Web of Science ®Google Scholar
• Chambers, C. D. (2013), “Registered Reports: a New Publishing Initiative at Cortex,” Cortex, 49, 609–610.
   PubMed Web of Science ®Google Scholar
• Chavalarias, D., Wallach, J. D., Li, A. H., and Ioannidis, J. P. (2016), “Evolution of Reporting P Values in the Biomedical Literature, 1990–2015,” JAMA, 315, 1141–1148.
   PubMed Web of Science ®Google Scholar
• Cristea, I. A., and Ioannidis, J. P. (2018), “P-values in Display Items are Ubiquitous and Almost Invariably Significant: A Survey of Top Science Journals,” PLoS ONE, 13, e0197440.
   PubMed Web of Science ®Google Scholar
• Fanelli, D. (2010), ““Positive” Results Increase Down the Hierarchy of the Sciences,” PLoS One, 5, e10068.
   PubMed Web of Science ®Google Scholar
• Fanelli, D., Costas, R., and Ioannidis, J. P. (2017), “Meta-assessment of Bias in Science,” Proceedings of the National Academy of Sciences U S A, 114, 3714–3719.
   PubMed Web of Science ®Google Scholar
• Gelman, A. (2014), “The Statistical Crisis in Science,” American Scientist, 102, 460–65.
   Web of Science ®Google Scholar
• Gøtzsche, P. C. (2006), “Believability of Relative Risks and Odds Ratios in Abstracts: Cross Sectional Study,” BMJ, 333, 231–234.
   PubMed Web of Science ®Google Scholar
• Ioannidis, J. P. (2018), “The Proposal to Lower P-value Thresholds to .005,” JAMA, 319, 1429–1430.
   PubMed Web of Science ®Google Scholar
• Ioannidis, J. P., and Trikalinos, T. A. (2005), “Early Extreme Contradictory Estimates may Appear in Published Research: the Proteus Phenomenon in Molecular Genetics Research and Randomized Trials,” Journal of Clinical Epidemiology, 58, 543–549.
   Google Scholar
• Khabsa, M., and Giles, C. L. (2014), “The Number of Scholarly Documents on the Public Web,” PLoS One, 9, e93949.
   PubMed Web of Science ®Google Scholar
• Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., and Ioannidis, J. P. (2017), “A Manifesto for Reproducible Science,” Nature Human Behaviour, 1, 0021.
   PubMed Web of Science ®Google Scholar
• Patel, C. J., Burford, B., and Ioannidis, J. P. (2015), “Assessment of Vibration of Effects due to Model Specification can Demonstrate the Instability of Observational Associations,” Journal of Clinical Epidemiology, 68, 1046–1058.
   PubMed Web of Science ®Google Scholar
• Patel, C. J., Ji, J., Sundquist, J., Ioannidis, J. P., and Sundquist, K. (2016), “Systematic Assessment of Pharmaceutical Prescriptions in Association with Cancer Risk: a Method to Conduct a Population-wide Medication-wide Longitudinal Study,” Scientific Reports, 10, 31308.
   Web of Science ®Google Scholar
• Pfeiffer, T., Bertram, L., and Ioannidis, J. P. (2011), “Quantifying Selective Reporting and the Proteus Phenomenon for Multiple Datasets with Similar Bias,” PLoS One, 6, e18362.
   PubMed Web of Science ®Google Scholar
• Simonsohn, U., Nelson, L. D., and Simmons, J. P. (2014), “P-curve: a Key to the File-drawer,” Journal of Experimental Psychology General, 143, 534–547.
   PubMed Web of Science ®Google Scholar
• Szucs, D., and Ioannidis, J. P. (2017a), “When Null Hypothesis Significance Testing is Unsuitable for Research: a Reassessment,” Frontiers in Human Neuroscience, 11, 390.
   PubMed Web of Science ®Google Scholar
• Szucs, D., and Ioannidis, J. P. (2017b), “Empirical assessment of Published Effect Sizes and Power in the Recent Cognitive Neuroscience and Psychology Literature,” PLoS Biology, 15, e2000797.
   PubMed Web of Science ®Google Scholar
• Szucs, D. (2016), “A Tutorial on Hunting Statistical Significance by chasing N,” Frontiers in Psychology, 7, 1444.
   PubMed Web of Science ®Google Scholar
• van Ravenzwaaij, D., and Ioannidis, J. P. (2017), “A Simulation Study of the Strength of Evidence in the Recommendation of Medications Based on Two Trials with Statistically Significant Results,” PLoS One, 12, e0173184.
   PubMed Web of Science ®Google Scholar
• Wasserstein, R. L., and Lazar, N. A. (2016), “The ASA's Statement on p-Values: Context, Process, and Purpose,” The American Statistician, 70, 129–133.
   Web of Science ®Google Scholar
• Young, N. S., Ioannidis, J. P., and Al-Ubaydli, O. (2008), “Why Current Publication Practices may Distort Science,” PLoS Medicine, 5, e201.
   PubMed Web of Science ®Google Scholar