Quality Control for Scientific Research: Addressing Reproducibility, Responsiveness, and Relevance

References

• Barlow, R. E., and Irony, T. Z. (1992), “Foundations of Statistical Quality Control,” in Current Issues in Statistics: Essays in Honor of D. Basu, Lecture Notes—Monograph Series (Vol. 17), New York: Springer.
   Google Scholar
• Berger, J. O., and Sellke, T. (1987), “Testing Point-Null Hypotheses: The Irreconcilability of p-Values and Evidence,” Journal of the American Medical Association, 82, 112–122. DOI: 10.2307/2289131.
   Web of Science ®Google Scholar
• Breiman, L. (2001), “Random Forests,” Machine Learning, 45, 5–32. DOI: 10.1023/A:1010933404324.
   Web of Science ®Google Scholar
• Brownstein, N., Louis, T., O’Hagan, A., and Pendergast, J. (2019), “The Role of Expert Judgment in Statistical Inference and Evidence-Based Decision Making,” The American Statistician.
   PubMed Web of Science ®Google Scholar
• Buckheit, J. B., and Donoho, D. L. (1995), “WaveLab and Reproducible Research,” Technical Report #474, Department of Statistics, Stanford University, Stanford.
   Google Scholar
• Casadevall, A., and Fang, F. C. (2010), “Reproducible Science,” Infection and Immunity, 78, 4972–4975. DOI: 10.1128/IAI.00908-10.
   PubMed Web of Science ®Google Scholar
• Deming, W. E. (2000), Out of the Crisis (1st ed.), Cambridge, MA: MIT Press.
   Google Scholar
• Dickersin, K. (1990), “The Existence of Publication Bias and Risk Factors for Its Occurrence,” Journal of the American Medical Association, 263, 1385–1389. DOI: 10.1001/jama.1990.03440100097014.
   PubMed Web of Science ®Google Scholar
• Fellner, G., and Krüger, S. (2012), “Judgmental Overconfidence: Three Measures, One Bias?,” Journal of Economic Psychology, 33, 142–154. DOI: 10.1016/j.joep.2011.07.008.
   Web of Science ®Google Scholar
• Ferrell, W. R., and Goey, P. J. (1980), “A Model of Calibration of Personal Probabilities,” Organizational Behavior and Human Decision Processes, 26, 32–56. DOI: 10.1016/0030-5073(80)90045-8.
   Web of Science ®Google Scholar
• Fisher, R. A. (1973), The Design of Experiments (9th ed.), New York: MacMillan.
   Google Scholar
• Fraser, D. A. S. (2017), “p-Values: The Insight to Model Statistical Inference,” Annual Reviews in Statistics and its Applications, 4, 1–14. DOI: 10.1146/annurev-statistics-060116-054139.
   Web of Science ®Google Scholar
• Friedberg, M., Saffran, B., Stinson, T. J., Nelson, W., and Bennett, C. L. (1999), “Evaluation of Conflict of Interest in Economic Analyses of New Drugs Used in Oncology,” Journal of the American Medical Association, 282, 1453–1957. DOI: 10.1001/jama.282.15.1453.
   PubMed Web of Science ®Google Scholar
• Goodman, S. N. (1999), “Toward Evidence-Based Medical Statistics: 1. The p-Value Fallacy,” Annals of Internal Medicine, 130, 995–1004. DOI: 10.7326/0003-4819-130-12-199906150-00008.
   PubMed Web of Science ®Google Scholar
• Howard, R. A., and Abbas, A. E. (2015), Foundations of Decision Analysis (1st ed.), New York: Pearson.
   Google Scholar
• Howard, R. A., and Matheson, J. E. (1977), Readings in Decision Analysis, Menlo Park, CA: SRI International.
   Google Scholar
• Howard, R. A., and Matheson, J. E. (1984), Readings on the Principles and Applications of Decision Analysis (2 vols.), Menlo Park, CA: Strategic Decisions Group.
   Google Scholar
• Hubbard, R. (2016), Corrupt Research: The Case for Re-conceptualizing Empirical Management and Social Science (1st ed.), Los Angeles, CA: Sage.
   Google Scholar
• Ioannidis, J. P. (2005), “Why Most Published Research Findings Are False,” PLoS Medicine, 2, e124, DOI: 10.1371/journal.pmed.0020124.
   PubMed Web of Science ®Google Scholar
• Jager, L. R., and Leek, J. T. (2014), “An Estimate of the Science-Wise False Discovery Rate and Application to the Top Medical Literature,” Biostatistics, 15, 1–12. DOI: 10.1093/biostatistics/kxt007.
   PubMed Web of Science ®Google Scholar
• Jaynes, E. T. (1958), “Prior Probabilities,” IEEE Transactions on Systems Science and Cybernetics, 4, 227–241. DOI: 10.1109/TSSC.1968.300117.
   Web of Science ®Google Scholar
• Jaynes, E. T. (2003), Probability Theory: The Logic of Science, London: Cambridge University Press.
   Google Scholar
• Kerr, N. L. (1998), “HARKing: Hypothesizing After the Results are Known,” Personality and Social Psychology Review, 2, 196–217. DOI: 10.1207/s15327957pspr0203_4.
   PubMedGoogle Scholar
• Lichteinstein, S., Fischhoff, B., and Phillips, L. (1981), Calibration of Probabilities: The State of the Art to 1980. Technical Report PTR-1092-81-6 to Offfice of Naval Research, Contract No. NOOO14-80-C-0150.
   Google Scholar
• Munafò, M. R., Nosek, B., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U.,Wagenmakers, E.-J., Ware, J. J, and Ioannidis, J. P. (2017), “A Manifesto for Reproducible Science,” Nature Human Behavior, 1, 0021. DOI: 10.1038/s41562-016-0021.
   PubMed Web of Science ®Google Scholar
• Murphy, A. H., and Winkler, R. L. (1977), “Reliability of subjective Probability Forecasts of Precipitation and Temperature,” Journal of the Royal Statistical Society, Series C, 26, 41–47. DOI: 10.2307/2346866.
   Google Scholar
• Nuzzo, R. (2014), “Scientific Method: Statistical Errors,” Nature, 506, 150–152. DOI: 10.1038/506150a.
   PubMed Web of Science ®Google Scholar
• O’Hagan, A. (2019), “Expert Knowledge Elicitation: Subjective But Scientific,” The American Statistician.
   PubMed Web of Science ®Google Scholar
• Open Science Collaboration (2015), “Estimating the Reproducibility of Psychological Science,” Science, 349, aac4716.
   PubMed Web of Science ®Google Scholar
• Popper, K. (1959), The Logic of Scientific Discovery, London: Routledge.
   Google Scholar
• Rochon, P. A., Gurwitz, J. H., Simms, R. W., Fortin, P. R., Felson, D. T., Minaker, K. L., and Chalmers, T. C. (1994), “A Study of Manufacturer Supported Trials of Non-steroidal Anti-inflammatory Drugs in the Treatment of Arthritis,” Archives of Internal Medicine, 154, 157–163. DOI: 10.1001/archinte.1994.00420020059007.
   PubMedGoogle Scholar
• Rosenthal, R. (1979), “The File Drawer Problems and Tolerance for Null Results,” Psychological Bulletin, 86, 638–641. DOI: 10.1037/0033-2909.86.3.638.
   Web of Science ®Google Scholar
• Sackett, D. L. (1979), “Bias in Analytic Research,” Journal of Chronic Diseases, 32, 51–63.
   PubMedGoogle Scholar
• Savage, L. J. (1954), The Foundations of Statistics, New York: Wiley.
   Google Scholar
• Sellke, T., Bayarri, M. J., and Berger, J. O. (2001), “Calibration of p-Values for Testing Precise Null Hypothesis,” The American Statistician, 55, 62–71. DOI: 10.1198/000313001300339950.
   Web of Science ®Google Scholar
• Sterling, T. (1959), “Publication Decisions and Their Possible Effects on Inferences Drawn From Tests of Significance,” Journal of the American Statistical Association, 54, 30–34. DOI: 10.2307/2282137.
   Web of Science ®Google Scholar
• Stodden, V. (2015), “Reproducing Statistical Results,” Annual Reviews of Statistics and its Application, 2, 1–19. DOI: 10.1146/annurev-statistics-010814-020127.
   Web of Science ®Google Scholar
• Stodden V., Guo, P., and Ma, Z. (2013), “Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals,” PLoS One, 8, e6711.
   Web of Science ®Google Scholar
• Ungar, L., Mellors, B., Satopää, V., Baron, J., Tetlock, P., Ramos, J., and Swift, S. (2012), “The Good Judgment Project: A Large Scale Test of Different Methods of Combining Expert Predictions,” AAAI Technical Report FS-12-06, Association for the Advancement of Artificial Intelligence, University of Pennsylvania.
   Google Scholar
• Von Neumann, J., and Morgenstern, O. (1944), The Theory of Games and Economic Behavior, Princeton, NJ: Princeton University Press.
   Google Scholar
• Wald, A. (1950), Statistical Decision Functions, New York: Wiley.
   Google Scholar