The Limited Role of Formal Statistical Inference in Scientific Inference

References

• Berk, R. A., and Freedman, D. (2003), “Statistical Assumptions as Empirical Commitments,” in Law, Punishment, and Social Control: Essays in Honor of Sheldon Messinger, (2nd ed.), eds. T. G. Blomberg and S. Cohen, Hawthorne, NY: Aldine de Gruyter, pp. 235–254.
   Google Scholar
• Berry, D. A. (2016), “P-values Are Not What They're Cracked Up to Be,” online supplemental comment to Wasserstein (2016).
   Google Scholar
• Bezeau, S., and Graves, R. (2001), “Statistical Power and Effect Sizes of Clinical Neuropsychology Research,” Journal of Clinical and Experimental Neuropsychology, 23, 399–406.
   PubMed Web of Science ®Google Scholar
• Blair, E., and Zinkhan, G. M. (2006), “Nonresponse and Generalizability in Academic Research,” Journal of the Academy of Marketing Science, 34, 4–7.
   Web of Science ®Google Scholar
• Bottai, M. (2014), “Inferences and Conjectures About Average and Conditional Treatment Effects in Randomized Trials and Observational Studies,” Journal of Internal Medicine, 276, 229–237.
   PubMed Web of Science ®Google Scholar
• Cartwright, N. (2007), “Are RCTs the Gold Standard?” BioSocieties, 2, 11–20.
   Google Scholar
• Chatfield, C. (2002), “Confessions of a Pragmatic Statistician,” The Statistician, 51, 1–20.
   Google Scholar
• Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences (2nd ed.), Hillsdale, NJ: Lawrence Erlbaum.
   Google Scholar
• Cook, T. D., and Campbell, D. T. (1979), Quasi-Experimentation: Design and Analysis Issues for Field Settings, Boston, MA: Houghton Mifflin.
   Google Scholar
• Demidenko, E. (2016), “The p-Value You Can't Buy,” The American Statistician, 70, 33–38.
   PubMed Web of Science ®Google Scholar
• Deming, W. E. (1975), “On Probability As a Basis For Action,” The American Statistician, 29, 146–152.
   Web of Science ®Google Scholar
• Ehrenberg, A. S. C. (1975), Data Reduction (Rev. reprint), London, UK: Charles Griffin.
   Google Scholar
• Fisher, R. A. (1926), “The Arrangement of Field Experiments,” Journal of the Ministry of Agriculture of Great Britain, 33, 503–513.
   Google Scholar
• Fisher, R. A. (1935), The Design of Experiments, Edinburgh: Oliver and Boyd.
   Google Scholar
• ——— (1966), The Design of Experiments (8th ed.), Edinburgh: Oliver and Boyd.
   Google Scholar
• ——— (1970), Statistical Methods for Research Workers (14th ed.), New York: Hafner.
   Google Scholar
• Freedman, D. (1999), “From Association to Causation: Some Remarks on the History of Statistics,” Statistical Science, 14, 243–258.
   Web of Science ®Google Scholar
• Gelman, A. (2016), “The Problems with P-Values are Not Just With P-Values,” online supplemental comment to Wasserstein (2016).
   Google Scholar
• Gigerenzer, G. (1993), “The Superego, the Ego, and the Id in Statistical Reasoning,” in A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues, eds. G. Keren and C. A. Lewis, Hillsdale, NJ: Lawrence Erlbaum, pp. 311–339.
   Google Scholar
• Gigerenzer, G. (2004), “Mindless Statistics,” Journal of Socio-Economics, 33, 587–606.
   Google Scholar
• Gigerenzer, G., and Marewski, J. N. (2015), “Surrogate Science: The Idol of a Universal Method for Scientific Inference,” Journal of Management, 41, 421–440.
   Web of Science ®Google Scholar
• Goodman, S. N. (1993), “p Values, Hypothesis Tests, and Likelihood: Implications for Epidemiology of a Neglected Historical Debate,” American Journal of Epidemiology, 137, 485–496.
   PubMed Web of Science ®Google Scholar
• Grice, J., Barrett, P., Cota, L., Felix, C., Taylor, Z., Garner, S., Medellin, E., and Vest, A. (2017), “Four Bad Habits of Modern Psychologists,” Behavioral Sciences, 7, 1–21.
   Web of Science ®Google Scholar
• Gunter, B., and Tong, C. (2016–2017), “A Response to the ASA Statement on Statistical Significance and P-values,” NV–ASA Newsletter, 14, 1–3.
   Google Scholar
• Guttman, L. (1985), “The Illogic of Statistical Inference for Cumulative Science,” Applied Stochastic Models and Data Analysis, 1, 3–10.
   Google Scholar
• Hahn, G. J., and Meeker, W. Q. (1993), “Assumptions for Statistical Inference,” The American Statistician, 47, 1–11.
   Web of Science ®Google Scholar
• Haig, B. D. (2005), “An Abductive Theory of Scientific Method,” Psychological Methods, 10, 371–388.
   PubMed Web of Science ®Google Scholar
• ——— (2014), Investigating the Psychological World: Scientific Method in the Behavioral Sciences, Cambridge, MA: MIT Press.
   Google Scholar
• Haig, B. D., and Evers, C. W. (2016), Realist Inquiry in Social Science, Thousand Oaks, CA: Sage.
   Google Scholar
• Harman, G. H. (1965), “Inference to the Best Explanation,” Philosophical Review, 74, 88–95.
   Web of Science ®Google Scholar
• Henry, G. T. (1990), Practical Sampling, Newbury Park, CA: Sage.
   Google Scholar
• Hubbard, R. (2016), Corrupt Research: The Case for Reconceptualizing Empirical Management and Social Science, Thousand Oaks, CA: Sage.
   Google Scholar
• Hubbard, R., and Bayarri, M. J. (2003), “Confusion Over Measures of Evidence (p's) Versus Errors (α's) in Classical Statistical Testing,” ( with comments), The American Statistician, 57, 171–182.
   Web of Science ®Google Scholar
• Hubbard, R., and Lindsay, R. M. (2013a), “From Significant Difference to Significant Sameness: Proposing a Paradigm Shift in Business Research,” Journal of Business Research, 66, 1377–1388.
   Web of Science ®Google Scholar
• Hubbard, R., and Lindsay, R. M. (2013b), “The Significant Difference Paradigm Promotes Bad Science,” Journal of Business Research, 66, 1393–1397.
   Web of Science ®Google Scholar
• Iacobucci, D., and Churchill, G. A., Jr. (2010), Marketing Research: Methodological Foundations (10th ed.), Mason, OH: South-Western.
   Google Scholar
• Jadad, A. R., and Enkin, M. W. (2008), Randomized Controlled Trials: Questions, Answers and Musings (2nd ed.), Malden, MA: Blackwell.
   Google Scholar
• Kass, R. E. (2011), “Statistical Inference: The Big Picture,” Statistical Science, 26, 1–9.
   PubMed Web of Science ®Google Scholar
• Kent, D., and Hayward, R. (2007), “When Averages Hide Individual Differences in Clinical Trials,” American Scientist, 95, 60–68.
   Web of Science ®Google Scholar
• Leviton, L. C. (2001), “External Validity,” in International Encyclopedia of the Social and Behaviour Sciences, eds. N. J. Smelser and P. B. Baltes, Oxford, UK: Elsevier, pp. 5195–5200.
   Google Scholar
• Meldrum, M. L. (2000), “A Brief History of the Randomized Controlled Trial: From Oranges and Lemons to the Gold Standard,” Hematology/Oncology Clinics of North America, 14, 745–760.
   PubMed Web of Science ®Google Scholar
• Mook, D. G. (1983), “In Defense of External Invalidity,” American Psychologist, 38, 379–387.
   Web of Science ®Google Scholar
• Nelder, J. A. (1986), “Statistics, Science and Technology,” ( with comments), Journal of the Royal Statistical Society, Series A, 149, 109–121.
   Web of Science ®Google Scholar
• Peng, R. (2015), “The Reproducibility Crisis in Science: A Statistical Counterattack,” Significance, 12, 30–32.
   Google Scholar
• Rosenbaum, P. R. (2001), “Replicating Effects and Biases,” The American Statistician, 55, 223–227.
   Web of Science ®Google Scholar
• Rosenthal, R., and Rosnow, R. L. (1984), Essentials of Behavioral Research: Methods and Data Analysis, New York: McGraw–Hill.
   Google Scholar
• Ruberg, S. J., Chen, L., and Wang, Y. (2010), “The Mean Does Not Mean as Much Anymore: Finding Sub-Groups for Tailored Therapeutics,” Clinical Trials, 7, 574–583.
   PubMed Web of Science ®Google Scholar
• Shadish, W. R., and Cook, T. D. (1999), “Comment—Design Rules: More Steps Toward a Complete Theory of Quasi-Experimentation,” Statistical Science, 14, 294–300.
   Google Scholar
• Shadish, W. R., Cook, T. D., and Campbell, D. T. (2002), Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Boston, MA: Houghton Mifflin.
   Google Scholar
• Shaver, J. P., and Norton, R. S. (1980a), “Populations, Samples, Randomness, and Replication in Two Social Studies Journals,” Theory and Research in Social Education, 8, 1–10.
   Google Scholar
• Shaver, J. P., and Norton, R. S. (1980b), “Randomness and Replication in Ten Years of the American Educational Research Journal,” Educational Researcher, 9, 9–15.
   Google Scholar
• Suppes, P. (1970), Probabilistic Theory of Causality, Atlantic Highlands, NJ: Humanities Press.
   Google Scholar
• Wasserstein, R. L. (2016), “ASA Statement on Statistical Significance and P-Values,” The American Statistician, 70, 131–133.
   Web of Science ®Google Scholar
• Wasserstein, R. L., and Lazar, N. A. (2016), “The ASA's Statement on p-Values: Context, Process, and Purpose,” The American Statistician, 70, 129–131.
   Web of Science ®Google Scholar
• Williams, B. A. (2010), “Perils of Evidence-Based Medicine,” Perspectives in Biology and Medicine, 53, 106–120.
   PubMed Web of Science ®Google Scholar
• Woodward, J. (1989), “Data and Phenomena,” Synthese, 79, 393–472.
   Web of Science ®Google Scholar