Hypothesis Testing in Scientific Practice: An Empirical Study

ABSTRACT

It is generally accepted among philosophers of science that hypothesis testing (or confirmation) is a key methodological feature of science. As far as philosophical theories of confirmation are concerned, some emphasize the role of deduction in confirmation (e.g. the H-D method), whereas others emphasize the role of induction in confirmation (e.g. Bayesian theories of confirmation). The aim of this paper is to contribute to our understanding of scientific confirmation (or hypothesis testing) in scientific practice by taking an empirical approach. I propose that it would be illuminating to learn how practicing scientists describe their methods when they test hypotheses and/or theories. I use the tools of data science and corpus linguistics to study patterns of usage in a large corpus of scientific publications mined from the JSTOR database. Overall, the results of this empirical survey suggest that there is an emphasis on mostly the inductive aspects of confirmation in the life sciences and the social sciences, but not in the physical and the formal sciences. The results also point to interesting and significant differences between the scientific subjects within these disciplinary groups that are worth investigating in future studies.

KEYWORDS:

• Bayesian confirmation
• confirmation
• corpus linguistics
• data science
• hypothesis testing
• hypothetico-deductive method

Acknowledgments

I am very grateful to two anonymous reviewers of International Studies in the Philosophy of Science for their helpful comments on earlier drafts of this paper.

Notes

1 Along the same lines, Potochnik, Colombo, and Wright (2019) define the H-D method as follows: “a method of hypothesis-testing; an expectation is deductively inferred from a hypothesis and compared with an observation; violation of the expectation deductively refutes the hypothesis, while a match with the expectation non-deductively boosts support for the hypothesis” (emphasis added).

2 For more on the application of the empirical methods of data science, such as text mining and corpus analysis, to philosophy of science, see Mizrahi (2013), (2016), and (2020). For a recent example of an application of survey and other empirical methodologies from the social sciences to philosophy of science, see Beebe and Dellsén (2020).

3 See also Salmon (1976) on “Deductive” versus “Inductive” Archeology.

4 It is worth noting that the JSTOR database does not contain the same number of publications in each subject. In other words, some subjects (e.g., Biological Sciences) contain more publications than other subjects (e.g., Chemistry) in the JSTOR database. This should not make a significant difference to the results of this empirical study because the comparisons made are between proportions rather than raw numbers of publications from each subject.

5 According to Machery (2016, 480), “Rational reconstructions reconstruct the way scientists use particular concepts [or methods].”

6 Although, according to Lakatos (1971, 91), “any rational reconstruction of history needs to be supplemented by an empirical (socio-psychological) ‘external history’.”