![Sample our Social Sciences journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/110822/TOC-Subject-Sample-2021-Social-Sciences.jpg"})
ABSTRACT
Although the null hypothesis significance testing procedure is problematic, many still favor the use of p-values as indicating the state of evidence against the model used to generate the p-value. From this perspective, p-values benefit science; or would benefit science if used correctly. In contrast, the novel argument to be presented introduces a taxonomy of assumptions included in the model; such as theoretical assumptions, auxiliary assumptions, statistical assumptions, and inferential assumptions. Careful attention is paid to the different categories of assumptions that models necessarily include to render p-value calculations sensible. Such careful attention suggests unappreciated limitations of p-values. Considering these limitations in the context of the descriptive statistics researchers routinely have available to them clarifies that p-values provide no added benefit to the scientist, above and beyond such descriptive statistics. The lack of added value, combined with the obvious harms documented in recent reviews, suggests that researchers in the sciences should rarely, or never, use p-values. Not even for indicating the state of evidence against models.
Disclosure statement
No potential conflict of interest was reported by the author.
Notes
1. Also, see the 2019 special issue in The American Statistician.
2. Most researchers who make this argument consider the typical null hypothesis significance testing procedure to be misused.
3. Bradley and Brand (Citation2016) provided an extensive discussion of distributional assumptions.
4. The assumption of random and independent sampling from a defined population should not be confused with the assumption of random assignment of participants to conditions.
5. Rothman, Galacher, and Hatch (Citation2013) presented an interesting argument against obtaining samples in this way. Their argument implies a further reason to eschew p-values, though the issue will not be pursued further here.
6. Even if a researcher were to find a way to derive p-value calculations without assuming random and independent sampling from a defined population, there would remain an issue of whether the assumptions used are perfectly true for the obtained sample.
7. An alternative is to maximize, but this only makes sense in a significance testing context (Trafimow, Citation2017a).
8. See Bradley and Brand (Citation2016) for a more complete discussion of this issue.
9. There is no justification here, even in the case of literal truth, for an inverse inference about the probability of the MODELTASI given the p-value.
10. Though psychology researchers sometimes do take scale values seriously even when the meaning of the unit is not clear.
11. McShane et al. (Citation2019) also recommended alternatives to p-value cutoffs.
12. This is not an argument that researchers always should use representative samples. There often are good reasons not to use representative samples (Rothman et al., Citation2013).
13. This not equivalent to power analyses. The goal of power analysis is to obtain the necessary sample size to have a good chance of rejecting the null hypothesis if it is false. In contrast, the goal of the a priori procedure is to obtain the necessary sample size to be confident that the sample statistics to be obtained will be close to their corresponding population parameters. These philosophical differences result in associated mathematical differences too.
14. For a discussion of the issue of exaggerated effect sizes, see an original piece by Locascio (Citation2017a); comments by Grice (Citation2017), Hyman (Citation2017), Kline (Citation2017), and Marks (Citation2017); and a rejoinder by Locascio (Citation2017b).
Additional information
Notes on contributors
David Trafimow
David Trafimow is a distinguished achievement professor of psychology at New Mexico State University, a fellow of the Association for Psychological Science, executive editor of the Journal of General Psychology and for Basic and Applied Social Psychology. He received his PhD in psychology from the University of Illinois at Urbana-Champaign in 1993. His current research interests include attribution, attitudes, cross-cultural research, ethics, morality, philosophy and philosophy of science, methodology, potential performance theory, and the a priori procedure.