Elusive Alpha and Beta Control in a Multicausal World

Abstract

Virtually all scientific outlets, including the most prestigious journals, have implemented strict rules of α and (1–β) control, supposed to quantify the probability of a significant result assuming H₀ and H₁, respectively. However, estimation of α and β rests on the untenable assumption that a systematic effect ΔY in the dependent variable cannot be brought about by any other causal influence than the influence ΔX stated in H₁ and negated in H₀. Yet, in a given study, empirical evidence on ΔY related to ΔX can always reflect extraneous causal influences, because no treatment or measurement tool affords a pure measure of X and Y, respectively. Consequently, α and β cannot quantify error probabilities in specific studies.

Notes

Notes

1 With regard to randomization as an allegedly omnipotent remedy, however, it is impossible to realize the ideal of sampling participants and all these causally relevant factors in a perfectly randomized, stochastically independent fashion (see Fiedler, 2017; Trafimow, 2019a, 2019b; Zhou & Fishbach, 2016).

2 Note in passing that by choosing an aggregation level – annual versus monthly statistics – an inferential assumption is introduced that can seriously compromise the validity of an empirical hypothesis test (Trafimow, 2019b).

Additional information

Funding

The work underlying the present article was supported by a grant provided by the Deutsche Forschungsgemeinschaft to the second author [FI 294/26-1].