Overreliance on Statistical Testing Logic in the Empirical Testing of Theories and Hypotheses

Abstract

This article explicates some conceptual and methodological problems involved in the tendency of overreliance on statistical testing logic, logic of disproof. Although statistical hypothesis testing is only a subset of the whole process of empirical testing, a number of researchers tend to misconceive that statistical testing logic is universally applicable to the whole process of empirical theory testing. The authors show that this overreliance on logic of disproof leads to many problems: (1) conceptual problems such as confusion between the research hypothesis and the statistical hypothesis, and oversight of the necessity of logic of proof for accepting the “substantive” null hypothesis containing the researcher's assertion per se, and (2) methodological problems such as a non-rigorous way of conducting empirical research (i.e., the breakdown of the boundary and the proper sequence between the deductive route and the inductive route in empirical theory testing), an incorrect interpretation of test results associated with the testing of the “substantive” null hypothesis, and confusion between logic of disproof (for testing the null hypothesis) and falsificationism (for testing theory).

To remove such conceptual and methodological points of confusion caused by overreliance on logic of disproof, the authors have proposed a seven-step model of empirical theory testing: Step 1 (Theory) → Step 2 (Setting up the research hypothesis: RHEF [i.e., the research hypothesis in existential form]/RHNF [i.e., the research hypothesis in non-existential form]) → Step 3 (Setting up the statistical hypothesis: Translating RHEF or RHNF into H0 and H₁) → Step 4 (Testing the statistical hypothesis by logic of disproof/proof) → Step 5 (Testing the research hypothesis: RHEF or RHNF is supported/not supported) → Step 6 (Testing theory empirically: The theory is empirically supported/not supported) → Step 7 (Interpretation of this empirical test result from the researcher's scientific standpoint, such as falsificationism or logical empiricism) (for better understanding, see Figure 1 in the conclusion section). If the research hypothesis is inductively derived from observations rather than theory, then Step 1, Step 6, and Step 7 become irrelevant.

With regard to the application of the seven step model, the authors have noted the following four points. Firstly, researchers should not intend to apply this seven step model to cases where statistical hypothesis testing is not valid (e.g., “H₀: The Sun revolves around the Earth” and “H₁: The Earth revolves around the Sun” [Johnson, 1999]; “H₀: The defendant is innocent” and “Ha: The defendant is guilty” [Anderson et al., 1999]). Secondly, a theory (T), the research hypothesis (RH) derived from the theory, and the statistical hypothesis (H₀ and H₁) translated from the research hypothesis should be distinguished from each other clearly. Thirdly, even in cases of empirically testable theories or models, two competing theories or models should not be regarded as being the direct objects of statistical testing such as “H₀: model A vs. H₁: model B” (e.g., “H₀: the hypothesized or simpler model” and “H₁: the more general model” [e.g., Arora 1982]; “H0: the random-walk model” and “H₁: the Markov switching model” (and vice versa) [e.g., Cheung and Erlandsson, 2005]). To test two competing theories or models, they usually have to use two separate seven-step or six-step models. Finally, the former Steps 1–3 correspond to the deductive route ruled by theory and the later Steps 4–6 stand for the inductive route ruled by empirical sample data. Thus, the boundary and the proper sequence between the deductive route and the inductive route must be observed in that order. If researchers observe these four points faithfully, unnecessary confusion can be avoided.

The authors believe that the seven step model of empirical theory testing can play a role in accelerating the sound development of scientific knowledge with regard to the empirical testing of theories and hypotheses.

摘要

本文阐述了有关ᅳ些概念和方法论上有关过度依赖统计测试 逻辑，逻辑反任的问题。尽管统计假设检测只是实任测伐整个 过程中 的ᅳ部分，但是有很多研究者把统计测试逻辑误解为 实 1；正理论測试整个过程中普遍适用 的工具。本文作者说明 了 这种 对逻辑反值的过分依赖会导致的诸多问题: (1) 概念性问题, 例如混清研究假设和统计假设，忽视了 为 了接受含有研究者论 断的“实质性”原假设而必要的逻辑任明(2)方法论的问题，如不 严谨的实任研究方法 (例如在实任理论检验中，打破了“演鋒” 和“归纳”的界限和适当 的顺序)，对“实廣性”零假设检验结果 的不正确的解释以及逻辑反(测试零假设) 和征伪主义 (测 试理论) 之间的混清。

为 了消除这些由过度依赖、逻辑反任造成的概念和方法论上的 混清，本文作者提出了 ᅳ个实任理论测伐的七步驟模型 ： 第ᅳ 步 (理论) ᅳ第二步 (建立研究假设: RHEF[存在形式的研究 假设]/RHNF[非存在形式的研究假设] ᅳ第三步 (建立统计学假 设 ： 把RHEF 或 RHNF 转换为 H0 和 Hi) — 第四步 (用逻辑 反 1；正或逻辑值明来测说统计学假设) — 第五步 (测试研究假 设: RHEF 或 RHNF 成立或不成立) — 第六步 (实任测试理 论 ： 该理论实 1：正成立或不成立) ᅳ第七步 (从研究者的科学的 角度来解释实 1：正测伐的结果，例如伪任主义或逻辑经验主义)

(为 了更好的理解，请参考结论部分的 图ᅳ)。如果研究假设 是通过观察得出而非理论的话，那么第ᅳ步，第六步和第七步 变得无关紧要

关于这个七步驟模型的运用，作者提出了 ᅳ下四点。第ᅳ， 统计假设检验不成立的话，研究者不要使用这个七步模型 (例 如，“H0: 太阳围绕地球” 和“피:地球围绕太阳”[Johnson 1999]; “Ho： 被告是无辜的” 和 uHa： 被告是有罪的” [Anderson et al. 1999])ᄋ 第二，理论 (T)，通过理论得出的 研究假设 (RH) 以 及把研 究假设转 化 成 的 统计 学假设之 间 有 显著 的 区 別。第 三，即使在理论或模型是可以进行实任测试的情况下，两个对 比的理论或模型不应该被看傲是统计测试的直接对象。例如 “Ho： 模型 A vs. Hi： 模型 B” (比如 “H0: 假设的或较简单的模 型” 和 “Hi： 更ᅳ般的模型” [例如, Arora 1982]; “H0: 无规行 走模型” 和 “Hi： Markov 机制转换模型” (反之亦然) [例如, Cheung 和 Erlandsson 2005])ᄋ 要测试两个对比理论或模型， 通常需要分別使用七步或六步模型。最后，前三步对应理论 的演鋒路线，而后面的第四到第六步代表实 1：正样本数据的归 纳路线。因此，研究者必须有序的观察演鋒路线和归纳路线之 间的界限和适当的顺序。如果研究者可以忠实的遭守这四点， 那 么就可 以避免不 必要的 混清。

笔者相信这个实值理论测试的七步驟模型能在理论和假设实 任测试方面起到加快理论和假设实任测试方面的科学知识的成 熟发展的作用。

Keywords:

• Null hypothesis
• Logic of disproof
• Logic of proof
• Research hypothesis in existential form
• Research hypothesis in non-existential form
• Falsificationism

失键词:

• 零假设
• 反记逻辑
• 逻辑记明
• 存在形式的研究 假设
• 非存在形式的研究假设
• 值伪主义