The Asymmetry of Predictive and Descriptive Capabilities in Quantitative Communication Research: Implications for Hypothesis Development and Testing

Abstract

The bulk of hypotheses in quantitative communication research are directional (e.g., the correlation is positive, the treatment mean is larger than the control mean). For testing such hypotheses, null hypothesis significance testing (NHST) and the use of effect sizes and confidence intervals (ES+CI) are functionally equivalent. ES+CI provides more precise descriptions of research results (effect sizes, confidence intervals) than does NHST, but that descriptive capability exceeds researchers' current predictive capabilities. Developing more refined predictive capabilities will require making good use of the additional information provided by ES+CI—and careful thinking about how such refined hypotheses might be tested.

Notes

¹Here, “null hypothesis” is used in its familiar sense, to refer to what is sometimes called a “nil hypothesis,” that is, a hypothesis of zero effect (no difference, no relationship). There is another broader meaning in which the “null hypothesis” is that hypothesis whose rejection (nullification) constitutes evidence for some particular alternative hypothesis.

²This is a simplified treatment, because statistical power depends on a variety of factors, including the putative size of the population effect. So an answer to the question “How likely is it that these data will permit identification of the population direction of effect?” depends in part on the population effect. The larger the population effect (everything else being equal), the more likely it is that a study's confidence interval will exclude zero, that is, the more likely it is that a study will be able to confidently identify the direction of effect.

³Notice that under this process-based interpretation of NHST, the null hypothesis is such as to not necessarily be a priori false. The null hypothesis is that random chance, not a substantive process, gave rise to the observed data. Thusly conceived, the null can indeed be plausibly entertained.

⁴At least where claims about effects of message varieties are concerned—a common kind of question in communication research—there is an argument to be made that message classes should not be conceived of as populations (in the usual sense); see Jackson (1992, pp. 132–136).