Message Generalizations That Support Evidence-Based Persuasive Message Design: Specifying the Evidentiary Requirements

Abstract

Evidence-based persuasive message design can be informed by dependable research-based generalizations about the relative persuasiveness of alternative message-design options. Five propositions are offered as specifying what constitutes the best evidence to underwrite such generalizations: (1) The evidence should take the form of replicated randomized trials in which message features are varied. (2) Results should be described in terms of effect sizes and confidence intervals, not statistical significance. (3) The results should be synthesized using random-effects meta-analytic procedures. (4) The analysis should treat attitudinal, intention, and behavioral assessments as yielding equivalent indices of relative persuasiveness. (5) The replications included in research syntheses should not be limited to published studies or to English-language studies.

ACKNOWLEDGMENTS

A version of this article was presented at the Kentucky Conference on Health Communication Preconference on Message Design in Health Communication, April 2014.

Notes

¹ There are meta-analytic methods that do not involve synthesizing effect sizes, such as vote-counting procedures (Bushman & Wang, 2009). But the most familiar meta-analytic methods—and the ones of natural interest in the present context—are ones that synthesize effect sizes.

² Thus the width of the confidence interval in a fixed-effect analysis is a function of the total human sample size across the various studies. It is not affected by the number of different studies (message pairs) or by the distribution of participants over those studies.

³ Because in random-effects analyses, each different message pair (implementation) is seen to have its own population effect, the mean effect in a random-effects analysis is sometimes described as the estimate of the mean across the universe of those various population effects, rather than itself being a population effect (see, e.g., Borenstein, Hedges, Higgins, & Rothstein, 2009, p. 79). “Random-effects models conceptualize a population distribution of effect sizes, rather than a single effect size as in the fixed-effects model” (Card, 2012, p. 230).

⁴ This brief exposition necessarily passes over a number of complexities; for fuller discussions, see Borenstein et al. (2009, pp. 77–86) and Card (2012, pp. 229–256).

⁵ The observed functional equivalence of attitude, intention, and behavior outcomes as indices of relative persuasiveness has implications beyond meta-analytic methodological choices. In particular, where formative persuasive campaign research compares two or more possible messages with the purpose of identifying the one most effective in influencing behavioral outcomes, message designers need not collect behavioral data (because, e.g., intention data will yield the same conclusion about relative persuasiveness).