Message Generalizations That Support Evidence-Based Persuasive Message Design: Specifying the Evidentiary Requirements

REFERENCES

• Abraham, C., & Kools, M. ( Eds.). (2012). Writing health communication: An evidence-based guide. Los Angeles, CA: Sage.
   Google Scholar
• Armstrong, J. S. (2010). Persuasive advertising: Evidence-based principles. New York, NY: Palgrave Macmillan.
   Google Scholar
• Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., … Wicherts, J. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27, 108 –119.
   Web of Science ®Google Scholar
• Basil, M., & Witte, K. (2012). Health risk message design using the extended parallel process model. In H. Cho ( Ed.), Health communication message design: Theory and practice ( pp. 41 –58). Los Angeles, CA: Sage.
   Google Scholar
• Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chicester, West Sussex, UK: Wiley.
   Google Scholar
• Brashers, D. E., & Jackson, S. (1999). Changing conceptions of “message effects”: A 24-year overview. Human Communication Research, 25, 457 –477.
   Web of Science ®Google Scholar
• Brown, C. H., Ten Have, T. R., Jo, B., Dagne, G., Wyman, P. A., Muthen, B., & Gibbons, R. D. (2009). Adaptive designs for randomized trials in public health. Annual Review of Public Health, 30, 1 –25.
   PubMed Web of Science ®Google Scholar
• Browning, C. J., & Thomas, S. A. ( Eds.). (2005). Behavioural change: An evidence-based handbook for social and public health. Edinburgh, UK: Elsevier Churchill Livingstone.
   Google Scholar
• Bushman, B. J., & Wang, M. C. (2009). Vote-counting procedures in meta-analysis. In H. Cooper, L. V. Hedges, & J. C. Valentine ( Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 207 –220). New York, NY: Russell Sage Foundation.
   Google Scholar
• Card, N. A. (2012). Applied meta-analysis for social science research. New York, NY: Guilford.
   Google Scholar
• Chan, A. W., Hrobjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. Journal of the American Medical Association, 291, 2457 –2465.
   Google Scholar
• Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997 –1002.
   Web of Science ®Google Scholar
• Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25, 7 –29.
   PubMed Web of Science ®Google Scholar
• Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., … Wilson, S. (2007). Statistical reform in psychology: Is anything changing? Psychological Science, 18, 230 –232.
   PubMed Web of Science ®Google Scholar
• Cushing, C. C., & Steele, R. G. (2010). A meta-analytic review of eHealth interventions for pediatric health promoting and maintaining behaviors. Journal of Pediatric Psychology, 35, 937 –949.
   PubMed Web of Science ®Google Scholar
• Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias: An updated review. PLoS ONE, 8(7), e66844.
   PubMed Web of Science ®Google Scholar
• Ernst, E., & Resch, K. L. (1994). Reviewer bias: A blinded experimental study. Journal of Laboratory and Clinical Medicine, 124, 178 –182.
   PubMed Web of Science ®Google Scholar
• Evanschitzky, H., Baumgarth, C., Hubbard, R., & Armstrong, J. S. (2007). Replication research’s disturbing trend. Journal of Business Research, 60, 411 –415.
   Web of Science ®Google Scholar
• Faulkner, C., Fidler, F., & Cumming, G. (2008). The value of RCT evidence depends on the quality of statistical analysis. Behaviour Research and Therapy, 46, 270 –281.
   PubMed Web of Science ®Google Scholar
• Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned action approach. New York, NY: Psychology Press.
   Google Scholar
• Fritz, A., Scherndl, T., & Kühberger, A. (2013). A comprehensive review of reporting practices in psychological journals: Are effect sizes really enough? Theory and Psychology, 23, 98 –122.
   Web of Science ®Google Scholar
• Gallagher, K. M., & Updegraff, J. A. (2012). Health message framing effects on attitudes, intentions, and behavior: A meta-analytic review. Annals of Behavioral Medicine, 43, 101 –116.
   PubMed Web of Science ®Google Scholar
• Gerber, A., & Malhotra, N. (2008). Do statistical reporting standards affect what is published? Publication bias in two leading political science journals. Quarterly Journal of Political Science, 3, 313 –326.
   Web of Science ®Google Scholar
• Glasgow, R. E. (2007). eHealth evaluation and dissemination research. American Journal of Preventive Medicine, 32, S119 –S126.
   PubMed Web of Science ®Google Scholar
• Goldstein, N. J., Martin, S. J., & Cialdini, R. B. (2008). Yes!: 50 Scientifically proven ways to be persuasive. New York, NY: Free Press.
   Google Scholar
• Goodall, C. E., Slater, M. D., & Myers, T. A. (2013). Fear and anger responses to local news coverage of alcohol-related crimes, accidents, and injuries: Explaining news effects on policy support using a representative sample of messages and people. Journal of Communication, 63, 373 –392.
   PubMed Web of Science ®Google Scholar
• Grabe, M. E., & Westley, B. H. (2003). The controlled experiment. In G. H. Stempel III, D. H. Weaver, & G. C. Wilhoit ( Eds.), Mass communication research and theory ( pp. 267 –298). Boston, MA: Allyn and Bacon.
   Google Scholar
• Harlow, L. L., Mulaik, S. A., & Steiger, J. H. ( Eds.) (1997). What if there were no significance tests? Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009). Feeling validated versus being correct: A meta-analysis of selective exposure to information. Psychological Bulletin, 135, 555 –588.
   PubMed Web of Science ®Google Scholar
• Hogan, K. (2011). The science of influence: How to get anyone to say yes in 8 minutes or less! (2nd ed.). Hoboken, NJ: Wiley.
   Google Scholar
• Igou, E. R., & Bless, H. (2007). Conversational expectations as a basis for order effects in persuasion. Journal of Language and Social Psychology, 26, 260 –273.
   Web of Science ®Google Scholar
• Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, 696 –701.
   Google Scholar
• Ioannidis, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology, 19, 640 –648.
   PubMed Web of Science ®Google Scholar
• Ioannidis, J. P. A., & Trikalinos, T. A. (2007). The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey. Canadian Medical Association Journal, 176, 1091 –1096.
   PubMed Web of Science ®Google Scholar
• Jackson, S. (1992). Message effects research: Principles of design and analysis. New York, NY: Guilford Press.
   Google Scholar
• Jackson, S., & Aakhus, M. (2014). Becoming more reflective about the role of design in communication. Journal of Applied Communication Research, 42, 125 –134.
   Web of Science ®Google Scholar
• Jackson, S., & Jacobs, S. (1983). Generalizing about messages: Suggestions for design and analysis of experiments. Human Communication Research, 9, 169 –181.
   Web of Science ®Google Scholar
• Jefferson, T., Rudin, M., Folse, S. B., & Davidoff, F. (2007). Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews 2007, issue 2, article MR000016.
   Web of Science ®Google Scholar
• Jensen, J. D. (2008). Scientific uncertainty in news coverage of cancer research: Effects of hedging on scientists’ and journalists’ credibility. Human Communication Research, 34, 347 –369.
   Web of Science ®Google Scholar
• Kay, E. J., & Richter, M. L. (1977). The category-confound: A design error. Journal of Social Psychology, 103, 57 –63.
   Web of Science ®Google Scholar
• Kim, H. S., Bigman, C. A., Leader, A. E., Lerman, C., & Cappella, J. N. (2012). Narrative health communication and behavior change: The influence of exemplars in the news on intention to quit smoking. Journal of Communication, 62, 473 –492.
   PubMed Web of Science ®Google Scholar
• Koole, S. L., & Lakens, D. (2012). Rewarding replications: A sure and simple way to improve psychological science. Perspectives on Psychological Science, 7, 608 –614.
   Web of Science ®Google Scholar
• Kromrey, J. D., & Rendina-Gobioff, G. (2006). On knowing what we do not know: An empirical comparison of methods to detect publication bias in meta-analysis. Educational and Psychological Measurement, 66, 357 –373.
   Web of Science ®Google Scholar
• Lee, S., Cappella, J. N., Lerman, C., & Strasser, A. A. (2011). Smoking cues, argument strength, and perceived effectiveness of antismoking PSAs. Nicotine and Tobacco Research, 13, 282 –290.
   PubMed Web of Science ®Google Scholar
• Levine, T. R., Weber, R., Hullett, C., Park, H. S., & Lindsey, L. L. (2008). A critical assessment of null hypothesis significance testing in quantitative communication research. Human Communication Research, 34, 171 –187.
   Web of Science ®Google Scholar
• Levine, T. R., Weber, R., Park, H. S., & Hullett, C. (2008). A communication researchers’ guide to null hypothesis significance testing and alternatives. Human Communication Research, 34, 188 –209.
   Web of Science ®Google Scholar
• Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1, 161 –175.
   Google Scholar
• Mann, T., Sherman, D., & Updegraff, J. (2004). Dispositional motivations and message framing: A test of the congruency hypothesis in college students. Health Psychology, 23, 330 –334.
   PubMed Web of Science ®Google Scholar
• Meyerowitz, B. E., & Chaiken, S. (1987). The effect of message framing on breast self-examination attitudes, intentions, and behavior. Journal of Personality and Social Psychology, 52, 500 –510.
   PubMed Web of Science ®Google Scholar
• Murayama, K., Pekrun, R., & Fiedler, K. (2014). Research practices that can prevent an inflation of false-positive rates. Personality and Social Psychology Review, 18, 107 –118.
   PubMed Web of Science ®Google Scholar
• Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011). Erroneous analyses of interactions in neuroscience: A problem of significance. Nature Neuroscience, 14, 1105 –1107.
   PubMed Web of Science ®Google Scholar
• O’Keefe, D. J. (2011a). The asymmetry of predictive and descriptive capabilities in quantitative communication research: Implications for hypothesis development and testing. Communication Methods and Measures, 5, 113 –125.
   Google Scholar
• O’Keefe, D. J. (2011b). Generalizing about the persuasive effects of message variations: The case of gain-framed and loss-framed appeals. In T. van Haaften, H. Jansen, J. de Jong, & W. Koetsenruijter ( Eds.), Bending opinion: Essays on persuasion in the public domain ( pp. 117 –131). Leiden, The Netherlands: Leiden University Press.
   Google Scholar
• O’Keefe, D. J. (2013). The relative persuasiveness of different message types does not vary as a function of the persuasive outcome assessed: Evidence from 29 meta-analyses of 2,062 effect sizes for 13 message variations. In E. L. Cohen ( Ed.), Communication yearbook 37 ( pp. 221 –249). New York, NY: Routledge.
   Google Scholar
• O’Keefe, D. J., & Jensen, J. D. (2006). The advantages of compliance or the disadvantages of noncompliance? A meta-analytic review of the relative persuasive effectiveness of gain-framed and loss-framed messages. In C. S. Beck ( Ed.), Communication yearbook 30 ( pp. 1 –43). Mahwah, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Porath-Waller, A. J., Beasley, E., & Beirness, D. J. (2010). A meta-analytic review of school-based prevention for cannabis use. Health Education and Behavior, 37, 709 –723.
   PubMed Web of Science ®Google Scholar
• Reeves, B., & Geiger, S. (1994). Designing experiments that assess psychological responses to media messages. In A. Lang ( Ed.), Measuring psychological responses to media messages ( pp. 165 –180). Hillsdale, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Rosenthal, R. (1991). Replication in behavioral research. In J. W. Neuliep ( Ed.), Replication research in the social sciences ( pp. 1 –30). Newbury Park, CA: Sage.
   Google Scholar
• Scott-Sheldon, L. A. J., DeMartini, K. S., Carey, K. B., & Carey, M. P. (2009). Alcohol interventions for college students improves antecedents of behavioral change: Results from a meta-analysis of 34 randomized controlled trials. Journal of Social and Clinical Psychology, 28, 799 –823.
   PubMedGoogle Scholar
• Sherman, D. K., Mann, T., & Updegraff, J. A. (2006). Approach/avoidance orientation, message framing, and health behavior: Understanding the congruency effect. Motivation and Emotion, 30, 165 –169.
   PubMed Web of Science ®Google Scholar
• Siegel, J. T., Alvaro, E. M., Crano, W. D, Lac, A., Ting, S., & Jones, S. P. (2008). A quasi-experimental investigation of message appeal variations on organ donor registration rates. Health Psychology, 27, 170 –178.
   PubMed Web of Science ®Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359 –1366.
   PubMed Web of Science ®Google Scholar
• Slater, M. D., Goodall, C. E., & Hayes, A. F. (2009). Self-reported news attention does assess differential processing of media content: An experiment on risk perceptions utilizing a random sample of US local crime and accident news. Journal of Communication, 59, 117 –134.
   Web of Science ®Google Scholar
• Sponberg, H. (1946). A study of the relative effectiveness of climax and anti-climax order in an argumentative speech. Speech Monographs, 13, 35 –44.
   Web of Science ®Google Scholar
• Sun, S., Pan, W., & Wang, L. L. (2010). A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology. Journal of Educational Psychology, 102, 989 –1004.
   Web of Science ®Google Scholar
• Thorson, E., Wicks, R., & Leshner, G. (2012). Experimental methodology in journalism and mass communication research. Journalism & Mass Communication Quarterly, 89, 112 –124.
   Web of Science ®Google Scholar
• Tsang, E. W. K., & Kwan, K.-M. (1999). Replication and theory development in organizational science: A critical realist perspective. Academy of Management Review, 24, 759 –780.
   Web of Science ®Google Scholar
• Uncles, M. D., & Kwok, S. (2013). Designing research with in-built differentiated replication. Journal of Business Research, 66, 1398 –1405.
   Web of Science ®Google Scholar
• Updegraff, J. A., Sherman, D. K., Luyster, F. S., & Mann, T. L. (2007). The effects of message quality and congruency on perceptions of tailored health communications. Journal of Experimental Social Psychology, 43, 249 –257.
   PubMed Web of Science ®Google Scholar
• Wanyonyi, K. L., Themessl-Huber, M., Humphris, G., & Freeman, R. (2011). A systematic review and meta-analysis of face-to-face communication of tailored health messages: Implications for practice. Patient Education and Counseling, 85, 348 –355.
   PubMed Web of Science ®Google Scholar
• Wells, G. L., & Windschitl, P. D. (1999). Stimulus sampling and social psychological experimentation. Personality and Social Psychology Bulletin, 25, 1115 –1125.
   Web of Science ®Google Scholar
• Witte, K. (1992). Putting the fear back into fear appeals: The extended parallel process model. Communication Monographs, 59, 329 –349.
   Web of Science ®Google Scholar
• Witte, K., & Allen, M. (2000). A meta-analysis of fear appeals: Implications for effective public health programs. Health Education and Behavior, 27, 591 –615.
   PubMed Web of Science ®Google Scholar
• Yzer, M. (2013). Reasoned action theory: Persuasion as belief-based behavior change. In J. P. Dillard & L. Shen ( Eds.), The Sage handbook of persuasion: Developments in theory and practice (2nd ed., pp. 120 –136). Thousand Oaks, CA: Sage.
   Google Scholar