‘Don’t know’ responding and estimates of perceived risk: failing to provide a ‘don’t know’ response systematically biases laypeople’s perceived risk estimates

References

• Allison, S. T., Messick, D. M., & Goethals, G. R. (1989). On being better but not smarter than others: The Muhammad Ali effect. Social Cognition, 7(3), 275–295.
   Web of Science ®Google Scholar
• Bartneck, C., Duenser, A., Moltchanova, E., & Zawieska, K. (2015). Comparing the similarity of responses received from studies in Amazon’s Mechanical Turk to studies conducted online and with direct recruitment. PloS One, 10(4), e0121595.
   PubMed Web of Science ®Google Scholar
• Beatty, P., Herrmann, D., Puskar, C., & Kerwin, J. (1998). “Don’t know” responses in surveys: Is what I know what you want to know and do I want you to know it? Memory, 6(4), 407–426.
   PubMed Web of Science ®Google Scholar
• Bruine de Bruin, W., & Carman, K. G. (2012). Measuring risk perceptions: What does the excessive use of 50% mean? Medical Decision Making, 32(2), 232–236.
   PubMed Web of Science ®Google Scholar
• Bruine de Bruin, W., Fischbeck, P. S., Stiber, N. A., & Fischhoff, B. (2002). What number is ‘fifty-fifty’?: Redistributing excessive 50% responses in elicited probabilities. Risk Analysis, 22(4), 713–723.
   PubMed Web of Science ®Google Scholar
• Buunk, B. P., Gibbons, F. X., & Visser, A. (2002). The relevance of social comparison processes for prevention and health care. Patient Education and Counseling, 47(1), 1–3.
   PubMed Web of Science ®Google Scholar
• Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29(6), 2156–2160.
   Web of Science ®Google Scholar
• Denman, D. C., Baldwin, A. S., Betts, A. C., McQueen, A., & Tiro, J. A. (2018). Reducing “I don’t know” responses and missing survey data: Implications for measurement. Medical Decision Making, 38(6), 673–682.
   PubMed Web of Science ®Google Scholar
• Epley, N., & Dunning, D. (2000). Feeling ‘Holier than Thou’: Are self-serving assessments produced by errors in self- or social prediction? Journal of Personality and Social Psychology, 79(6), 861–875.
   PubMed Web of Science ®Google Scholar
• Fagerlin, A., Zikmund-Fisher, B. J., & Ubel, P. A. (2005). How making a risk estimate can change the feel of that risk: Shifting attitudes toward breast cancer risk in a general public survey. Patient Education and Counseling, 57(3), 294–299.
   PubMed Web of Science ®Google Scholar
• Fischhoff, B., & Bruine De Bruin, W. (1999). Fifty-fifty=50%? Journal of Behavioral Decision Making, 12(2), 149–163.
   Web of Science ®Google Scholar
• Fiske, S. T., & Tablante, C. B. (2015). Stereotyping: Processes and content. In M. Mikulincer, P. R. Shaver, E. Borgida, J. A. Bargh, M. Mikulincer, P. R. Shaver, … J. A. Bargh (Eds.), APA handbook of personality and social psychology, volume 1: Attitudes and social cognition (pp. 457–507). Washington, DC, USA: American Psychological Association.
   Google Scholar
• French, D. P., & Marteau, T. M. (2008). Different measures of risk perceptions and distress yield different patterns of results: The importance of specificity in measurement and theory. Psychology, Health & Medicine, 13(5), 591–596.
   PubMedGoogle Scholar
• Gardner, R. M., Brown, D. L., & Boice, R. (2012). Using Amazon’s Mechanical Turk website to measure accuracy of body size estimation and body dissatisfaction. Body Image, 9(4), 532–534.
   PubMed Web of Science ®Google Scholar
• Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive judgment. New York, NY, USA: Cambridge University Press.
   Google Scholar
• Glanz, K., & Bishop, D. B. (2010). The role of behavioral science theory in development and implementation of public health interventions. Annual Review of Public Health, 31, 399–418.
   PubMed Web of Science ®Google Scholar
• Glanz, K., Rimer, B. K., & Viswanath, K. (2008). Health behavior and health education: Theory, research, and practice (4th ed., pp. 45–96). John Wiley & Sons, San Francisco, CA.
   Google Scholar
• Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576.
   PubMed Web of Science ®Google Scholar
• Hauser, D. J., & Schwarz, N. (2016). Attentive turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407.
   PubMed Web of Science ®Google Scholar
• Hay, J. L., Coups, E., & Ford, J. (2006). Predictors of perceived risk for colon cancer in a national probability sample in the United States. Journal of Health Communication, 11, 71–92.
   PubMed Web of Science ®Google Scholar
• Hay, J. L., Orom, H., Kiviniemi, M. T., & Waters, E. A. (2015). “I don’t know” my cancer risk: Exploring deficits in cancer knowledge and information-seeking skills to explain an often-overlooked participant response. Medical Decision Making, 35(4), 436–445.
   PubMed Web of Science ®Google Scholar
• Holden, C. J., Dennie, T., & Hicks, A. D. (2013). Assessing the reliability of the M5-120 on Amazon’s Mechanical Turk. Computers in Human Behavior, 29(4), 1749–1754.
   Web of Science ®Google Scholar
• Howell, J. L., & Shepperd, J. A. (2013). Reducing health-information avoidance through contemplation. Psychological Science, 24, 1696–1703.
   PubMed Web of Science ®Google Scholar
• Kiviniemi, M. T., Orom, H., Waters, E. A., McKillip, M., & Hay, J. L. (2018). Education‐based disparities in knowledge of novel health risks: The case of knowledge gaps in HIV risk perceptions. British Journal of Health Psychology, 23(2), 420–435.
   PubMed Web of Science ®Google Scholar
• Kiviniemi, M. T., Snyder, M., & Johnson, B. C. (2008). Motivated dimension manipulation in the processing of social comparison information. Self and Identity, 7(3), 225–242.
   PubMed Web of Science ®Google Scholar
• Klein, W. M. (1997). Objective standards are not enough: Affective, self-evaluative, and behavioral responses to social comparison information. Journal of Personality and Social Psychology, 72(4), 763–774.
   PubMed Web of Science ®Google Scholar
• Klein, W. P. (2003). Self-prescriptive, perceived and actual attention to comparative risk information. Psychology & Health, 18(5), 625–643.
   Web of Science ®Google Scholar
• Kroh, M. (2006). Taking ‘don’t knows’ as valid responses: A multiple complete random imputation of missing data. Quality and Quantity, 40(2), 225–244.
   Google Scholar
• Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213–236.
   Web of Science ®Google Scholar
• Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, W. M., Kopp, R. J., … Smith, V. K. (2002). The impact of” no opinion” response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66(3), 371–403.
   Web of Science ®Google Scholar
• Lenzner, T., Kaczmirek, L., & Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003–1020.
   Web of Science ®Google Scholar
• Manisera, M., & Zuccolotto, P. (2014). Modeling “don’t know” responses in rating scales. Pattern Recognition Letters, 45, 226–234.
   Web of Science ®Google Scholar
• Misovich, S. J., Fisher, J. D., & Fisher, W. A. (1997). Social comparison processes and AIDS risk and AIDS preventive behavior. In B. P. Buunk & F. X. Gibbons (Eds.), Health, coping, and well-being: Perspectives from social comparison theory (pp. 95–123). Mahwah, NJ: Lawrence Erlbaum.
   Google Scholar
• Nelson, D., Kreps, G., Hesse, B., Croyle, R., Willis, G., Arora, N., … Alden, S. (2004). The Health Information National Trends Survey (HINTS): Development, design, and dissemination. Journal of Health Communication, 9, 443–460.
   PubMed Web of Science ®Google Scholar
• Orom, H., BIddle, C., Waters, E. A., Kiviniemi, M. T., Sosnowski, A. N., & Hay, J. L. (in press). Causes and consequences of uncertainty about illness risk perception. Journal of Health Psychology. doi: 10.1177/1359105317745966
   Google Scholar
• Orom, H., O’Quin, K. E., Reilly, S., & Kiviniemi, M. T. (2014). Perceived cancer risk and risk attributions among African-American residents of a low-income, predominantly African-American neighborhood. Ethnicity & Health, 1–14. doi:10.1080/13557858.2014.950197
   Web of Science ®Google Scholar
• Orom, H., Schofield, E., Kiviniemi, M. T., Waters, E. A., BIddle, C., Chen, X., … Hay, J. L. (2018). Low health literacy and health information avoidance but not satisficing help explain don’t know responses to questions assessing perceived risk. Medical Decision Making, 38(8), 1006–1017.
   PubMed Web of Science ®Google Scholar
• Pendry, L. F., & Macrae, C. N. (1994). Stereotypes and mental life: The case of the motivated but thwarted tactician. Journal of Experimental Social Psychology, 30(4), 303–325.
   Web of Science ®Google Scholar
• Rothman, A. J., & Kiviniemi, M. T. (1999). Treating people with information: An analysis and review of approaches to communicating health risk information. Journal of the National Cancer Institute Monographs, (25), 44–51.
   Google Scholar
• Rothman, K. J., Greenland, S., & Lash, T. L. (Eds.). (2008). Modern epidemiology (3rd ed.). Philadelphia, PA: Lippincott, Williams, and Wilkins.
   Google Scholar
• Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.
   PubMed Web of Science ®Google Scholar
• Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys: Experiments on question form, wording, and context. New York: Academic Press.
   Google Scholar
• Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1(2), 213–220.
   Google Scholar
• Shoemaker, P. J., Eichholz, M., & Skewes, E. A. (2002). Item nonresponse: Distinguishing between don’t know and refuse. International Journal of Public Opinion Research, 14(2), 193–201.
   Web of Science ®Google Scholar
• Sung, J. J. Y., Choi, S. Y. P., Chan, F. K. L., Ching, J. Y. L., Lau, J. T. F., & Griffiths, S. (2008). Obstacles to colorectal cancer screening in Chinese: A study based on the health belief model. The American Journal of Gastroenterology, 103(4), 974–981.
   PubMed Web of Science ®Google Scholar
• Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York, NY: Cambridge University Press.
   Google Scholar
• Waters, E. A., Hay, J. L., Orom, H., Kiviniemi, M. T., & Drake, B. F. (2013). “Don’t know” responses to risk perception measures: Implications for underserved populations. Medical Decision Making, 33(2), 271–281.
   PubMed Web of Science ®Google Scholar
• Weinstein, N. D. (1980). Unrealistic optimism about future life events. Journal of Personality and Social Psychology, 39(5), 806–820.
   Web of Science ®Google Scholar
• Zajac, L. E., Klein, W. M. P., & McCaul, K. D. (2006). Absolute and comparative risk perceptions as predictors of cancer worry: Moderating effects of gender and psychological distress. Journal of Health Communication, 11(Suppl 1), 37–49.
   PubMedGoogle Scholar