Comparing cultural theory and cultural cognition theory survey measures to each other and as explanations for judged risk

Abstract

Different approaches to operationalizing the cultural theory (CT) developed by Douglas, Thompson, Wildavsky, and others in survey research on risk perceptions are rarely compared, never for the same people outside of China. We compare for US respondents the construct validity of cultural worldview measures developed by Jenkins-Smith and colleagues—including both indices of items refining the Wildavsky and Dake approach, and short paragraphs (cultural ‘statements’)—to those developed by Kahan and colleagues based on cultural cognition theory (CCT). Correlational analyses reveal moderate convergent and discriminant validity among these measures, and along with regression analyses controlling for demographic variables similarly moderate predictive validity across measures for judgments of personal risk for 10 hazards. CT statements better discriminate between individualists and hierarchists, and CT indices explain more variance in judged risk (predictive validity) when controlling for demographic variables in regression analyses, but no cultural measure was consistently more predictive for the expected sign of regression coefficients. We discuss theoretical and methodological implications of our findings to foster further scholarly comparisons and improvements in these survey-based cultural approaches to explaining risk judgments.

Keywords:

• Cultural theory
• cultural cognition theory
• culture
• worldviews
• risk perception

Notes

Acknowledgments

Marcus Mayorga supervised data collection and cleaning. This material is based upon work supported by the U.S. National Science Foundation under Grant No. 1644853. We appreciate comments of an anonymous reviewer.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 What constitutes explanation of course varies across these studies, which use a variety of research methods, including case studies and survey research, which tend to have strengths that offset each other’s weaknesses (Johnson and Swedlow 2019). Studies relying on surveys generally report modest explanatory effects (Xue et al. 2014), which may be due to challenges operationalizing CT and CCT for survey research (Johnson and Swedlow 2019).

2 Hierarchists should be more willing than other cultural types to delegate responsibility to an expert, the pilot. Individualists might prefer people competing to work the controls or piloting their own airplanes. Egalitarians might like passengers to vote on the pilot’s next action or take turns at the controls. But it is likely that the latter two cultural types will in practice (like fatalists) resign themselves to pilot control, a loss of control which may register as heightened fear in a survey response.

3 Fatalists might expect risks to be randomly distributed, due to good and bad luck from diverse sources, so lack concern about any particular hazard, or see any hazard as threatening, and thus rate all as high risk (Verweij, Luan, and Nowacki 2011). The meta-analysis hypothesized fatalists as indifferent toward risk ‘given their generally high level of disengagement and external locus of control,’ but found fatalists tended to be afraid of natural hazards (Xue et al. 2014, pp. 250, 254–256).

4 Some theoretical disagreement and empirical divergence about whether different cultures are orthogonal leave open whether to expect negative, or no, correlations.

5 The CCT–CT comparison was not envisioned during study design, otherwise data on all cultural measures probably would have been collected in the same wave, although as we note in the text that this would pose problems given overlap in the content of different measures.

6 Technically 4.5 expectations were met for GMOs, as by definition the positive correlation for the HE scale means a negative, therefore unexpected, correlation for egalitarianism.

7 Analyses were re-run with Wave 3 responses (excluding only the fatalism statement, mistakenly omitted from that wave) by Wave 4 respondents. Correlations’ signs did not differ, with small changes in magnitude and significance, suggesting the 3-month temporal gap in collecting different CT measures did not drive unexpected results there. The 5-month gap between CT data collection, and CCT and risk judgment data in Wave 1, cannot be assessed this way.

8 All CT indices met Cohen’s small-effects threshold, with the medium-effects threshold met for airplane crashes, Ebola virus, abortion and climate change; all met Ferguson’s minimal-practical-effects criterion, and abortion’s R² change almost met his medium-effects criterion (and Cohen’s large-effects threshold). CT statements all met Cohen’s small effects threshold, with airplane crashes and abortion meeting the medium threshold; all but gun control, pesticides and food additives met Ferguson’s practical minimum significance threshold, with none meeting the medium effect threshold. CCT indices met Cohen’s threshold for even small effects four times (airplane crashes, Ebola, gun control, abortion), and for medium effects once, an issue which met Ferguson’s threshold for minimal practical significance (climate change).

9 As fatalism has tended to be ignored in theoretical discussions, the largely positive correlations between fatalist measures and others observed here are difficult to interpret. Kahan (2012) argues that fatalism is a psychological condition, not a way of life, and might argue that our results indicate a free-floating anxiety, anomie, and other worries shared across cultures.

10 Kahan found early in his use of the two CCT measures with U.S. samples that correlations were r ∼ 0.3, but in later studies correlations rose to r ∼ 0.4 (Xue et al. 2014, p. 255) and could rise to r > 0.5 even when using the CCT short form survey (Kahan 2015). A study using the short form with Australians found a correlation of only 0.22 (Phillips, Hine, and Marks 2018), but a U.S. study using the long form with the same online panel as used here found a correlation of 0.74 (Finucane and Holup 2005). Other studies, using the four separate CT indices rather than CCT measures, also found separating hierarchy and individualism difficult (Dake and Wildavsky 1991; Grendstad 2003; Kiss, Montpetit, and Lachapelle 2016; Marris, Langford, and O’Riordan 1998).

11 Fatalists’ tendency to rate all risks as high suggests, as we speculated earlier, that fear of ‘everything’ might shape their lives, but we need a better rationale for why fatalists might lack a specific set of issues that threaten them, and broader empirical data to test that hypothesis.

12 Overall respondents on average rated 7 of 10 hazards as posing less than ‘moderate’ personal risk, with only food additives, climate change and pesticides featuring 52.4–60.4% rating these as ‘moderate,’ ‘high’ or ‘very high’ risk.

13 Acquiescence bias is measured here as the proportion agreeing minus those disagreeing for each item (for CT statements, 0–4 and 6–10 ratings on the 11-point scale) following Olli (2012, 331–338). Some 83.3% of CT items, 76.5% of IC items, and 75% of CT statements, versus 23.1% of HE items, featured more agreement than disagreement. Assuming zero as the ideal and 100 as the maximum imbalance in response (Olli 2012), ‘high’ imbalance (defined here as > 30 percentage points difference) occurred for 38% (HE), 59% (IC), 58% (CT indices), and 75% (CT statements) of items.

14 Researchers usually also need to consider practical survey administration issues, such as cost and response rates. One-item measures like the CT statements may be preferable on both grounds. But the time it takes to answer questions also influences response rates, perhaps making CT items or CCT ‘short form’ measures more attractive.