Active versus passive: evaluating the effectiveness of inoculation techniques in relation to misinformation about climate change

ABSTRACT

Objective

The current study evaluated whether an active inoculation (interactive skill development) or a passive inoculation message (provision of information) were effective tools for conferring resistance to misinformation about climate science in the context of extreme weather events.

Method

Participants were randomly assigned to one of the three conditions: a control condition (no training); a passive inoculation condition; or an active inoculation condition. Participants completed demographic questions followed by training or no training and then evaluated a misinformation and factual article for reliability and persuasiveness.

Results

Participants in the active inoculation condition rated the reliability and persuasiveness of the misinformation article and the reliability of the factual article lower than participants in the control condition. Participants in the passive inoculation training did not rate the reliability and persuasiveness of a misinformation and factual article significantly differently to those in the control condition. When factors such as ideological worldview and climate change beliefs were controlled for however, the inoculation interventions had no significant effect on ratings of reliability and persuasiveness for a misinformation or factual article.

Conclusion

Inoculation seems to be a promising method of preventing the acceptance of misinformation on climate science. However, this analysis highlights that more investigation is required in order to determine the most effective inoculation training design.

KEY POINTS

What is already known about this topic:

• (1) Misinformation about the occurrence of anthropogenic climate change has led to a lack of support for policies which address climate change.

• (2) Inoculation theory and its application have been extensively studied and are considered an effective method for conferring resistance to persuasion.

• (3) There is some conflict within the literature as to whether an active or passive inoculation is the most effective method for conferring resistance.

What this topic adds:

• (1) Further evidence for the effectiveness of inoculation interventions in the recent application to climate science misinformation.

• (2) Active inoculation was more effective than passive inoculation.

• (3) Need for further evaluation into the mechanisms which facilitate resistance to persuasion and therefore evoke attitude/behaviour change.

KEYWORDS:

• Active inoculation
• climate change
• critical thinking
• inoculation theory
• misinformation
• resistance

Receptivity to persuasive misinformation has become an increasingly pressing issue, particularly for policymakers. Persuasive mis/disinformation in the media has meant that certain members of the general population are not engaging with necessary mitigation strategies which seek to protect public and environmental health. Misinformation and disinformation are both defined as information that is false and misleading (Lazer et al., 2018). The difference between misinformation and disinformation lies in the intention of the information provider. That is, specifically in relation to disinformation, the information provider knows the information is false but still shares it as their intention is to deceive their audience (Lazer et al., 2018). With this said, as the intention of an information provider is not a variable of interest for the current study, the term “misinformation” will be used to denote both types of information.

One area where the prevalence of misinformation is of particular concern is in relation to climate science. Among climate science experts, there is a 97% consensus about the occurrence of anthropogenic climate change (Anderegg et al., 2010; Cook et al., 2013, 2016). Misinformation regarding this scientific consensus alongside people’s political ideologies is thought to have contributed to an observed lack of engagement with climate change policies (Carmichael et al., 2017; Cook et al., 2018; Lewandowsky et al., 2015). Climate change misinformation is commonly spread by conservative media outlets who employ persuasive techniques which deliberately misconstrue the consensus on the occurrence of anthropogenic climate change (Elsasser & Dunlap, 2013). The spread of such misinformation is concerning, as cooperation from the general public is required for policies which address climate change to be effective (Ding et al., 2011; McCright et al., 2013). To address this issue, research such as that by Cook et al. (2017) and van der Linden et al. (2017) has explored the effectiveness of passive inoculation interventions which aim to equip individuals with the skills to critically evaluate information. This, in turn, is argued to provide individuals with a defence against persuasive misinformation. The current study aimed to add to this evidence base by comparing the effectiveness of active and passive inoculation interventions in conferring resistance to misinformation about extreme weather events that are a linked consequence of climate change.

Factors contributing to misinformation receptivity

Research has identified several factors, both internal and external to the individual, which contribute to some members of the publics’ propensity to contest the occurrence of anthropogenic climate change. Internal predictors include individuals’ political ideologies and worldviews. Specifically, it has been identified that individuals are more likely to contest the occurrence of anthropogenic climate change if they express a strong desire for a social and economic system which is stratified and where individuals have control over their well-being (Kahan et al., 2011; Lewandowsky & Oberauer, 2016; Nilsson et al., 2019). Individuals who hold such desires are argued to hold hierarchical and individualist worldviews (Kahan et al., 2007, 2011). For example, for those whose political beliefs tend to be right of centre, the implementation of policies which seek to address global greenhouse gas emissions are perceived as a threat to privatisation of resources and free trade as well as national sovereignty (Jacques et al., 2008).

Contributing external factors, on the other hand, include persuasive techniques employed by the media. One of the most common techniques used by climate denialists is to employ a fake expert (Cook et al., 2017; Diethelm & McKee, 2009). Readers tend to believe the merit of information based upon the endorsement of an “expert”. However, as seen in such examples of misinformation, the “expert” quoted does not have appropriate expertise or authority to be making claims or providing advice about such topics. The use of a fake expert is also commonly paired in disinformation examples with the defaming of actual experts or the diversion of readers' attention to other unrelated information (Diethelm & McKee, 2009; Elsasser & Dunlap, 2013). Equal time given to both sides of an issue (or false balanced reporting), is another technique commonly employed in relation to climate science (Cook et al., 2017). False balance reporting misconstrues the level of scientific agreement from the view of the public (Lewandowsky et al., 2012). For the issue of climate change, this balanced reporting is particularly inappropriate considering that there is a 97% consensus among climate science experts of the occurrence of anthropogenic climate change (Cook et al., 2016). Another technique employed is the creation of impossible expectations of what research can deliver (Diethelm & McKee, 2009). For example, climate deniers may argue that temperature records are inaccurate before the invention of the thermometer, therefore global warming is not a reality (Diethelm & McKee, 2009).

Repeated exposure or familiarity with misinformation is another factor contributing to continued misinformation receptivity. Familiarity with a piece of information has been shown in several studies to strengthen an individual’s belief in said information, even if it is false (Pennycook & Rand, 2019; Pennycook et al., 2018; Polage, 2012). In such cases, individuals are not assessing the plausibility of the information presented to them, but rather trusting the information simply because it is familiar (Lewandowsky et al., 2012; Pennycook et al., 2018). Further, the influence of the misinformation on individuals’ thinking has been shown to still be pervasive in many situations even once a correction has been provided. This effect is known as the continued influence effect (Lewandowsky et al., 2020). Therefore, in order to address receptivity to misinformation, messaging techniques which teach individuals to recognise persuasive techniques and consider the plausibility of statements need to be evaluated. Such a technique will need to be skilful in the way in which it warns individuals of impending challenges to beliefs so as to avoid continued influence effects. As alluded to, techniques based on inoculation theory show promise for designing such interventions.

Inoculation theory

Inoculation theory was developed by McGuire (1964) and is a theory which describes a messaging technique and provides a theoretical explanation for how resistance to persuasion is facilitated. Inoculation messages comprise two components: a forewarning of impending attitudinal challenge and a refutational pre-emption. The refutational pre-emption presents weakened examples of counterattitudinal information which are then refuted. McGuire proposed two core mechanisms that are required to facilitate resistance to persuasion: threat and counterarguing. Threat is a message response whereby an individual begins to recognise the vulnerability of their position or attitude. Threat can be induced explicitly in an inoculation message through a forewarning such as “the attitude you hold is about to be challenged”. Or threat can be induced implicitly through recognising and perceiving the vulnerability of one’s position or attitude as a result of exposure to weakened examples of counterattitudinal information (Compton, 2012; McGuire, 1964). The second mechanism, counterarguing, occurs post inoculation treatment and is proposed to be a result of the individual feeling threatened and therefore motivated to counterargue. Indeed, counterarguing is also a message component; however, it is counterarguing which occurs post treatment which is said to facilitate and sustain resistance to persuasive counterattitudinal information.

McGuire made the distinction between active and passive inoculation interventions. The difference between the two methods is by the way counterarguing is encouraged. Passive interventions have been operationalised as requiring the participant to passively read counterarguments provided in the inoculation message. However, in active inoculations, participants are required to generate their own counterarguments in response to refutations presented in the message (Compton & Pfau, 2005). Active inoculation is proposed to be more effective than passive inoculation as the “internal” counterarguing employed in active inoculation is a more involved cognitive process, and therefore, generalised resistance to persuasion is suggested to be more likely to occur (Roozenbeek & van der Linden, 2018). However, some studies have suggested that passive inoculation is the superior method at conferring resistance as less of a cognitive demand is required for counterarguing due to the information needed already being provided as well as being of potentially better quality compared to the self-produced counterarguments of an active inoculation (Banas & Rains, 2010; Rogers & Thistlethwaite, 1969).

In McGuire (1961b) original research on inoculation interventions, comparisons of passive versus active conditions suggested that the methods are at least differentially effective. McGuire presented participants with inoculation messages which raised and corrected misinformation in relation to a cultural truism. Cultural truisms are defined as an attitude, belief or position which is widely held in a cultural group and is seldom challenged. It was found that those who actively generated arguments following training displayed a stronger resistance to counterattitudinal attacks that were novel to the initial cultural truism, whereas those in the passive condition had a stronger resistance to counterattitudinal attacks that were identical to what was presented during the inoculation training. Therefore, though passive inoculation was more effective at conferring resistance to persuasion for identical counterarguments, active inoculation in this case was more effective at conferring a generalised resistance to persuasion. Generalised resistance, also known as cross, blanket or umbrella protection, is hypothesised to occur as participants are prompted to counterargue in the initial inoculation interventions and such motivation to counterargue then carries over to subsequent attacks (Parker et al., 2012, 2016). It should be noted that in the literature, scholars have differentiated between blanket or umbrella protection and cross protection. Blanket or umbrella protection refers to an inoculation resulting in protection against novel challenges on the same topic encountered during training, as was evaluated by McGuire and Papageorgis in their early experiments (McGuire, 1961b; Papageorgis & McGuire, 1961). Cross-protection, however, refers to those who have undergone inoculation treatment can then resist challenges to novel information on different but related issues to what was encountered during training. Cross-protection was evaluated in more recent studies by Parker et al. (2012) and Parker et al. (2016) where it was found that passive inoculation interventions were also effective at providing cross protection to persuasive attacks for topics which were related but were not identical to topics discussed in the initial inoculation training. However, it remains unclear as to how each method differs in effectiveness at developing resistance against persuasive misinformation.

The current study

Given the politicisation of climate science and the increasing need for imminent action to address the impacts of climate change, identifying means to counter misinformation in the public discourse is imperative. Inoculating people against climate science misinformation may provide an avenue for people to more critically evaluate and subsequently endorse appropriate climate policies. Therefore, the purpose of the current study is to compare the effectiveness of an active inoculation versus a passive inoculation message on participants’ ability to identify unreliable information in news articles about the increased frequency and intensity of extreme weather events. The current study adapts the methodologies of Roozenbeek and van der Linden (2018) and van der Linden et al. (2017).

van der Linden et al. (2017) compared the ability of a general versus a specifically worded passive inoculation message at conferring resistance to misinformation about climate change. The messages warned that some politically motivated groups employ persuasion techniques to misrepresent the scientific consensus of climate change in public discourse. The claim of there being no scientific consensus was then debunked by reiterating that there was almost no dispute within the scientific community regarding the scientific consensus. The second, more specific, inoculation message included additional arguments to those just outlined, such as an example of persuasive techniques used in a common source of misinformation. It was found that compared to those who did not receive inoculation training, both general and specific passive inoculation messages resulted in significant increases in ratings in agreement with the scientific consensus on climate change. Analysis also showed that the specific inoculation message was more effective than the general message. Indeed, as van der Linden et al. (2017) also aimed to evaluate whether the passive inoculations would have an impact on participants across the political spectrum, the greater effectiveness of specific passive inoculation messaging was demonstrated for those who supported right leaning parties. As stated earlier, those who support right leaning parties tend not to support/believe in anthropogenic climate change (McCright et al., 2013). Therefore, not only does this study highlight the effectiveness of specific message design for passive inoculation but also provides evidence for the effectiveness of inoculation techniques in shifting attitudes in relation to climate science across the political spectrum by using inoculation techniques which explain how those external persuasion techniques are employed.

Taking the idea of exposing how persuasion techniques are used, Roozenbeek and van der Linden (2018) extended the traditional active inoculation methodology to be more interactive and experiential. The researchers designed an active inoculation “game” where 95 high school students from the Netherlands were recruited. Participants in the experimental condition (active inoculation; n = 57) were first taught about journalistic techniques (e.g., hyperbole) and then asked to create their own news articles regarding immigration using the techniques they had just learnt. The participants in the control condition (n = 38) were directed to watch an unrelated film whilst the participants in the active condition undertook their training. Analysis revealed that those who undertook the active inoculation training rated a fake news article which contained many of the same techniques students used during training as significantly less reliable than the control group. A later publication by Roozenbeek and van der Linden (2019) further showed the efficacy of the active gamified method, with this study showing that participants who undertook inoculation training via an online “fake news game” significantly improved their ability to correctly identify when a news headline was unreliable and employing a persuasive technique. The content of the news headlines presented to participants was varied, suggesting that the “fake news game” provided a blanket protection to persuasive misinformation.

Based on the findings from previous research and the provision that the active experiential inoculation training seemingly provides more of an opportunity for critical thinking skills to be developed through more active recognition of persuasion techniques, the following hypotheses were developed:

H1:

It is expected that compared to participants who did not receive any inoculation training (control group)

1. participants in both the active and passive conditions will rate the misinformation articles as less reliable and less persuasive.

2. participants in the active condition will rate the reliability and persuasiveness of a misinformation article lower than those in the passive condition.

H2:

Participants in both the active and passive inoculation conditions will not rate the reliability and persuasive of the factual article significantly differently to the control group.

H3:

Similarly to van der Linden et al. (2017), it is expected that when controlling for other factors such as age, sex, worldview, climate change beliefs, agreement and familiarity with information encountered there will be a similar pattern of results for the three conditions on ratings of reliability and persuasiveness for the misinformation and factual article.

Method

Participants

After removing the missing data (n = 117), there were a total of 137 participants (49 males, 88 females; M = 36.63 years of age, SD = 13.01 years) recruited for this study from Queensland, Australia. Participants were recruited via social media and snowballing techniques. A link to the online experiment was posted on the researchers’ social media accounts and accompanied by a short explanation of the study. Links to the survey were shared on relevant community pages, such as Townsville Community Notice Board and Central and North Queensland Weather. The result of this recruitment process meant that the sample was predominately from North Queensland whose residents are particularly susceptible to experiencing extreme weather events such as tropical cyclones and storm surges (Queensland Government, 2019).

Materials and procedure

This study received approval from the James Cook University Human Research Ethics Committee (Approval number: H7428). This online Mixed experiment was hosted by Qualtrics. The first page of the online survey included information regarding the study and participants were asked to provide consent before continuing to the survey. The study was conducted in three parts. All participants completed Part One of the study. Prior to Part Two, participants were randomly assigned to either the control condition, passive inoculation condition, or the active inoculation condition. Those assigned to the control condition moved directly from Part One to Part Three of the study. Those in the passive and active inoculation conditions completed the training described below for Part Two before moving on to Part Three. The study took approximately 10–20 min to complete, depending on the condition assigned.

Part one

This study was part of a larger study conducted, and subsequently, not all variables were used for analysis in this paper. In Part One of the study, as well as demographic items, participants were asked to answer questions which assessed political alignment, cultural cognition (worldview), environmental beliefs, weather event risk perception, and climate change beliefs. For analysis of hypothesis three, participants' age, sex, climate change beliefs and cultural cognition were included as covariates. Political ideology was not used as a covariate for analysis as the cultural cognition measure also captures political outlook and is more comparable to other populations.

Climate change beliefs

Climate change beliefs were assessed via two items which were treated as separate scales. The first item was developed by the research team and asked participants to indicate on a 5-point Likert scale how likely they think it is that global warming has contributed to a change in the frequency and intensity of extreme weather events (1 = extremely unlikely and 5 = extremely likely). The second item was developed by Bain et al. (2012) and asked participants to indicate which one of the following statements best described their beliefs:

• Humans are contributing substantially to climate change

• Climate change is occurring, but humans are not contributing substantially to it

• The climate is not changing

Cultural cognition

Participants' cultural cognition was measured through two sets of six items used by Kahan et al. (2011). The first set of items assessed whether a person has a hierarchical or egalitarian worldview (Cronbach's alpha = .657) and the second set of items assessed whether the person has an individualist or communitarian worldview (Cronbach's alpha = .835). Egalitarian and communitarian worldviews are indicated by scores which fall below the median of each scale. Hierarchical and egalitarian worldviews are indicated by scores which fall above the median of each scale. Participants who had scores which fell on the exact midpoint were excluded from the analysis (n = 23). From here, participants were able to be further classified into cultural groups with those with low scores on both scales being coded as 1 = Egalitarian Communitarian. Those with high scores on both scales being coded as 2 = Hierarchical Individualist. Those with a low score on scale 1 and a high score on scale 2 being coded as 3 = Egalitarian Individualist. Finally, those with a high score on scale 1 and a low score on scale 2 are coded as 4 = Hierarchical Communitarian.

Part two

Please see Appendix A for complete experimental materials.

Passive inoculation condition

Participants were asked to read a passive inoculation passage. The threat component in the passage stated people can be complacent with their preparations for extreme weather events as individuals perceive the risks posed by such events to be exaggerated. The second stage of the message provided an example of how this problem is exacerbated as media techniques are employed to gain the attention of audiences. For example, multiple extreme weather events such as Tropical Cyclone Yasi are described as “one-in-a-hundred-year” events. The final part of this passive inoculation passage provided an explanation of how this statistic, which is legitimately used to describe extreme weather events, is often misused by reporters, and therefore can be easily misunderstood by the public. The participant is then encouraged to identify “attention-seeking” persuasion techniques in the information they see and hear so that they can make better decisions.

Active inoculation condition

Participants in this condition were first asked to read the same threat message as those in the passive inoculation condition. Then, for the misinformation example, participants read a brief passage explaining persuasion techniques commonly used in the media. The techniques explained were hyperbole; common man, which attempts to convince audience by appealing to common sense; and thirdly, appeals to authority, which asserts that an argument is true based upon the credentials of the person making it (i.e., a fake expert). The influence of misinformation was explained to participants, and it was emphasised that in the context of extreme weather preparation it is important to have the correct information. For the debunking training, participants were presented with three tweets and asked to match the correct persuasion technique with the tweet. For instance, the hyperbole tweet read as follows: “#Breaking: Monster Cyclone Debbie tracking towards Qld coast #GetReady #Bringit”. Following this exercise, participants were then asked to generate their own tweets for each persuasion technique.

Part three

Participants were asked to read two edited articles written by the same journalist for the same news media company. The article characterised by misinformation was titled, Cyclones are slowing down but increasing in destructive power (Marsh, 2018a). This article discusses decreasing cyclone speeds and the effect this may have on communities in the path of such an event. The article was classified as a misinformation article as the “expert” the author interviews and quotes is considered an unreliable source of information due to not having appropriate qualifications. The factual article is titled, Queensland weather: Tourism “unfavourable” due to extreme weather (Marsh, 2018b). This article discusses the effect extreme weather events could have on Queensland tourisms and was classified as factual due to the use of reliable sources such as the United Nations and The Climate Council. Please see Appendix B for full factual and misinformation article texts. After reading each article, participants were asked the same four outcome questions as asked by Roozenbeek and van der Linden (2018). They are as follows:

• How familiar are you with the topic in general addressed in the article? (1 = not familiar at all; 7 = very familiar)

• How persuasive did you find the article? (1 = not persuasive at all, 7 = very persuasive)

• How much do you personally agree with the article? (1 = completely disagree, 7 = completely agree)

• Do you think the information in the article is reliable? (1 = not reliable at all, 7 = very reliable)

For this particular investigation, the reliability and persuasiveness ratings were used as dependent variables for both hypotheses. Ratings of familiarity and agreement were included as covariates for analysis of hypothesis three.

Results

Analysis was conducted using IBM SPSS. One-way ANOVAs were conducted to evaluate the impact of type of inoculation training on ratings of reliability and persuasiveness of both a factual and misinformation article (Hypotheses 1 & 2). ANCOVAs were conducted to evaluate whether there was still an effect on ratings of reliability and persuasiveness when controlling for age, sex, climate change beliefs, worldviews and perceived familiarity and agreement with the articles.

Table 1 depicts the mean (SD) ratings of reliability and persuasiveness by condition.

Table 1. Mean (SD) persuasiveness and reliability ratings for misinformation and factual articles by condition.

	Part 3 variables
	Misinformation			Factual
	N	Persuasive	Reliability	N	Persuasive	Reliability
Control	44	4.66 (1.22)	4.61 (1.32)	43	3.72 (1.56)	3.88 (1.45)
Passive	53	4.38 (1.71)	4.30 (1.65)	52	3.50 (1.70)	3.31 (1.62)
Active	40	3.75 (1.50)	3.73 (1.48)	40	3.23 (1.59)	2.90 (1.43)

N = 137.

The assumption for homogeneity of variance was met for all ANOVAs conducted for hypotheses 1 and 2 (p > .05).

Hypothesis one

The results of a one-way ANOVA indicated that there was a significant difference between the mean reliability ratings (see Table 1) for the three conditions for the misinformation article, F(2,134) = 3.77, p = .026, η² = .053. A Bonferroni Post Hoc Test (chosen for its ability to control the Type I error rate; Aron et al., 2013) showed that there was a significant difference in mean ratings of reliability of the misinformation article between those in the control and the active inoculation condition (p = .023). There was no significant difference in the ratings between those in the passive and active inoculation conditions (p = .206) nor in the passive and control conditions (p = .930).

A one-way ANOVA evaluated whether there was a differential impact of training on the perceived persuasiveness of the misinformation article. The analysis revealed there to be a differential response F (2,134) = 3.98, p = .021, η² = .056. A Bonferroni Post Hoc Test revealed a significant difference in the average ratings of persuasiveness between those in the control condition and those in the active inoculation condition (p = .020). There was no significant difference in average ratings between those in the passive and control conditions (p = 1.000). There was also no significant difference between mean persuasiveness ratings of those in the active and passive conditions (p = .146).

Hypothesis two

Another one-way ANOVA showed there was a significant difference between the means of the three condition ratings on the reliability of the factual article F (2,132) = 4.47, p = .013, η² = .063. Further analysis showed there to again be a significant difference between the ratings of those in the control condition and the active inoculation group (p = .011), and there to be no significant difference between the ratings in the passive inoculation condition and the control condition (p = .199) and the passive and active inoculation condition (p = .605). There were no significant differences between the mean ratings of persuasiveness of the factual article between each of the groups F (2,132) = 0.97, p = .383, η² = .014.

Hypothesis three

One-way between-groups ANCOVAs were conducted to evaluate whether the passive and active inoculation trainings were still effective once perceived familiarity, agreement, worldviews, climate change beliefs, age and sex were controlled for. Adjusted means (SE) for each dependent variable can be seen in Table 2. Table 3 summarises the results of the four ANCOVAs conducted. The assumption of equality of variances (p > .05) was met for all proceeding ANCOVAs.

Table 2. Adjusted mean (SE) persuasiveness and reliability ratings for misinformation and factual articles by conditions.

	Part 3 variables
	Misinformation			Factual
	N	Persuasive	Reliability	N	Persuasive	Reliability
Control	35	4.41 (0.18)	4.32 (0.15)	34	3.40 (0.20)	3.44 (0.14)
Passive	44	4.30 (0.16)	4.27 (0.13)	43	3.65 (0.17)	3.53 (0.12)
Active	35	4.15 (0.18)	4.08 (0.15)	35	3.39 (0.19)	3.19 (0.13)

Table 3. One-way analysis of covariance for each dependent variable.

Dependent variable	ηp^2	F	p
Misinformation Article - Persuasiveness	.009	.469	.627
Misinformation Article - Reliability	.013	.698	.500
Factual Article - Persuasiveness	.014	.715	.492
Factual Article – Reliability	.036	1.893	.156

As can be seen in Table 3, after adjusting for age, sex, worldviews, climate change beliefs and perceived familiarity and agreement, there were no longer significant differences between conditions on the ratings of reliability and persuasiveness of a misinformation article nor the reliability ratings of a factual article. Ratings of persuasiveness were also non-significant in relation to the factual article.

Discussion

The current study aimed to assess whether inoculation theory is an effective tool for conferring resistance to misinformation about climate science. Further, this study aimed to provide additional evidence for the effectiveness of passive versus active inoculation methods. Mixed results were found across the hypotheses. The first hypothesis was supported partially in that participants who undertook an active inoculation training were more likely to rate a misinformation article as less reliable and persuasive than the control condition. However, there was no difference between those who undertook a passive inoculation training and those in the control condition or passive and active conditions’ ratings of reliability and persuasiveness. Furthermore, although there were no significant differences between ratings of persuasiveness between conditions, there were significant differences in ratings of reliability of the factual article. Similarly, to the misinformation article, those in the active training condition rated the factual article as less reliable than those in the control condition, with no differences between active and passive or passive and control condition ratings. The final hypothesis was not supported as once factors such as worldviews, age, sex, climate change beliefs, and perceived familiarity and agreement were controlled for, there was no significant effect by condition for ratings of reliability and persuasiveness for either article.

The findings of the current study do not support those of van der Linden et al. (2017), as no significant difference between control and passive conditions was found. However, this study adds to the preliminary findings by Roozenbeek and van der Linden (2018) by showing promise for the use of active inoculation as an effective method for conferring resistance to persuasion. Those in the active inoculation condition rated a misinformation article as significantly less reliable and persuasive when compared to those who did not receive any inoculation training. These findings suggest that an active inoculation training provided individuals with the tools to be more resistant to persuasive misinformation in the current context of extreme weather events. The literature on inoculation theory suggests that greater resistance to the persuasion presented in the misinformation article was conferred due to the more involved counterarguing process. That is, as more extensive training was provided in regard to recognition of persuasive techniques, participants were able to recognise the techniques employed and then make the appropriate judgement in regard to the overall reliability and persuasiveness when presented with a misinformation article (Banas & Rains, 2010; Roozenbeek & van der Linden, 2018, 2019). Whilst not specifically evaluated in this study, this supports the idea that a blanket protection to persuasive misinformation is likely to be developed through the use of an active inoculation methodology. This is consistent with the findings of Roozenbeek and van der Linden (2018, 2019). As discussed, a technique which can facilitate individuals being resistant to multiple forms of persuasive misinformation is desirable considering the current media landscape and prolific presence of misinformation encountered by individuals daily.

However, the preliminary nature of these findings needs to be stressed and work into the conditions which will facilitate the most effective delivery of active inoculation needs to be further researched. This is especially so considering the findings in this study that those who received an active inoculation training then went on to rate the reliability of a factual article lower than those in the other two conditions. Furthermore, those who underwent active inoculation training rated the reliability of the factual article lower than what they rated the reliability of the misinformation article. It should be noted that the order in which the misinformation article and factual article were presented to participants was counterbalanced, so results observed are not due to practice effects. It is possible that participants took the use of the word “unfavourable” and “inhospitable” in the factual article for the use of a hyperbole as opposed to simply being a descriptive. This implies that upon recognition of any persuasive technique, participants may evaluate information as being unreliable.

These findings suggest that perhaps the active inoculation training fostered a general scepticism towards news media. Instances of where such exposure to media education has led to scepticism can be observed in other studies such as Van Duyn and Collier (2019). Van Duyn and Collier found that when participants were exposed to tweets (or primed) by elite discourse about fake news, participants were subsequently less accurate in their identification of whether a news article was real compared to participants who were not exposed to discourse about fake news. Elites in this context are defined as “politicians, higher-level government officials, journalists, some activists, and many kinds of experts and policy specialists” (Zaller 1992, p.6, as cited in Van Duyn & Collier, 2019). Furthermore, a study by Ashley et al. (2010) found that there were statistically significant differences between participants who had and had not been exposed to information about media ownership on judgements of general accuracy and superficiality of news articles. Specifically, when participants are first provided with information about media ownership, participants were subsequently more critical of later encountered news articles. This suggests that participants who learned of news media ownership were more attuned than those who had not been exposed to the tendency of the news to sensationalise and trivialise information. As can be seen, the findings of the current study add to the literature discussing the careful line between teaching individuals to be more attuned and critical of news media techniques as opposed to just breeding cynicism or general scepticism in relation to the media.

Indeed, it would be remiss not to acknowledge that the results, especially in relation to the factual article, may also be due to other factors such as a continued influence effect as was suggested by the findings of analysis from hypothesis two. The four ANCOVAs revealed that once worldviews, climate change beliefs, age, sex and perceived familiarity and agreement were accounted for, there was no significant effect from training condition on participants’ ratings of reliability and persuasiveness for either article. It is likely that participants are using heuristics and relying on past beliefs when encountering information. For instance, conservative media regularly discredits organisations such as the United Nations and the Intergovernmental Panel for Climate Change (IPCC), due to their goals being in opposition to the conservative agenda (Elsasser & Dunlap, 2013). Perhaps, this continued influence was “activated” when the author quoted the United Nations in the factual article, which resulted in participants thinking the information was unreliable as they readily recalled an instance(s) of when the reliability of such organisations had been questioned by sources they trust. Considering that past passive inoculation interventions designed by Cook et al. (2017) and van der Linden et al. (2017) and an active inoculation by Roozenbeek and van der Linden (2019) showed that, irrespective of political ideologies and worldviews, inoculation training was effective at conferring resistance to misinformation, research into message framing so as to avoid familiarity and continued influence effects within an inoculation message should be further explored.

In relation to the lack of significant findings regarding the passive inoculation, they are surprising as prior research has exemplified the rigour of passive inoculations and shown passive inoculation effectiveness in conferring resistance to persuasive misinformation in relation to contentious topics before. However, the results of the current study may be due to the lack of attitudinal and content matching between the passive inoculation training and the misinformation text. As found in Ivanov et al. (2009), inoculation training is more effective when the framing of the training message and the attitude of the individual are matched. The inoculation training likely elicited more of a cognitive response towards extreme weather events due to the type of misinformation debunked (i.e., statistical formula of the 1 in-100-year-event). It should be noted that the predominately North Queensland sample of this study has experienced two significant extreme weather events (Cyclone Yasi and Cyclone Debbie) in the several years leading up to the dissemination of this study. As discussed in Bergquist et al. (2019), experiencing an extreme weather event has marked affects to individuals physical and psychological wellbeing. This mismatch here between individuals’ affect and the framing of the inoculation interventions may account for the lack of significant effect of the passive inoculation. Furthermore, as found by McGuire (1961b), for passive inoculation interventions, resistance to subsequent persuasion is more effective when the content of the training message and encountered misinformation are identical. This was not the case for this study. Instead, this study was based on Cook et al. (2017) which utilised a passive inoculation message which did not specifically mention the same information again from training to misinformation to provide resistance to persuasion about climate change. This suggests that in Cook et al. a blanket protection to persuasion was conferred. Considering that this was not observed in the current study for potential reasons which have already been discussed, such as confirmation bias and the need for matching, overall suggests that further investigation in this area is needed.

A limitation of this study was the lack of measure of elicited threat. This measure was omitted as the primary aim of this study was to assess participants perceived persuasiveness and reliability of factual and misinformation articles following exposure to active and passive conditions, as opposed to assessing the mechanisms facilitating resistance to persuasion. Without such a measure, the presence of threat can only be inferred. Including a measure of elicited threat would be an important inclusion in future research which compares the efficacy of active versus passive inoculation methodologies to see if threat functions differently between the different methods. A potential confound of the current study design is the different content and activity focus of the active and passive trainings. That is, the passive training focused more specifically on how the statistic 1-in-a-100-year event can be misrepresented and the active training focuses on certain misinformation techniques which participants may encounter. With this said however, the purpose of active trainings is that they are more in-depth and as such, more content would need to be presented. The difference in training was also due to the types of inoculation, and specifically the gamified inoculation (Roozenbeek & van der Linden, 2018) the active condition was modelled after. It should further be noted that the explicit threat component was the same across both of the trainings.

It would also be remiss not to mention that this study was potentially limited by the recruitment strategy of utilising social media platforms which may lead to self-selection bias as inherently only those with access to the internet and those who self-select to participate in the survey are captured (Bethlehem, 2010). This study was also limited by its high attrition rate, particularly after part one. Given that participants were entering the survey under the impression they would be answering questions about how information is communicated about extreme weather events, participants may have been disconcerted by the political alignment and cultural worldview measures. Moreover, for those who then went on to receive active inoculation training particularly, the time taken to participate was upwards of 15–20 min. It lends to question whether there would be efficacy in a shorter active inoculation intervention which could be easily accessible online to individuals. This is something that could be of interest for future research.

In all, the results of this study provide promising evidence for the use of an active inoculation methodology as a tool to confer resistance to persuasive misinformation about climate change and extreme weather events. However, particularly, the findings in relation to the factual article highlight the potential for unintended, or the iatrogenic effects as coined by Compton (2021), of inoculation interventions. That is, it appears that the active intervention trained a generalised scepticism of information as opposed to developing critical evaluation skills. General scepticism appeared to develop as participants essentially evaluated information as unreliable upon recognition of any persuasive technique. Furthermore, the effects of prior beliefs are strong, suggesting that the concept of matching message and attitude may be an important consideration in the design of inoculation messages, particularly in relation to a salient topic such as extreme weather events. Provided that the aforementioned factors can be addressed, this study shows promise for the use of inoculation methods in conferring resistance to persuasion about climate-related topics which will encourage much needed engagement from the general population with relevant policies.

Acknowledgments

The authors would like to acknowledge Dr Meegan Kilcullen for their feedback. The authors would also like to acknowledge the time and contribution of the participants to this study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author, Madison Green, upon reasonable request.

Additional information

Funding

No external financial grants or funding was received for this study. In-kind support was provided by James Cook University.

Appendices Appendix A.

Condition 2 – Passive Inoculation

Instruction: Please read the following information carefully.

Threat: Statement of misinformation occurring

There has been much public debate around the perceived exaggerated risks posed by some extreme weather events. These exaggerations are problematic as they often result in people not believing risk information and becoming complacent in their preparations for these extreme weather events.

Example of Misinformation: Example of media misinformation on extreme weather events

Exaggerations occur due to the persuasion techniques that are used in media and social media to gain the attention of their audience. For example, some news stories have described weather events as “one-in-a-hundred-years” events, like Cyclone Yasi. The problem is that these events have been followed by events also described as “one-in-a-hundred-years”.

Refutational Pre-emption/Debunking: Statement on correction. http://www.bom.gov.au/water/designRainfalls/rainfallEvents/why100years.shtml

The reason why these events seem to occur more frequently than every 100 years is that the term “100 year event” is used as a part of a statistical formula for estimating the likelihood of an event rather than the actual event occurring. For instance, “the odds that this event will happen in this year is one in one-hundred” rather than “this event will only happen once every 100 years”. If this is applied to each year, then it is entirely possible to have a one-in-a-hundred year event in sequential years.

This is an example of how sometimes the source of the information is misunderstood and miscommunicated, resulting in people believing risks are exaggerated and becoming complacent in their preparations. Therefore, it is important that we can identify these attention-seeking and persuasion techniques in the information we see and hear so that we can make better decisions.

Condition 3 – Active Inoculation

Instruction: Please read the following information carefully. Your knowledge will be tested on the following page.

Threat: Discussion on the techniques used by information sources to influence opinion

There has been much public debate around the perceived exaggerated risks posed by some extreme weather events. These exaggerations are problematic as they often result in people not believing risk information and becoming complacent in their preparations for these extreme weather events.

Exaggerations occur due to the persuasion techniques that are used in media and social media to gain the attention of their audience.

Example of Misinformation: shown examples of the different techniques used.

Some of these techniques include:

Hyperbole: exaggerated statements or claims not meant to be taken literally. For example, “Jenny has a million friends”. Jenny does not literally have a million friends, what is implied is that she has many friends.

Common Man: attempts to convince readers or audience by appealing to common sense. For example, politicians often talk about policies not meeting the “pub test”. Another example is a person making a statement implying that everyone would agree “Everybody knows it is common sense not to walk under a drop bear tree”. This makes the audience think that most people would agree with this statement and that they should know about drop bear risks.

Arguments from Authority: asserts an argument is true based on the credentials of the person making it. For example, “James X, who graduated with Honours in Public Health from the prestigious University of Oxford, said that the structural integrity of the apartment building was fine”. This statement makes the audience think that the “expert’s” opinion is valid even though the “expert” does not have any expertise in engineering.

Refutational Pre-emption/Debunking: Explain how these are wrong. Train how to write using these techniques. Test knowledge via MCQ examples. Asked to generate a tweet using different persuasion techniques.

These techniques often result in information being misunderstood or miscommunicated, which can influence the way that people think, feel and behave. Again, this is particularly concerning in the context of appropriate risk awareness and preparation for extreme weather events. Therefore, it is important that we can identify these attention-seeking and persuasion techniques in the information we see and hear so that we can make better decisions.

So this is what we want to do today. We want to train you to better identify the quality of the information you see and hear. First, we are going to show you a list of tweets using different persuasion techniques. We would like you to indicate which persuasion technique is being used. Secondly, we will ask you to generate your own tweets using a persuasion technique. Ready?

About tweets: if you do not have a Twitter account and do not understand how tweets work, here is a simple instruction. You need to generate a statement using 240 characters (characters include letters, numbers, symbols, and spaces). Hashtags are often included in tweets to shorten or simplify the communication, linking the tweet with an existing conversation or topic. The answer box is limited to 240 characters to help you identify when you have reached your limit.

For the following tweets, please indicate if the persuasion technique being used is a hyperbole, common man or appeal to authority.

Hyperbole: #Breaking: Monster Cyclone Debbie tracking towards Qld coast #GetReady #Bringit

Common Man: Floodwater not a concern for #NthQld if there is enough cold #beer to go around #trueblue #aussie #battler

Appeal to Authority: #ICYMI Disaster Management director says Cairns is “well overdue” for a direct hit from #tropicalcyclone

Now we would like you to make up your own tweets using each persuasion technique. In each of the spaces below, please provide a tweet using a hyperbole, common man and appeal to authority as directed. The topic of the tweet needs to be in relation to an extreme weather event (cyclone, storm, flood, heatwave, bushfire, etc.).

Appendix B.

Article 1: Misinformation (Vanessa Marsh, 07/06/2018, Courier Mail/Cairns Post Reprint)

http://www.bom.gov.au/water/designRainfalls/rainfallEvents/why100years.shtml TROPICAL cyclones in Australia are becoming more destructive than ever, with steadily slowing travel speeds causing extreme rainfall, flooding and storm damage.

Weather experts have found that, globally, the translation speed at which cyclones travel across the ocean or land has decreased by 10% over the past 70 years, and in Australia, the speeds have slowed by an astonishing 19%.

Meteorologists say the change means rainfalls during cyclones can more than double and that communities are exposed to drastically longer periods of destructive winds.

Higgins Storm Chasing weather expert Jeff Higgins said a reduction in cyclone translation speeds could help weaken the severity of the systems.

“The slower a cyclone goes over the ocean can cause slight weakening due to the cooler waters being churned up off the bottom”, Mr Higgins said.

“But the downside is that while they might be slightly weaker, when it does get to land that slow travel speed means more heavy rain, more damaging winds for a longer period of time and then your risk of flooding and damage rises very quickly, so it’s a catch 22.

“I’d sooner have a category five race through over a smaller area than a category four go slow and cause thousands of square kilometres of damage through flooding.”

Mr Higgins said Cyclone Debbie, which caused damaged estimated at $2.67 billion in March last year, was an example of the havoc a slow-moving system could wreak.

“The extent of the flooding was enormous and it caused a lot of damage,” he said.

“With a cyclone, most of the deaths occur from flash flooding and unfortunately, a lot of those are caused by people driving into floodwaters.

“The wind and structural collapse of cyclones don’t kill many people, so we need to be making sure people are aware of their flood zones and don’t become complacent.”

Article 2: Reliable Information (article adapted from V. Marsh, Courier Mail, 08/02/2018 http://www.bom.gov.au/water/designRainfalls/rainfallEvents/why100years.shtml)

QUEENSLAND could be deemed “unfavourable” for tourism with extreme weather events, such as heatwaves, storms and bushfires, predicted to affect the viability of the state’s booming tourism industry.

The United Nations has identified Australia as one of the five tourism hotspots vulnerable to climate change. The nation’s top attractions, including beaches, wildlife, the Great Barrier Reef and national parks, could be at risk from changing weather patterns.

The Climate Council will today reveal the extent of the threat to Australia’s tourism industry from climate change, warning that northern states such as Queensland are at highest risk of becoming “inhospitable” during peak tourism times.

Analysis for the United Nations found Australian tourism was particularly vulnerable due to hotter summers, warmer winters, water scarcity, marine biodiversity loss, sea-level rise, an increase in disease outbreaks and an increase in extreme weather events.

“Analysis indicated that … much of Queensland … could become inhospitable during substantial parts of the year, especially in the summer months, the peak season for international visitors,” the report said.