“I was Right about Vaccination”: Confirmation Bias and Health Literacy in Online Health Information Seeking

Abstract

When looking for health information, many people turn to the Internet. Searching for online health information (OHI), however, also involves the risk of confirmation bias by means of selective exposure to information that confirms one’s existing beliefs and a biased evaluation of this information. This study tests whether biased selection and biased evaluation of OHI occur in the context of early-childhood vaccination and whether people’s health literacy (HL) level either prevents or facilitates these processes. Vaccination beliefs were measured for 480 parents of young children (aged 0–4 years) using an online survey, after which they were exposed to a list of ten vaccine-related message headers. People were asked to select those headers that interested them most. They also had to evaluate two texts which discussed vaccination positively and negatively for credibility, usefulness, and convincingness. The results showed that people select more belief-consistent information compared to belief-inconsistent information and perceived belief-confirming information as being more credible, useful, and convincing. Biased selection and biased perceptions of message convincingness were more prevalent among people with higher HL, and health communication professionals should be aware of this finding in their practice.

For the third year in a row, immunization rates among newborns and young children have decreased in the Netherlands, and other Western countries face the same problem (Omer, Salmon, Orenstein, Dehart, & Halsey, 2009; van Lier et al., 2017). This is problematic since decreasing immunization rates increase the likelihood of a disease outbreak that also affects young children who have been vaccinated but are not yet completely immunized. Although the role of online information in vaccination decisions is not completely clear, research has shown that 41% of information-seeking parents use the Internet as an information source about vaccines (Harmsen et al., 2013). It has been shown, however, that parents who exempt their children from vaccinations are more likely to have obtained information from the Internet compared to parents who choose to vaccinate their children (Jones et al., 2012), and people’s scores on the VAX scale, which measures vaccination resistance, appeared to be associated with people’s preferences for online information (Martin & Petrie, 2017).

Online health information (OHI) seeking is easy, but it may also facilitate confirmation bias. This means that people predominantly read information that is consistent with their existing beliefs and ignore inconsistent information (i.e., selective exposure); people also tend to evaluate consistent information as more accurate than inconsistent information (i.e., biased evaluation). Although the role of confirmation bias has been extensively studied in the domain of politics and news selection (e.g., Jang, 2014; Knobloch-Westerwick & Meng, 2009), little is known about the occurrence of confirmation bias processes in the context of health information (Hastall & Knobloch-Westerwick, 2013). Research on confirmation bias is important since the nature of the Internet requires people select the information they encounter. Online information is not centrally controlled and therefore differs from many other information sources (Flanagin & Metzger, 2000). Everyone on the Internet can be an author, with or without medical qualifications, which means that lay-people can also easily spread information on a very large scale. This is in contrast to the pre-Internet era, in which controversy over medical topics also existed, but the platforms to spread this information were much more limited. While information seekers should ideally test their pre-existing beliefs by objectively consulting the most accurate and least biased sources of evidence available and by evaluating these sources as such, information seekers often, in practice, attempt to find answers by looking at sources where their assumptions are most likely to be confirmed (St. Jean et al., 2015). A biased selection and evaluation of OHI based on existing beliefs might jeopardize well-informed decision-making.

The extent to which people are able to properly use information sources in making informed health decisions is called health literacy (HL). This concept is of great societal importance since health-literate people are hospitalized less often, participate more often in preventive health services, and have a better overall health status compared to people with lower HL levels (Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011). Having a higher level of HL, however, does not mean that someone complies with professional health advice. In the context of vaccination, for example, it has been shown that parents with higher HL perceive OHI as more reliable than people with lower HL, which subsequently decreases compliance with the immunization program (Aharon, Nehama, Rishpon, & Baron-Epel, 2017). It is still unclear whether this finding is the result of confirmation bias and only applies to parents who hold negative beliefs about early-childhood vaccination. Another study in the same context showed that people with higher HL are in fact more willing to vaccinate their children (Wang, Zhou, Lin, & Mantwill, 2018).

The aim of this study is twofold. First, we aim to investigate the role of confirmation bias in information seeking with respect to early-childhood vaccination. Second, this study explores the role of people’s HL in this relationship. Insights into the mechanisms underlying the relationship between OHI seeking, existing beliefs and HL could serve as a basis for the development of effective communication strategies to prevent immunization rates from decreasing any further.

Confirmation Bias in OHI Seeking

As the Internet is such an important source of health information, it is essential that people possess the crucial skills to critically evaluate and weigh the information they encounter. However, individuals generally ignore messages that challenge their beliefs and behaviors and seek out reinforcing content instead (Festinger as cited in Westerwick, Johnson, & Knobloch-Westerwick, 2017). Confirmation bias in OHI seeking could take place in several different stages of the search process. People usually begin their search at a search engine (Diviani, Van Den Putte, Meppelink, & Van Weert, 2016), which means that they have to select information from a list of results. Regarding selection, one of the few studies on selective exposure in the health domain showed that information preferences relate to personal values and goals (Pease, Brannon, & Pilling, 2006). In the context of heart disease, it was found, for example, that online search behavior is affected by inaccurate or limited knowledge. Participants explored irrelevant sites and also sought out information that confirmed their incorrect assumptions (Keselman, Browne, & Kaufman, 2008). Other studies examined the message characteristics that may foster selective exposure to health information (Hastall & Knobloch-Westerwick, 2013; Kim, Forquer, Rusko, Hornik, & Cappella, 2016) or the effects of selective exposure on attitudes towards health behaviors (Westerwick et al., 2017).

After a selection has been made, people need to evaluate the information they encounter. Confirmation bias could also play a role in this stage of the search process. People are not always motivated to critically verify the reliability and accuracy of online information (Metzger, 2007). Instead, people rely on subjective characteristics to determine whether a certain website provides reliable information. One important evaluation criteria involves the degree to which the information confirms people’s existing ideas or beliefs (Diviani et al., 2016; Wathen & Burkell, 2002). To test the role of confirmation bias in both phases, the following hypotheses are formulated:

People are more likely to select belief-consistent than belief-inconsistent vaccination messages (H1). People perceive belief-consistent vaccination messages as more credible (H2a), useful (H2b), and convincing (H2c) than belief-inconsistent vaccination messages.

The Role of Health Literacy

An important individual factor that is relevant to OHI seeking is HL. In recent decades, the HL concept has rapidly evolved and has been conceptualized and defined in multiple ways (Sørensen et al., 2012). In our study, we adhere to the frequently applied definition: “the ability to obtain, process, understand, and communicate about health-related information needed to make informed health decisions” (Berkman, Davis, & McCormack, 2010, p. 16). There are two opposing perspectives on the relation between HL and online confirmation bias. First, people with higher HL are generally better informed about health issues and make better-informed health decisions compared to people with low HL skills (e.g., Meppelink, Smit, Buurman, & van Weert, 2015). They also use better criteria to assess the quality of OHI (Diviani et al., 2016), which could prevent them from selecting health information that confirms their own beliefs. The second perspective, however, suggests that people with higher HL could be at risk of a biased selection and evaluation of OHI. Particularly when people perceive themselves as highly knowledgeable, they can overestimate their judgement skills, which can make them less receptive to information that is perceived as incorrect (Schulz & Nakamoto, 2013). This second perspective could explain the finding that people with higher levels of HL seem to be more open to – even inaccurate – confirming health information (Aharon et al., 2017). Moreover, a certain level of understanding of the material is required for recognizing whether messages are in line with or against people’s own beliefs, which could also increase confirmation bias among people with higher HL. To further investigate the role of HL in the OHI-seeking process, the following research questions are formulated:

Does HL influence the selection of belief-consistent versus belief-inconsistent vaccination messages in OHI seeking? (RQ1). Does HL influence the evaluation of belief-consistent versus belief-inconsistent vaccination messages in terms of perceived credibility (RQ2a), perceived usefulness (RQ2b), and perceived convincingness (RQ2c)?

Methods

Survey Procedure

We used an online survey to investigate the relationships between existing vaccination beliefs, HL, and the selection and evaluation of online vaccine-related information. The Amsterdam School of Communication Research provided ethical approval for this study. Data were collected in September 2017. Considering the topic of early-childhood vaccination, only parents of children aged between 0 and 4 years old could participate. The survey started with some demographical questions, questions about early-childhood vaccination beliefs, and OHI use-related questions. After some filler questions to prevent people from guessing the study goal, participants were exposed to a list of ten headers about early-childhood vaccination. Five of those headers described vaccinations negatively, and five described them positively. Respondents were told that they saw the result of an Internet search for ‘early-childhood vaccination’ and were asked to select the five headers that they were most interested in reading further. After this selection, people were exposed to two full-text messages of approximately 200 words each discussing early-childhood vaccination either positively or negatively. Both texts were the same for everyone and not necessarily the ones people had selected. The texts were presented in random order and people had to evaluate their credibility, usefulness, and convincingness. All materials used in this study were carefully developed and extensively pretested for tone-of-voice, readability level, and writing style, which is described in Appendix A. People were properly debriefed afterwards.

Data Collection

The market research institute PanelClix (ISO certified) sent a link to our questionnaire to a randomly selected sample of their online panel members. Of the 591 people who started the questionnaire, 18 (3%) discontinued their participation on the introduction page, six (1%) quit when they were asked to provide informed consent, nine (1.5%) did not complete the entire survey, and 78 (13.2%) were screened out because they did not report having a child aged between 0 and 4, resulting in a final sample of 480 people.

Measures

Health Literacy

HL was measured with the Dutch version of the Newest Vital Sign (NVS: Fransen et al., 2014; Weiss et al., 2005). This validated measure shows people the nutrition label of an ice cream container, followed by six open-ended questions that require interpretation and calculation skills related to the nutrition label. The six answers are coded as incorrect (0) or correct (1) according to the measure’s guidelines, resulting in a sum score ranging from 0 to 6 (M = 4.06, SD = 1.92).

Functional, Communicative, and Critical Vaccine-Related Health Literacy

We also assessed functional, communicative, and critical HL (Ishikawa, Takeuchi, & Yano, 2008; van der Vaart et al., 2012). Our items were identical to the ones used by Aharon et al. (2017), who adapted the measure to fit the context of early-childhood vaccination. It consists of 13 statements with answer options ranging from “never” to “often” on a 4-point scale. Five items measured functional HL (e.g., In reading instructions or leaflets regarding your child’s vaccines, how often do you need a long time to read and understand the text), five items measured communicative HL (e.g., Since your child was born, how often have you collected information on vaccination from various sources) and three items measured critical HL (e.g., Since your child has needed vaccines, how often have you considered the credibility of the information about vaccines). The item “since your child was born, to what extent have you understood all the information you obtained about vaccination?” was excluded from the communicative HL scale based on factor analysis. The scores for functional HL were reversed coded so that higher scores on all subscales indicate higher HL. All subscales were reliable: functional: α = .908, M = 3.08, SD = 0.69; communicative: α = .815, M = 2.58, SD = 0.60; critical: α = .892, M = 2.38, SD = 0.82.

Vaccination Beliefs

The Vaccination Confidence Scale (VCS: Gilkey et al., 2014, 2016) was used to measure existing beliefs towards early-childhood vaccination. The scale consists of eight items (e.g., vaccines are necessary to protect the health of children) with answer options ranging from totally disagree (1) to totally agree (11) (α = .832, M = 8.00, SD = 1.63).

Message Selection

Participants were exposed to a list of ten headers about early-childhood vaccination, five of which were clearly negative and five clearly positive (see Appendix A). They were asked to select the five headers that they were most interested in reading more about. A sum score was calculated indicating the valence of the selected headers, ranging from 0 (all negative) to 5 (all positive), M = 2.76, SD = 1.30. Thus, a higher score indicates the selection of more positive vaccination messages.

Perceived Information Credibility

Perceived credibility of the vaccine-related information was measured for both full-texts, using a five-item message credibility scale (Flanagin & Metzger, 2000). The items were, “To what extent is the information believable – accurate – trustworthy – biased - complete”. Answer options ranged from (1) not at all to (7) very much (positive text: α = .748, M = 4.58, SD = 0.87; negative text: α = .843, M = 3.92, SD = 1.12). Subsequently, perceived credibility scores for the text describing vaccinations negatively were subtracted from the credibility scores of the text discussing vaccinations positively, resulting in a difference score. As a result, a negative score on this measure indicates that the participant perceived the negative text as more credible, whereas a positive score shows that the positive text was perceived as more credible (M = 0.65, SD = 1.31).

Perceived Information Usefulness

For both full-texts, perceived information usefulness was measured using the following items on a 7-point Likert scale: This information is relevant – important – useful - helpful; (1) not at all, (7) very much. (positive text: α = .937, M = 4.97, SD = 1.19; negative text: α = .944, M = 4.06, SD = 1.44). Then, the usefulness scores of the negative text were subtracted from the positive ones, resulting in a difference score. A negative score on this measure indicates that the participant perceived the negative text as more useful, whereas a positive score shows that the positive text was perceived as more useful (M = 0.90, SD = 1.70).

Perceived Information Convincingness

The extent to which both texts were perceived as convincing was measured with the following item: “to what extent do you think that the text is convincing?”. Answer options ranged from (1) not at all to (7) very much (M_pos = 5.06, SD_pos = 1.21, M_neg = 4.09, SD_neg = 1,57). By subtracting one’s score on the negative text from the positive score, a difference score was created. Thus, a negative score indicates that the participant perceived the negative text as more convincing, whereas a positive score indicates that the positive text was perceived as more convincing (M = 0.97, SD = 1.85).

Statistical Analysis

All variables were checked for normality before they were used in the analysis (skewness between −1 and 1, kurtosis between −3 and 3). We used PROCESS (model 1, 5,000 bootstrapped samples) to test our moderation hypotheses. PROCESS specifies the conditional effects of X on Y for different levels of the moderator by means of bootstrapping (Hayes, 2013). Existing vaccination beliefs and HL were mean centered prior to the analysis.

In this study, the NVS is used as a measure of HL since this measure is validated and performance based and has been widely applied to assess HL in different contexts (Weiss et al., 2005). By using this measure, we are able to draw conclusions about the effects of HL on our dependent variables rather than self-reported HL specific to the context of early-childhood vaccination. For reasons of comparability with other studies that may apply context-specific HL measures (e.g., Aharon et al., 2017; Wang et al., 2018), we also tested our hypotheses using the self-reported functional, communicative, and critical HL scales as predictors. The results obtained with the functional HL scale are comparable with the results presented below (using the NVS). The other two HL scales showed somewhat different patterns. See Appendix B for a report of these extra tests and differences. To make sure that our findings regarding HL were not spuriously caused by education, all analyses were repeated with education as a covariate. This did not change the results as these models showed comparable outcomes indicated by the same coefficients and levels of significance.

Results

Table 1 shows the characteristics of the study participants. About half of the sample was female, and half of the participants had obtained a higher professional education or university degree. Most participants searched online for health-related information only a few times per year or once per month. Over 80% of the participants sought vaccine-related OHI a few times per year or less. A quarter of the participants reported having never sought vaccine-related information online. A large majority (87%) of the participants had vaccinated their children according to the Dutch National Immunization Program, and for most participants, the decision to vaccinate their children was barely affected by religion.

Table 1. Participant characteristics

Variable		n	%	Mean	SD
Gender	Male	221	46
	Female	259	54
Age (years)				34.02	6.56
Education level	Low	61	12.7
	Middle	179	37.3
	High	240	50.0
Health-related online	Never	16	3.3
information seeking	Once per year	36	7.5
	Few times per year	164	34.2
	Once per month	111	23.1
	Few times per month	73	15.2
	Once per week	35	7.3
	Few times per week	45	9.4
Online health information seeking about vaccinations	Never	127	26.5
	Once per year	143	29.8
	Few times per year	133	27.7
	Once per month	34	7.1
	Few times per month	16	3.3
	Once per week	11	2.3
	Few times per week	16	3.3
Professional medical	Yes	93	19.4
background	No	387	80.6
Perceived medical knowledge				3.86	1.30
(1 not at all −7 very much)
e-Health literacy (eHEALS)				3.67	0.61
(1 low – 5 high)
Children vaccinated according to the Dutch National	Yes, standard vaccination scheme	421	87.7
Immunization Program?	Yes, adapted vaccination scheme	38	7.9
	No	9	1.9
	Other	12	2.5
Extent to which people are in favor or against the Dutch National Immunization Program (1 completely against – 11 completely in favor)				8.98	2.28
Extent to which religion plays a role in vaccination decisions (1 not at all - 7 very much)				2.23	2.00

Vaccination Beliefs, Information Selection, and Health Literacy

To test our first hypothesis and answer RQ1, a moderation analysis was conducted using PROCESS with existing vaccination beliefs as an independent variable, HL as a moderator, and selection of vaccine-related information as a dependent variable. The results showed that vaccination beliefs significantly predicted the selection of messages b = 0.32, t(476) = 9.60, 95% CI [0.25;0.38], p < .001. People holding positive beliefs towards vaccination selected more positive messages, whereas people holding negative beliefs selected more negative messages. Furthermore, a significant interaction effect was observed between existing vaccination beliefs and HL b = 0.04, t(476) = 2.77, 95% CI [0.01;0.08], p = .006. To interpret this interaction, the Johnson and Neyman technique (J/N analysis) was used to identify regions on the moderator continuum where the effect of the independent variable is significant and where it is not (Hayes, 2013). As shown in Figure 1, confirmation bias was found on both sides of the belief-spectrum. People holding negative beliefs about vaccination tend to select more negative texts, whereas people holding positive beliefs select more positive texts. Although this confirmation bias appears to exist among all people, supporting H1, these effects are stronger when people have a higher level of HL (b = 0.40, t(476) = 9.12, 95% CI [0.31;0.49], p < .001) compared to average HL (b = 0.32, t(476) = 9.60, 95% CI [0.25;0.38], p < .001) or lower HL (b = 0.23, t(476) = 5.06, 95% CI [0.14;0.32], p < .001). The J/N analysis showed that for VCS scores of 6.21 or lower (VCS scale midpoint is 6), HL was negatively associated with the selection of positive messages, whereas for VCS scores of 9.45 or higher, HL is positively related to the selection of positive messages. No main effect for HL was found b = 0.01, t(476) = 0.52, 95% CI [−0.04;0.07], p = .61. Thus, to answer RQ1, people with higher HL tend to select more belief-consistent information than people with lower HL.

Figure 1. Interaction-effect of health literacy and vaccination beliefs on the selection of positive versus negative messages (RQ1). A higher score indicates the selection of more positive messages. Zones marked with p < .05 indicate where the relationship between HL and message selection depends on vaccination beliefs.

Vaccination Beliefs, Message Evaluation, and Health Literacy

Three moderation analyses were conducted in PROCESS to test to what extent existing vaccination beliefs influence the evaluation of belief-consistent versus belief-inconsistent messages (H2a, H2b, and H2c) and to investigate the role of HL in this process (RQ2a, RQ2b, RQ2c). For the first evaluation measure, perceived information credibility, the results showed that existing vaccination beliefs were associated with perceived credibility b = 0.32, t(476) = 8.74, 95% CI [0.25;0.39], p < .001. Since the perceived credibility score is a function of the difference between the perceived credibility of the positive text and the negative text, a positive association means that people who hold positive beliefs towards vaccination perceive positive information as more credible, whereas people who hold negative beliefs find negative information more credible, supporting H2a. Additionally, a significant interaction effect was observed between vaccination beliefs and HL b = 0.06, t(476) = 3.33, 95% CI [0.02;0.09], p = .001. As shown in Figure 2, the relationship between HL and a biased evaluation of belief-consistent information was found only on the positive side of the belief-spectrum. Although this confirmation bias appeared to exist among all people, these effects were stronger when people have a higher level of HL (b = 0.43, t(476) = 7.50, 95% CI [0.32;0.55], p < .001) compared to average HL (b = 0.32, t(476) = 8.73, 95% CI [0.25;0.39], p < .001) or lower HL (b = 0.21, t(476) = 5.05, 95% CI [0.13;0.29], p < .001). The results of the J/N analysis showed that among people holding positive vaccination beliefs (belief score of 6.78 or higher), there is a positive relationship between HL and perceived credibility of belief-consistent information. However, if vaccination beliefs are less positive or even negative, HL is not associated with a biased perception of message credibility. Thus, to answer RQ2a, people with higher HL tend to perceive belief-consistent information as being more credible when they hold positive beliefs. However, this relationship was not observed for negative beliefs and negative messages. Furthermore, the results also showed a main effect of HL (b = 0.13, t(476) = 5.56, 95% CI [0.08;0.18], p < .001), meaning that people with higher HL tend to perceive positive vaccine-related messages as more credible compared to negative vaccine-related messages.

Figure 2. Interaction-effect of HL and vaccination beliefs on the perceived credibility of information (RQ2a). The zone marked with p < .05 indicates where the relationship between HL and perceived information credibility depends on vaccination beliefs.

The same analyses were conducted for usefulness, the second evaluation measure. We tested whether people find belief-consistent information more useful compared to belief inconsistent information (H2b) and explored the role of HL (RQ2b) in this respect. Existing vaccination beliefs were positively associated with perceived information usefulness b = 0.44, t(476) = 9.24, 95% CI [0.35;0.53], p < .001. This means that people who hold positive beliefs towards vaccination find messages in favor of vaccination more useful, whereas people who hold negative beliefs towards vaccination find information against vaccination more useful, supporting H2b. Beliefs and HL also appeared to interact b = 0.06, t(476) = 2.84, 95% CI [0.02;0.11], p = .005. Figure 3 shows that the relationship between HL and confirmation bias was only found on the positive side of the belief-spectrum. Although confirmation bias appeared to exist among all people, these effects were stronger when people had a higher level of HL (b = 0.57, t(476) = 8.09, 95% CI [0.43;0.70], p < .001) compared to average HL (b = 0.44, t(476) = 9.24, 95% CI [0.35;0.53], p < .001) or lower HL (b = 0.32, t(476) = 5.34 95% CI [0.20;0.43], p < .001). The results of the J/N analysis showed that when people hold positive vaccination beliefs (belief score of 6.93 or higher), HL is positively related to the perceived usefulness of belief-consistent information. Thus, to answer RQ2b, people with higher HL find belief-consistent information more useful than belief-inconsistent information, but only when they are in favor of vaccination. Additionally, a main effect of HL was found (b = 0.14, t(476) = 4.34, 95% CI [0.07;0.20], p < .001), suggesting that people with higher HL generally find information in favor of vaccination more useful than information against it.

Figure 3. Interaction-effect of HL and vaccination beliefs on the perceived usefulness of information (RQ2b). The zone marked with p < .05 indicates where the relationship between HL and perceived information usefulness depends on vaccination beliefs.

To test the final hypothesis (H2c) and answer RQ2c, the third evaluation measure (perceived information convincingness) was used as the dependent variable in the moderation analysis. The results showed a main effect for vaccination beliefs (b = 0.48, t(476) = 9.64, 95% CI [0.38;0.57], p < .001), indicating that belief-consistent information is perceived as more convincing than belief-inconsistent information. Vaccination beliefs and HL also appeared to interact b = 0.09, t(476) = 3.47, 95% CI [0.04;0.13], p = .001. As shown in Figure 4, the perception that belief-consistent information is more convincing than belief-inconsistent information was found on both sides of the belief-spectrum. People holding negative beliefs about vaccination found negative information more convincing, whereas people holding positive beliefs found positive information more convincing, supporting H2c. Although this confirmation bias appeared to exist among all people, these effects were stronger when people had a higher level of HL (b = 0.64, t(476) = 8.60, 95% CI [0.49;0.79], p < .001) compared to average HL (b = 0.48, t(476) = 9.64, 95% CI [0.38;0.57], p < .001) or lower HL (b = 0.31, t(476) = 5.05, 95% CI [0.19;0.43], p < .001). The J/N analysis showed that higher HL is associated with the perception that belief-consistent information is more convincing than belief-inconsistent information. This applied to people who hold stronger positive beliefs about vaccination (belief score of 6.98 or higher), or strong negative beliefs (belief score of 3.78 or lower). In the latter case, people perceived the negative text as more convincing than the positive one, answering RQ2c. The results also showed a main effect of HL b = 0.16, t(476) = 4.67, 95% CI [0.09;0.23], p < .001.

Figure 4. Interaction-effect of HL and vaccination beliefs on the perceived convincingness of information (RQ2c). Zones marked with p < .05 indicates where the relationship between HL and perceived information convincingness depends on vaccination beliefs.

Discussion

The aim of this paper was to investigate the role of confirmation bias and health literacy in OHI seeking. One of the main findings of our study is that parents generally prefer to select belief-consistent information regarding early-childhood vaccination. When people are presented with a list of messages, they tend to selectively expose themselves to information that confirms their prior beliefs rather than information that opposes their ideas. This confirmation bias was found not only with respect to message selection but also with respect to message evaluation. People consistently perceived belief-confirming information as being more credible, useful, and convincing. With this finding, our study contributes to the body of knowledge about the occurrence of selective exposure and biased evaluation in online health communication.

Furthermore, our study showed that confirmation bias in message selection was stronger among parents with higher HL compared to parents with lower HL. People with higher HL selected more messages that were in line with their own beliefs, and they did this regardless of whether they were in favor of or strongly against vaccination. We also found a stronger confirmation bias with respect to message evaluation among people with higher HL, but this mainly applied to people who had positive vaccination beliefs. The HL differences in message evaluation on the positive side of the belief spectrum could be explained by the fact that the participants in our study were generally quite positive about early-childhood vaccination. More than 87% of parents in this group had their children vaccinated in line with the Dutch National Immunization Program. Therefore, the group of parents who are strongly against vaccination might have been underrepresented in our sample, which may have influenced our findings. Nevertheless, immunization coverage below 95% is already problematic for public health, since lower rates increase the risk of a measles outbreak, for example (van Lier et al., 2017). Future research should therefore focus on this specific minority to see how OHI influences their decision to – or not to - vaccinate their children and identify strategies that are useful to ensure this group can become more open to correct but belief-inconsistent information.

Our study showed that people with higher HL and negative beliefs towards early-childhood vaccination perceive negative information as more convincing than positive information. This could possibly be explained by the discourse that is frequently used on anti-vaccination websites. Vaccine-critical websites generally promote alternative medicine, encourage parental autonomy and responsibility, and openly distrust medical expertise (Kata, 2010). These are complex themes, which especially health literate parents could be attracted to.

The findings of this study, using the performance-based NVS as the HL measure, are comparable to outcomes provided by the self-reported functional HL scale. Communicative and critical HL, in contrast, were positively associated with the preference of anti-vaccination messages. The formulation of the items for the communicative and critical HL scales possibly fits people who are skeptical towards vaccination better and therefore might not only measure HL but also vaccination hesitancy. Consulting multiple information sources when a child needs vaccines, for example, increases critical HL. However, the majority of our participants reported never seeking out vaccine-related information online, or only doing so once per year. It is possible that people only consult OHI in this context when they question the default option, namely, vaccinating children according to the immunization program. Since the choice of HL measure might impact the study results (Kiechle, Bailey, Hedlund, Viera, & Sheridan, 2015), this is something researchers need to be aware of, especially because increasing numbers of context-specific HL measures are being developed and the variety of measures is increasing (Mackert, Champlin, Su, & Guadagno, 2015).

Despite different initiatives and calls for action to improve HL on a population level (Nutbeam, 2000; Pleasant, Cabe, Patel, Cosenza, & Carmona, 2015), our study showed that under some conditions (i.e., deviating beliefs and OHI seeking), higher HL can also be problematic. If higher HL also means that people feel more certain and knowledgeable about certain health-related topics and therefore become less open to information that challenges their own ideas, this could impact health communication effectiveness. Future research is needed to investigate whether people with low and high HL differ in attitude strength towards certain health topics and whether one group is more open to attitude change than another.

A limitation of this study is that the presentation of the messages could have influenced our findings. To prevent people from evaluating the messages based on characteristics other than the text, no source or website interface was provided. In natural settings, however, people take the message source into account when judging information credibility (e.g., Metzger, 2007). Future research should therefore investigate to what extent the source influences people’s receptiveness to belief-consistent versus inconsistent information and whether the level of expertise of the information source could prevent people from confirmation bias in OHI seeking.

Conclusion

Our study showed that people selectively expose themselves to belief-consistent information rather than disconfirming information when they seek online information about early-childhood vaccination. People also find information more credible, useful and convincing when it is in line with their ideas. Biased selection and biased perceptions of message convincingness were more prevalent among people with higher HL, indicating that the phenomenon of confirmation bias is quite important in the context of vaccination, especially among parents with high levels of HL.

Appendix A

To develop the study materials, 18 texts were retrieved from real Dutch webpages about early-childhood vaccination. The keywords “early-childhood vaccination,” “vaccination benefits,” “vaccine side effects,” and “vaccination dangers” were entered into Google (cookies and history deleted beforehand) to find these webpages. Half of the selected texts described early-childhood vaccines in a positive way, e.g., by discussing the benefits for public health and people’s responsibility to keep immunization coverage high. The other articles were negative and discussed the – suggested – poison content of vaccines and the severity of side effects. The 18 selected articles were rewritten by the research team to make them comparable in terms of length (200–210 words each), reading level (all level B1/B2, assessed using the Accessibility leesniveau tool (2017)), and quality of writing. The 18 texts and accompanying headers were pretested in a sample of 48 participants (79.2% female, M_age = 39.3, SD_age = 14.7 range = 23–78, education level = 47.9% university degree). In this pretest, each participant was asked to evaluate a random sample of five of those 18 texts and headers. People rated the extent to which the header and text were positive or negative about early-childhood vaccination on a 9-point Likert scale. They also rated the quality of writing of the text, varying from very bad (1) to very good (9) on a Likert scale. Based on this pretest, ten headers were selected for the main study (see table below). The five selected negative headers were significantly more negative about vaccination than the five positive ones (p < .05). Furthermore, two full texts were selected based on the pretest; these were shown to all participants in the main study. The selected negative text was perceived as significantly more negative about vaccination (M = 2.38, SD = 1.39) than the positive one (M = 8.25, SD = 1.22, p < .001), but the texts did not significantly differ in quality of writing (M_neg = 5.77, SD_neg = 2.05, M_pos = 6.67, SD_pos = 1.97, p = 1.00).

Results of the pretest to select the vaccine-related headers (translated to English).

Table

Number	Header	N	Mean	SD	Sign. differs from (Bonferroni applied)
6	Vaccines, a wolf in sheep’s clothing	13	1.62	0.77	4, 10 to 18
7	The dangerous content of vaccines	13	2.00	1.22	10 to 18
5	Doctor stresses that vaccines are harmful	15	2.27	1.33	10 to 18
8	Which poisons are used in vaccines?	13	2.31	1.55	10 to 18
2	The disadvantages of vaccines are hardly mentioned	15	3.07	2.12	10, 11, 12, 14 to 18
3	Damage caused by side effects of vaccination	12	3.25	0.97	10, 11, 12, 14 to 18
9	Vaccines are no longer needed	14	3.29	1.38	10, 11, 12, 14 to 18
1	Vaccines are unnatural	14	3.36	1.95	10, 11, 12, 14 to 18
4	Pharmaceutical industry makes billions in profits through vaccines	15	3.93	1.94	10, 12, 15 to 18
13	Myth about vaccines and autism is still alive	13	5.15	2.27	5, 6, 7, 8, 10, 16, 17
14	Parents unfamiliar with the risks associated with not vaccinating when children go to daycare	13	6.31	2.14	1, 2, 3, 5 to 9
11	Anti-vaccination lobby uses incorrect arguments	12	6.33	2.15	1, 2, 3, 5 to 9
18	Scary stories, but vaccines are not poisonous	14	7.00	1.24	1 to 9
15	Not vaccinating puts others’ babies in danger	10	7.20	2.35	1 to 9
12	Vaccine-preventable diseases are certainly not harmless	12	7.42	1.24	1 to 9
17	Vaccines have prevented much harm	12	8.08	0.79	1 to 9, 13
16	Pediatricians: high vaccination rates are extremely important	16	8.19	1.11	1 to 9, 13
10	Vaccines protect your child and the population	12	8.33	0.98	1 to 9, 13

Note. The headers in bold (numbers 6, 7, 5, 8, 2 are negative; 15, 12 17, 16, 10 positive) were selected for the study based on this pretest. All positive headers are perceived as significantly more positive than the negative ones. Within their own category (negative/positive), the headers do not differ from each other.

Appendix B

Comparison of Results using the NVS Versus the Subjective HL as Predictors

The correlations between the NVS and the self-reported scales were: functional HL: r = .48, p < .001; communicative HL: r = −.09, p = .041; and critical HL: r = −.175, p < .001.

Dependent variable: Selection of vaccine-related information

Functional HL: Main effect of vaccination beliefs b = 0.31, t(476) = 8.90, p < .001. Interaction between functional HL and vaccination beliefs b = 0.10, t(476) = 1.83, p = .067. Among people holding positive vaccination beliefs, functional HL is significantly related to the selection of belief consistent messages. No main effect of functional HL b = 0.08, t(476) = 0.96, p = .34.

Communicative HL: Main effect of vaccination beliefs b = 0.30, t(476) = 8.96, p < .001. Interaction between communicative HL and vaccination beliefs b = 0.13, t(476) = 2.66, p = .008. Among people holding negative vaccination beliefs, communicative HL is significantly related to the selection of belief consistent messages. Main effect of communicative HL b = −.26, t(476) = −2.80, p = .005.

Critical HL: Main effect of vaccination beliefs b = 0.28, t(476) = 7.88, p < .001. Interaction between critical HL and vaccination beliefs b = 0.09, t(476) = 2.34, p = .019. Among people holding negative vaccination beliefs, critical HL is significantly related to the selection of belief consistent messages. Main effect of critical HL b = −.18, t(476) = −2.51, p = .012.

Dependent variable: Perceived information credibility

Functional HL: Main effect of vaccination beliefs b = 0.30, t(476) = 9.00, p < .001. Interaction between functional HL and vaccination beliefs b = 0.13, t(476) = 2.58, p = .010. Among people holding positive vaccination beliefs, functional HL is significantly related to the perception of information credibility. Main effect of functional HL b = 0.44, t(476) = 5.71, p < .001.

Communicative HL: Main effect of vaccination beliefs b = 0.34, t(476) = 9.09, p < .001. No interaction between communicative HL and vaccination beliefs b = 0.04, t(476) = 0.62, p = .535. No main effect of communicative HL b = −.00, t(476) = −0.01, p = .994.

Critical HL: Main effect of vaccination beliefs b = 0.32, t(476) = 8.71, p < .001. No interaction between critical HL and vaccination beliefs b = 0.05, t(476) = 1.10, p = .271. No main effect of critical HL b = −.11, t(476) = −1.57, p = .117.

Dependent variable: Perceived information usefulness

Functional HL: Main effect of vaccination beliefs b = 0.41, t(476) = 8.54, p < .001. Interaction between functional HL and vaccination beliefs b = 0.21, t(476) = 2.59, p = .010. Among people holding positive vaccination beliefs, functional HL is significantly related to the perception of information usefulness. Main effect of functional HL b = 0.52, t(476) = 5.47, p < .001.

Communicative HL: Main effect of vaccination beliefs b = 0.46, t(476) = 9.64, p < .001. No interaction between communicative HL and vaccination beliefs b = 0.02, t(476) = 0.25, p = .803. No main effect of communicative HL b = −.09, t(476) = −0.73, p = .465.

Critical HL: Main effect of vaccination beliefs b = 0.43, t(476) = 8.79, p < .001. No interaction between critical HL and vaccination beliefs b = 0.04, t(476) = 0.73, p = .467. No main effect of critical HL b = −.15, t(476) = −1.58, p = .115.

Dependent variable: Perceived information convincingness

Functional HL: Main effect of vaccination beliefs b = 0.44, t(476) = 9.01, p < .001. Interaction between functional HL and vaccination beliefs b = 0.25, t(476) = 3.12, p = .002. Among people holding positive vaccination beliefs, functional HL is significantly related to the perception of information convincingness. Main effect of functional HL b = 0.58, t(476) = 6.05, p < .001.

Communicative HL: Main effect of vaccination beliefs b = 0.50, t(476) = 10.19, p < .001. No interaction between communicative HL and vaccination beliefs b = −0.07, t(476) = 0.82, p = .412. No main effect of communicative HL b = −.14, t(476) = −1.14, p = .256.

Critical HL: Main effect of vaccination beliefs b = 0.46, t(476) = 9.05, p < .001. No interaction between critical HL and vaccination beliefs b = 0.05, t(476) = 0.81, p = .421. For people holding moderate vaccination beliefs, critical HL is significantly negatively related to perceived information convincingness. Main effect of critical HL b = −.27, t(476) = −2.67, p = .008.