The Association Between the We Can Do This Campaign and Vaccination Beliefs in the United States, January 2021–March 2022

Abstract

Public health campaigns addressing COVID-19 vaccination beliefs may be effective in changing COVID-19 vaccination behaviors, particularly among people who remain vaccine hesitant. The “We Can Do This” COVID-19 public education campaign (the Campaign) was designed to increase COVID-19 vaccine confidence and uptake. This study aims to evaluate whether Campaign dose was associated with changes in vaccination beliefs related to COVID-19 vaccine concerns and perceived risks, the importance of COVID-19 vaccines, the perceived benefits of COVID-19 vaccination, normative beliefs about COVID-19 vaccination, and perceptions about general vaccine safety and effectiveness. The study linked data from four waves of a nationally representative longitudinal panel of U.S. adults (January 2021–March 2022) with Campaign paid digital media data (April 2021–May 2022). We used mixed-effects linear regressions to examine the association between Campaign paid digital impressions and changes in vaccination beliefs. The results provide evidence that Campaign digital impressions were significantly associated with changes in respondent beliefs regarding COVID-19 vaccine concerns and perceived risks, perceived benefits of COVID-19 vaccination, and perceptions about general vaccine safety and effectiveness. Findings suggest that public education campaigns may influence vaccine confidence and uptake by increasing positive vaccination beliefs and reducing vaccine concerns.

Vaccine hesitancy, defined by the World Health Organization as “the delay in the acceptance or refusal to vaccinate despite the availability of vaccine services” (Dorman et al., 2021; MacDonald, 2015; Schuster et al., 2015), poses a significant global public health challenge (Dhama et al., 2021; Dubé et al., 2013; Galagali et al., 2022). The risk of morbidity and mortality from vaccine-preventable infectious diseases increases with vaccine delays and refusals (Frenkel, 2021; Larson et al., 2015; Phadke et al., 2016; WHO, 2015), underscoring the importance of efforts to reduce vaccine hesitancy and increase vaccine uptake. Despite the availability of vaccines in the United States since early December 2020 and their proven safety and effectiveness in reducing symptomatic COVID-19 cases, hospitalizations, and deaths (Bernal et al., 2021; Hall et al., 2022; Plumb et al., 2022), as of April 22, 2023, less than 70% of the U.S. population had completed their primary vaccination series, and only 17% had received an updated (bivalent) vaccine dose (Centers for Disease Control and Prevention [CDC], 2023).

Correlates of COVID-19 Vaccine Hesitancy

COVID-19 vaccine hesitancy has been associated with a range of factors, including sociodemographic characteristics such as age, sex, education, geographic region, income, race/ethnicity, and political ideology (Callaghan et al., 2021; Khubchandani et al., 2021; Viswanath et al., 2021; Wang & Liu, 2022). In addition to sociodemographic characteristics, beliefs, including those about vaccination, are among the strongest correlates and determinants of COVID-19 vaccine hesitancy (Kricorian et al., 2022; Lueck & Callaghan, 2022; Nguyen et al., 2023). Vaccination beliefs can be categorized into positive behavioral factors (beliefs and attitudes that facilitate vaccination and increase the likelihood of getting vaccinated) and negative behavioral factors (beliefs and attitudes that constrain or decrease the likelihood of vaccination) (Lueck & Callaghan, 2022; Padamsee et al., 2022). Vaccine-confident individuals—those who are the least hesitant about vaccination—are more likely to endorse positive vaccination beliefs and to report trust in government (Lueck & Callaghan, 2022). Conversely, vaccine-hesitant individuals are more likely to endorse negative beliefs, such as concerns about vaccine safety, government distrust, medical mistrust and concerns with the speed of testing and approval of the vaccines, confusion over conflicting information, and conspirative thinking (Ciardi et al., 2021; Doherty et al., 2021; Lindholt et al., 2021; Walker et al., 2021).

Public Education Campaigns

Public education campaigns present an opportunity to raise awareness about health conditions or behaviors, influence beliefs, and promote specific behaviors (Wakefield et al., 2010). Evaluations indicate that campaigns that address behavioral beliefs are effective in changing health behaviors (Anker et al., 2016; Noar, 2006; Noar et al., 2009; Snyder & Hamilton, 2002). Specifically, emerging research demonstrates that public health campaigns can change COVID-19 beliefs and behaviors regarding vaccination (Athey et al., 2023; Marcell et al., 2022; Williams et al., 2023), as COVID-19 vaccination decisions are strongly associated with behavioral beliefs (Bonner et al., 2023; Kikut et al., 2022).

In 2021, the U.S. Department of Health and Human Services (HHS) introduced the “We Can Do This” COVID-19 public education campaign (the Campaign) to increase COVID-19 vaccine confidence and uptake (HHS, 2022). One of the Campaign’s goals was to overcome COVID-19 vaccine hesitancy by strengthening “public confidence in the vaccines so people have the science-based information they need to make decisions about getting vaccinated and get protection from the worst outcomes of COVID-19” (HHS, 2022). A detailed description of the Campaign has been published elsewhere (Denison et al., 2023; Kranzler et al., 2023; Williams et al., 2023).

Theories of Behavior Change

Health behavior change theories provide valuable frameworks for identifying the determinants of health-promoting behaviors, including the uptake of vaccines (Piltch-Loeb & DiClemente, 2020). The Campaign was informed by empirically supported theories of behavior change, including the theory of planned behavior (TPB), the theory of reasoned action (TRA), and the health belief model (HBM), which propose that beliefs, among other factors, are key determinants of health behavior. The TRA posits that individuals’ behavioral intentions, which precede behavior, are shaped by their behavioral, normative, and self-efficacy beliefs (Ajzen & Fishbein, 1977); the TPB builds upon the TRA by incorporating perceived behavioral control as an additional determinant of intention and behavior (Ajzen, 1991). Similarly, the HBM suggests that beliefs regarding susceptibility to health conditions and the benefits of preventive actions predict behavioral performance (Rosenstock, 1974). Empirical support suggests the utility of these theories in guiding interventions that aim to increase COVID-19 vaccination uptake and intentions (Coe et al., 2022; Guidry et al., 2021; Limbu et al., 2022; Mercadante & Law, 2021; Tostrud et al., 2022).

These theories suggest that increased exposure to Campaign messages can positively influence COVID-19 vaccination by increasing individuals’ positive beliefs and attitudes (e.g., the belief that vaccines are effective) and normative beliefs (e.g., the belief that other people who are important to them will get vaccinated), and by reducing negative beliefs and attitudes (e.g., vaccination concerns). For example, messaging that communicates COVID-19 risks and the effectiveness of vaccines in preventing illness can increase risk perceptions and/or pro-vaccination beliefs, thereby motivating individuals to get vaccinated. Similarly, messaging that addresses COVID-19 vaccination concerns can allay fears that contribute to vaccine hesitancy among campaign audiences. With a goal of increasing vaccine uptake, the Campaign developed messages targeting both positive and negative beliefs about COVID-19 vaccination in line with this theoretical framework, including COVID-19 vaccine concerns and perceived risks, the importance and benefits of COVID-19 vaccination, and COVID-19 general vaccine effectiveness and safety to influence COVID-19 vaccination beliefs and corresponding behavior.

Broadly, changes in these classes of beliefs are expected to influence cognitions such as vaccination attitudes and subjective norms (e.g., perception of whether other people want an individual to perform a behavior), resulting in downstream effects on COVID-19 vaccination intentions and uptake. However, meaningful changes in campaign-targeted beliefs and behaviors are contingent upon adequate levels of campaign exposure, as insufficient exposure may fail to engage and persuade audiences (Hornik, 2002). Accordingly, we expected the Campaign to influence campaign-targeted beliefs and, ultimately, behaviors, under the condition that audiences were adequately exposed to Campaign messaging. This line of thought is consistent with evaluations of previous health campaigns, which have demonstrated dose-response relationships between campaign exposure and targeted outcomes, such that the magnitude of the change in campaign outcomes corresponded with the amount of campaign exposure (Davis et al., 2015; Farrelly et al., 2005; Huhman et al., 2007). Evidence of null effects for campaigns that do not garner sufficient exposure (Hornik, 2002) underscores the critical role of exposure in prompting campaign effects. Other campaign evaluations that have been guided by these or similar theories of behavior change have demonstrated associations between campaign exposure—measured exogenously (e.g., campaign dose) or endogenously (e.g., self-reported recall)—and campaign-targeted beliefs (Duke et al., 2018; Kranzler & Hornik, 2019; Kranzler et al., 2017; MacMonegle et al., 2022; Vallone et al., 2018), suggesting that Campaign exposure may have prompted similar effects on beliefs about vaccination.

Current Study

Prior evaluations of the Campaign have shown evidence that probable and recalled exposure to the Campaign is positively associated with increased COVID-19 vaccine confidence and uptake (Denison et al., 2023; Kranzler et al., 2023; Luchman et al., 2023; Williams et al., 2023). However, the relationship between exposure to the Campaign and changes in vaccination beliefs—a potential pathway through which the Campaign influenced behavior—has not been fully examined. This study examined the relationship between the Campaign and changes in individual-level vaccination beliefs among U.S. adults ages 18 and older, with a focus on the digital component of the Campaign (Campaign digital media). To validate these findings, this study further examined the relationship between Campaign digital media dose and vaccination beliefs that received comparatively less emphasis in Campaign advertisements during the study period.

Materials and Methods

Data Source and Study Sample

This study used panel data consisting of the first four waves (waves 1–4) of a longitudinal survey and media market-level Campaign digital impressions data. The longitudinal survey was administered every 4 months from January 2021–June 2023 to a nationally representative sample of U.S. adults ages 18 years and older and covered topics such as adherence to COVID-19 preventive behaviors, including COVID-19 vaccination; beliefs about COVID-19, vaccination, and COVID-19 vaccination; views on science, research, and medical professionals; and sociodemographic characteristics. Survey participants were recruited through NORC at the University of Chicago’s AmeriSpeak research panel (NORC, 2022). A comprehensive timeline of the survey waves and other important dates is provided in the Supplement. The study sample included respondents who completed all first four waves of the survey.

Campaign digital dose data were composed of aggregate Campaign impressions on digital platforms (e.g., site direct, programmatic, and social media advertisements; see the Supplement for more details) within a designated market area (DMA), or media market, from the launch of the Campaign (April 2021) to the end of Wave 4 fielding (March 2022). Additionally, by using survey items, we geo-referenced respondents into specific DMAs, thereby calculating estimated Campaign digital dose for each individual based on their DMA of residence for each survey wave. Therefore, our panel used a two-level hierarchical data structure with respondent-level survey data nested within DMAs. Further details regarding the study sample can be found in the Supplement. This study obtained approval from the Biomedical Research Alliance of New York’s (BRANY) institutional review board (IRB), an external IRB service accredited by the Association for the Accreditation of Human Research Protection Programs.

Dependent Variables

The dependent variables consisted of five belief scales reflecting the following categories of vaccination beliefs: (1) COVID-19 vaccine concerns and perceived risks, (2) importance of COVID-19 vaccines, (3) perceived benefits of COVID-19 vaccination, (4) normative beliefs about COVID-19 vaccination, and (5) perceptions about general vaccine safety and effectiveness. Hereafter, the belief scale names have been abridged to Concerns, Importance, Benefits, Norms, and General Effectiveness, respectively. These vaccination belief scales were developed using 22 individual belief items assessed in the survey, with a different scale per respondent for each wave. For each belief item, respondents reported the extent of their agreement or disagreement with the item on a 5-point Likert scale (1 = Strongly disagree, 5 = Strongly agree). We used factor analysis to determine which belief items had sufficiently high factor loadings to warrant inclusion in each belief scale, and computed Cronbach’s alpha for each belief scale to assess scale reliability. Further details regarding the descriptive statistics, computation, definitions, itemized vaccination beliefs, and psychometric results are documented in the Supplement.

Independent Variable

Campaign digital impressions reflect the cumulative reach of digital advertisements within specific DMAs, aggregated into respondent-waves as follows. Prior to aggregation, we applied the following specifications to construct cumulative Campaign digital impressions within each DMA. First, we calculated cumulative reach by using a 3-week decay function with an exponential half-life to account for both the impacts of initial Campaign media exposure and diminishing returns over time. Previous studies have indicated that the cumulative impact of advertising is most pronounced immediately following exposure, although individuals may recall campaign messages at a decaying rate for several weeks (Chong & Druckman, 2010; Gerber et al., 2011). Zhang et al. (2018) also found that compounded messaging can elicit responses despite diminishing message retention.

Next, we employed a 2-week lag to evaluate whether the Campaign provided sufficient information to influence changes in vaccination beliefs in the weeks preceding each respondent-wave (see Sherman et al., 2020). Lagging Campaign digital impressions ensured that changes in vaccination beliefs were not attributed to Campaign doses received after each survey completion date. Finally, we aggregated the weekly-level DMA data so that each respondent was assigned a cumulative Campaign dose for each of the four survey waves analyzed, respective of cumulative Campaign digital impressions in their DMA of residence. Additional information regarding variable construction and sensitivity analyses is available in the Supplement.

Covariates

Additional respondent- and DMA-level characteristics were included in the models to account for individual-level and exogenous effects that may confound the associations between cumulative Campaign digital dose and vaccination beliefs. We controlled for respondent-level sociodemographic factors, including several ordinal (age, education, income, rurality), dichotomous (gender, essential worker status, preexisting health condition), and categorical (race/ethnicity and political ideology) variables. We also controlled for initial COVID-19 vaccine dose administered within each respondent’s DMA as a percentage of DMA population per 100,000, using publicly available CDC data (CDC, 2023). Table 1 presents weighted descriptive statistics of the analysis sample (n = 4,109). Details about these covariates and their coding are provided in the Supplement.

Table 1. Descriptive statistics of the study sample (n = 4,109) for survey waves 1–4 (January 2021-March 2022) among U.S. Adults

Individual Respondent-Level Variables		Weighted (%)	Unweighted (%)
Income	Less than $50,000	45.7%	44.0%
	$50,000–$74,999	19.9%	20.7%
	$75,000–$100,000	12.9%	13.3%
	Greater than $100,000	22.2%	22.1%
Gender	Male	48.5%	48.7%
	Female	51.8%	51.3%
Age	Age 18–24	11.3%	5.7%
	Age 25–44	35.1%	43.6%
	Age 45–64	32.7%	29.5%
	Age 65 and Older	21.6%	21.2%
Education	No College	37.5%	21.1%
	Some College	27.9%	44.0%
	College Graduate or Beyond	35.1%	34.9%
Essential Worker Status	Essential Worker	32.1%	34.9%
	Non-Essential Worker	68.1%	65.2%
Political Ideology	Liberal	27.9%	30.1%
	Moderate	37.9%	36.4%
	Conservative	34.7%	33.5%
Health Condition	Pre-Existing Health Condition	63.8%	64.2%
	No Pre-Existing Health Condition	36.5%	35.8%
Rurality	Urban	38.8%	42.3%
	Suburban	43.6%	41.3%
	Rural	18.1%	16.4%
Race/Ethnicity	Non-Hispanic White	63.0%	61.8%
	Non-Hispanic, Another Race	8.8%	7.4%
	Non-Hispanic Black	12.4%	10.6%
	Hispanic/Latino	16.6%	20.2%

Number of respondents is 4,109. Wave 1 respondents were used. Percentages may not sum to 100% due to rounding.

Statistical Analysis

We estimated five separate mixed-effects linear regression models for which cumulative Campaign digital impressions were used to predict each of the five vaccination belief scales. Predicted probabilities were calculated using predictive margins tests at both mean-level (µ = 212,587) and 1 million digital impressions (within 2 standard deviations of µ). Due to the balanced nature of our panel data, each respondent’s vaccination beliefs were assessed across four time periods, corresponding with their completion of each survey wave. Consequently, each time period also corresponded with a specific Campaign digital dose: zero impressions for all respondents prior to Campaign initiation (Wave 1), and the cumulative Campaign digital impressions assigned to each respondent, given their DMA of residence and date of survey completion, thereafter (waves 2–4). This approach provided a comprehensive examination of the relationship between cumulative Campaign digital impressions and vaccination belief scales for all respondents across each survey wave.

The models were constructed using a multilevel structure, with respondent-wave observations nested within both respondents and DMAs and the respondent-survey wave as the unit of analysis. Survey wave fixed effects were included in the models to capture any time-specific variations in the vaccination belief scales. To focus on Campaign impact, respondents were excluded from the sample once they received their initial COVID-19 vaccine dose, ensuring that the analysis primarily captured the relationship between the Campaign and respondent vaccination beliefs prior to individuals receiving their initial vaccine dose. Respondents with missing vaccination belief data were excluded from the analysis. The Supplement provides additional information on the attrition of the sample size due to missing data and how it was accounted for in the analysis. Lastly, we conducted a series of robustness checks to assess whether regression results differed for models using alternative half-life measures, lags, nested structures, and for models that excluded COVID-19 vaccinations administered by DMA. These models are described in the Supplement.

Results

Main Regression Models

The results for each of the mixed-effects models assessing the relationship between Campaign paid media dose and vaccination belief scales are provided in Table 2. Models 1 to 5 reflect the following vaccination belief scales: (1) Concerns, (2) Importance, (3) Benefits, (4) Norms, and (5) General Effectiveness, respectively. Model fit statistics indicate that the models were both useful and a reasonable fit to the data on which they were built. Overall, the results indicate a statistically significant association between cumulative Campaign digital impressions and vaccination beliefs for three of the five vaccination belief scales. Increases in cumulative Campaign digital impressions were associated with a decrease in the Concerns belief scale (β = −1.01e-07, p = .003, 95% CI [−1.69e-07, −3.33e-08]), and increases in both the Benefits (β = 1.14e-07, p = .003, 95% CI [3.78e-08, 1.90e-07]) and General Effectiveness (β = 5.70e-08, p = .03, 95% CI [5.38e-09, 1.09e-07]) belief scales. Results demonstrate no significant association between cumulative Campaign digital impressions and the Importance belief scale, nor between cumulative Campaign digital impressions and the Norms belief scale.

Table 2. Cumulative campaign digital impressions and vaccination belief scales

	(1) COVID-19 Vaccine Concerns and Perceived Risks	(2) Importance of COVID-19 Vaccination	(3) Perceived Benefits of COVID-19 Vaccination	(4) Normative Beliefs about COVID-19 Vaccination	(5) Perceptions about General Vaccine Safety and Effectiveness
	β (SE)	β (SE)	β (SE)	β (SE)	β (SE)
Main Independent Variable
Cumulative Campaign Digital Impressions	-1.01e-07** (3.46e-08)	5.60e-08 (4.11e-08)	1.14e-07** (3.87e-08)	6.69e-08 (3.55e-08)	5.70e-08 *(2.64e-08)
Covariates
Income	−0.081*** (0.012)	0.063*** (0.014)	0.061*** (0.012)	0.059*** (0.012)	0.046*** (0.009)
Female	0.204*** (0.267)	−0.184*** (0.032)	−0.134*** (0.028)	−0.108*** (0.027)	−0.064** (0.022)
Age	−0.245*** (0.016)	0.285*** (0.019)	0.201*** (0.016)	0.207*** (0.016)	0.143*** (0.013)
Education	−0.206*** (0.019)	0.163*** (0.023)	0.170*** (0.020)	0.151*** (0.019)	0.147*** (0.016)
Essential Worker	0.054* (0.024)	−0.054 (0.029)	−0.028 (0.026)	−0.013 (0.024)	−0.006 (0.019)
Political Ideology	0.468*** (0.017)	−0.561*** (0.020)	−0.449*** (0.018)	−0.431*** (0.017)	−0.389*** (0.014)
Pre-Existing Health Issues	−0.045* (0.019)	0.064** (0.023)	0.030 (0.021)	0.059** (0.020)	0.056*** (0.015)
Rurality	0.073*** (0.019)	−0.118*** (0.024)	−0.090*** (0.020)	−0.102*** (0.019)	−0.052** (0.016)
Non-Hispanic Black	0.335*** (0.044)	−0.149** (0.054)	−0.122** (0.046)	−0.200*** (0.044)	−0.263*** (0.036)
Hispanic	0.187*** (0.036)	0.089* (0.044)	0.058 (0.038)	0.067 (0.036)	−0.067* (0.029)
Non-Hispanic, Another Race	0.118* (0.052)	0.128* (0.063)	0.058 (0.054)	0.084 (0.052)	−0.051 (0.042)
At Least One COVID-19 Dose Population %	−0.001* (0.001)	0.001 (0.001)	0.001 (0.001)	0.000 (0.001)	0.000 (0.000)
Intercept	2.371*** (0.095)	4.181*** (0.115)	3.947*** (0.099)	3.978*** (0.095)	3.919*** (0.044)
DMA Variance	0.001 (.002)	0.006 (.005)	0.003 (.004)	0.000 (.002)	0.000 (.002)
Respondent Variance	0.568 (0.016)	0.827 (0.023)	0.588 (0.017)	0.558 (0.016)	0.386 (0.010)
Number of Waves	4	4	4	4	4
Number of DMAs	207	207	207	207	207
Number of IDs	4,132	4,132	4,123	4,125	4,122
N	8,978	8,955	8,859	8,907	8,836

*p < .05, ** p < .01, ***p < .001; two-tailed tests. The values for each variable are coefficients and the standard errors are in parenthesis. DMA stands for designated market area. The values for DMA variance and ID variance are random effects parameters and not coefficients.

The predictive margins tests in Figure 1 confirmed these relationships. When cumulative Campaign digital impressions were held at their mean, there was a 0.8% decrease in the Concerns belief scale. When cumulative Campaign digital impressions were increased from their mean to 1 million cumulative impressions, there was a 3.9% decrease in the Concerns belief scale. Further, there was a 0.7% increase in the Benefits belief scale when cumulative Campaign digital impressions were held at their mean. When cumulative digital impressions were increased from their mean to 1 million cumulative impressions, there was a 3.2% increase in the Benefits belief scale. Finally, there was a 0.3% increase in the General Effectiveness scale when cumulative Campaign digital impressions were held at their mean. When this quantity was increased to 1 million digital impressions, the General Effectiveness belief scale increased by 5.7%. Figure 1 also validates that there was no significant association between cumulative Campaign digital impressions and the Importance belief scale, nor between cumulative Campaign digital impressions and the Norms belief scale.

Figure 1. Predictive Margins of the Vaccination Belief Scales.

95% confidence intervals and predicted probabilities at means and 1 million cumulative Campaign digital impressions (within 2 standard deviations).

The results for respondent-level characteristics and vaccination-related covariates illuminated several statistically significant effects. Table 1 describes units for each covariate, which provides important context for unit changes. Across all models, the following covariates were significantly associated with the vaccination belief scales: income, gender, age, education, political ideology, preexisting health condition, rurality, and race/ethnicity. Higher income, age, and education, and having a preexisting health condition were positively associated with the Importance, Benefits, Norms, and General Effectiveness belief scales, and were negatively associated with the Concerns belief scale. In contrast, being female, non-Hispanic Black (compared to non-Hispanic White), more ideologically conservative, and living in a more rural locale were negatively associated with the Importance, Benefits, Norms, and General Effectiveness belief scales, and were positively associated with the Concerns belief scale. In contrast, Hispanic/Latino respondents (compared to non-Hispanic White respondents) showed a more positive association with the Concerns and Importance belief scales and a negative association with the General Effectiveness belief scale. Table 2 characterizes each covariate in more detail across models, and the Supplement includes more detailed descriptions of these differences.

A series of robustness checks to validate the reliability of the presented findings, including a combined mixed model, sensitivity analysis, alternative nests and clusters, and alternative measures for the independent variables can be found in the Supplement. Collectively, results from these additional tests were not substantively different from the main models, providing further validation that these results were robust to alternative model specifications.

Robustness Checks

To address concerns about unmeasured confounders, we examined relationships between cumulative Campaign digital dose and beliefs that were less focal to the Campaign during the study period. This approach, also referred to as discriminant validation, aimed to determine whether factors other than the Campaign influenced the relationship between cumulative Campaign digital dose and the main vaccination belief scales (Kranzler & Hornik, 2019). For example, if factors such as non-Campaign COVID-19 ads, or other media coverage relevant to COVID-19, covaried with cumulative Campaign digital dose, then they could influence a range of beliefs, including those explicitly targeted by the Campaign (e.g., the belief that COVID-19 vaccines are safe and effective) and those that were not central to Campaign ads (e.g., the belief that one is burned out from hearing that COVID-19 is a serious problem). If this were the case, then we would expect similarly significant relationships between (1) cumulative Campaign digital dose and beliefs that were focal to Campaign ads (focal beliefs), and (2) cumulative Campaign digital dose and beliefs that were less focal to Campaign ads (non-focal beliefs). The absence of significant relationships between cumulative Campaign digital dose and non-focal beliefs would suggest that our main Campaign-focal-belief results were less subject to the influence of unmeasured confounders.

To assess the relationship between Campaign digital media dose and non-focal beliefs, we conducted complementary regression analyses using scales reflecting the following sets of beliefs: (1) trust in science and experts, (2) COVID-19 information burnout, and (3) COVID-19 indifference (hereinafter referred to as Trust in Science, Information Burnout, and Indifference, respectively). As reported in the Methods section, we used factor analysis to determine which belief items had sufficiently high factor loadings to warrant inclusion in each belief scale, and computed Cronbach’s alpha for each belief scale to assess scale reliability. Please refer to the Supplement for more information about these belief scales, including definitions, itemized questions, and psychometric results. All models controlled for the same respondent- and DMA-level covariates as in previous models and included wave fixed effects.

Table 3 presents the findings from three mixed-effects regression models examining the associations between cumulative Campaign digital impressions and non-focal beliefs. Results across all three models indicate that cumulative Campaign digital impressions were not significantly associated with changes in Trust in Science, Information Burnout, and Indifference belief scales. The significance levels for covariates varied across all three models but showed similar patterns to those described for the main focal-belief regression models.

Table 3. Main results of vaccination belief scales that were less focal to the campaign during the study period (January 2021–March 2022)

	Trust In Science	COVID-19 Information Burnout	(3) COVID-19 Indifference
	β (SE)	β (SE)	β (SE)
Main Independent Variable
Cumulative Campaign Digital Impressions	−8.41e-09 (1.33e-08)	1.51e-09 (2.99e-08)	5.52e-09 (3.45e-08)
Covariates
Income	−0.004 (0.003)	−0.002 (0.008)	−0.012 (0.012)
Female	−0.006 (0.007)	−0.069*** (0.017)	−0.089** (0.027)
Age	−0.010* (0.004)	−0.127*** (0.011)	−0.228*** (0.016)
Education	−0.016** (0.005)	0.004 (0.012)	−0.106*** (0.020)
Essential Worker	0.012 (0.007)	0.081*** (0.017)	0.092*** (0.024)
Political Ideology	0.054*** (0.004)	0.238*** (0.011)	0.601*** (0.017)
Pre-Existing Health Issues	0.005 (0.006)	−0.009 (0.015)	−0.072*** (0.019)
Rurality	−0.004 (0.005)	0.014 (0.013)	0.069** (0.021)
Non-Hispanic Black	0.039*** (0.011)	−0.219*** (0.029)	−0.257*** (0.046)
Hispanic	0.063*** (0.009)	−0.052* (0.024)	−0.017 (0.038)
Non-Hispanic, Another Race	0.054*** (0.013)	−0.054 (0.035)	−0.088 (0.054)
At Least One COVID-19 Dose Population %	0.000 (0.000)	0.000 (0.001)	0.000 (0.000)
Intercept	3.035*** (.024)	1.714*** (.063)	1.977*** (.098)
DMA Variance	0.000 (0.000)	0.000 (0.000)	0.007 (0.004)
ID Variance	0.023 (0.001)	0.204 (0.007)	0.609 (0.017)
Number of Waves	4	4	4
Number of DMAs	206	207	207
Number of IDs	4,112	4,122	4,123
N	8,783	8,919	8,878
AIC	60492.09	48623.68	62627.37
BIC	60634.15	48765.73	62769.43

* p < .05, ** p < .01, *** p < .001; two-tailed tests. The values for each variable are coefficients and standard errors are in parenthesis. The values for DMA variance and ID variance are random effects parameters.

Discussion

This study examined the relationship between the digital component of the We Can Do This COVID-19 public education campaign and changes in individual-level vaccination beliefs among U.S. adults. Findings indicate that cumulative Campaign digital media dose was significantly associated with changes in vaccination beliefs, such that greater Campaign dose was associated with greater endorsement of positive vaccination beliefs (e.g., perceived benefits of COVID-19 vaccination and perceptions of general vaccine safety) and lesser endorsement of negative vaccination beliefs (e.g., COVID-19 vaccination concerns). These findings are consistent with those from previous evaluations of the Campaign (Denison et al., 2023; Kranzler et al., 2024; Luchman et al., 2023; Williams et al., 2023), which demonstrated significant associations between Campaign exposure and Campaign-targeted outcomes, and with research showing that COVID-19 campaigns are associated with positive changes in vaccination beliefs (Athey et al., 2023; Coffie et al., 2022; Hong, 2023). Moreover, findings align with the theories of behavior change that guided the Campaign (Ajzen, 1991; Ajzen & Fishbein, 1977; Rosenstock, 1974), which imply that changes in vaccination-relevant beliefs due to Campaign exposure result in downstream effects on COVID-19 vaccination intentions and uptake.

Taken together, the results from this and other evaluations suggest that Campaign messaging may have influenced COVID-19 vaccination intentions and uptake through changes in Campaign-targeted beliefs. Among individuals who were hesitant to get vaccinated against COVID-19, Campaign exposure may have reduced their endorsement of vaccination concerns, thereby increasing the likelihood they would get vaccinated. Similarly, Campaign exposure may have increased perceptions of the benefits of COVID-19 vaccination, general vaccine safety, and pro-vaccination attitudes, leading to greater confidence in COVID-19 vaccination and subsequent vaccine uptake. Recent research has shown that endorsement of these belief classes is associated with an increased likelihood of COVID-19 vaccine uptake (Kranzler et al., 2024), highlighting a potential pathway through which the Campaign influenced vaccination—from exposure to belief change to behavior change—that aligns with the Campaign’s foundational theories of behavior change.

Results from this study indicate non-significant relationships between Campaign dose and the Importance beliefs scale and the Norms beliefs scale. These findings contrast with Athey et al. (2023), whose meta-analysis found that social media advertising campaigns had significant effects on beliefs about the importance of vaccination and normative beliefs about COVID-19 vaccination. The absence of significant findings for these belief scales might indicate that the Campaign did not influence these classes of beliefs, or that respondents were not sufficiently exposed to Campaign messaging that targeted these classes of beliefs, as sufficient campaign exposure is crucial to eliciting campaign effects (Hornik, 2002).

Furthermore, the lack of statistical significance for these belief scales may be attributed to a restriction of variance given the selection on vaccination. Specifically, the dataset in this study was structured to remove respondent-wave observations once respondents reported receiving an initial COVID-19 vaccine, potentially skewing the sample toward more vaccine-hesitant individuals. Notably, recent research demonstrates that beliefs about the importance of COVID-19 vaccines and normative beliefs about COVID-19 vaccination are the strongest predictors of COVID-19 vaccine uptake (Kranzler et al., 2024). However, their dataset was constructed to omit observations from respondents who, after reporting high belief scores for these belief scales, subsequently got vaccinated. Such exclusion criteria were not applied in this study, and thus the nonsignificant effects of those beliefs in this study may indicate that our sample was insufficiently powered to detect effects for these specific belief scales given restricted belief scale variance.

Results indicate that vaccination beliefs varied by sociodemographic factors. Non-Hispanic Black respondents reported lower levels of the perceived benefits and greater levels of COVID-19 vaccine concerns and risks compared to non-Hispanic White respondents (Table 2). Such differences align with extant literature showing that Black populations, indigenous populations, and people of color in the United States are less likely to trust health authorities and institutions (Majee et al., 2023; Martinez Leal et al., 2023; Nguyen et al., 2021; Strully et al., 2021); this sentiment may be rooted in their historical mistrust of health care due to discriminatory experiences such as the Tuskegee Syphilis Study (Carpio, 2004; Scharff et al., 2010) and persisting discrimination in health care settings (Nong et al., 2020; Thorburn & Lindly, 2022). Essential workers, rural residents, and politically conservative respondents also reported more negative vaccination beliefs due to vaccine safety and effectiveness concerns, echoing similar findings from relevant COVID-19 research (Callaghan et al., 2020; Soorapanth et al., 2023; Williams et al., 2023). Together, these findings highlight the continued need for culturally appropriate and targeted messaging in future campaign efforts.

Robustness checks demonstrated that Campaign digital dose was not significantly associated with trust in science and experts, COVID-19 information burnout, and COVID-19 indifference belief scales (Table 3). The absence of significant relationships between Campaign dose and these non-focal beliefs offers discriminant validation of our main findings—that Campaign dose was associated with Campaign-targeted beliefs. Furthermore, results from the non-focal belief analysis reduce the likelihood that observed associations between Campaign dose and Campaign-targeted beliefs could be attributed to third-variable influence (e.g., exposure to a campaign that increased COVID-19-relevant beliefs across the board).

Study findings contribute to empirical research on campaign effects, offering practical insights for understanding the role of campaign exposure in shaping individuals’ beliefs about COVID-19 vaccination. In comparison to past literature which has predominantly relied on cross-sectional studies, the longitudinal design adapted for this study enables a deeper exploration into how Campaign exposure may lead to behavioral changes over time. Findings help clarify the pathways through which campaigns can influence individuals’ beliefs and behaviors, thereby enhancing our understanding of implications for future vaccination campaigns. Additionally, results suggest that vaccination campaigns that target a range of beliefs relevant to vaccination can complement other public health interventions to increase vaccination uptake.

Limitations

We acknowledge several limitations that should be considered when interpreting study findings. Potential inconsistencies in respondent self-reports across study waves could introduce measurement error and affect the reliability of the results. Self-reported data may be subject to errors due to recall bias or social desirability bias. This study relies on Campaign digital impressions, which represent opportunities for exposure rather than achieved exposure to the Campaign. It is possible that respondents did not see or pay attention to these Campaign ads and results may not reflect the effect of achieved exposure to the Campaign on vaccination beliefs.

The observed effects are specific to the study period and populations analyzed, and may not apply to other periods and/or populations. However, other evaluations have demonstrated Campaign effects over different periods using different measures of Campaign exposure (Denison et al., 2023; Kranzler et al., 2023; Williams et al., 2023), suggesting that Campaign effects are not limited to the period under study. Lastly, the analysis examined data from four survey waves with an average of 4 months between the initial fielding of each survey wave. Although past health campaign evaluations suggest that a 4-month lag between survey waves is adequate for detecting changes in beliefs (Emery et al., 2005; Terry McElrath et al., 2007), it is possible that a longer period between survey waves is required to observe substantive belief change. These four waves of data provide preliminary insights, but more extended studies may be needed to capture and assess the full dynamics of belief changes. Future research could address these limitations by employing more robust data collection methods, extending the observation period, and exploring the generalizability of the results to different populations and time frames.

Conclusion

Despite evidence of the safety and effectiveness of COVID-19 vaccines, many adults have been hesitant to get vaccinated, prompting efforts to increase vaccination through the provision of science-based information in public education campaign messaging. This study offers evidence that exposure to such information through the digital component of a COVID-19 public education campaign corresponds with positive changes in the perceived benefits of COVID-19 vaccination, perceptions about general vaccine safety and effectiveness, and negative changes in COVID-19 vaccine concerns and perceived risks. These classes of beliefs have been shown to associate with vaccination (Harada & Watanabe, 2021; Kikut et al., 2022; Kranzler et al., 2024; Latkin et al., 2022), lending support to a growing body of literature demonstrating public education campaign influence on COVID-19 vaccine confidence and uptake (Kranzler et al., 2023; Williams et al., 2023). Furthermore, findings from this study and others highlight the utility of public education campaigns as one of several strategies for encouraging health-promoting behavior in the face of a public health crisis.

Disclaimer

This publication represents the views of the authors and does not represent U.S. Department of Health and Human Services (HHS) position or policy.

This study was approved by the Biomedical Research Alliance of New York (protocol 20-077-821).

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/10810730.2024.2373159.

Additional information

Funding

This work was supported by HHS using National Institutes of Health (NIH) contract#75N98019D00007, task orders 75N98020F00001, 75N98021F00001, and 75N98022F00001.