Identifying beliefs driving COVID-19 vaccination: Lessons for effective messaging

ABSTRACT

Increasing vaccination acceptance has been essential during the COVID-19 pandemic and in preparation for future public health emergencies. This study aimed to identify messaging strategies to encourage vaccine uptake by measuring the drivers of COVID-19 vaccination among the general public. A survey to assess COVID-19 vaccination acceptance and hesitancy was advertised on Facebook in February–April 2022. The survey included items asking about COVID-19 vaccination status and participant demographics, and three scales assessing medical mistrust, perceived COVID-19 risk, and COVID-19 vaccine confidence (adapted from the Oxford COVID-19 vaccine confidence and complacency scale). The main outcome was vaccination, predicted by patient demographics and survey scale scores. Of 1,915 survey responses, 1,450 (75.7%) were included, with 1,048 (72.3%) respondents reporting they had been vaccinated. In a multivariable regression model, the COVID-19 vaccine confidence scale was the strongest predictor of vaccination, along with education level and perceived COVID-19 risk. Among the items on this scale, not all were equally important in predicting COVID-19 vaccination. The items that best predicted vaccination, at a given score on the COVID-19 vaccine confidence scale, included confidence that vaccine side effects are minimal, that the vaccine will work, that the vaccine will help the community, and that the vaccine provides freedom to move on with life. This study improved our understanding of perceptions most strongly associated with vaccine acceptance, allowing us to consider how to develop messages that may be particularly effective in encouraging vaccination among the general public for both the COVID-19 pandemic and future public health emergencies.

KEYWORDS:

• COVID-19 vaccine
• vaccine hesitancy
• public health
• survey
• communication

Introduction

COVID-19 vaccines have been a critical tool in the global response to the COVID-19 pandemic. Evidence strongly supports the ability of COVID-19 vaccines to prevent severe illness and death.¹ In one year following the first administered dose of a COVID-19 vaccine in December 2020, 14.4–19.8 million deaths are estimated to have been prevented due to vaccination.² The widespread presence of false and misleading information, however, complicates efforts to increase vaccine uptake.³ Therefore, understanding how to promote vaccination is critical in response to the evolving COVID-19 pandemic and in preparation for future public health emergencies caused by novel infectious diseases.

Many studies have uncovered individuals’ concerns that have prevented or delayed vaccination during the COVID-19 pandemic, including questions about the necessity, efficacy, and safety of COVID-19 vaccines.⁴ Social media-fueled misinformation has played a significant role in amplifying these concerns, prompting a need for more effective messaging strategies. As a result, numerous organizations have explored new messaging strategies to convince vaccine-hesitant individuals to get vaccinated.^5,6 Existing strategies for communicating to vaccination-hesitant populations include highlighting negative consequences of foregoing vaccination,⁷ preemptively warning individuals about vaccination misinformation (e.g., inoculation theory),^3,8 and utilizing vaccination endorsement from trusted leaders within a particular group (e.g., elite party cues).⁹ These strategies adapt messages to draw on communication theory; however, they do not consider the unique characteristics of a population’s values, concerns, and knowledge to identify messages that will be most effective.

Among the vast possibilities for messages to address individuals’ concerns, we need to understand what messages may be the most useful to potentially impact vaccination acceptance. The objective of this study was to understand beliefs driving acceptance of COVID-19 vaccines in order to identify messaging targets to promote COVID-19 vaccinations. We conducted a survey of the general public to evaluate the key perceptions associated with COVID-19 vaccination in this population. Identifying these perceptions through our survey allowed us to better understand what messaging strategies may be most impactful in encouraging vaccination among the general public.

Methods

Study design and population

We designed a survey to assess vaccination acceptance and hesitancy among the general adult population in the United States (US). Study data were collected and managed using REDCap (Research Electronic Data Capture), a secure, web-based platform for designing and managing online surveys and data.¹⁰ This survey was designed in REDCap with the public-survey-link feature enabled, which creates a singular survey URL that does not track internet provider addresses and automatically generates a random subject ID for each respondent upon submitting their responses. To maintain anonymity, no identifiers (name, address, phone number, e-mail, etc.) were collected in the survey, and respondents could voluntarily refuse to take the survey or not answer the limited demographic questions. An informed consent form was provided for participants to review the purpose of the study and details regarding the confidentiality of their answers. Documentation of informed consent was not required for this study, but participants were prompted to download and read the consent form prior to continuing to the survey. This study was approved by The Ohio State University’s Institutional Review Board (protocol 2022E0109), including approval of all advertising materials, consent, and survey questions before launching the advertisement.

Eligibility for the survey included asking the participant to self-disclose being 18+ years of age and currently living in the US. The survey would close if respondents answered “No” to either question. Two unique stock images were used for advertising the survey on Facebook. Targeted audience features for the ad included Facebook members who were 18+ years of age and located in the US. The Facebook ad was launched and managed by Recruitment and Retention Services in The Ohio State University’s Center for Clinical and Translational Science. The survey was advertised to random selections of thousands of users daily on Facebook (auto-generated through Facebook’s advertising platform), and ran from February 17, 2022–March 3, 2022, and from March 25, 2022–April 8, 2022. No incentives were offered to complete the survey. The advertisement reached 47,968 users (the number of unique Facebook users who saw the ad), with 2,959 clicking on the advertisement to read about the study opportunity.

Measures

The survey included items assessing COVID-19 vaccination status (having received at least one dose of a COVID-19 vaccine) and participant demographics as well as three scales including a medical mistrust scale,¹¹ a scale of perceived COVID-19 risk,¹² and an adapted COVID-19 vaccine confidence scale.¹³ The medical mistrust scale included seven items that assessed mistrust of healthcare organizations. Response options were on a 5-point scale from “strongly disagree” to “strongly agree.” A mean medical mistrust score was calculated as a mean of responses to all items, where a higher score indicated greater medical mistrust. The perceived COVID-19 risk scale included eight items that assessed the likelihood of getting COVID-19 and being negatively affected by a COVID-19 infection. Responses were reported on a 4-point scale from “almost no chance” to “very high chance.” A mean perceived risk score was calculated from the mean of responses to all items, where a higher score indicated higher perceived risk of negative effects from COVID-19. The adapted COVID-19 vaccine confidence scale included six items from the 14-item Oxford COVID-19 vaccine confidence and complacency scale (Table 1). This adapted scale included items that covered all four dimensions of the Oxford COVID-19 vaccine confidence and complacency scale: collective importance of a COVID-19 vaccine, beliefs that the vaccine will work, speed of vaccine development, and side effects. Previous work has compared this adapted COVID-19 vaccine confidence scale to other measures of vaccine hesitancy,¹⁴ and this adapted scale has also been utilized in the characterization of COVID-19 vaccine hesitancy among medical professionals.¹⁵ Items in the adapted vaccine confidence scale included response options on 5-point scales. A mean COVID-19 vaccine confidence score was calculated as a mean of all items in the adapted scale, where a higher score indicated higher confidence in a COVID-19 vaccine.

Table 1. Components of the three survey scales.

Survey Component	Survey Question	Response Options
Medical mistrust scale
1	You’d better be cautious when dealing with health care organizations	ο Strongly disagree ο Disagree ο Neither agree nor disagree ο Agree ο Strongly agree
2	Patients have sometimes been deceived or misled by health care organizations
3	When health care organizations make mistakes, they usually cover it up
4	Health care organizations have sometimes done harmful experiments on patients without their knowledge
5	Health care organizations don’t always keep your information totally private
6	Sometimes I wonder if health care organizations really know what they are doing
7	Mistakes are common in health care organizations
Perceived COVID-19 risk scale
1	You will get COVID-19	ο Almost no chance ο Small chance ο Moderate chance ο Very high chance
2	You will have to go to the hospital if you get COVID-19
3	You will experience symptoms of COVID-19 for over 5 weeks
4	You will experience symptoms of COVID-19 for over 12 weeks
5	You will develop chronic or long-term health effects from COVID-19
6	If you get COVID-19, you will give it to someone else
7	If you get COVID-19, you will die
8	Someone in your direct environment (family, friends, colleagues) will die from COVID-19
Adapted COVID-19 vaccine confidence scale
1 – side effects	Vaccinated: The side effects of a COVID-19 vaccine are Unvaccinated: The side effects of getting a COVID-19 vaccine will be	ο None ο Mild ο Moderate ο Significant ο Life-threatening
2 – guinea pig	Vaccinated: Taking a new COVID-19 vaccine makes me feel like a guinea pig Unvaccinated: Taking a new COVID-19 vaccine will make me feel like a guinea pig	ο Do not agree ο Agree a little ο Agree moderately ο Agree a lot ο Completely agree
3 – speed of development	Vaccinated and unvaccinated: The speed of developing and testing a COVID-19 vaccine means it is	ο Really unsafe ο Unsafe ο Does not affect how safe it is ο Safe ο Really safe
4 – efficacy	Vaccinated and unvaccinated: A COVID-19 vaccine is likely to	ο Definitely not work for me ο Probably not work for me ο May or may not work for me ο Probably work for me ο Definitely work for me
5 – community benefit	Vaccinated: Getting a COVID-19 vaccine is Unvaccinated: If I get a COVID-19 vaccine it will be	ο Really unhelpful for the community around me ο Unhelpful for the community around me ο Neither helpful nor unhelpful for the community around me ο Helpful for the community around me ο Really helpful for the community around me
6 – freedom	Vaccinated and unvaccinated: Taking a COVID-19 vaccine	ο Completely restricts my freedom to get on with life ο Restricts my freedom ο Has no effect on my freedom ο Gives me greater freedom ο Gives me complete freedom to get on with my life

Data analysis

We limited the data to US residents aged ≥ 18 and ≤ 85 who were not missing age or rural/urban/suburban status and who were not missing more than three of the following six demographic variables: gender, race, Hispanic status, educational level, flu vaccination, and political affiliation. Out of 1,915 survey responses, we had 1,450 responses (75.7%) meet these criteria. Within this subsample, there was less than 4% missing data on predictors that would potentially be used in our model of COVID-19 vaccination.

We fit univariable logistic regression models for all predictors. Age was used to determine that a natural cubic spline outperformed age quartiles. We then fit a multivariable logistic regression model with gender, age, race, educational level, rural/urban/suburban status, previous COVID-19 infection, perceived COVID-19 risk, medical mistrust, and COVID-19 vaccine confidence. Finally, we used backward selection to minimize the Bayes Information Criterion. At each stage, we checked goodness-of-fit using the Hosmer-Lemeshow test with 10 groups and the calibration belt. Variables were retained if they were highly statistically significant or needed to improve the model’s fit. This method led to a model with age, educational level, previous COVID-19 infection, perceived COVID-19 risk, and COVID-19 vaccine confidence. We then added orthogonal polynomials for perceived COVID-19 risk and COVID-19 vaccine confidence and continued backward selection. This addition led to a model without age or previous COVID-19 infection, with a linear term for perceived COVID-19 risk, and a quadratic term for COVID-19 vaccine confidence.

A model that used the individual components of the vaccine confidence scale did not fit the data well, so we used the linear residuals for each component (i.e., each component minus the mean of the components). After additional backward selection, the only residuals remaining in the model were components two (feeling like a guinea pig) and three (concern about the speed of vaccine development). Finally, we checked interaction terms, retaining an interaction term for perceived COVID-19 risk and the linear term for COVID-19 vaccine confidence. The final model had a Hosmer-Lemeshow goodness-of-fit p-value of .85 and an excellent calibration belt (p = .19).

Results

We received 1,915 survey responses. Of the total respondents, 1,450 (75.7%) met our inclusion criteria. The majority of the population was female, White, non-Hispanic, and lived in a suburban setting. Representation across age groups was nearly equal, with the majority having a bachelor’s, master’s, or doctoral degree. Most respondents reported a Democratic political affiliation. The majority believed they had not had COVID-19 and had received a flu vaccination in the previous flu season. Differences in demographics between vaccinated and non-vaccinated individuals are shown in Figure 1 and Supplementary Table S1.

Figure 1. Demographics and characteristics of study participants.

The results of univariable logistic regression are reported in Table 2. All variables were significantly associated with COVID-19 vaccination, except for ethnicity. Males were less likely to be vaccinated compared to females. Individuals who were 41–56 or 57–65 years of age were less likely to be vaccinated compared to individuals 40 years of age or younger. Individuals of all other races were less likely to be vaccinated than White individuals. There was no statistical difference in vaccination between Hispanic and non-Hispanic individuals. Individuals with education levels less than a master’s degree were less likely to be vaccinated when compared to those with a master’s degree or higher. Those living in rural settings were less likely to be vaccinated than those in urban settings. Individuals who previously had COVID-19 were less likely to be vaccinated compared to those who had not experienced a COVID-19 infection. Those who had received a flu vaccine in the previous flu season had much higher odds of receiving a COVID-19 vaccine. Among our survey scales, higher scores on the perceived COVID-19 risk and COVID-19 vaccine confidence scales increased the likelihood of vaccination. In contrast, higher scores on the medical mistrust scale were associated with a decreased likelihood of vaccination.

Table 2. Estimated odds ratios for COVID-19 vaccination from univariable regression.

Predictor	Odds ratio	p-value	95% CI
Gender
Female	Ref	Ref	Ref
Male	0.225	<.001	(0.176, 0.289)
Other/prefer not to answer	0.796	.598	(0.341, 1.860)
Age
<41 years	Ref	Ref	Ref
41–56 years	0.659	.014	(0.473. 0.920)
57–65 years	0.666	.016	(0.478, 0.928)
66–85 years	0.895	.522	(0.637, 1.257)
Race
White	Ref	Ref	Ref
All others/prefer not to answer	0.274	<.001	(0.188, 0.399)
Ethnicity
Not Hispanic or Latino	Ref	Ref	Ref
Hispanic or Latino	0.981	.955	(0.501, 1.921)
Prefer not to answer	0.273	<.001	(0.161, 0.461)
Education level
HS/GED	0.113	<.001	(0.068, 0.189)
Some college	0.201	<.001	(0.143, 0.284)
Associate	0.285	<.001	(0.187, 0.433)
Bachelor’s	0.543	<.001	(0.395, 0.746)
Master’s/Doctorate	Ref	Ref	Ref
Urbanicity
Urban	Ref	Ref	Ref
Suburban	0.935	.716	(0.651, 1.343)
Rural	0.440	<.001	(0.305, 0.636)
Political affiliation
Democrat	Ref	Ref	Ref
Republican	0.031	<.001	(0.019, 0.050)
Other/none	0.049	<.001	(0.032, 0.076)
Previous COVID-19 infection
No (or I don’t think so)	Ref	Ref	Ref
Yes (or I think so)	0.227	<.001	(0.178, 0.290)
Flu vaccination status
No	Ref	Ref	Ref
Yes	69.70	<.001	(47.04, 103.29)
Don’t know	8.555	<.001	(4.079, 17.94)
Scales
Perceived COVID-19 risk	6.048	<.001	(4.545, 8.048)
COVID-19 vaccine confidence	27.32	<.001	(19.18, 38.91)
Medical mistrust	0.109	<.001	(0.086, 0.138)

The odds ratio estimates, p-values, and 95% log-transformed Wald confidence intervals of the multivariable analysis are shown in Table 3. The strongest predictor of COVID-19 vaccination by far was COVID-19 vaccine confidence, with an odds ratio of 167.4 (95% CI: 65.37, 428.8) per one-unit increase at the mean perceived COVID-19 risk and mean COVID-19 vaccine confidence. The odds ratio significantly greater than one on the quadratic term for COVID-19 vaccine confidence indicates that the odds ratio associated with a one-unit increase in COVID-19 vaccine confidence increases as we start at higher confidence levels. The odds ratio significantly below one on the interaction between perceived COVID-19 risk and COVID-19 vaccine confidence indicates that vaccine confidence plays a more important role in the decision to get vaccinated among individuals with lower perceived risk. Equivalently, those with higher perceived risk have the same odds of vaccination at a lower level of vaccine confidence.

Table 3. Estimated odds ratios for COVID-19 vaccination from multivariable regression.

Predictor	Odds ratio	p-value	95% CI
Educational level
High school/GED	0.190	.014	(0.051, 0.713)
Some college	0.452	.054	(0.201, 1.015)
Associate	0.979	.963	(0.395, 2.424)
Bachelor’s	0.659	.263	(0.318, 1.367)
Master’s/Doctoral	Ref	Ref	Ref
Perceived COVID-19 risk (centered)	1.019	.939	(0.633, 1.640)
COVID-19 vaccine confidence
Linear (centered)	167.4	<.001	(65.37, 428.8)
Quadratic (orthogonal)	6.588	<.001	(3.918, 11.08)
Perceived risk * vaccine confidence (linear)	0.534	.007	(0.339, 0.843)
Vaccine confidence residual component 2	0.439	<.001	(0.294, 0.655)
Vaccine confidence residual component 3	0.510	.017	(0.293, 0.886)

Among the six components of the COVID-19 vaccine confidence scale, components two and three had estimated odds ratios significantly less than one. At a given COVID-19 vaccine confidence score, individuals who were particularly confident in not feeling like a guinea pig (component 2) or particularly confident that vaccine safety is not compromised by rapid development (component 3) were less likely to be vaccinated than we would expect. This implies that the primary drivers of vaccination within the COVID-19 vaccine confidence scale were the other four components: confidence that side effects are minimal (component 1), confidence that the vaccine is likely to work (component 4), confidence that vaccination helps protect the community (component 5) and confidence that vaccination provides freedom to get on with life (component 6).

Beyond vaccine confidence, education level remained an important predictor of COVID-19 vaccination, with the estimated odds of vaccination generally increasing with greater education. Conversely, at a given COVID-19 vaccine confidence score, there was a significantly lower odds of vaccination among those with no college than those with master’s or doctorate degrees.

Discussion

We utilized survey responses regarding perceptions about COVID-19 vaccination in the general public to improve our understanding of beliefs that are driving COVID-19 vaccine acceptance. An adapted version of the Oxford COVID-19 vaccine confidence and complacency scale was found to be the most important predictor of vaccination in our study population. Within this scale, however, not all items were equally important. At a given scale value, vaccination was best predicted by higher confidence in the following items: that vaccine side effects are minimal, that the vaccine will work, that the vaccine will help the community, and that the vaccine provides freedom to move on with life.

Our findings can be applied to identify several potential messaging topics that may be more impactful in influencing an individual’s vaccination decision. This list of topics demonstrates the possibility of influencing vaccine acceptance by addressing information/misinformation and morals/values.^16,17 These topics included vaccine side effects, efficacy, societal benefits, and freedom. The present study’s findings are consistent with past research conducted before COVID-19 vaccines were available. Regarding survey data collected in July 2020, Kreps and colleagues found that Americans reported greater likelihood of getting a COVID-19 vaccination in the future based on two possible attributes: high efficacy and low rates of serious side effects.¹⁸ Taken together, these two studies suggest promising messages to promote COVID-19 vaccinations, including their high effectiveness in preventing hospitalizations/deaths and their safety profiles.

When formulating messaging approaches around these topics, it is essential to identify the population these messages are intended to benefit and what strategies may be most successful at reaching these groups. For example, messaging related to freedoms afforded by vaccination (e.g., giving people greater freedom to travel or to participate in in-person social events) may particularly resonate with people with strong individualist values.¹⁹ Similarly, a study of over 15,000 individuals in the United Kingdom demonstrated that messages focusing on personal benefits of vaccination, as opposed to societal benefits, were more successful at reducing vaccine hesitancy among individuals who were highly hesitant to receive a COVID-19 vaccination.²⁰

Our findings also identified topics that may be less effective in supporting vaccination by revealing that higher confidence in not feeling like a guinea pig and that safety wasn’t compromised by the rapid development of the vaccine were associated with a decreased likelihood of vaccination at a given value of the adapted COVID-19 vaccine confidence scale. These two topics both relate to the development and testing of the vaccine. Based on this finding, messages aimed toward convincing individuals about the safety of vaccine development may be less effective in supporting vaccination. While these topics were determined to be less important in predicting COVID-19 vaccination than other components in the COVID-19 vaccine confidence scale, they were nevertheless positively predictive of accepting COVID-19 vaccination. Messages could still be developed to address these two less important concerns, including 1) a social norms approach²¹ highlighting that “Hundreds of millions of other Americans have already received the COVID-19 vaccination;” and 2) information regarding the decades-long research in developing mRNA vaccines.

Beyond the COVID-19 vaccine confidence scale, education level was also identified as an important predictor of vaccination in our model. There was a general trend of decreasing likelihood of COVID-19 vaccination with decreasing education level, with individuals having a high school or GED education level being significantly less likely to be vaccinated as compared to individuals with master’s or doctoral level education. This finding, which has been corroborated in other studies,^22–24 reiterates the need to develop messaging that is accessible to populations with diverse educational backgrounds. For example, better efforts should be made to disseminate findings from research studies on vaccine efficacy and side effects using language at a reading level inclusive of the general public.

It is also important to recognize the role of trust in vaccine acceptance. Although elements of trust were included in the COVID-19 vaccine confidence scale, the medical mistrust scale used in our study did not contribute significantly to our model of vaccine acceptance. Our chosen measure of trust (i.e., mistrust in medical organizations) does not encompass trust in all organizations that may impact individuals’ decisions to be vaccinated, such as trust in government and pharmaceutical companies. Rebuilding declining trust in these institutions has been recognized as an important additional aspect of messaging strategies to encourage vaccination.²⁵

Limitations

Several limitations should be considered when interpreting the results of our study. First, our study population is limited by our Facebook recruitment strategy. One limitation created by this strategy is a lack of diversity, particularly for participant race, as has been reported in other studies.²⁶ A second limitation of this strategy is potential bias in participant sampling, in that we cannot guarantee true randomness in sampling when using Facebook’s auto-generated advertisements. A third limitation of this approach is that the study population may not be representative of the general US adult population, as it excludes individuals who do not use Facebook or have limited access to the Internet. Our choice to utilize Facebook for recruitment was purposeful to elicit perspectives from a platform that has been immensely impactful in spreading information and misinformation regarding the COVID-19 pandemic and COVID-19 vaccination. Nonetheless, investigating perceptions regarding COVID-19 vaccination among diverse populations is critical in developing appropriate and effective messaging strategies and should be addressed with future research.

Our decisions related to data exclusion and potential confounders also present a limitation in our analysis. We chose to limit our analysis according to demographic characteristics (including only individuals aged 18–85 years) and the availability of demographic information, which may introduce bias into our findings. In addition, our study was not designed to comprehensively evaluate all possible confounding factors that may influence vaccination acceptance or hesitancy. Factors such as income, access to healthcare, cultural beliefs, and pressures to receive a vaccination, including vaccination requirements imposed by educational organizations or employers, are important confounders to consider in future studies.

Additionally, our analysis focused on COVID-19 vaccination as defined by receiving at least one COVID-19 vaccine dose. Therefore, our analysis did not consider accepting COVID-19 boosters or completing a primary series for certain mRNA vaccines. Individuals’ perceptions of booster shots may differ from perceptions of initial vaccination and may therefore require identifying different topics for messaging to support acceptance of COVID-19 boosters. Finally, the timing of our study represents a snapshot of the evolving landscape of the COVID-19 pandemic. At the time of our study, COVID-19 vaccinations and boosters were widely available and recommended for adults in the US. However, the rapid changes in the availability of and eligibility for COVID-19 vaccination over the course of the pandemic may have influenced participants’ perceptions. Furthermore, vaccination options have continued to change since our study’s completion, including the release of the bivalent booster in October 2022. Continued assessment of vaccination perceptions is warranted to keep a pulse on public perceptions as the pandemic transitions to an endemic stage.

Conclusions

Through a survey of public perceptions about COVID-19 vaccination, this study improved our understanding of beliefs that are drivers of COVID-19 vaccine acceptance, thereby helping us to identify messaging topics that may be most effective in supporting COVID-19 vaccination. Studies that evaluate the impact of messaging strategies on health beliefs (e.g., vaccination perceptions) and health behaviors (e.g., vaccination acceptance/refusal) can be used to inform interventions to address vaccine hesitancy. Focusing efforts to communicate messages that connect with perceptions important to vaccine acceptance continues to be important in response to the COVID-19 pandemic and in preparing for future public health emergencies.

Data sharing statement

De-identified participant data is available upon request from Sarah R. MacEwan.

Acknowledgments

We would like to thank Recruitment and Retention Services in The Ohio State University’s Center for Clinical and Translational Science for their assistance managing the Facebook advertisement of our survey.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary data

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2023.2266929.

Additional information

Funding

This work was supported by a grant from the National Cancer Institute (U54CA260582) and the National Center for Advancing Translational Sciences (UL1TR002733).