COVID-19 vaccine literacy: A scoping review

ABSTRACT

To address vaccine hesitancy, specific self-rated tools have been developed to assess vaccine literacy (VL) related to COVID-19, including additional variables, such as beliefs, behavior, and willingness to be vaccinated. To explore the recent literature a search was performed selecting articles published between January 2020 and October 2022: 26 papers were identified using these tools in the context of COVID-19. Descriptive analysis showed that the levels of VL observed in the studies were generally in agreement, with functional VL score often lower than the interactive-critical dimension, as if the latter was stimulated by the COVID-19-related infodemic. Factors associated with VL included vaccination status, age, educational level, and, possibly, gender. Effective communication based on VL when promoting vaccination is critical to sustaining immunization against COVID-19 and other communicable diseases. The VL scales developed to date have shown good consistency. However, further research is needed to improve these tools and develop new ones.

KEYWORDS:

• Vaccine literacy
• health literacy
• COVID-19
• vaccine uptake
• vaccine hesitancy

Introduction

Health literacy (HL) relates to the ability to meet the complex demands of health: it entails people’s knowledge, motivation, and competence to find, understand and use health information to make decisions on healthcare, disease prevention, and health promotion.¹ It can be considered a tool for people’s empowerment.² Limited HL has been independently associated with poor use of health services and outcomes, and it is a major source of economic inefficiency.^3–5 On the contrary, high levels of HL can facilitate communication between healthcare professionals and the public,^6,7 which is critical in terms of vaccination adherence because people with low HL and perceived distrust of the health care system are more reluctant to be vaccinated.^8,9

Although active immunization has proven to be effective in controlling several vaccine-preventable diseases, vaccination coverage has remained steady or even decreased in the past few years, while vaccine hesitancy has emerged, resulting in a refusal or delay in vaccine acceptance, including COVID-19 vaccines, at least in specific segments of the population.^10–12 Such behavior results from a complex decision-making process, that is influenced by different factors summarized into the so-called “3 Cs” model, including the domains of complacency, confidence, and convenience, that has been evolving recently into the “4 Cs” and “5c” models, comprising additional domains of calculation and collective responsibility.^13,14 Limited HL is considered a component of the convenience domain of vaccine hesitancy and a contributing factor to the low uptake of vaccines. Indeed, information about vaccines is complex, and its understanding requires certain literacy skills. Even when high proportions of the population have an adequate level of HL, many people report difficulties processing information about vaccines.¹⁵ This is particularly relevant during times of crisis, such as the COVID-19 outbreak.¹⁶ People with limited HL have also contributed generate and increase negative rumors on the media about vaccines and other interventions aimed at containing the spread of the pandemic, because of the lack of reliable knowledge of scientific references. This has contributed to unhealthy, nonsocial behaviors such as not wearing masks, not washing hands regularly, and avoiding SARS-CoV-2 vaccination.^17,18

To address the issues related to vaccine hesitancy, the concept of “Vaccine Literacy” (VL) has been proposed,¹⁹ based on the same idea of HL. VL is not simply knowledge about vaccines, but it entails motivation and competence to deal with information about immunization, disease prevention, and health promotion. VL has also been defined as “the ability to find, understand and judge immunization-related information to make appropriate immunization decisions,”¹⁵ or “a process of providing vaccine information, building communication, and increasing people’s engagement about vaccines.”²⁰ Indeed, disease prevention and health promotion share many goals and there are overlaps between the two realms. The relevance of VL in the domain of disease prevention is obvious, as vaccination aims at preventing infectious diseases in individuals and within the population (herd immunity). In addition, it is also relevant to health promotion, as a process of empowering people to increase control over their health.^21,22 VL helps people to recognize the reasons behind recommendations and consider the outcomes of their possible actions. Thus, it is important for the public, but also for all healthcare workers to be “vaccine literate” to understand the meanings and the effects linked to newer and older vaccines, when communicating the relevance of immunization.

Limited HL is associated with low adoption of preventive measures such as immunization,^9,23 but, using different general HL measures, the association between HL and vaccine hesitancy, was shown to be inconsistent (positive or negative).²⁴ Therefore, the development of specific VL tools was undertaken to further advance the vaccination field and provide useful data to better understand the determinants of vaccine hesitancy regarding children’s and adults’ immunization. In particular, a self-rated tool has been proposed (HL Vaccines for adults in Italian – HLVa-IT)²⁵ –later translated into English and called HLVa – aimed at measuring VL levels associated with adulthood vaccination. Based on the same construct, a measure has also been developed to assess specifically COVID-19 VL (from this point forward, COVID-19-VLS): this scale also includes other variables of relevance, such as opinions, attitudes, behaviors, and willingness to be vaccinated against SARS-CoV-2.²⁶

HL and VL have received growing attention through research during the pandemic. Emerging literature has proposed different measures that explore public and individual attitudes and behaviors about COVID-19 and vaccine acceptance.²⁷ Yet, the role of HL on outcomes is still controversial.²⁸ In the context of COVID-19, assumptions have been made about the relevance of the mediating role of HL, for instance between distrust of the healthcare system and vaccine hesitancy.⁸ It has also been shown that a higher HL is associated with better health behaviors, and suggested that the effects of HL can be partially mediated through reduced perceived barriers to behavioral action.²⁹ Conversely, according to others, there was no direct relationship between HL and COVID-19 vaccine acceptance, which was rather the outcome of positive attitudes toward general vaccination and self-efficacy.³⁰

In light of these considerations, this paper aimed to review – among the large volume of publications on HL and COVID-19 - articles specifically assessing people’s VL skills using specific tools, as well as related determinants and outcomes, primarily the acceptance of the COVID-19 vaccines. In addition, we intended to potentially compare VL levels reported in the various studies, as well as their association with beliefs, attitudes, and behaviors toward COVID-19 vaccines within the general and selected populations, and describe methods of validation of the tools used in local languages.

Methods

This review was conducted in two steps. Given the correlation between HL and VL, the first step involved a non-systematic exploratory search of the literature on the role of HL during the COVID-19 pandemic. For the second step, we conducted a scoping review according to Arksey and O’Malley’s five-stage scoping review framework,³¹ refined with the Joanna Briggs Institute methodology³² to identify and describe all VL assessment tools in the literature.

Research questions

The research questions addressed in this review were:

• “What are the VL levels in the population in the context of COVID-19?”

• “What are the determinants and outcomes of VL in the context of COVID-19?”

Search strategy

To capture all studies to contribute to a wide review, a search strategy on MEDLINE/Pubmed was built by using the following search string: “vaccine literacy” OR “vaccination literacy” OR “vaccination health literacy” OR “vaccine health literacy.” Database searches were also conducted in five other databases (Embase, Web of Science, Cinahl, Scopus, and Psycinfo) using the following terms: “vaccine literacy” OR “vaccine health literacy.” The last search was completed on 31 October 2022. No other date limits were applied. Citations from selected articles were also reviewed for possible additional references and a supplementary manual search on Google search was conducted.

Inclusion and exclusion criteria

To be included in this review studies should have:

• described a tool/questionnaire that explicitly assessed VL;

• reported a VL score;

• reported at least one determinant or outcome of VL. For determinants, we considered any sociodemographic variables that could influence the VL score. By outcomes, we considered any variable that can be influenced by the VL, particularly “attitudes,” and/or “behavior,” and/or “beliefs,” and/or knowledge” of participants about COVID-19, and/or their “vaccine acceptance,” or “vaccine uptake,” or “willingness/intention to get vaccinated,” or “vaccine hesitancy.”

All electronic database search results were combined in Endnote, and duplicate records were removed. The Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) flow diagram guidance was used to display studies that were identified by the database search and met the inclusion and exclusion criteria.

Data extraction and presentation of results

To answer the research questions, we created a data charting form with the following elements: Authors, year of publication, country of the study, study design, sample characteristics, assessment tool, VL scores, determinants, outcomes, other variables, and main findings and statistical methods used. Data extraction was performed by three reviewers, and findings were verified by other three reviewers.

We used information from the data charting form to describe and summarize the overall number of studies, years of publication, countries where studies were conducted, and the focus and purpose of the studies. All vaccine literacy instruments were categorized according to their characteristics including their purpose, instrument design, and scoring method. This review was not intended to evaluate the quality of evidence from the selected publications (for example, using quality rating scales), which is not within the purposes of scoping studies.³¹

The results of studies that used VL tools of various and different nature have been analyzed and compared, also to describe the association between VL levels, determinants, and outcomes. A descriptive analysis was performed to summarize the data reported in the publications. Due to the marked variety of the studies in terms of demographics, methods, and results, a meta-analytical approach could not be applied. Anyway, additional analyses have been conducted to verify the homogeneity of population samples, as well as the difference and association between the aggregate results, in addition to their distribution and potential predictive value in identifying limited VL. In particular, chi-square, Grubb’s, Cochran’s Q, Begg’s, Shapiro-Wilk, Wilcoxon, Friedman, Spearman’s correlation, ROC curve have been used, considering 0.05 as alpha value. MedCalc (ver. 18.2.1)³³ and NCSS 2022³⁴ statistical software were used for analysis.

Results

In total, 39 studies were selected, of which 26 were original publications (24 from bibliographic databases and two from Google Search) assessing VL about COVID-19 through the use of specific VL tools – i.e. HLVa or COVID-19-VLS (Figure 1). The two publications retrieved from Google^35,36 are in the Turkish language but were included because English abstracts were available, containing data relevant to the objectives of the review. These 26 VL papers were ultimately considered for review and reported in Table 1. The other 13 screened publications addressed the use of these same questionnaires to assess VL in situations other than COVID-19, as well as of other tools exploring VL about COVID-19 (Table 2).

Figure 1. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flow diagram.

Table 1. Original published research about COVID-19 VL skills and association with determinants and outcomes, using specific VL tools, before and after SARS-CoV-2 vaccines approval and deployment (as of Jan 2021) – Studies using COVID-19-VLS are reported in Italic.

First Author, Ref #, Year	Country, Study population	Subjects N, Gender, Mean age (y.rs)	Study design, Methods, Execution period	Tool Name	Functional and Interactive-critical VL score, Notes	Determinants (common determinants: age, gender, education, information sources)	Outcomes	Other variables and tools	Main findings and assessments, Statistic tests
Achrekar^Citation37 2022	India, General population	687, F 462, Age 25.41 ± 9.1	Cross-sectional online survey, December 2021– February 2022	HLVa Scale	Non-hesitant participants Functional: 15.29 ± 3.19 Commun.: 16.02 ± 2.79 Critical: 12.85 ± 2.48 Hesitant participants Functional: 13.87 ± 3.37 Commun.: 14.93 ± 2.99 Critical:11.66 ± 2.75 Summative score used	Marital status, income, residence, contacts tested COVID-19 positive, living with vulnerable individuals	Intention to get COVID-19 vaccine booster dose	Vaccine Confidence Index (VCI), multi-theory model (MTM)	Higher functional, communicative, and critical VL (p < .001) in vaccine non-hesitant participants. Chi-square; independent samples t-test; linear regression
Alshehry^Citation38 2022	Saudi Arabia, nursing students	1,170, F 639, Age: 21,31 ± 1.88	Multi‐university online survey, November-December 2020	COVID-19-VLS	Functional: 2.98 ± 0.72 Interac.-crit: 2.70 ± 0.65	Previous COVID-19 infection	Intention to get vaccinated against COVID-19	COVID-19 Vaccine Attitude Scale, Vaccine Belief Scale	Interactive-critical VL: significant predictor of intentions to get vaccinated. Multinomial logistic regression
Batra^Citation39 2022	U.S.A, general population	501, F 255, Age: 51.21 ± 18.8	Cross-sectional Web-Based Survey, October 2021	HLVa Scale	Non-hesitant participants Functional= 15.82 ± 3.5 Commun.= 15.45 ± 3.25 Critical= 13.1 ± 2.73 Hesitant participants Functional=12.8 ± 3.99 Commun.= 14.71 ± 3.26 Critical= 12 ± 2.81 Summative score used	Marital status, income, residence, contacts with COVID-19 positive, religion, political affiliation	Intention to get COVID-19 vaccine booster dose	Vaccine Confidence Index (VCI) and multi-theory model (MTM)	Lower VL scores in hesitant participants (p < .001). Hierarchical multiple regression
Biasio^Citation40 2021	Italy, general population	160, F 62% Age 18–65 (1% 65+)	Cross-sectional online survey, January 2021	COVID-19-VLS	Functional: 2.99 ± 0.63, Interactive-critical: 3.38 ± 0.46	Beliefs about vaccination, Seasonal flu vaccination	Willingness to be vaccinated, Opinions, behavior	-	Interactive-critical VL significantly higher than in June 2020 survey. Spearman’s correlation; chi-square, Kruskal-Wallis, Mann–Whitney tests
Biasio^Citation26 2021	Italy, general population	885, F 443, Age between 18 and 65 (9% 65+)	Cross-sectional online survey, June 2020	COVID-19-VLS	Functional: 2.92 ± 0.70 Interactive-critical: 3.27 ± 0.54	Beliefs about vaccination, Seasonal flu vaccination	Willingness to be vaccinated, Opinions, behavior	-	Association (p < .001) between interactive-critical VL, attitudes and behavior vs Covid, and vs seasonal flu vaccination Spearman’s correlation; chi-square, Kruskal-Wallis, Mann–Whitney, PCA
Bulca Acar ^Citation41 2022	Turkey, adults visiting COVID-19 clinic	388, F 59% Age 37.55 ± 14	Face-to-face interview July - September 2021	COVID-19-VLS	Functional: 3.02 ± 0.81 Interactive-critical: 2.92 ± 0.71	Having child, marital status, history of COVID-19, chronic diseseases	Vaccine acceptance (uptake)	-	VL higher in individuals voluntarily vaccinated (p < .001). Multiple linear regression
Carter ^Citation42 2022	Australia, rural women unvaccinated against COVID-19	F 90 (10 in focus group), Age 30–44	Cross-sectional questionnaire June- July 2021 In-depth interviews July- August 2021	COVID-19-VLS	Functional: 2.92 Interactive-critical: 3.06 Mainly qualitative study	Geographical area,, occupation, children, pregnancy, preventive behavior	Intention to be vaccinat. against COVID-19, three-point Likert scale	Individual interviews, p= 10	Intention to be vaccinated not associated with VL. Logistic regression
Correa-Rodriguez ^Citation43 2022	Spain. Patients with systemic autoimmune disease	319, F 92%, Age 31–50	Cross-sectional study May-June 2021	HLVa Scale	Functional 2.59 ± 0.74; Interactive-critical 3.07 ± 0.60	Area of residence, civil status and socio-economic status;	Attitudes perceptions and beliefs about COVID-19 vaccines	-	Intention to be vaccinated In 96.7% of patients Mann-Whitney, Kruskal Wallis, Fisher’s
Costantini^Citation44 2021	Japan, Family carers	292, F 44%, Age 53 ± 9.1	Cross-sectional survey through an online research service company, May–June 2021	Nine HL items and nine focused on COVID-19 vaccines	Mean scores across the nine items range: VL from 2.55 to 2.84 HL from 2.69 to 2.94 Nine questions administered	Hours of care provided, having children, income, marital status	Patterns of VL relative to HL	-	VL lower in older participants Student’s t-test, chi-square, single-variable and multivariable regressions
Durmus^Citation36 2021	Turkey, General population	596, F 60.6%, Age 32.53 ± 14.96	Cross-sectional online survey Early 2021	COVID-19 VLS (defined as HLVa)	Functional: 2.40 ± 0.75 Inter-critical: 2.60 ± 0.69	-	VL scale Construct validation in Turkish	-	Two dimensions of the scale Exploratory (EFA) and Confirmatory Factor Analysis (CFA)
Engelbrecht ^Citation45 2022 Engelbrecht ^Citation46 2022	South Africa South Africa	10,466, F 34.9% Age 43.5 ± 13.68, 40% received COVID-19 vaccine	Cross-sectional, online survey September 2021	COVID-19 VLS (defined as HLVa)	Functional: 2.841 ± 0.799 Interactive-critical: 3.331 ± 0.559 Functional score >2.5: 59.8% Inter.-critical >2.5: 91.8%	Income, race, employment, Government’s ability to roll out vaccines, flu vaccine, vaccination status	Factors associated with limited VL Covid-19 Vaccination uptake	-	Vaccination status and poor health significantly associated with low VL. CFA confirmed two-factor construct Limited functional (p = .31) VL and interactive–critical (p = .002) VL more likely observed in unvaccinated individuals Binomial logistic regression
								-
Gendler ^Citation45 2021	Israel, parents) of children aged 12–15 y.s	520, F 401, Age 44.76 ± 8.09	Cross-sectional online survey, June 2021	COVID-19 VLS	Hesitant population, Intercritical: 2.80 ± 0.62 Functional: 3.18 ± 0.60 Non-hesitant, Inter-critical: 3.07 ± 0.44 Functional: 3.27 ± 0.61	Place of birth, residency,, occupation, religion, COVID-19 vaccination status	Willingness to vaccinate children, COVID-19 vaccine perception	5c model vaccine behavior, COVID-19 vaccine hesitancy	Higher VL (p = .06) and more positive perception of the vaccine (p < .001) in non-hesitant parents. Willingness to vaccinate associated with parents’ vaccination status (vaccine uptake) Binary logistic regression
Gusar ^Citation47 2021	Croatia, >18 y.s	1227, F 935, Mainly Aged between 18 and 44	Cross-sectional online survey, January 2021	COVID-19 VLS	Functional 2.86 ± 0.71, Interactive-critical 2.12 ± 0.75	Occupation, alcohol and tobacco consumption, chronic diseases, medicine use, vaccination status	Perceptions and attitudes about COVID-19 vaccination	-	VL level associated with education (p = .031) and lower age (p < .001). Intention to get vaccinated associated with higher VL (p < .01); two-factor loading confirmed t-test; PCA
Khiari ^Citation48 2021	Tunisia, Cancer patients	200 54.4 ± 12.7	Cross-sectional study February 2021	COVID-19 VLS	Functional : 3.2 ± 1.0 inter-critical : 1.7 ± 0.9	Occupation, chronic diseases, COVID-19 history	Willingness to get the COVID-19 vaccine (named acceptance)	-	Acceptance to get vaccine associated with VL score (p = .02) and Interactive VL score (p = .001) multivariate logistic regression
Kittipimpanon ^Citation49 2022	Thailand, ≥60 ys	408 F 72.5%, 83.3% of participants 60–69 years	Online cross-sectional survey between May and July 2021	COVID-19 VLS	Functional 2.88 ± 0.68; Interactive-critical 3.34 ± 0.49	Income, underlying diseases	Attitudes and Intention to get COVID-19 vaccine	-	Higher VL in females and persons aged 60–69 (p < .001) Pearson correlation, Independent t-tests
Krishnamurthy ^Citation50 2021	Barbados, Survey of healthcare professionals	343, 76% F, 144 Aged 18–34; 199 Aged 35+	Online Survey, February 2021	Six items from the HLVa Scale	Interactive critical score: 3.46 (non-hesitant HCWs) Six questions administered	Marital status, occupation, seasonal flu vaccine, nationality, chronic illness, perceived knowledge of COVID-19	COVID-19 vaccine intention	Recommending vaccination for relatives and patients	Vaccine hesitancy: 44.9%, chi-square. multivariable logistic regression; Spearman’s correlation
Li^Citation51 2022	China, resident population	362, F 53.9% Age 34.49 ±10.05	Online survey, May 2022	HLVa and two more items based on domestic and foreign literature	Functional 4.41 ± 0.73 Interactive 3.55 ± 0.95 Critical 3.28 ± 1.09 16 items administered, assessed by five-point Likert scale	Residence, marital status, occupation, income, vaccination status	Tool validation in the Chinese language	-	Chinese COVID-19 Vaccine Literacy Scale: three latent factors EFA and CFA
Maki ^Citation52 2022	Japan; Health Care Workers	1,519, F 76%, Age 38 95.3% vaccinated against COVID-19	Cross-sectional online survey, July 2021	COVID-19 VLS	Functional= 2.65 ± 0.73 Inter.-critical= 2.96 ± 0.62	Occupation, vaccination status	Intention to recommend the COVID-19 vaccine	Kessler Psychological Distress Scale (K6) questionnaire	95.3% vaccinated against COVID-19. VL scores divided into three groups per tertiles. Upper tertile more likely to recommend vaccine Poisson multivariable regression
Maneesriwongul ^Citation53 2022	Thailand, aged 18+	1,002, 502 Exploratory Factor Analysis 500 Confirmatory Factor Analysis F 85%, Most <50 old	Cross-sectional survey according to STROBE checklists, May 2021	COVID-19 VLS	EFA group Functional 2.87 (0.69) Inter/critical 3.32 (0.53) CFA group Functional 2.78 (0.73) Inter/critical 3.39 (0.51)	Marital status, occupation, income adequacy, underlying diseases, vaccination status	Tool construct validation in Thai	Vaccine uptake, intention to get vaccinated	Three groups identified: a) have got COVID-19 vaccine; b) will get the vaccine for sure; c) not sure/will not get the vaccine. Higher VL in vaccinated persons (p < .001). Two-factor dimensions construct EFA and CFA
Nath^Citation54 2021	Bangladesh, Young	343, F 41.40% Age 18–30	Cross-sectional study, through the Secondary & Intermediate Level Students’ Welfare Assoc. (SILSWA)	Eight items from the HLVa Scale	Mean 33.86 (SD = 7.21) Mean eHealth literacy 39.39 (SD = 7.87) Eight questions administered; summative score used	-	Vaccine uptake intention (VUI) measured by level of agreement	eHealth literacy scale (eHEALS), Vaccine Hesitancy “5C model”	Vaccine hesitancy strongest predictor of VUI. No significant association between VL and eHL. Pairwise correlations
Omidvar ^Citation55 2022	Iran, people > 18 y.s,	1564, F 70%, mean Age 37,45	Cross-sectional web-based study May 2021	COVID-19 VLS	37.77 Summative score used	Residency	Vaccination Acceptance	-	Predictors of vaccine acceptance: risk perception, disease knowledge, trust in health system, attitude towards vaccination, VL (p < .001)
Siewchaisakul^Citation56 2022	Thailand; health volunteers	5312 F 84% Age: <40–60+ balanced between age groups	Cross-sectional - online questionnaire September 2021	Three questions from the HLVa Scale	VL score 2.92 Three VL items assessed	Comorbidity, marital status, occupation, income, religion	Acceptance of COVID-19 vaccination assessed by a VA score based on 6 items	Fear of COVID-19 scale (FCoV-19S), Vaccine Hesitancy Scale (VHS)	VL not significantly associated with increased vaccine acceptance multivariable logistic regression
Takahashi ^Citation572022	Japan, pregnant women and mothers of young children	7,32, 1,639 pregnant 5,688 mothers Age 31.0–32.5	Cross-sectional internet survey July-August 2021	COVID-19 VLs	Pregnant women VL Functional 2.62 ± 0.75 Inter-critical 2.76 ± 0.61 Mothers VL Functional VL 2.62 ± 0.77 Inter-critical VL 2.69 ± 0.62	Occupation, income, Infection prevention practices	Vaccine hesitancy and vaccine uptake, selecting one of four options	Communicative and Critical Health Literacy (CCHL) scale	Vaccine hesitancy higher among pregnant women with lower interactive-critical skills and mothers with lower functional VL and lower interactive-critical VL (p < .001); two different latent factors Adjusted regression; PCA
Yadete^Citation58 2022	USA, general population 18+ Midwest, Northeast, South, and West	2138 Group 1n=1322 Group 2 p=816 M 47.6% Age group 1 47.62 ± 19.3 Age group 2 42.47 ± 17.7	Cross-sectional web-based survey July 2021 performed by a marketing research company	HLVa Scale	Intending to Take Booster Functional VL=3.1 ± 0.75 Commun. VL = 3.1 ± 0.60 Critical VL= 3.21 ± 0.68 Not- Intending to Take Functional VL=2.61 ± 0.74; Commun. VL= 2.70 ± 0.66; Critical VL=2.8 ± 0.74	Vaccination status, ethnicity, marital status, health insurance, contacts positive, region, religion, political affiliation	intention to take booster dose, Intention to have children vaccinated	Vaccine Confidence Index (VCI)	VCI and VL scores higher among primed people and intending to vaccinate children (p < .001). chi-square (post hoc contingency table), independent-sample p-/Welch tests
Yilmaz^Citation59 2022	Turkey, Nursing students	391, F (82.69%), Age 20.23 ± 1.93	Online survey January-March 2022	COVID-19 VLS	Functional VL 2.41 ± 0.58 Interactive-critical 2.99 ± 0.55	COVID-19 vaccination status, residence, Income, COVID-19 positivity	Assessment of VL levels	Student Introduction Form	Functional VL higher in vaccinated people (96.4% vaccinated against COVID-19) (p = .030) Student t-test, ANOVA
Yuksekol ^Citation35 2022	Turkey, Women of reproductive age	609 Age: 33.17 ± 9.30	Cross-sectional web-based survey October 2021	COVID-19 VLS	Total VL: 2.83 ± 0.47 Functional 2.61 ± 0.70 Inter-critical 2.94 ± 0.61	Marital status, nal background, employment status, chronic conditions, Covid-19 vaccination status	Assessment of VL levels	-	intermediate VL levels observed independent sample t-test, one-way ANOVA, Tukey test

Table 2. Original published research about VL skills and association with determinants and outcomes, using different tools.

First Author, Ref #, Year	Country Study population	Subjects N, Gender, Mean age (y.rs)	Study design, Methods, Execution period	Tool Name	Functional and Interactive-critical VL score	Determinants (common determinants: age, gender, education, information sources)	VL Outcomes	Other variables and tools	Main findings and assessments, Statistics
Amit Aharon^Citation60 2017	Israel, parents of children aged 3–4 y,s	731, Completed vaccinations 422 Not-completed vaccinations 309	cross-sectional survey; 2012–2013	vaccine health literacy scale	Completed vaccination Functional 1.92; ± 0.71 Communic. 2.92; ± 0.67 Critical 2.59 ± 0.86 Not Completed vaccination Functional 1.94 ± 0.77 Communic. 3.22 ± 0.59 Critical 3.06 ± 0.78	Number of children, religion, income	Vaccination attitudes, compliance with vaccination	Knowledge, beliefs, and attitudes towards vaccination	Significant negative association between communicative HL and compliance with vaccinations (p < .05). Chi-square, ANOVA, and MANOVA
Arriaga^Citation61 2022	Portugal, 16 y.s+	1247, M 604, F 643 Age 46 ± 16.7	Telephone or mobile phone survey, 2019–2020	HLS19-VAC (part of HLS 19-Q47), Digital HL, Navigational HL, VL (HL-VAC)	HL-VAC: 8.8% excellent 62.4% sufficient 15.1% problematic 13.7% inadequate	Economic capacity to pay for medical examinations, profession	HL and VL levels	-	Significant correlations between general HL and digital HL (p<.001), navigational HL (p < .001), and VL (p < .001) Pearson’s correlation, CFA
Biasio^Citation62 2020	Italy, general population aged 50–75 y.s	200, M 66%, Mean age 63.25	Cross-sectional, face-to-face survey, April-May 2019	HLVa-IT Scale	Functional: 3.23 Interactive 2.92 Critical 2.86	Occupation,, vaccination status	Vaccine acceptance. vaccination knowledge	Vaccine quiz	Face and construct validity, two latent factors identified (functional and interactive-critical) PCA
Cadeddu^Citation15 2022	Italy, general population	3550, F 51,9% (1815) Age 18–65+ balanced between classes	Cross-sectional study, May 2021	Four items of the HLS19-Q47 to measure VL scale	VL levels	Migration background, financial deprivation, geographical area	Explanation of VL levels	-	Good VL level in 47.5% sufficient VL in 20.1% limited VL in 32.4% Frequent difficulties in dealing with vaccination information. Ordinal logistic regression
Fadda^Citation63 2022	Italy, adolescents aged 14–18 y.s	391, M 48%, F 52%, Mean age 16	Paper-and-pencil survey, August 2018 - January 2019	Questions about opinions and behaviors regarding vaccination	Mean score: 6.51 ± 3.40 Range from 0 to 14	Types of school, parents’ and profession, siblings	Explanation of VL and knowledge	Knowledge about vaccination: reply to 14 statements about vaccines	Generally, poor vaccine knowledge Descriptive analysis
Lorini^Citation64 2022	staff of Tuscany nursing homes, Italy	1,794 858 analyzed for the aims of the study F 86.7%	Cross-sectional survey conducted online in July–August 2020	HLVa Scale	HLVa total mean score 3.19 ± 0.49 functional 3.17 ± 0.69 interact.-critical 3.21 ± 0.59	Mother language, profession	Sources of information on vaccines and vaccination	-	Interactive-critical VL: positive predictor of the use of official vaccination campaigns and health professionals; negative predictor of the use of social media. Univariate logistic regression.
Meppelink^Citation65 2019	Netherlands, September 2017	480 parents of 0–4 children 46% M, 54% F, Mean age 34	Cross-sectional, online survey	Vaccine health literacy scale (Aharon’s)	Functional VL 3.08 ± 0.69 Interactive VL 2.58 ± 0.60 Critical VL 2.38 ± 0.82	Online seeking about vaccinations, children vaccinated, vaccination beliefs, Medical job background	Perceptions of message convincingness about vaccines, credibility and usefulness	Newest Vital Sign	Biased selection and biased perceptions of message convincingness more prevalent among people with higher HL and health communication PROCESS model analysis
Montagni^Citation66 2022	France, adult population in the context of COVID-19	1,640, F 78.4%, Age: 18–34 84.5% 34+ 15.5%	Online cohort study April-May 2020, according to STROBE checklist	CONFINS questions: 77 items for HL, knowledge and beliefs about vaccines, digital VL	Digital vaccine literacy: hesitant 21.7 ± 2.8 pro-vaccination 22.7 ± 2.4	Seasonal flu vaccination, marital status, having children, being a student, working in health domain, self-perceived health, medical history	Define the hesitant population toward COVID-19 vaccines	-	Hesitant persons: more frequently females (p = .044), not annually vaccinated against flu (p = .026), less optimistic about treatment against COVID-19 (p < .001), less ready to undergo treatment (p < .001). Univariate and multivariate analyses
Rosano^Citation15 2022	Italy, 3500 men, and women aged 18 y,s and older	3,500, of which: 2949: online survey 551: phone interview 1685 M, 1815 F	Cross-sectional Study, From 8 April to 8 May 2021, a questionnaire including a 47-item section (HLS19-Q47)	HLS19-VAC (part of HLS 19)	Vaccination HL coronavirus-related HL	Financial deprivation, geographic area, vaccine coverage	Define Vaccination HL levels	GEN-HL	Excellent levels in 49%, limited in 30%. High overall HL-COVID level, difficulties dealing with COVID-19 info in elderly, those living in southern regions, and with high financial deprivation. Regression analysis
Suitner^Citation67 2022	Italy, elderly	600, 311 M 289 feM; Age 67.51 ± 1.46	Cross-sectional - phone interview	Questions about Pneumococcus and Pneumo vaccines	Majority aware of Pneumo vaccine availability	Region of residence	Vaccine knowledge	-	Very low reported vaccination status (11.2%), Descriptive analysis
Sumile^Citation68 2020	Philippines, mothers	200, Different age groups	Cross-sectional survey	vaccine health literacy scale (Aharon’s)	Functional: 3.14 ± 0.76 Communicative: 2.95 ± 0.87 Critical: 3.65 ± 0.74	-	Vaccine acceptability	-	Mothers’ functional HL (p:0.006) and critical HL (p:0.000) significantly related to vaccine acceptability. Multiple linear regression
Turhan^Citation8 2021	Turkey, general population	620, F 66.9%, Age: 35.06 ± 12.95	Cross-sectional, online survey December 2020	COVID-19 VL: two questions about COVID-19 vaccine information access l	COVID-19 VL score: 5.66 ± 2.76	Mask-wearing, hygienic rules	Healthcare system distrust, vaccine hesitancy	Health Care System Distrust Scale (HCSDS), Vaccine Hesitancy Scale (VHS)	HL mediation role of the relationship between healthcare system distrust and vaccine hesitancy Structural Equation Model (SEM) a
Wang^Citation69	China, parents	M 520 F 1344, Age: 32.2 ± 4.1	Cross-sectional, face to face survey April 2016	Three questions about a vaccination scandal	-	Having a child between the 0 and 3	Trust and choose domestically produced vaccines	-	Parents with better vaccine literacy (<0.05) more likely to trust and choose domestically produced vaccines Binary logistic regression

The HLVa scale and COVID-19-VLS share the same psychometric construct (Table 3), including functional, interactive (otherwise known as communicative), and critical questions. HLVa includes 14 items (questions) (Annex 1), while in COVID-19-VLS there are 12 questions overall (Annex 2). Five functional and nine interactive-critical items are listed in the HLVa questionnaire, whereas in COVID-19-VLS they were reduced to four and eight, respectively, to avoid redundancy, merging questions that resulted repetitive during the validation process of HLVa.⁶² HLVa and COVID-19-VLS answers are rated on a forced 4-point Likert scale describing frequency: a mean (± SD) score is calculated (ranging from 1 to 4), a higher value corresponding to a higher VL level. For both HLVa and COVID-19-VLS, a total VL score (i.e. the mean of the functional, interactive and critical subscales) can be reported, whereas many investigators prefer to describe functional and interactive-critical scores separately. The variables are treated as numerical data, like in prior studies where similar scales had been validated ⁷⁰ and used in vaccination realms.^60,64,71 Despite a cutoff not being identified so far, a ‘limited’ VL score has been proposed corresponding to a value ≤ 2.50.²⁶ Other Authors refer to the low tertile bound of the locally observed scores as a limited VL identification threshold.^52,57

Table 3. Constructs and items of VL scales used in the surveys; Items’ description is reported in Annex 1.

Subscales	Scales
	HLVa	COVID-19-VLS
Functional items	5	4
Interactive (communicative) items	5	8 (interactive and critical items merged)
Critical items	4
Total VL items	14	12
Other items included in the questionnaire	None	Opinions, attitudes, and behavior toward COVID-19 and other vaccines

HLVa was initially used and validated in 2019,⁶² while COVID-19-VLS was first utilized in mid-2020 when SARS-CoV-2 vaccines were being developed. Afterward, the scale was adapted to the second version in January 2021 (Annex 3), after the approval and deployment of the first vaccines, at the very start of the Italian vaccination campaign against SARS-CoV-2. ⁴⁰ The validation studies have identified two distinguished latent components (factors) defining the construct, corresponding to the functional and interactive-critical VL subscales, that explain quite high percentages of the total variance. Both scales allow comparisons between populations, as have been adapted and translated into various languages, in addition to Italian⁶² and English,²⁵ and administered in surveys carried out in different countries (Tables 1 and 4). The construct of the COVID-19 VLS, already validated in Italy,²⁶ has been re-tested and validated in Thailand⁵³ and Turkey³⁶ - before administering it locally to assess COVID-19 VL in samples from resident populations -, in addition to Croatia,⁴⁷ Japan⁵⁷ and South Africa.⁴⁵ The tool has also been translated into seven frequently spoken dialects in South Africa.⁴⁶

Table 4. Functional and interactive-critical VL scores from 17 surveys about COVID-19, using COVID-19-VLS or HLVa. In the included studies VL score was calculated uniformly based on tool instructions, as described in the validation studies. Some study populations have been split, as reported in the Results section of the selected publications..

Author, Country	Tool used	N=	Average Functional VL score	SD	Average Intercritical VL score	SD	Paired difference	Average Total VL score	Notes
1 - Alshehry et al. Saudi Arabia^Citation38	Covid-19-VLS	1,170	2.98	0.72	2.70	0.65	−0.28	2.84 *	-
2 - Biasio et al. Italy^Citation26	Covid-19-VLS	885	2.92	0.7	3.27	0.54	0.35	3.10 *	Interactive-critical vs Functional VL: p < .001
3 - Biasio et al. Italy^Citation40	Covid-19-VLS	160	2.99	0.63	3.38	0.46	0.39	3.19 *	Inter-critical VL higher in Jan 2021 than Jun 2020 (p = .021)
4 -Bulca Acar et al. Turkey^Citation41	Covid-19-VLS	388	3.02	0.81	2.92	0.71	−0.1	2.95	-
5 - Correa-Rodriguez et al. Spain^Citation43	HLVa	319	2.59	0.74	3.07	0.6	0.48	2.83 *	-
6 - Durmus et al. Turkey^Citation36	Covid-19-VLS	596	2.4	0.75	2.6	0.69	0.2	2.54	-
7 - Engelbrecht et al. South Africa^Citation45 8 - Engelbrecht et al. South Africa	Covid-19-VLS	4,190	2.95	0.77	3.45	0.51	0.51	3.21 *	Vaccinated persons	Unpublished data, kindly provided by the Author
		6,275	2.77	0.81	3.26	0.58	0.51	3.02 *	Unvaccinated persons
9 - Gendler et al. Israel^Citation74 10 - Gendler et al. Israel	Covid-19-VLS	366	3.27	0.61	2.86	0.66	−0.41	3.07	Parents intending to vaccinate their children
		154	3.18	0.60	2.80	0.62	−0.38	2.99	Parents not intending to vaccinate their children
11 - Gusar et al. Croatia^Citation47	Covid-19-VLS	1,227	2.86	0.71	2.12	0.75	−0.74	2.49 *	p < .001 Interactive-critical vs Functional VL
12 - Khiari et al. Tunisia^Citation48	Covid-19-VLS	200	3.2	1	1.7 §	0.9	−1.5 §	2.45 *	§ outlierp = .026 and p = .008, Grubb’s test, respectively
13 - Kittipimpanon et al. Thailand^Citation49	Covid-19-VLS	408	2.88	0.68	3.34	0.49	0.46	3.19	-
14 - Maki et al. Japan^Citation52	Covid-19-VLS	1,519	2.65	0.73	2.96	0.62	0.31	2.81 *	-
15 - Maneesriwongul et al. Thailand^Citation53 16 - Maneesriwongul et al. Thailand	Covid-19-VLS	500	2.78	0.73	3.39	0.51	0.61	3.19	Participants in Confirmatory Factor Analysis (CFA)
		502	2.87	0.69	3.32	0.53	0.45	3.17	Participants in Exploratory Factor Analysis (EFA)
17 - Takahashi et al. Japan^Citation57 18 - Takahashi et al. Japan	Covid-19-VLS	1,639	2.62	0.75	2.76	0.61	0.14	2.69 *	Pregnant women
		5,688	2.62	0.77	2.69	0.62	0.07	2.66 *	Mothers of young children
19 - Yadete et al USA^Citation58 20 - Yadete et al USA	HLVa	1,322	3.10	0.75	3.10°	0.60	0	3.1 *	Non-hesitant persons, °=communicative literacy score
		816	2.61	0.74	2.70°	0.66	0.09	2.66 *	Hesitant persons, ° =communicative literacy score
21 - Yilmaz et al. Turkey^Citation59	Covid-19-VLS	391	2.41	0.58	2.99	0.55	0.58	2.8	-
22 -Yuksekol et al. Turkey^Citation35	Covid-19-VLS	609	2.61	0.7	2.94	0.61	0.33	2.83	-
Mean		1333	2.83 ^	0.25	2.92 ^	0.42	0.09	2.88	^ p = .1305, Wilcoxon
Median, tertiles		603	2.87, 2.64–2.96	-	2.95, 2.78–3.13	-	0.08	2.91, 2.8–3.07	--

*: Total VL scores were calculated using the functional and interactive critical scores reported in the respective studies.

^: Excluding the outlier, functional mean score was 2.81 ± 0.24, and interactive-critical 2.98 ± 0.33 (p = .0502, paired samples t-test); percentile distribution did not change (N = 7 in the lower tertile for both subscales); the distribution of the functional VL data was normal (Shapiro-Wilk p = .5925), while the distribution of the interactive VL data became normal when the outlying value was excluded (Shapiro-Wilk p = .222).

The HLVa scale was used In nine of the studies included for review. Yet, in some of these publications, the score was not calculated as per the original tool’s instructions:⁷² for example, in some surveys, a summative score was reported, rather than the mean score^37,39,56 (Table 1). In other studies, only a few questions derived and/or adapted from the HLVa scale were administered to the participants.^50,54 In addition, this scale was translated into Chinese⁵¹ also adding two supplementary interactive items to the 14 original questions of the tool, and using a five-point instead of a four-point Likert scale for frequency. Thus, despite the relevance of these publications to the scientific community, only two articles using HLVa^43,58 reported scores “comparable” to other studies.

COVID-19-VLS was utilized in 17 studies. Actually, this tool was used in 18 studies, but two^45,46 were conducted with different objectives on the same population sample, reporting the same results, and were therefore considered as a single study. In addition, in two surveys where COVID-19-VLS was used, the VL score was not calculated according to the original instructions,⁵⁵ and/or the objectives were more focused on qualitative than quantitative aspects,⁴² thus they were not considered “comparable” to other studies, in terms of observed score. As a result, a total of 15 COVID-19-VLS studies have been selected for review, to be added to two HLVa studies (17 surveys in total).

HLVa and COVID-19-VLS scores, determinants, and outcomes

Table 4 and Figure 2 list the 17 studies comparable in terms of tool and scoring methods used, although they differ in terms of population sample size, demographic characteristics, and execution period. Table 4 does not include surveys administering questions and/or using rating methods other than those described in the instructions of the tools.⁷² In some of the 17 surveys, participants were divided into sub-groups, as reported by the respective Authors, for a total of 22 study populations, which were significantly dissimilar in sample size, ranging from 154 to 14,466 (chi-square for homogeneity, p = .000). The total VL score was not reported in all studies, therefore it was considered preferable to describe the results of the two subscales separately. However, where the total VL score was not reported, it was computed using the data from the corresponding studies, as described in Table 4, to get a more comprehensive picture of the findings. Heterogeneity was significant in both functional and interactive-critical sub-scales (Q test p < .0001; I²>98%, for both subscales), while publication bias was not (Begg’s test p = .523 and .732 for the functional and the interactive-critical subscale, respectively). An interactive critical subscale value was a low, significant outlier (Grubb’s, p = .026), which we decided not to remove from the analysis to be consistent with the descriptive purpose of the review, but pointing out where its exclusion could alter the significance. Actually, extremely low VL values had already been observed in other studies using the same tools.⁶⁰

Figure 2. Functional and interactive-critical VL score means (CI 95%), and lower tertile distribution of 22 study populations using COVID-19-VLS or HLVa, and respective references; lower tertiles bounds (functional = 2.64, interactive-critical = 2.78,) are represented by dotted lines; arbitrary cutoff value is set at 2.50 (solid lines). Authors’ names and study populations are numbered and listed alphabetically on the left side of the graph; markers dimension reflects variability in the sample size of the studies.

Comparing the aggregate data, the mean functional score was lower than the interactive-critical score (2.83 ± 0.25 and 2.92 ± 0.42 respectively; median 2.87, and 2.95). This difference was not significant (Wilcoxon, p = .1305), but it became marginally significant by removing the outlier (t-test p = .0502) (Table 4). There was no difference concerning the distribution in percentile, as one-third of both subscales scores were in the lower tertile (Figure 2). The correlation between the two subscales was not significant (Spearman, p = .724), even if it became significant (p = .001) by excluding the populations where the functional VL score was higher than the Interactive-critical one (Figure 3). This was not the case for the opposite, i.e. in the studies where the paired difference was in favor of the functional VL (p = .844), and remaining non significant even when removing the outlier (p = .166) (Figure 3).

Figure 3. Spearman’s correlation (CI 95%) between functional and interactive-critical VL (refer to Figure 2 for study numbering identification).

The left-hand graph shows studies where functional scores observed exceed interactive critical scores; the graph on the right shows those reporting a paired difference in favor of interactive-critical VL.

The most frequently considered determinants were age, gender, educational level, occupational status, income, and information sources, followed by race/ethnicity, marital status, vaccination status, medicine consumption, testing positive for COVID-19, geographical region, religion, health insurance, consuming tobacco and/or alcohol, having been in self-isolation, and political affiliation. In most studies participants were asked to indicate the main sources of information, to evaluate what primarily influenced people’s understanding and opinions on coronavirus and vaccines.

Age was not reported uniformly across studies: while some Authors indicated the mean age (± SD), others reported participants’ frequencies according to different age classes. Participants were mostly between 30 and 50 years, with extremes of about 20 years (Saudi and Turkish nursing students),^38,59 and 70 years and older (Thai general population).⁴⁹ Based on 21 study populations for which the mean age of respondents was described, or information could be approximated from the calculation of the reported age classes frequencies, the mean has been estimated at 38.40 years (SD 10.16). Functional VL appeared to increase significantly depending on aging (weighted linear regression, p = .025); it was not the same for interactive-critical VL (p = .925) (Figure 4). Gender was rather balanced, although with a predominant proportion of female participants in some studies,^43,53,56,59 and with three surveys enrolling only women.^35,42,57

Figure 4. Weighted linear regression (95% CI) between age and functional VL, and between age and interactive-critical VL; for studies where average values were not reported, age was estimated from the age class distribution - Refer to Figure 2 for study numbering identification.

Most investigated outcomes were COVID-19 vaccine “acceptance,” sometimes called “uptake” - although the two terms could not be equivalent in specific situations⁷³ -, and “vaccine hesitancy,” often assessed through the “5c” model vaccine behavior,⁷⁴ the Vaccine Hesitancy Scale (VHS),⁷⁵ or the “Vaccine Confidence Index” (VCI).³⁹ Another frequent outcome was “willingness” or “intention to get vaccinated.” Even if vaccine intention does not necessarily equate to vaccine acceptance or uptake, it still represents a relevant variable, as it is demonstrated the association between higher hesitancy and lower uptake.⁷⁶ Other relevant outcomes were about vaccine booster acceptance, in addition to a willingness to vaccinate children, opinions and attitudes toward COVID-19 and other vaccines (such as against influenza), perceived advantages of preventive measures recommended by the Governments, and intention to provide positive advice/counsel on vaccination against COVID-19.

Additional publications reviewed and related tools

Among the papers screened, HLVa was also used outside of the COVID-19 context (Table 2).

A face-to-face cross-sectional study was conducted in Italy to validate the tool (HLVa-IT) in people attending public health offices (mean age 63 years, females 66%): the mean functional VL score was 3.23, whereas the interactive and critical scores were 2.92 and 2.8, respectively.⁶² The study allowed the face and construct validation of the instrument, by associating the VL score with the objective measure of the participants’ vaccine knowledge.

Using the same tool, an online survey was carried out among nursing home workers in Tuscany:⁶⁴ the mean value of the functional scale was 3.17, while the interactive-critical score was slightly higher (3.21). The higher values of both subscales observed in this study compared to other surveys may be related to the occupational status of respondents. It was also shown that the interactive-critical score was a significant positive predictor of participants’ use of official vaccination campaigns, healthcare professionals, and search engines, and a negative predictor of social media usage.

Aharon et al⁶⁰ utilized the same scale adapted from the Ishikawa tool developed for chronic patients⁷⁰ to assess VL in a population of parents about the intention to vaccinate their children. Contrary to expectations, among those who completed the vaccination schedule, the functional score (1.92), and the interactive (2.92) were lower than among those who did not complete it (1.94 and 3.22, respectively), showing that higher HL skills may be not predictive of vaccine acceptance (p < .05), as also confirmed by others using the same scale.⁷¹

These findings are also consistent with results from a Dutch survey using the same HLVa scale in an online questionnaire aimed to assess the VL of parents of children aged 0–4 years, in addition to their beliefs about vaccines.⁶⁵ After reading texts containing positive or negative advice about vaccination, their answers revealed that they saw information that was consistent with their beliefs as more credible and useful. Biased selection and perceptions of message convincingness were more frequent among those with higher HL.

The same HLVa scale was utilized in the Philippines to evaluate parents’ awareness, VL, and Dengue vaccine acceptance,⁶⁸ in addition to a survey carried out about VL and the degree of information and awareness of Italian adolescents regarding teens’ vaccination.⁶³

In other studies, VL was assessed using other tools such as HLS19-VAC,⁷⁷ which is part of a family of instruments measuring different types of HL, including four vaccination-related items selected to assess the vaccination-specific HL. This instrument was used to assess levels of VL in the general Portuguese population⁶¹ together with other questions about digital and navigational HL, and to identify the determinants of VL of the Italian population through two online and telephone surveys, performed before¹⁵ and during the COVID-19 pandemic.⁷⁸

Within the same context, in France, a survey was initiated in April 2020 (CONFINS cohort⁷⁹ to monitor people’s well-being and mental health during the pandemic lockdown, and to define the population’s hesitancy toward vaccines. HL assessment was performed using five items from the 5th dimension of the Health Literacy Questionnaire (HLQ),⁸⁰ and seven items about Digital vaccine literacy, in addition to knowledge and beliefs about vaccination and capabilities to detect COVID-19-related fake news.⁶⁶

In Japan, three consecutive surveys were conducted in 2021,⁸¹ using a 14-item questionnaire among the general population, based on validated scales and indicators, to measure vaccine confidence and literacy, although containing only two functional and one interactive VL item. The first survey was conducted in January 2021 (before vaccine approval), the second in June (start of vaccination of the elderly), and the third in September (when about 70% of the target population was vaccinated), corresponding to the end of the 3^rd, 4^th and 5^th waves of the COVID-19 epidemic in Japan, respectively.

Another study⁵¹ assessed the validity and reliability of the Chinese version of COVID-19-VLS, in 362 residents. Factor analyses indicated that the scale consisted of three dimensions. Although different from investigations revealing two dimensions of functional and interactive-critical VL,^26,36,53 the theoretical basis of all these studies was the same, founded on the three-level HL model proposed by Nutbeam.⁸² In the Chinese study, the mean COVID-19 VL functional score was 4.41 ± 0.73, while those scores of interactive and critical VL were 3.55 ± 0.95 and 3.28 ± 1.09, respectively, out of a range from 1 to 5. The results indicated significant differences between functional, interactive, and critical VL (p < .001), which was consistent with previous studies,^26,46,47 despite the different scoring methods.

Discussion

The COVID-19 pandemic has been causing major health, economic and social impact. Uncertainty among people has been exacerbated by an enormous overload of conflicting information, resulting in a veritable infodemic.^83,84 Debates between individuals and organizations with a strong presence on the web and in the media have often led to contradictory opinions and negative beliefs. Scientific evidence is not always useful for interpreting information, even for literate individuals, and can be counterproductive if too much information is released with consequent saturation with conflicting data. It has also been shown that searching online for health information carries a risk of confirmation bias also for literate individuals, by selecting information that supports their own opinions.⁶⁵

Misinformation and disinformation negatively impact immunization programs and contribute to increased vaccine hesitancy, as was the case with COVID-19 vaccines during their development in 2020 and after they were approved and deployed in early 2021, although there was variability across countries as a result of different local determinants, such as socio-demographic factors. Furthermore, the emergence of the Omicron BA.1 variant and various newer sub-lineages has made it difficult for the public to understand the evolution of the pandemic, and the rationale for booster immunization.^85,86

In this context, the relevance of VL skills, and related determinants and outcomes, were addressed in response to the review’s research questions, by analyzing the studies using comparable tools and methods. However, the review was not intended for meta-analysis, nor was it a systematic review. In line with the methodological framework of scoping reviews,^31,32 the data reported in the selected studies were analyzed not to assess the quality of the publications, but to contribute to the knowledge on a relevant health topic, i.e. the relevance of VL in the field of immunization against SARS-CoV-2.

HL and COVID-19

HL is key in preparing populations for situations that require a rapid response, such as amidst a pandemic.^16,87 In particular, HL is critical for navigation in the coronavirus context, as shown by the vast number of online surveys published, aimed at evaluating the abilities of people to collect and understand information about COVID-19, using various scoring scales on online questionnaires.⁸⁸ Specific HL measures have been developed, such as HLS-COVID-Q22,⁸⁹ adapted from the HLS-EU questionnaire⁹⁰ and used for the first time before vaccine availability in a German cross-sectional survey of participants aged 16 years and over. It was shown that half of the interviewees had adequate COVID-19 HL levels, whereas 15% had problematic and 35% had inadequate HL skills: confusion about coronavirus information was significantly higher among those who had lower HL.

Also in other surveys conducted in early 2020, attitudes toward immunization and its relation to HL levels were not explored as often as other outcomes. In a few studies, simple questions were asked to participants,⁹¹ such as their opinion about the statement “Data about the effectiveness of vaccines is often made up” to which those with inadequate HL levels were significantly more likely to agree. More complex questionnaires were also utilized, during the various pandemic phases. As noted, in Japan⁸¹ vaccine hesitancy was assessed before COVID-19 vaccine approval, at the start of vaccination, and when 70% of the elderly had received at least one dose of vaccine: hesitancy was detected in 17.5%, 65.3%, and 19.4% of participants, respectively, and was significantly associated with limited HL.

Limited HL was also related to reduced adoption of protective behaviors toward COVID-19,⁹² and it was often significantly associated with negative beliefs and attitudes. There were few exceptions, where no significant difference was found in vaccine hesitancy as defined by HL levels.⁹³ The role of HL as a predictor of COVID-19 vaccine hesitancy was shown in studies where specific behavioral questions about vaccination were addressed.^94–98 This was confirmed by a review including 47 articles,⁹⁹ although the tools used to measure HL were not described for all the reviewed publications.

Attitudes toward COVID-19 vaccination have often been assessed by tailoring existing HL tools. As mentioned, in some studies an updated version of the European Health Literacy Questionnaire (HLS19-Q47) was used, developed in the framework of the M-POHL collaboration (WHO Action Network on Measuring Population and Organizational Health Literacy),¹⁰⁰ also including four vaccination-related questions to assess the vaccination-specific HL.^15,78 In other surveys, participants’ HL was measured using the 12-item short version of HLS-EU-Q, integrated with three vaccine-related items extracted from the full version of HLS-EU-Q, to assess vaccine literacy among people with different levels of perceived stress related to the pandemic. As expected, people with higher HL showed lower vaccine hesitancy.¹⁰¹

Findings about COVID-19 VL

The assessment of HL skills about COVID-19 vaccines has been made more accurate using specific self-rated VL tools, such as HLVa and COVID-19-VLS,²⁶ which were adapted from self-rated scales developed for chronic patients,⁷⁰ and specific to parents’ VL.⁶⁰ The items included in both tools engage the semantic system, i.e. the ability to read and understand information (functional subscale), whereas the interactive-critical subscale regards more the cognitive efforts (i.e. the ability to engage with information and use it to make decisions). COVID-19-VLS is more comprehensive than HLVa, including also questions on immunization beliefs in general, attitudes toward COVID-19 vaccines, in addition to behaviors toward other vaccines for adults, such as influenza, thus allowing evaluation of association with common outcomes without the need to use other instruments.

The vast majority of studies using HLVa and COVID-19-VLS were cross-sectional online surveys. These studies are easier and quicker to complete, particularly during the frequent periods of restriction (lockdown) since the start of the pandemic. However, cross-sectional studies have various limitations. Since they correspond to a one-time measurement of exposure and outcome, it is difficult to infer causality. In particular, in the surveys selected for this review, data reflect snapshots taken at a different time of the outbreak (between 2020 and 2022), when, in reality, individuals’ attitudes are dynamic and changing, in particular during a time as complex as a pandemic. Furthermore, different distribution channels were used in the studies reviewed, whose participants were generally not randomly selected, but a convenience sampling method was used, except for some investigations where marketing research companies were in charge of recruiting a representative sample by gender, race, ethnicity, and geographical distribution.^44,58 In addition, only a few studies were conducted following the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) guidelines.¹⁰² In general, Internet users do not represent the entire population. Online surveys are subject to some well-known multiple risks of polarization, such as selection, self-selection, non-response, and social desirability biases,^88,103 although the latter may be more common in telephone-based and face-to-face interviews than in self-administered web surveys.¹⁰⁴ In addition, online surveys may exclude persons who do not have access to or do not frequently use the web due to different reasons, including limited literacy.¹⁰⁵ This may be an additional bias that is relevant for studies – such as those reviewed – that specifically assess literacy skills, as it increases the probability of overestimating people’s VL levels.

Although the literature on HLVa or COVID-19-VLS is limited (26 publications have been identified for review), survey respondents were heterogeneous, including all adult age groups and both genders. Participants were distributed across multiple countries, including general populations (Italy, the USA, Croatia, and South Africa), young and older adults (Bangladesh and Thailand, respectively), parents in Israel, women in Australia, Turkey, and Japan, in addition to healthcare workers in Barbados, Japan, and Saudi nursing students, Tunisian oncologic and Spanish patients affected with autoimmune diseases. Sample sizes varied considerably from one study to another, ranging from 90 to 10,666. However, the VL score was not affected by the dimension of the population under assessment (Spearman, p = .898). Most of the investigations were carried out when some COVID-19 vaccines were already approved and administered in several countries, or immunization campaigns were close to starting (i.e. beginning-mid 2021), while the first survey conducted in Italy was carried out at an early stage of development, in June 2020. Three studies ^37,39,58 extended the assessment of COVID-19 VL to the booster dose. Differently than other common vaccines for adults (such as flu, pneumo, zoster, dTaP), COVID-19 vaccines are on a multiple-dose schedule. As a result, the experience with priming shots (such as experiencing side effects,¹⁰⁶ or self-rating a lower health status after the primary vaccination¹⁰⁷ may be considered to be one of the determinants of booster acceptance.

Several differences were observed between the studies considered in terms of geographical location and population demography, sample size, and enrollment procedures, in addition to the different execution periods of the surveys, and methods of online administration of questionnaires. Due to this heterogeneity, it has been possible to perform a direct comparison between them only about the score of the VL subscales. The average values observed were generally relatively high for both subscales, although with differences between studies: the reported average functional scores ranged from 2.4 to 3.27, whereas the highest score for the interactive review was 3.39, and the lowest score was 1.7, which represents a significant outlying value, but, if excluded, did not alter significantly the mean interactive-critical score (2.98 instead of 2.92), neither the distribution of scores in tertiles.

This variability is worth discussion. In the survey conducted among oncologic patients in Tunisia,⁴⁸ the average functional VL score (= 3.2) was much higher than the interactive-critical one which, as said, was extremely low (= 1.7). A significantly higher functional VL score was also observed among the Croatian population⁴⁷ (functional VL = 2.86, interactive-critical VL = 2.12), in Israel⁷⁴ (3.27 vs 2.86), Saudi Arabia³⁸ (2.98 vs 2.70), and Turkey⁴¹ (3.02 vs 2.92, respectively). Moreover, in studies conducted in Tunisia and Croatia the mean interactive-critical score was below the cutoff value of “limited” VL (≤2.50). Consistent with these observations, also another study using HLVa found a paired difference in favor of the functional subscale.⁵¹ Therefore, these populations seem to have sufficient skills in reading and comprehending COVID-19 vaccine information, but their ability to be actively involved in making their own decisions about COVID-19 vaccination appears to be potentially reduced. These studies were conducted at different points in the pandemic, mostly during 2021, when the incidence of COVID-19 cases progressively increased in all geographic areas, and vaccination campaigns were implemented with substantial differences across Countries in terms of coverage rates. However, differences in VL scores between studies appeared to depend more on local cultural and socio-demographic determinants than related to epidemiological factors and/or vaccine availability or uptake during the conduct of the surveys.

In one study⁵⁸ the scores of the two subscales were equal, whereas, in all other investigations, the observed functional values were below the interactive-critical VL values, even if these were dispersed in a wider range (Figure 2). In the investigation conducted in South Africa a functional VL score≤2.50 was identified in 40% of respondents, compared to 8% with limited interactive-critical literacy.⁴⁶ Similarly, in Italy²⁶ a limited VL score was observed in 33% of persons for the functional and 11% for the interactive-critical scale. Notably, when considering only studies reporting mean interactive-critical scores higher or equal to the functional scores, the correlation between the two subscales was significant. It is unclear whether this can be an indicator of the sensitivity of the tool when assessing the association between VL and vaccine acceptance. This would be worth exploring further.

While in Italy interactive-critical COVID-19 VL was higher than functional VL, the results of an earlier study conducted in the same country using HLVa before the pandemic had shown the opposite,⁶² with a lower interactive-critical score. Moreover, the level of interactive critical VL observed in the second Italian survey was higher than that performed in the middle of 2020.⁴⁰ The rapid development of the COVID-19 outbreak has called for people to acquire and apply health information, and adapt their behavior at a fast pace. Likely, while the enormous quantity and variety of news have produced an overload, on the other hand, it has also led many people to seek precise and reliable information, check the credibility of sources, and discuss with other people, thus increasing their interactive and critical skills, although the ability to search for accurate information can also be related to people’s characteristics and educational level. On the other hand, functional skills were challenged by many complex terminologies and technical information, which may explain the lower functional score, also among highly educated individuals. In addition, in some countries the levels of functional VL may have remained lower as information about the vaccines is generally available in English: participants with a non-English first language may have difficulties in reading and understanding the information.⁴⁵ Notably, not only interactive-critical but also basic functional VL is relevant:¹⁰⁸ if it is low, there may be a risk that individuals do not always understand the information they are interacting with. A certain level of semantic understanding of the data is needed to determine if the information is consistent with people’s choice to be vaccinated.

The VL levels of selected populations studied in some of the surveys (caregivers of elderly parents, health professionals, patients) were not significantly different from those of the general populations, except for cancer patients in Tunisia, as mentioned,⁴⁸ whose average interactive-critical score was very low and associated to the acceptance to get the COVID-19 vaccine, which was also low (35.0%). However, this is difficult to investigate given the limited number of studies assessing these aspects, especially in low- and middle-income countries. Such differences between countries are an obstacle to understanding the association between VL skills and willingness to get vaccinated.

Finally, in two surveys where HLVa was used in the general population of India and the USA, findings were in agreement with the other studies, although a summative score was calculated, instead of a mean one. Non-hesitant participants showed significantly higher scores of functional, communicative, and critical literacies as opposed to hesitant participants, regarding COVID-19 booster vaccination.^37,39 Notably, the score values were very similar between the two studies, although the surveys were carried out in two different countries. Similarly, in another survey performed in Iran⁵⁵ COVID-19-VLS was not administered following the tool’s instructions, but VL was identified as a significant predictor of vaccine acceptance. On the contrary, in another survey⁵⁴ where eight items from HLVa were administered to young adults from Bangladesh, VL failed to have any influence on the vaccine uptake Intention, while eHealth literacy shared a positive association with it, and vaccine hesitancy was identified as the strongest predictor of vaccine uptake intention. These observations raise the question of the usefulness of using a limited number of items to assess literacy, considering the complexity of the dimensions underlying the VL domains, such as disease prevention and health promotion. It is a frequent practice to reduce the length of questionnaires by selecting items from an existing validated scale, but shortening may impact on the construct validity and consequently affect the assessing potential of the scale.⁸⁸

Determinants

The causal relationship between antecedents, HL, and outcomes is still unclear.²⁸ Moreover, it is not easy to understand the relationships between the different factors considered in the surveys and VL, as causality cannot be established in cross-sectional studies. This is also relevant to the vaccination status, which is usually considered an outcome of VL, but, as mentioned, it could also be regarded as a determinant: people may have improved their VL skills through primary immunization, which may have contributed, among other factors, to their choice of getting booster doses, or other vaccines in the future. Some factors are traditionally considered antecedents of HL, including personal determinants.¹ In the literature reviewed, the main factors associated with VL included age,^{39,44,47,49,55} gender,^43,45,49 educational attainment,^{26–45–47–49–55} marital,⁵⁰ and socioeconomic status.⁴⁵ The relation between lower levels of education and lower income groups and VL was also reported in the survey of patients with autoimmune diseases conducted in Spain.⁴³ The positive association between income, education level, and VL is somewhat expected, as persons with higher levels of education are more likely to have a higher socio-economic status and could be expected to have better access to knowledge and be able to comprehend the information available to them. However, as said, higher levels of education do not always correspond with the ability to critically interpret information, as information overload can occur even in people with higher levels of functional and interactive-critical VL, who may not properly assess the available data. Ethnicity can be an additional factor associated with functional VL, particularly in multi-ethnic countries, such as South Africa where vaccine illiteracy was highest among Black Africans.⁴⁵ In the same study age, gender, education, income, health, and vaccination status were also independently significantly associated with limited interactive-critical VL.

As mentioned, frequent factors influencing VL are also age and gender. When examining the findings reported in the reviewed studies, aging was significantly associated with a higher VL, despite the mean age of the population examined being quite low (median 37.55 years). in a research carried out in Thailand⁴⁹ older adults aged under 69 years had higher VL than those aged over 70, which is consistent with progressive decline in the capacity for processing information.^108,109 These findings may be related to seniors paying more attention to information on disease management and prevention. While in younger people HL may be related to higher education levels, in older individuals HL skills, mainly assessed on functional tests, are likely associated with memory performance ¹¹⁰ and linked to higher morbidity rates and frequent use of medications, as well as medical visits, and a consequent better recollection of medical terms. In fact, crystallized cognitive skills, like generalized knowledge and vocabulary, are more stable with age, while fluid cognitive abilities may decline.¹¹¹

Thai females had a higher mean score for COVID-19 VL than males, which was consistent with what has been reported in Spanish patients, among whom women had higher interactive critical VL than males.⁴³ Also in a study carried out in China,⁵¹ women showed higher VL, than men. These findings are supported by previous research, which had shown that women tend to have a higher HL, for reasons that are unclear.¹¹² On the contrary, Italian females’ mean VL score was lower than males,’ as also reported by others,^26,45 while other researches have found no significant association between gender and levels of interactive-critical VL.^47,50 These uneven data are probably related to the different characteristics and cultures of the populations studied. Some surveys were unbalanced in terms of gender representation, with women being more numerous, up to 76% in the study carried out in the Croatian population⁴⁷ or 92% among the Spanish patients.⁴³ Furthermore, in three surveys,^35,42,57 only females were enrolled, including a population of pregnant women. Further studies are needed to better understand these observations, as the gender imbalance may have had an impact on VL levels.

Outcomes: vaccine uptake and intention to be vaccinated

Results across several studies indicate that higher levels of both VL subscales are often significantly associated with uptake,^{41,45,53,57,59} or the intention to get vaccinated against COVID-19.^{26,37,38,48,55,58} Notably, in the largest of the reviewed studies⁴⁵ unvaccinated persons were 1.3 times more likely to have limited functional and interactive-critical VL than those vaccinated (p = .001). Three studies on repeat-dose immunization have been carried out in India³⁷ and the USA³⁹ to assess the acceptability of COVID-19 vaccine booster dose among adult individuals: among the booster-hesitant groups VL was significantly lower, and they were more likely to be unvaccinated; in addition, they did not intend to have their children vaccinated.⁵⁸ It was also observed that women were significantly more hesitant than men to receive the booster.⁵⁸ The investigation conducted in Barbados in early 2021,⁵⁰ at the start of the national vaccination campaign, administering six VL interactive-critical questions (adapted from HLVa) to healthcare workers, showed that those willing to be vaccinated had a higher VL score. In this survey overall vaccine hesitancy was high (44.9%) even among healthcare workers, but those with higher VL scores believed that coronavirus vaccinations were safe and should have been recommended. This is extremely important, as the attitudes and behavior of health care providers are important to support the value of immunization to their patients. HL is required both for those who need information and for the health services that provide it and ensure its accessibility to the general population.⁷ Interactions between health professionals and patients are essential processes by which people are informed, educated, persuaded, and motivated to take care of their health. Indeed, VL skills are also relevant for health care workers because of their role as trusted sources of health information and advocates, in particular during vaccination campaigns.

Conversely, in Bangladesh, it was found that VL did not have any influence on young people’s intention to get a COVID-19 vaccine.⁵⁴ Also, in the study conducted in Australia among a sample of rural women the intention to be vaccinated against COVID-19 was not associated with high VL levels.⁴² In a survey of Israeli parents,⁷⁴ it was observed that willingness to vaccinate children was associated with their vaccination status, and also with their VL levels, although not significantly. Interestingly, in another survey conducted in the same country before the pandemic, parents with higher VL skills appeared to be more at risk of not vaccinating their children.⁶⁰ As mentioned, good educational attainment does not always equate to an appropriate ability to critically interpret information: people with appropriate levels of functional, interactive, or even critical literacy, can risk incurring errors of evaluation due to an overload of information. In addition, when people perceive themselves to be very knowledgeable, they may overestimate their judgment, which may make them less receptive to misinformation,⁶⁵ as could happen with the COVID-19 infodemic.

Vaccination status, in particular receiving the seasonal influenza vaccine, was a statistically significant determinant of VL levels⁴⁵ and a predictor of reported intention to get the coronavirus vaccine. Other studies confirmed that the behavior in the previous year’s seasonal influenza vaccination predicted COVID-19 vaccine acceptance.^113,114 Concerning these vaccines, discussions were already intense, as they were in the early stages of development, by 2020: the willingness to be vaccinated varied from 55% to 90% of the general population of many countries.⁹⁵ In some countries, such as Italy, during the early stages of COVID-19 vaccine development,²⁶ and also close deployment,⁴⁰ about 90% of people stated they were willing to be vaccinated. This is consistent with high coverage rates achieved in 2021,¹¹⁵ although local mandates and restrictions may have contributed to this result consistently, and possibly more than the actual confidence in vaccines. In fact, differently than for the common recommended vaccinations in children, achieving high levels of adult vaccine coverage has always been difficult, despite the efforts of healthcare professionals and the evident health and societal benefits to get vaccinated,^116,117

Information sources

The “infosphere” is regarded as a social determinant of health,¹¹⁸ and a situational determinant of HL.¹ At the same time, the ability to understand information is a primary indicator of HL,¹¹⁹ and its correct use can be considered as an outcome, as higher levels of literacy enable a better selection of valid information. In the reviewed studies, the information sources differed among countries and study populations. The internet was the most used source in Italy by the respondents (72%), followed by social media (47%), and television (49%),²⁶ whereas it was television (56%), followed by social media (37%) among Japanese family caregivers,⁴⁴ with about 30% of information sources being healthcare professionals. In the same country, a survey of pregnant women and mothers⁵⁷ confirmed these data, while those with higher skills tended to rely more on doctors and government and academic websites than women with lower interactive-critical VL. In other countries, information sources were more balanced, such as in South Africa ⁴⁵ and Australia,⁴² while sources most frequently used by the oncologic patients in Tunisia were television and radio (95.5%) followed by the internet (52.8%). Surprisingly, these patients did not include health professionals among their sources of information.⁴⁸ Doctors were not frequently mentioned also in other situations, such as in South Africa,⁴⁵ where they represent only 12% of the information sources, whereas in Australia⁴² and Japan⁴⁴ they represented around 30%. In contrast, the information source most frequently used by Israeli parents who intended to vaccinate their children was healthcare professionals (40%), whereas hesitant participants mostly retrieved information from the Internet (24%),⁷⁴ which confirms the importance of the information source to counter vaccine hesitancy. In an Italian study, using the HLVa tool before the pandemic⁶⁴ total VL was significantly higher among nursing staff who had declared to use official recommendations and healthcare professionals as the main sources of information. The literature on the role of healthcare workers in patient knowledge and HL is extensive. Receiving information also about COVID-19 from physicians was associated with higher HL among both young people and seniors.¹²⁰ However, the role of the different information sources as determinants, and the association with VL and other variables, deserve further investigation to be carried out on a dedicated review.

Consistency, validity, and reliability of VL tools

The VL tools developed so far have demonstrated good consistency in the scores observed in various countries while showing differences linked to sociodemographic factors. Their construct has been validated in the general population of different regions , based on participants’ knowledge of vaccines, and vaccination status (VL scores among vaccinated individuals were higher than those who were not or were not willing to do so). Principal Component Analysis (PCA) and Exploratory Factor Analysis (EFA), were used to extract the latent factors defining the construct of VL skills. Both techniques have identified two separate components (i.e. factors) underlying the functional and Interactive-critical VL items, explaining high and comparable percentages of the total variance, between 73.5%,²⁶ and 58.1%⁵³ In some studies, Confirmatory Factor Analysis (CFA) has been performed, in addition to EFA, in equivalent samples of the same population, to verify the factorial structure of the questionnaire.^36,53 Looking at the factor loading values reported in studies applying the same factor extraction technique (PCA) no significant differences were revealed between the populations, although using different statistical software (Friedman test, between studies: p = .257) (Figure 5).

Figure 5. VL factor loading, using PCA extraction method, in populations from Itay (Biasio et al., 2020), Japan (Takahashi et al. 2022), Croatia (Gusar et al. 2021), and Thailand (Maneesriwongul et al. 2002, unpublished data kindly provided by the Author).

Items measuring functional VL (# 1-4) load on the second factor, while interactive-critical items (# 5-12) load more on the first factor. Values reported in Gusar’s publication have been reversed to consent comparison. Refer to Annex 2 for Item number identification.

These results, showing a two-component construct, are consistent with those of a study of nursing homes staff (explained variance = 56.75%) using the HLVa tool, although VL assessment was not specifically related to COVID-19.⁶⁴ In a survey carried out in China,⁵¹ using an adapted version of HLVa, as mentioned, analyses identified three dimensions (instead of two), i.e. functional, interactive, and critical VL, explaining 63.3% of the total variance. In addition to the methods used for factor extraction, there may be other explanations for these differences: in the Chinese study the number of items included in the tool was higher, and a five-point Likert scale was used to rate the responses, instead of a forced four-point scale used in the other surveys. Moreover, the cultural backgrounds of the countries were different, and, although the process of translating the tools was certainly accurate and according to customary procedures, such as back-translations performed by professionals, it may have been difficult to avoid subtle variations in expression between the various versions of the questionnaires.

In addition to the validation of the construct, face validity has been performed for HLVa, and criterion validity has also been sought for the same tool, verifying its relation with acceptance of vaccines recommended in the adult/senior age.⁶² A positive association with vaccine acceptance had been observed on the functional scale in people aged 65 and over, which did not allow to accept a predictive validity, confirming that the outcomes of VL are more clearly verifiable on knowledge than behaviors. However, as mentioned, at least five of the reviewed publications have demonstrated a significant association between VL levels and COVID-19 vaccine uptake,^{41,45,53,57,59} in addition to the other publications demonstrating a significant association with the willingness to be immunized. Reliability, evaluated by Cronbach’s alpha, resulted in high or acceptable values in all the studies examined, sometimes with values for the functional above the interactive-critical subscale values, whereas in other cases, the coefficients were similar or corresponding.

All these observations, in addition to the consistency of the scores observed in the population samples and the association with various relevant outcomes, support the suitability of the VL tools, and will possibly provide information that can lead to redefining which cutoff score should be used to define limited levels of VL, which is currently proposed to be a score of 2.50 or lower. Currently, when looking at the distribution by percentiles of mean score estimates, one-third lies in the lower 33^rd percentile, which could likely correspond to “low literate” individuals, as suggested by some Authors.^52,57 In fact, before having defined a cutoff value universally applicable, referring to local average scores to identify limited VL seems valuable. Assuming the lower tertile bound of the mean scores reported in the reviewed articles as a threshold of limited VL, the ROC curve analysis has shown that association criteria ≤ 2.78 (for the interactive-critical subscale) and ≤ 2.64 (for the functional subscale) obviously provide the highest (100%) predictivity in identifying limited VL (p < .0001), but sensitivity decreases sharply by reducing both criteria by a single decimal point, to reach extremely low predictivity at ≤ 2.50. Thus, a specific analysis is required to provide a more accurate definition of cutoff values, based on additional data that will become available in the future.

Strengths and limitations of this review

Arksey’s, and Briggs Institute’s well-established definitions for scoping reviews were used to guide study selection, which represents a strength of this review, together with the utilization of various electronic bibliographic databases. Search strings were precisely tailored to the specific questions, including an exhaustive list of inclusion/exclusion criteria regarding VL determinants and outcomes. Furthermore, citations and articles were reviewed and controlled by two independent groups of persons. The same occurred for the content and format of the charting tables.

Despite the use of various databases and attempts to be as comprehensive as possible, this review may not have identified all relevant articles in the published and gray literature, as the overall search strategy may have been biased toward public health. Searches of other bibliographic databases may have resulted in other relevant published studies, given the number and variety of publications in the context of COVID-19. Furthermore, while this review included any article published in English, the search was conducted using only English terms, which possibly could have led to missing some studies, despite that it was able to retrieve two relevant papers published in Turkish via their English abstract. Reported limitations and the heterogeneity of online cross-sectional studies in terms of the period of completion, methods used, and reporting of results may have affected the interpretation of the data, despite the accuracy of the research questions. Particularly, in some studies, participants were divided into subpopulations according to different criteria, making it more complex to evaluate the results. In addition, the studies that were not included in the score comparison, due to the different methods used for the assessment, could have provided additional relevant data, although the reported results were in line with those of the other surveys, confirming the positive association between high VL skills and positive behavior toward COVID-19 primary vaccination and booster.

Despite differences, most of the reviewed studies used the same scale validated in different populations and languages, which is a strength, considering the wide variety in rating scales of online questionnaires administered in the COVID-19 context. Thus, by associating studies, we believe that it has been possible to obtain a fairly accurate image of the use of tools and of the measure of VL skills, even in the diversity of values reported in the various regions and populations, probably linked to methodological and/or local cultural differences. Due to this, we addressed the research questions mainly descriptively, despite statistical interpretations that have been proposed, which should be treated with caution because they rely on reported heterogeneous data, and were not directly observed.

Future research

In addition to a possible definition of the cutoff values of the VL scale, as mentioned above, other important areas require further investigation, the main of which remains the role of VL as a mediator in influencing vaccine acceptance – despite some evidence suggesting this -, given that the research undertaken so far is limited and heterogenous, and predominantly focused on COVID-19. Moreover, while VL appears to be associated with education, income, and social status, more research is needed to determine whether and why gender differences occur, and how to address them. More generally, the causal relationships between determinants, VL, and health outcomes – and the mediating role of VL – should be further investigated. Future research should also be focused to develop other VL tools and/or fine-tune the construct of existing instruments and items, concomitantly with administering objective questions on vaccine knowledge. Indeed, using VL tools may be not only psychometric in scope but also “educational,” as by answering the questions participants may find the motivation to increase their ability to obtain accurate information, thus increasing knowledge about health topics.

The likelihood that respondents often use inaccurate information is supported by the fact that they frequently rely on media, such as television and social media that are less likely to be objective than other sources such as healthcare workers. Therefore, with the widespread use of television and social media for COVID-19 information, future VL surveys should be designed to tailor communication strategies to information channels which ensure a better understanding of news and increase confidence in vaccines. However, despite mixed information on the Internet, different from official media, the interactivity of social media cannot be underestimated, since they encourage educated people to analyze and compare information from different sources, thus improving literacy skills

As is the case with vaccine hesitancy, VL is dynamic and context-dependent. Thus, measuring and tracking it over time is essential and is an important objective for further studies. Repeated cross-sectional and longitudinal surveys have already been conducted to identify changes in vaccine acceptance over time and factors influencing it. Some authors described a decrease in intentions of getting COVID-19 vaccines when they became available, often related to concerns about the vaccine’s safety,^121,122 whereas others have reported an increase in vaccine acceptance over time associated with an increase in HL skills,¹²³ and others have documented changing trends during the various phases of the pandemic.⁸¹ An increase in the interactive-critical VL levels was observed in Italy in 2021 in comparison with the previous investigation conducted in 2020, administering the same questions, although to a different smaller population sample. More longitudinal research should be conducted administering specific VL tools, to further confirm the consistency of results and check individuals’ ability to understand and use information over time, also after the end of the pandemic.

Longitudinal surveys may also help to evaluate the knowledge, attitudes, and behaviors of the population about the different COVID-19 vaccines approved and available to date. Indeed, people’s literacy skills may have also been affected by conflicting information and debates about the various manufacturing platforms (mRNA, viral vector, proteic, etc.) and the respective safety and efficacy data released by the scientific and lay press – although in some surveys these aspects have been partially explored ⁴² -, in addition to the number of doses to be administered, and the decision to make them mandatory by some Governments. Moreover, as noted above, the HLVa scale was also used to assess VL in studies carried out outside the COVID-19 context,^63–65 also including literacy specific to other communicable diseases, such as Dengue.⁶⁸ This may represent another interesting area for further research about VL.

Finally, adolescents are another area for future research, since all participants in the studies of this review were adults. The pandemic has had many negative effects on teenagers, especially in low- and middle-income countries,¹²⁴ while vaccine coverage rates in younger age groups are insufficient even in developed realms.^125,126 The controversial nature of COVID-19 vaccination may put additional pressure on parents making decisions about immunizing their sons. In recommending vaccination, it is important to consider parents’ beliefs, attitudes, and behaviors to increase vaccine uptake, but also adolescents’ knowledge of self-consent rules, and of the relevance of taking part in vaccination decisions.¹²⁷ Likely, adolescents’ awareness about the importance of vaccination against SARS-CoV-2 and self-consent is related to VL: these aspects should be worth exploring.

Conclusions

To be health literate refers to people’s ability to understand and process health information in a way that enables them to make appropriate decisions related to achieving positive health outcomes. HL is even more relevant during pandemics, to prepare people for situations that require rapid responses, and an understanding of the relevance of prevention, especially in terms of vaccine acceptance. To address the issues related to vaccine hesitancy, the concept of VL has been proposed on the same concept of HL. VL entails motivation and competence to deal with information about immunization, disease prevention, and health promotion. Tools developed to assess VL in adulthood, subsequently adapted specifically for COVID-19, have been proven to be useful in different countries in measuring VL and its association with knowledge, attitudes, and adherence to preventive measures. This scoping review enabled us to reply to the research questions regarding the VL levels in the context of COVID-19, as well as the main determinants and outcomes. The VL scores reported in the different surveys were variable but allowed to perform comparisons, although mostly descriptive. Pending the definition of a generalizable cutoff score, limited VL can be identified based on local average values. The observed interactive-critical score was often higher than the functional one, suggesting that the COVID-19-related infodemic may have boosted people’s motivation and improved their ability to interpret and use information. Preparing the population for possible future communicable disease outbreaks is mandatory and urgent, and it is a complex topic. In this regard, it will be relevant to further develop the concept of VL and related tools for monitoring the evolution of people’s knowledge about immunization and promoting vaccination uptake. Considering the uncertainties around the virus variants that will probably be the primary driver of possible future outbreaks, and how severe these variants may be, it is key to continue and adapt the immunization programs against SARS-CoV-2, based on data available over time on the viral evolution and the characteristics of current and new vaccines. The results of this review confirm the relevance of assessing and considering people’s VL skills when promoting vaccine adoption and establishing related communications. The reviewed tools have shown a significant potential to assess VL levels, which may help tailor and target public health interventions to address vaccine hesitancy.

Authors’ contribution

Conceptualization: L.R.B., P.Z., M.D.R., C.L., G.B; Methodology: P.Z. Investigation: P.M., A.F., V.G., G.C., G.A., M.D.R., P.Z.; Data curation: P.Z.; Data analysis: L.R.B.; Writing – Review and editing: L.R.B., P.Z., C.L., S.P., and G.B.; Supervision: P.Z., C.L., G.B.

Acknowledgments

Special thanks to Michelle Engelbrecht ⁴⁵ and Wantana Maneesriwongul ⁵³ for providing additional unpublished data, to Katie Palmer for editing the manuscript, and to Giovanni Lorenzini Foundation for providing scientific support to the initiative.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Annex 1. HLVa, tool to assess Health Literacy about Vaccination in adulthood

Table

Variable	Measure and Items	Assessment (score)
Vaccine Literacy functional skills	When reading or listening to information about future vaccines or current vaccines for adults: (1) Did you find that the material as a whole (texts and/or images) was difficult to read? (2) Did he find words you didn’t know? (3) Did you find that the texts were difficult to understand? (4) Did you need much time to understand them? (5) Did you or would you need someone to help you understand them?	Ordinal 4 points Likert scale for ferquency Often (1) Sometimes (2) Rarely (3) Never (4)
Vaccine Literacy interactive (communicative) skills	When looking for information about vaccines for adults: (6) Have you consulted more than one source of information? (7) Did you find the information you were looking for? (8) Did you understand the information found? (9) Have you had the opportunity to use the information? (10) Did you discuss what you understood about vaccinations with your doctor or other people	Ordinal 4 points Likert scale for frequency Often (4) Sometimes (3) Rarely (2) Never (1)
Vaccine Literacy critical skills	(11) Did you consider whether the information collected was about your condition? (12) Have you considered the credibility of the sources? (13) Did you check whether the information was correct? (14) Did you find any useful information to make a decision on whether or not to get vaccinated?	Ordinal 4 points Likert scale for frequency Often (4) Sometimes (3) Rarely (2) Never (1)

Annex 2. COVID-19-VLS, tool to assess VL skills, perceptions, attitudes, behavior, and beliefs

Table

Variable	Measure and Items	Assessment (score)
Vaccine Literacy functional skills	When reading or listening to information about future COVID-19 vaccines or current vaccines: Did you find words you didn’t know? Did you find that the texts were difficult to understand? Did you need much time to understand them? Did you or would you need someone to help you understand them?	Ordinal 4 points Likert scale for frequency Often (1) Sometimes (2) Rarely (3) Never (4)
Vaccine Literacy interactive/critical skills	When looking for information about future COVID-19 vaccines or current vaccines: (5) Have you consulted more than one source of information? (6) Did you find the information you were looking for? (7) Have you had the opportunity to use the information? (8) Did you discuss what you understood about vaccinations with your doctor or other people? (9) Did you consider whether the information collected was about your condition? (10) Have you considered the credibility of the sources? (11) Did you check whether the information was correct? (12) Did you find any useful information to make a decision on whether or not to get vaccinated?	Ordinal 4 points Likert scale for frequency Often (4) Sometimes (3) Rarely (2) Never (1)
Beliefs about vaccination	How much do you agree with the following statements: ‘I am not favorable to vaccines because they are unsafe’ ‘There is no need to vaccinate because natural immunity exists’	Ordinal 4 points Likert scale for agreement Totally (1) A little (2) Partially (3) Not at all (4)
COVID-19 vaccines perceptions and attitudes	About future COVID-19 vaccines: Will be possible to produce safe and efficacious vaccines? Will you get vaccinated, if possible? Will Authorities succeed in vaccinating the entire population? Would you pay a fee to be vaccinated? Should children be vaccinated too?	Nominal YES/NO
Other vaccines behavior	About current vaccines: Have you been vaccinated against flu last season? Will you get vaccinated against flu this year? Do you plan to be vaccinated against other infectious diseases?	Nominal YES/NO

Annex 3. COVID-19-VLS, reviewed in January 2021, subsequent to the approval of the first SARS-CoC-2 vaccines

Table

Variable	Measure and Items	Assessment
COVID-19 vaccines perceptions and attitudes	About future COVID-19 vaccines: …do you think the vaccines developed so far are safe? …do you think they are efficacious? ….do you think they overlap, regardless of the production technique used? …do you intend to get vaccinated against COVID-19? …. if you could, would you choose which vaccine to take? …will the Government be able to offer the vaccine against COVID-19 free for everyone? …would you pay a fee to be vaccinated? …should vaccination against COVID-19 be made mandatory for everyone? …should vaccination against COVID-19 be made compulsory for the most at-risk groups? …do you think children should be vaccinated too?	Nominal YES/NO
Other vaccines behavior	About current vaccines: … have you been vaccinated against flu? …you wanted to be vaccinated against the flu, but you couldn’t? …in 2020 you have been vaccinated and/or do you intend to vaccinate yourself soon against other infectious diseases, in addition to influenza and COVID-19?	Nominal YES/NO