International adaptation and validation of the Pro-VC-Be: measuring the psychosocial determinants of vaccine confidence in healthcare professionals in European countries

ABSTRACT

Background

Healthcare professionals (HCPs) play an important role in vaccination; those with low confidence in vaccines are less likely to recommend them to their patients and to be vaccinated themselves. The study’s purpose was to adapt and validate long- and short-form versions of the International Professionals’ Vaccine Confidence and Behaviors (I-Pro-VC-Be) questionnaire to measure psychosocial determinants of HCPs’ vaccine confidence and their associations with vaccination behaviors in European countries.

Research design and methods

After the original French-language Pro-VC-Be was culturally adapted and translated, HCPs involved in vaccination (mainly GPs and pediatricians) across Germany, Finland, France, and Portugal completed a cross-sectional online survey in 2022. A 10-factor multigroup confirmatory factor analysis (MG-CFA) of the long-form (10 factors comprising 34 items) tested for measurement invariance across countries. Modified multiple Poisson regressions tested the criterion validity of both versions.

Results

2,748 HCPs participated. The 10-factor structure fit was acceptable to good everywhere. The final MG-CFA model confirmed strong factorial invariance and showed very good fit. The long- and short-form I-Pro-VC-Be had good criterion validity with vaccination behaviors.

Conclusion

This study validates the I-Pro-VC-Be among HCPs in four European countries; including long- and short-form tools for use in research and public health.

KEYWORDS:

• Europe
• healthcare professionals
• international tool: vaccines
• vaccine confidence
• vaccine hesitancy

1. Introduction

Healthcare professionals (HCPs), particularly general practitioners (GPs) and pediatricians, play an important role in the vaccination of the general population and provide reliable information about available vaccines (i.e. benefits and risks, production of vaccines, etc.) to their patients [1–3]. While the great majority of HCPs endorse vaccinations, several studies have shown they can also be hesitant about certain vaccines [4–7].

Research has also shown that HCPs with lower confidence in vaccines are less willing to recommend them to their patients [5–10]. This can create problems, as a recommendation from an HCP frequently drives patients’ acceptance of vaccination for both themselves and their children [11–13]. Additionally, HCPs with lower vaccine confidence have been found less likely to accept vaccinations for their children and themselves [5–7,9,14]; this result increases the risk of transmitting infection to vulnerable individuals.

Several psychosocial factors contribute to vaccine confidence among HCPs, including their trust in health authorities [10] and their perceived collective responsibility for vaccinating patients to contribute to community immunity [15], to name a few. Better understanding these determinants in HCPs is needed to identify areas where intervention could increase their vaccine confidence, thus improving their recommendations to patients and self-vaccination behaviors [7]. The original Professionals Vaccine Confidence and Behaviors (Pro-VC-Be) questionnaire is a validated tool to measure the determinants of vaccine confidence among several types of HCPs in French-speaking countries. This original version of the Pro-VC-Be was also designed to measure the efficacy of interventions aimed at improving vaccine confidence in HCPs [16]. The purpose of this study was to adapt the original Pro-VC-Be to be applicable in a wider range of contexts and to validate the adapted international version (I-Pro-VC-Be) across European countries.

2. Materials and Methods

First, we briefly present the theoretical foundations of the original validated version (Pro-VC-Be) and describe its dimensions. We then present the steps we took to adapt and validate it in several European countries in order to propose an international version.

2.1. Theoretical foundation of the original Pro-VC-Be

The original instrument development and validation was carried out in French-speaking populations of GPs and nurses in France, Belgium, and Quebec, a French-speaking province in Canada. The first version of the questionnaire was created based on previous quantitative and qualitative studies on vaccine hesitancy (VH) among GPs and nurses in France [10,17,18] and aimed to measure vaccine confidence and vaccination behaviors in HCPs, including the psychosocial determinants underlying these behaviors. The theoretical bases of the questionnaire have been described in a previous publication, along with more detailed information related to its development and validation [19]. Briefly, the Pro-VC-Be questionnaire was constructed by combining complementary frameworks to obtain a global and public health oriented view of the psychosocial determinants of HCPs’ vaccination behavior. The questionnaire was informed by the 5C tool, a reference model for measuring psychological antecedents of vaccination behaviors in the general public [20,21]: the proximity of vaccine-hesitant HCPs’ attitudes to those of the public [7,22,23] supported the adoption of this framework, which in particular addresses the central VH issues of confidence, and collective responsibility. We also used the Health Belief Model (HBM), particularly its central hypothesis that adoption of preventive behaviors depends on their risk and benefit perceptions [24,25] as well as the Theoretical Domains Framework (TDF), a synthesis of several theories of behavior and behavior change relevant to vaccine-related intervention studies of HCPs [26,27]. Thus we added into the Pro-VC-Be the dimensions of self-efficacy and commitment, which constitute essential psychosocial resources for HCPs to address public VH. Finally, two other dimensions were borrowed from other theoretical frameworks to better understand the limits of HCPs’ trust in institutions and their communication behaviors towards patients (see paragraph 2.2).

2.2. Description of the original Pro-VC-Be

The original Pro-VC-Be consists of two parts: questions probing recommendation (HCPs’ tendency to recommend vaccines to their patients) and self-vaccination behaviors (HCPs’ personal vaccine uptake), and questions measuring the psychosocial determinants of vaccine confidence and vaccination behaviors. Recommendation behavior is measured with questions, which probe how often HCPs recommend certain vaccines to targeted patients; these questions can be used to focus on vaccines with suboptimal uptake in some populations and contexts.

The psychosocial determinants of HCPs’ vaccine confidence and vaccination behaviors measured in the original Pro-VC-Be include ten factors, divided into two categories: (1) six core determinants of vaccination behaviors and (2) four possible intermediary factors behind these behaviors. The six core determinants (5C and HBM models) are: perceived risks of vaccines (i.e. how safe HCPs perceive certain vaccines to be), complacency (i.e. the perception of lack of usefulness of vaccines), perceived benefit-risk balance of vaccines (i.e. the degree to which HCPs perceive that their benefits outweigh their potential risks), perceived collective responsibility (i.e. the extent to which HCPs recommend vaccines to contribute to community immunity), trust in authorities (i.e. trust in institutions and health authorities to provide reliable vaccine information and to define the vaccination strategy), and perceived constraints (i.e. such as cost of or access to vaccines) to patient vaccination. There are also four intermediary factors; the first two are informed by the TDF model: commitment to vaccination (i.e. the extent to which HCPs are proactive in motivating their patients to accept vaccinations), and self-efficacy (i.e. how prepared HCPs feel in terms of knowledge and skills to address vaccination with patients). The two remaining intermediary factors are theoretically-based dimensions: reluctant trust (i.e. the ‘leap of faith’ to trust vaccines and policies even if HCPs have doubts, a sociological concept developed by Giddens [28]), and openness to patients (i.e. attitudes toward [hesitant] patients and viewing their concerns regarding vaccination as legitimate, an attitude drawn from the theoretical frame of motivational interviewing [29]). These latter two factors help us to understand the limits of HCPs’ trust in institutions and their vaccination behavior toward patients. These factors might directly influence these behaviors or could mediate associations between the core determinants and vaccination behaviors. The 10 core and intermediary factors are measured with several statements to which respondents are asked to report their agreement on a 4-point Likert scale from ‘strongly disagree’ ( = 1) to ‘strongly agree’ ( = 4), with an additional ‘I don’t know’ option outside of the Likert scale.

2.3. Development of I-Pro-VC-Be: cross-cultural adaptation to several European countries

Based on this original 10-factor French-language questionnaire, the I-Pro-VC-Be was adapted to be used cross-culturally in the vaccination settings of Finland, Germany, Portugal, and the United Kingdom (UK), in addition to French-speaking countries. The original version was first translated and back-translated into English (by two separate, professional translators). Then, the adaptation process consisted of four steps: 1) initial assessment of the cultural appropriateness of the questionnaire in each country, 2) translation/back translation, 3) cognitive validation, 4) and pilot testing of the adapted questionnaire. For a complete table detailing all modifications made between the original Pro-VC-Be and the I-Pro-VC-Be, see the Appendix (Table A1).

2.3.1. Initial assessment of cultural appropriateness

Several adaptations were made based on expert reasoning and discussion among experts. The vaccine-specific recommendation questions were reformulated to minimize the risk that response variance might stem from differences in HCPs’ working conditions (i.e. ‘How often do you recommend the following vaccines?’ was changed to ‘When you treat [patient group] who have not had the [vaccine], what is the percentage of these patients for whom you actively recommend the vaccine?’ to account for the varying frequencies that HCPs could encounter patients needing certain vaccinations). The original response scale of the vaccine recommendation frequency questions was changed (4 levels, with an additional option ‘I don’t know’) to a scale ranging from 0=‘0%’ to 10=‘100%’ with the option ‘I do not treat patients within this age/target group’. When participants chose this response, questions were added to measure their intentions to vaccinate in hypothetical situations in order to decrease missing responses. Questions were also added to measure COVID-19 related attitudes and behaviors. Some questions were reformulated to allow for target age ranges of vaccines and names of relevant authorities to be inserted in accordance with the country in which the questionnaire is administered to make the questionnaire easily adaptable to different contexts. For the measurement of psychological determinants, the ‘I don’t know’ response option was placed in the middle of the response scale and renamed ‘undecided’, creating a 5-point Likert response scale. Finally, various adaptations were made related to the wording of questions to facilitate the translation of terms into each language and cultural setting (Appendix Table A1). Two items measuring professional norms were added, outside of the I-Pro-VC-Be, to take potential cultural differences and social desirability bias into account.

2.3.2. Translation

The adapted English version of the questionnaire was double-back translated into Portuguese, German, French, and Finnish and Swedish (the two official languages of Finland) by two independent, professional translators, for each language, from English to the target language [30,31]. A consensus version for each language was created among researchers and back-translated from the target language to English by two additional independent translators. If the back-translations revealed any translation issues, the consensus version was revised to solve them.

2.4. Cognitive validation and pilot test of the I-Pro-VC-Be

2.4.1. Cognitive validation

Following guidelines for cognitive validation of questionnaires [32], each country conducted interviews with HCPs (N = 28, at least five from each country) of various types from October to November 2021 to assess the meaningfulness and appropriateness of the items within each cultural context and to obtain information on whether the items were interpreted as intended. Several adaptations were made based on results of these cognitive interviews (Appendix Table A1).

2.4.2. Pilot test

The I-Pro-VC-Be was pilot-tested between December 2021 and February 2022 and administered to HCPs in participating countries via electronic survey. The total sample size was N = 272, while the expected population for the pilot analyses was N = 250. Cronbach’s alpha was calculated for each of the 10 factors to assess internal consistency, which was good or excellent (α=.72 to .99 according to the factor) except for the factor of openness to patients (α=.37) (Appendix Table A2). Given the interest of the items associated with this factor, however, we kept them to test their construct validity among a larger number of HCPs. Finally, the separate questions to measure recommendation behaviors for each COVID-19 vaccine in the pilot-version were replaced by one general question related to COVID-19 vaccines in order to limit the number of items in the final questionnaire (Appendix Table A1).

2.5. Final statistical validation

2.5.1. Cross-sectional survey in four European countries

An electronic, cross-sectional survey collected data from HCPs involved in vaccination in Finland, France, Germany, Portugal, and the UK in March-June 2022. The version of the I-Pro-VC-Be consisted of 44 items (Appendix Table A3): 10 concerning vaccination behaviors (recommendations to patients and self-vaccination) and 34 comprising the psychosocial determinants (10 factors); with the two added items measuring professional norms of HCPs, a dimension outside of the I-Pro-VC-Be. Invitation emails were sent to recruit relevant HCPs in each country (see Appendix Table A4).

The ethics boards of the University of Aix-Marseille (reference 16 December 2021-01), the Health Care Division of the Ethics Committee for Human Sciences at the University of Turku (1/2022), the advisory board on ethical issues of the University of Erfurt (20210713), the Ethics Committee of the Faculty of Medicine of the University of Coimbra (093-CE-2021), and the School of Psychological Sciences Research Ethics Committee (SREC) of the University of Bristol (119594) approved the original study protocol and questionnaires in their respective countries. The survey was conducted exclusively online, participants were asked to give consent before being directed to the platform where the survey took place in each country.

Upon analysis, the research team discovered that distributions of the UK responses deviated from expected distributions of similar variables in previous studies. Moreover, response times were significantly faster for the UK sample than for the other countries, as well as for the pilot samples from the UK and the other participating countries for the same questions. This unreliable data quality led us to exclude the data collected from the UK from the main statistical analyses. Only 135/999 participants in the UK sample took at least as long as the minimum response time (compared to pilot items) to answer these items in the survey: 118 of them were included in the sensitivity analyses, while 17 were excluded because of missing data for gender, age, or profession.

2.5.2. Sample size

Sample sizes were determined according to requirements for confirmatory factor analyses (CFA, see 2.5.3): computing a CFA with a maximum likelihood robust to non-normality of data and missing values requires a minimum number of 400 participants ([33], cited by [34]), which was (almost) the case for the participating countries.

2.5.3. Structural validity and measurement invariance of the I-Pro-VC-Be

First, we sought to determine if the 10-factor first-order latent structure of the core and intermediary determinants of vaccine confidence established in the original Pro-VC-Be [19] was appropriate for its international adaptation, in each country separately. We thus performed separate CFAs to examine the fit of the 10-factor first-order latent structure of the I-Pro-VC-Be in each country. We treated Likert scales as continuous variables and used maximum likelihood with standard errors robust to non-normality to estimate factors [35].

Then, to examine whether the 10-factor structure of the I-Pro-VC-Be was invariant across countries, we included all countries in the same 10-factor multigroup-CFA (MG-CFA). Measurement invariance testing assesses whether a construct has the same structure and is measured the same way across groups (i.e. countries), thus allowing cross-country comparisons in factor means and correlations [36]. We considered three measurement invariance steps in our analyses, increasingly constraining parameters to be equal between them [36]: 1) configural invariance (i.e. the same 10-factor structure is observed in each country): the same factor structure was specified for all countries, with factor loadings and item intercepts allowed to be freely estimated across countries; 2) then metric invariance (i.e. each item contributes to the same extent to its corresponding factor in each country): factor loadings were held equal across countries; and finally 3) scalar invariance (i.e. each item intercept is the same across countries, implying that factors are measured in the same way in each country and can thus be compared) [36–38].

Model fit was assessed with the following criteria: root mean square error of approximation (RMSEA), comparative fit index (CFI), Tucker-Lewis index (TLI), and standardized root mean square residual (SRMR). Models with CFI ≥ 0.90, TLI ≥ 0.90, RMSEA < 0.06, and SRMR < 0.08 were considered to fit reasonably or well [39]. Configural invariance was tested by evaluating the overall fit of the model; metric and scalar invariance were evaluated by testing the difference in robust ${\chi }^2$ of the increasingly constrained models [36].

2.5.4. Tests of construct validity: convergent and discriminant validity

We assessed convergent and discriminant validity of the 10 latent factors based on the MG-CFA parameter estimates. We used item factor loadings as indicators of convergent validity (cut-off criteria: ≥0.71 (excellent), 0.63–0.70 (very good), 0.55–0.62 (good), 0.45–0.54 (fair), 0.32–0.44 (poor) [40], and factor correlations as indicators of discriminant validity (≥0.80 indicating poor discriminant validity) [41].

2.5.5. Short-form version

We developed a 10-item short-form tool on the basis of the results of the procedures used in the original Pro-VC-Be. For the I-Pro-VC-Be short-form validation, the 10 items selected (one to represent each of the retained dimensions) were those recommended in the original Pro-VC-Be short-form [42]. Criterion validity for the short-form tool was tested with the methods described below in 2.5.6; the factorial structure was not tested as this had previously been done for the original Pro-VC-Be short-form [42].

2.5.6. Test of criterion validity: associations with behavioral outcomes

We considered three behavioral outcomes as criteria to test the extent to which the 10 psychosocial determinants of vaccine confidence measured in the I-Pro-VC-Be were associated with HCPs’ different kinds of vaccination behaviors [43]. The first outcome was a score for self-reported vaccine recommendation frequency/intention for eight specific vaccine situations, created by averaging responses to recommendation frequency/intention items [19,42]. The other two behavioral outcomes were self-vaccination status against 1) influenza over the previous three years and 2) COVID-19. Because the distributions of these three outcomes were markedly skewed to the right, we dichotomized them by using the following thresholds: 1) near-systematic vaccine recommendation: score > 90%; 2) up-to-date with influenza vaccinations: score = 3/3 (3 vaccinations in 3 preceding years); 3) up-to-date with COVID-19 vaccination: score = 4/4 (fully vaccinated and having received a booster dose) (Appendix Table A3).

Explanatory scores were created for each core and intermediate determinant factor by summing the corresponding items; these factor scores, as well as the selected single-items to represent them in the short-form, were then used to test for associations with behavioral outcomes in both the long- and short-form versions of the I-Pro-VC-Be, respectively. The scores of each factor were dichotomized at the scale mean, to assess the extent to which HCPs with above-average scores reported more frequent vaccination behaviors than those with below-average scores. We performed multiple modified Poisson regressions with robust error variances to correct the error overestimation of estimated relative risks that can occur when Poisson regression is applied to binomial data [44]. All regressions tested each factor separately and were adjusted for gender, age, profession, and country.

2.5.7. Sensitivity analyses

The sample size of 118 valid UK participants was insufficient to run a separate CFA for this country; however, sensitivity analyses were performed by including them in the MG-CFA models.

All analyses were based on two-sided P-values, with P ≤ 0.05 indicating statistical significance. They were conducted with Mplus, version 7.2 for factor analyses, and SAS, version 9.4 for the others.

3. Results

The study population included 2,748 HCPs involved in vaccination (Table 1): 1,213 from France (44%), 375 from Finland (14%), 557 from Portugal (20%), and 603 from Germany (22%). In all, 40% of the surveyed HCPs were men; 33% were aged under 40 years; 44% were 50 years or older; 78% were GPs and 18% pediatricians (Table 1). The distributions of I-Pro-VC-Be items and the professional norm items can be found in the Appendix (Table A5). The distribution of items varied significantly between countries in many cases; this was especially marked for professional norms. Compared with much lower proportions of HCPs in France and Germany (24–38%, P < 0.0001), the majority of HCPs in Portugal and Finland (68–78%) reported strong agreement with the corresponding items. Additionally, some items showed large variations within individual countries, such as the items for trust in authorities (Appendix Table A5).

Table 1. Characteristics of HCPs^a, by country (n = 2,748).

	All (n = 2748)	France (n = 1213)	Finland (n = 375)	Portugal (n = 557)	Germany (n = 603)	χ^2 p-value
What is your gender identity? (Please select all that apply)						<.0001
Male	40.3	44.6	21.3	19.9	62.2
Female	59.7	55.4	78.7	8.1	37.8
Age category						<.0001
Under 40	32.8	32.6	14.1	66.4	13.9
40–49	22.9	26.8	21.3	19.4	19.1
50 and over	44.3	40.6	64.5	14.2	67.0
What is your profession?						<0.0001
General practitioner	77.5	100.0	69.9	44.0	68.0
Pediatrician	18.3	0.0	30.1	56.0	13.1
Gynecologist	4.2	0.0	0.0	0.0	18.9

Abbreviations. HCPs: healthcare professionals; GPs: general practitioners; UK: United Kingdom.

^aFor information, of the 118/135 HCPs from the UK with no missing data: 26 (22%) were male, 92 (78%) were female; 32 (27%) were under 40, 27 (23%) were 40–49, 59 (50%) were 50 or over; 16 (14%) were GPs, 100 (85%) were nurses, and 2 (<2%) were pediatricians.

3.1. Structural validity of the I-Pro-VC-Be in each country

After the exclusion of one item regarding COVID-19 vaccine safety, which was highly correlated with the item concerning the benefit-risk balance of COVID-19 vaccines, the 10-factor CFAs conducted on each individual country showed good fit in France, Germany, and Finland (Appendix Table A6). The fit of the model was somewhat poorer in the Portuguese sample (RMSEA = 0.043 [0.039;0.047]; CFI = 0.89; TLI = 0.87; SRMR = 0.05), but became acceptable after allowing two pairs of within-factor item residuals to be correlated (RMSEA = 0.040 [0.036;0.044]; CFI = 0.90; TLI = 0.89; SRMR = 0.05) (Appendix Table A6).

3.2. Measurement invariance of the I-Pro-VC-Be

The configural invariance model fit the data well (Table 2), thus suggesting that the factor structure was equivalent across all groups, including Portugal, without any correlation between pairs of item residuals. In the metric invariance model, robust ${\chi }^2$ difference testing indicated no significant change in the model fit when factor loadings of the items were constrained to be equal across countries, compared to the configural model (P = 0.27). The scalar model (Figure 1) also indicated invariance in model fit, compared to both configural (P = 0.91) and metric (P = 1.00) models, when additionally constraining item intercepts to be equal across countries. This finding implies that factors are measured in the same way in each country and can be compared (Table 2). This was further confirmed in the sensitivity analysis including the 118 valid UK participants, which indicated configural, metric, and scalar invariance again (Appendix Table A7).

Figure 1. Multigroup confirmatory factor analysis^a with scalar invariance: structure and factor loadings (n = 2,748).Abbreviations. resp. = responsibility. RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis index; SRMR = standardized root mean square residual. ^aFactors were allowed to correlate. All item intercepts and factor loadings were set to be held equal across countries, and the metrics of the factors were defined by fixing their variances to one in one country while freely estimated in the others. *P ≤ 0.05 **P ≤ 0.01 ***P ≤ 0.001. Models with CFI ≥0.90, TLI ≥0.90, RMSEA <0.06, and SRMR <0.08 were considered to fit reasonably or well [39].

Table 2. Test of measurement invariance of the I-Pro-VC-Be: comparison of configural, metric, and scalar invariance models (n = 2,748; 33 items^a.).

				Comparison to configural model			Comparison to metric model
	χ^2 value	Df	Scaling correction factor for MLR	Robust χ^2 difference^b	df difference	P-value	Robust χ^2 difference^b	df difference	P-value	RMSEA (90% CI)	CFI	TLI	SRMR
Configural	4009.6	1800	1.22							.042 (.041–.044)	0.93	0.91	0.05
Metric	3961.7	1869	1.30	75.7	69	0.27				.040 (.039–.042)	0.93	0.92	0.05
Scalar	3994.2	1938	1.29	116.6	138	0.91	0.13	69	1.00	.039 (.038–.041)	0.93	0.93	0.05

Abbreviations. I-Pro-VC-Be: International Professionals’ Vaccine Confidence and Behaviors; Df: degrees of freedom; MLR: maximum likelihood with robust standard errors; RMSEA (90% CI): root mean square error of approximation with 90% confidence interval; CFI: comparative fit index; TLI: Tucker-Lewis index; SRMR: standardized root mean square residual; MG-CFA: multigroup confirmatory factor analysis.

^aThe MG-CFA structure included 10 factors allowed to correlate: safety (4 items after exclusion of the item regarding COVID-19 vaccines), complacency (3 items), benefit/risk balance (5 items), collective responsibility (2 items), trust in authorities (3 items), commitment to vaccination (3 items), self-efficacy (4 items), openness to patients (3 items), perceived constraints (3 items), and reluctant trust (3 items).

^bRobust ${\chi }^2$ proposed in [37; 38] for testing a restricted model against a less restricted one when estimating models using maximum likelihood with robust standard errors.

Models with CFI ≥ 0.90, TLI ≥ 0.90, RMSEA < 0.06, and SRMR < 0.08 were considered to fit reasonably or well [39].

Lecture: The configural model showed good fit (RMSEA = .042 (.041–.044); CFI = 0.93, TLI = 0.91, SRMR = 0.05). The comparison of the metric to the configural model, based on robust ${\chi }^2$ difference at alpha = 0.05, indicated no significant difference between these two models (P = 0.27), implying invariance between the two models and thus metric invariance. The comparison of the scalar to the metric model, based on robust chi-square difference, indicated no significant difference (P = 1.00), implying invariance between the two models and thus scalar invariance.

3.3. Tests of construct validity: convergent and discriminant validity

In the scalar MG-CFA, with item intercepts and factor loadings held equal across countries, eight of the 10 factors had good to excellent convergent validity (measured by the magnitude of item factor loadings) with all loadings ≥ 0.62 or ≥ 0.71 (P < 0.001, Figure 1). Convergent validity of the constructs for openness to patients (loadings from 0.47 to 0.67, P < 0.001) and perceived constraints (loadings from 0.43 to 0.76, P < 0.001) was fair.

Between-factor correlations in the scalar MG-CFA, which concerned discriminant validity, varied across countries: France, Finland, and Portugal showed only weak to moderate factor correlations, indicating good discriminant validity. The factor correlations were higher in Germany (Table 3), where vaccine safety perception, complacency, benefit-risk balance, collective responsibility, trust in authorities and, to a lesser extent, commitment to vaccination were moderately to highly correlated with each other (absolute ρ ranging from 0.45 to 0.82, P < 0.001); self-efficacy was also highly correlated with commitment to vaccination (ρ = 0.71, P < 0.001). Other correlations were moderate or weak.

Table 3. Correlations of the factors from the multigroup confirmatory factor analysis^a with scalar invariance, by country (n = 2,748).

	S	C	BRB	CR	TA	CV	SE	OP	PC	RT
France
S	1.00
C	−0.17	1.00
BRB	0.34	−0.49	1.00
CR	0.24	−0.26	0.40	1.00
TA	0.27	−0.30	0.42	0.33	1.00
CV	0.19	−0.29	0.48	0.29	0.28	1.00
SE	0.14	−0.18	0.34	0.25	0.26	0.58	1.00
OP	−0.11	0.38	−0.23	−0.22	−0.25	−0.02 (NS)	−0.05 (NS)	1.00
PC	−0.02 (NS)	0.26	−0.12	−0.05 (NS)	−0.11	−0.01 (NS)	−0.05 (NS)	0.23	1.00
RT	−0.21	0.44	−0.34	−0.16	−0.28	−0.27	−0.29	0.25	0.27	1.00
Finland
S	1.00
C	−0.37	1.00
BRB	0.43	−0.46	1.00
CR	0.19	−0.19	0.28	1.00
TA	0.40	−0.55	0.51	0.34	1.00
CV	0.05 (NS)	−0.28	0.18	0.32	0.18	1.00
SE	0.01 (NS)	−0.14	0.16	0.15	0.09 (NS)	0.65	1.00
OP	−0.12	0.21	−0.16	−0.06 (NS)	−0.13	0.15 (NS)	0.35	1.00
PC	−0.08 (NS)	0.14	−0.20	0.01 (NS)	−0.11	−0.04 (NS)	−0.08 (NS)	0.03 (NS)	1.00
RT	−0.12	0.10	−0.07 (NS)	−0.06 (NS)	−0.06 (NS)	−0.20	−0.16	−0.17	0.23	1.00
Portugal
S	1.00
C	−0.27	1.00
BRB	0.43	−0.50	1.00
CR	0.09 (NS)	−0.11 (NS)	0.25 (NS)	1.00
TA	0.17	−0.29	0.46	0.22	1.00
CV	0.22	−0.28	0.47	0.46	0.34	1.00
SE	0.13	−0.11 (NS)	0.29	0.24	0.22	0.42	1.00
OP	−0.09 (NS)	0.34	−0.12 (NS)	−0.06 (NS)	−0.10 (NS)	−0.03 (NS)	−0.01 (NS)	1.00
PC	−0.08	0.22	−0.10 (NS)	−0.08 (NS)	−0.08 (NS)	−0.08 (NS)	−0.14	0.31	1.00
RT	−0.06 (NS)	0.27	−0.14	−0.08 (NS)	−0.16	−0.11	−0.25	0.23	0.29	1.00
Germany
S	1.00
C	−0.55	1.00
BRB	0.75	−0.82	1.00
CR	0.54	−0.58	0.73	1.00
TA	0.55	−0.55	0.69	0.76	1.00
CV	0.52	−0.45	0.66	0.67	0.50	1.00
SE	0.31	−0.28	0.47	0.40	0.34	0.71	1.00
OP	−0.26	0.66	−0.37	−0.22	−0.32	−0.02 (NS)	0.12 (NS)	1.00
PC	−0.04 (NS)	0.22	−0.12	−0.03 (NS)	−0.09 (NS)	−0.11 (NS)	−0.13	0.33	1.00
RT	−0.13	0.28	−0.23	−0.14	−0.14	−0.27	−0.29	0.36	0.48	1.00

Abbreviations. S: Safety; C: Complacency; BRB: benefit/risk balance; CR: collective responsibility; TA: trust in authorities; CV: commitment to vaccination; SE: self-efficacy; OP: openness to patients; PC: perceived constraints; RT: reluctant trust; NS: not significant.

^aAll item intercepts and factor loadings were set to be held equal across countries, and the metrics of the factors were defined by fixing their variances to one in one country while freely estimated in the others. All correlations are significant at P ≤ 0.05 except when (NS) stated.

3.4. Test of criterion validity: associations with behavioral outcomes

Modified Poisson regression models showed, for the long-form version of the I-Pro-VC-Be, that HCPs with higher scores of safety perception, benefit/risk balance, collective responsibility, commitment to vaccination, self-efficacy, and trust in authorities were significantly more likely to report that they recommended vaccines to their patients nearly systematically (score >90%) than those with below-average scores (Table 4). For example, the adjusted relative risk (aRR) of reporting near-systematic vaccine recommendations was 2.46 (95% confidence interval (95% CI): [2.11, 2.87]) for HCPs with an above-average benefit-risk balance score. On the other hand, lower likelihoods of near-systematic vaccine recommendations were associated with above-average scores of complacency (aRR: 0.50; 95% CI: [0.42;0.60]), reluctant trust (aRR: 0.78; 95% CI: [0.71;0.86]) and perceived constraints (aRR: 0.89; 95% CI: [0.82;0.98]) (Table 4). Associations similar to those observed for near-systematic vaccine recommendations were found with self-vaccination against influenza, apart from perceived constraints, which were not associated with influenza vaccination. Associations between I-Pro-VC-Be factors and self-vaccination against COVID-19 (with booster) were notably weaker, albeit statistically significant, than for other vaccination behavior outcomes. Additionally, perceived constraints and self-efficacy were not associated with COVID-19 vaccination.

Table 4. Associations between vaccination behaviors and I-Pro-VC-Be factors among HCPs: results from multiple modified Poisson regressions with robust standard errors^a (n = 2,748).

	Long-form	Short-form^b
	aRR [95% CI]
I-Pro-VC-Be factors
Self-reported near-systematic (>90%) vaccine recommendation or intent-to-recommend
Safety perception > mean (ref. No)	1.79 [1.47;2.17]	1.78 [1.47;2.14]
Complacency > mean (ref. No)	0.50 [0.42;0.60]	0.53 [0.45;0.62]
Benefit/risk balance > mean (ref. No)	2.46 [2.11;2.87]	2.74 [2.11;3.56]
Collective responsibility > mean (ref. No)	1.85 [1.61;2.13]	1.82 [1.54;2.15]
Commitment to vaccination > mean (ref. No)	2.03 [1.77;2.33]	2.48 [2.13;2.90]
Self-efficacy > mean (ref. No)	1.60 [1.44;1.77]	1.49 [1.36;1.63]
Trust in authorities > mean (ref. No)	1.71 [1.48;1.97]	1.80 [1.57;2.06]
Openness to patients > mean (ref. No)	0.90 [0.81;0.98]	0.93 [0.85;1.02]
Reluctant trust> mean (ref. No)	0.78 [0.71;0.86]	0.77 [0.70;0.85]
Perceived constraints > mean (ref. No)	0.89 [0.82;0.98]	1.00 [0.90;1.11]
Self-vaccination against influenza: 3 times over the past 3 years (score = 3/3)
Safety perception > mean (ref. No)	1.27 [1.20;1.35]	1.23 [1.16;1.30]
Complacency > mean (ref. No)	0.74 [0.69;0.79]	0.80 [0.76;0.84]
Benefit/risk balance > mean (ref. No)	1.32 [1.26;1.39]	1.23 [1.15;1.32]
Collective responsibility > mean (ref. No)	1.17 [1.12;1.22]	1.24 [1.18;1.31]
Commitment to vaccination > mean (ref. No)	1.17 [1.12;1.22]	1.21 [1.15;1.27]
Self-efficacy > mean (ref. No)	1.17 [1.12;1.21]	1.14 [1.09;1.18]
Trust in authorities > mean (ref. No)	1.16 [1.11;1.22]	1.17 [1.12;1.23]
Openness to patients > mean (ref. No)	0.92 [0.88;0.96]	0.94 [0.90;0.98]
Reluctant trust> mean (ref. No)	0.90 [0.86;0.94]	0.92 [0.88;0.96]
Perceived constraints > mean (ref. No)	0.98 [0.94;1.02]	1.01 [0.96;1.05]
Self-vaccination against COVID-19: fully vaccinated + booster (score = 4/4)
Safety perception > mean (ref. No)	1.09 [1.06;1.13]	1.08 [1.05;1.12]
Complacency > mean (ref. No)	0.91 [0.89;0.94]	0.94 [0.92;0.97]
Benefit/risk balance > mean (ref. No)	1.10 [1.07;1.13]	1.08 [1.05;1.12]
Collective responsibility > mean (ref. No)	1.04 [1.02;1.06]	1.07 [1.05;1.10]
Commitment to vaccination > mean (ref. No)	1.04 [1.02;1.06]	1.08 [1.05;1.10]
Self-efficacy > mean (ref. No)	1.02 [1.00;1.04]	1.01 [0.99;1.03]
Trust in authorities > mean (ref. No)	1.05 [1.03;1.08]	1.06 [1.03;1.08]
Openness to patients > mean (ref. No)	0.97 [0.95;0.99]	0.98 [0.97;1.00]
Reluctant trust> mean (ref. No)	0.96 [0.94;0.98]	0.95 [0.93;0.97]
Perceived constraints > mean (ref. No)	0.99 [0.97;1.00]	0.99 [0.97;1.01]

Abbreviations. I-Pro-VC-Be: International Professionals’ Vaccine Confidence and Behaviors; HCPs: healthcare professionals; aRR [95% CI]: adjusted relative risk and 95% confidence interval.

^aAll models were run separately and adjusted for country, gender, age, and profession. Explanatory variables (I-Pro-VC-Be factors) were dichotomized at the sample mean. Alpha risk = 0.05; statistically significant results in bold.

^bItems included in the short-form in Table 5.

Table 5. Final short-form I-Pro-VC-Be tool to measure determinants of vaccine confidence in HCPs^a,b.

Dimension	Item
Perceived safety of vaccines	1. Vaccines are safe.
Complacency	2. Today, some vaccines recommended by [insert relevant authority] are not useful, because the diseases they prevent are not serious.
Perceived benefit/risk balance	3. The benefits of vaccines outweigh their potential risks.
Perceived collective responsibility	4. I recommend the vaccines on the vaccination schedule to my patients because it’s essential to contribute to protection of the population (community immunity).
Commitment to vaccination	5. I am committed to ensuring that my patients are vaccinated.
Self-efficacy	6. I feel sufficiently trained on how to bring up the question of vaccines with hesitant patients.
Trust in authorities	7. I trust the [insert relevant authority] to ensure that vaccines are safe.
Openness to patients	8. I inform my patients about the benefits and risks of vaccines without trying to influence them.
Perceived constraints	9. The cost of some vaccines is a problem for some patients and can keep me from prescribing them.
Reluctant trust	10. I may sometimes recommend vaccines from the official schedule even if I feel the vaccination policy is not sufficiently clear.

Abbreviations. I-Pro-VC-Be: International Professionals’ Vaccine Confidence and Behaviors; HCPs: healthcare professionals.

^aThe following response scale applies to all items: 1: strongly disagree, 2: somewhat disagree, 3: undecided, 4: somewhat agree, 5: strongly agree.

^bThe final long-form I-Pro-VC-Be can be found at [45]. Two generalized items for perceived vaccine (‘Vaccines are safe’) and the perceived benefit/risk balance of vaccines (‘The benefits of vaccines outweigh their potential risks’) were recommended after this analysis was completed to offer an adaptable tool for various country and cultural contexts. In the present short-form criterion validity test, the corresponding items used were 'Vaccines against human papillomaviruses are safe' and 'The benefits of the vaccine against hepatitis B outweigh its potential risks'

With the short-form version, similar associations were observed between vaccination behavioral outcomes and the I-Pro-VC-Be dimensions, with few exceptions: perceived constraints and openness to patients were not associated with near-systematic vaccine recommendations and COVID-19 (with booster) self-vaccination; nor was self-efficacy for the COVID-19 self-vaccination model (Table 4).

The final, validated I-Pro-VC-Be long-form tool with 43 items (10 regarding vaccination behaviors and 33 regarding psychosocial determinants) is publicly available for use in six languages [45]. Methods related to the calculation of scores used in this study for vaccine confidence determinants and vaccination behaviors can be found in section 2.5.6. The final, recommended I-Pro-VC-Be short-form tool consists of 10 items, one for each dimension of the long-form tool, as was recommended for the original Pro-VC-Be short-form [42]. Two generalized items for perceived vaccine (‘Vaccines are safe’) and the perceived benefit/risk balance of vaccines (‘The benefits of vaccines outweigh their potential risks’) were recommended after this analysis was completed to offer an adaptable tool for various country and cultural contexts (Table 5) [42].

4. Discussion

This study provides the adaptation and validation, across four European countries, of an international long- and short-form tool to measure vaccine confidence and behaviors in HCPs. The long-form I-Pro-VC-Be showed good to acceptable fit of the 10-factor CFA structure in each participating country (Finland, France, Germany, and Portugal), as well as strong factorial invariance between countries. These results indicate that it accurately measured the same determinants of the core and intermediary factors in the same manner across various country and cultural contexts, including among countries with differing professional norms surrounding vaccination. The results thus suggest that the questionnaire is suitable for conducting cross-national comparisons of determinants of vaccine confidence among HCPs. The existence and use of a standardized measure will also facilitate future meta-analyses of HCPs’ vaccine confidence and related topics, such as the effectiveness of interventions. Furthermore, the long- and short-form I-Pro-VC-Be vaccine confidence determinants showed good criterion validity with vaccination behavior outcomes regarding patients and HCPs themselves, but with lower effect estimates regarding HCP COVID-19 vaccination. The 10-item short-form tool of the I-Pro-VC-Be provides an efficient and less time-consuming method for measuring vaccination attitudes and behaviors in HCPs. These tools have been made easily accessible in their entirety and a template version is adaptable for use in other countries.

4.1. Sensitivity to professional norms: cultural variations

Researchers added items related to HCPs’ professional norms to this international survey to measure the extent to which potential social desirability bias varied across the participating countries. Previous studies have also shown the importance of professional norms in HCPs’ vaccination behaviors and suggest that pro-vaccine professional norms are associated with more frequent vaccine recommendations [46–48]. In our study, HCPs from Portugal and Finland showed the strongest acceptance of professional norm statements. Including these items to complement the I-Pro-VC-Be makes it possible to measure the variance of sensitivity to these norms between countries and cultures.

4.2. Reluctant trust: a new determinant of HCPs’ vaccine confidence

Another interesting result of the I-Pro-VC-Be was the presence of reported reluctant trust in all four participating countries (Table A5). Reluctant trust is considered the ‘leap of faith’ that lay people take when it comes to trusting health authorities, vaccination policies, and vaccines themselves, even if they still have doubts. Unlike other well-known determinants of HCPs’ vaccine confidence, such as trust in authorities or perceived risks of vaccines, this reluctant trust in vaccination was first explored among French GPs [49]; its negative influence on the likelihood of recommending certain vaccines to patients was then observed among French and Belgian GPs, and Quebec nurses [19]. The findings of the present study confirm that reluctant trust is present even among highly educated groups (physicians), is cross-cultural, and exists in societies with strong professional vaccination norms. This finding is important as it reflects HCPs’ hidden uncertainties or misunderstandings regarding vaccination [49] that could lead to VH if they are not adequately addressed.

4.3. The Pro-VC-Be: an instrument measuring constructive psychosocial resources of HCPs’ vaccination behavior

Both the original Pro-VC-Be and the I-Pro-VC-Be show good construct validity according to validated cut-offs and similar criterion validity for their corresponding long- and short-form tools [19,42]. Similar to the original Pro-VC-Be, the factors of perceived risks of vaccines, complacency, perceived benefit-risk balance of vaccines, perceived collective responsibility, commitment to vaccination, self-efficacy, and trust in authorities were consistently associated with vaccine behaviors in the long- and short-form version of the I-Pro-VC-Be. Many of these factors are examples of the ‘constructive’ psychosocial resources that HCPs can rely on in their daily immunization practices, whereas complacency, reluctant trust, and perceived constraints, could be considered barriers to these practices. The latter three factors may also be more sensitive to differing cultural contexts, health systems, and interventions, such as those targeting HCP educational training [42].

4.4. Short form version

For the dimensions of perceived vaccine safety and perceived benefit-risk balance, we propose the use of more generalized items within the I-Pro-VC-Be short-form tool rather than items focusing on specific vaccine situations, as we also did for the original Pro-VC-Be short-form [42]. Although vaccine-specific items are useful for representing a latent dimension when considering several vaccine situations, this specificity appears less consistent when only one item represents a dimension. For example, the use of a more general statement ‘Vaccines are safe,’ for the dimension of perceived vaccine safety is more appropriate for a single international item, due to potential variation in acceptance and perceived safety of specific vaccines across countries [50].

4.5. Strengths and Limitations

This work has several strengths, including most notably the large sample size of HCPs in our population across various countries in Europe. This study offers the first cross-cultural validation of both a long- and short-form tool, public available for use, to measure vaccine confidence and its determinants in HCPs. Researchers have made both the validated long- and short-form tools easily adaptable (i.e. by specifying where certain adaptations, such as target age group for a vaccine, should be made in the template version). The rigorous adaptation of the original Pro-VC-Be questionnaire through multidisciplinary expert opinion, professional language translation, cognitive validation efforts and pre-survey testing, allowed us to develop this cross-cultural version of the Pro-VC-Be. This is reflected in the strong factorial invariance of the instrument between the four participating countries. Additionally, the statistical analyses involved in the validation of this international tool, as well as of its original long and short-form tools [19,42], provide detailed steps that can be replicated by future researchers. Lastly, another strength of this work is the inclusion of hypothetical ‘vaccination recommendation intention’ situations, which help to capture the attitudes and behaviors of HCPs who are not directly involved in prescribing vaccines to patients, but still have an important supporting and educating role.

This study also has some limitations, such as potential differences in vocabulary due to the need for translations for all participating countries. However, double back-translation of surveys and preceding materials was completed to control for this as much as possible. Future researchers should pay careful attention to translation of items to maintain these tools’ integrity. The I-Pro-VC-Be, while validated in several European countries, has not been validated in nonWestern populations. Accordingly, it may require new validation studies to assess the suitability of this instrument for use in these countries where the weight of psychosocial factors that contribute to vaccine confidence in HCPs, and the difficulties in access to vaccination, may be very different [51]. Because of insufficient sample sizes for some HCP types, the I-Pro-VC-Be analyses simply controlled for profession rather than stratifying by profession, as was done in the original Pro-VC-Be to validate its use among GPs and nurses. Another limitation is the self-reported behavioral measurements of HCPs, and more particularly the self-reported hypothetical behaviors in the vaccine recommendation intention questions, which might be influenced by desirability bias. Due to data quality issues, the data collected in the UK had to be excluded from the principal analyses. However, this did not pose a substantial issue for overall analyses as the sample sizes from other participating countries were large enough to ensure adequate power. Thus, analyses remained robust enough for cross-national comparisons across four different countries. Additionally, sensitivity analyses including 118 UK participants with reliable data showed similar results in the MG-CFA model. There is also potential for selection bias due to the recruitment of participants solely online [52]. Participants in all countries had high vaccine confidence with generally little response variation between participants, especially in Finland and Portugal. This might have resulted in underestimating the associations between vaccine attitudes and behaviors. Finally the discriminant validity for the German sample was lower than in the other participating countries, with higher factor correlations between vaccine safety perception, complacency, benefit-risk balance, collective responsibility, and trust in authorities, on the one hand, and between self-efficacy and commitment to vaccination on the other. Similar results in the original Pro-VC-Be led us to construct two second-order factors with, this time, good discriminant validity: ‘vaccine confidence’ included vaccine safety perception, complacency, benefit-risk balance, and collective responsibility, while ‘proactive efficacy’ was measured by self-efficacy and commitment to vaccination. Thus, we recommend that researchers apply a similar logic when analyzing factor correlations: a second-order tool may be more appropriate to their data than a first-order one.

5. Conclusion: use of I-Pro-VC-Be

The ability to assess the psychosocial factors of vaccine confidence and their influence on vaccination behaviors in a systematic and culturally aware manner is important for the development and selection of interventions to increase HCPs’ confidence in vaccines. The original Pro-VC-Be has already been used to assess the impact of motivational interview interventions on vaccine confidence in French medical interns and showed a strong impact on self-efficacy and trust in authorities [16].

Our final, international short-form tool provides a timely and cost-effective solution for measuring these factors across HCP types. Due to its cross-country adaptability, the long- and short-form I-Pro-VC-Be can be used for cross-national comparisons of vaccine attitudes and behaviors among HCPs: they allow their reliable measurements even in the presence of cultural and other contextual variations, such as differing professional norms among HCPs. This aspect is useful for investigating, for example, how country characteristics (i.e. health policy, education, social cohesion) may affect HCPs’ vaccine confidence and behaviors.

Declaration of interests

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

PV, ED, and AG were involved in the conception and design of the original, long-form Pro-VC-Be. AmG, LK, AF, FR, PS, FT, DH, SL, PN, EA, AS, and PV were involved in the international adaptation of the original tool; including translation, cognitive interviews, and the pilot study. PV, AS, LK, and AmG were involved in the organization and implementation of data collection in the international populations. LF, PV, and AmG were involved in the statistical analysis, interpretation of the data, and initial drafting of the paper. AG, ED, AF, LK, FR, FT, DH, PN, EA, SL, AS, and PS were involved in the interpretation of the data and revising the paper critically for intellectual content; they approved it for publication. All authors agree to be accountable for all aspects of the work.

Ethical Considerations

The ethics boards of the University of Aix-Marseille (reference 16 December 2021-01), the Health Care Division of the Ethics Committee for Human Sciences at the University of Turku (1/2022), the advisory board on ethical issues of the University of Erfurt (20210713), the Ethics Committee of the Faculty of Medicine of the University of Coimbra (093-CE-2021), and the School of Psychological Sciences Research Ethics Committee (SREC) of the University of Bristol (119594) approved the original study protocol and questionnaires in their respective countries. The survey was conducted exclusively online, participants were asked to give consent before being directed to the platform where the survey took place in each country.

Acknowledgments

We thank the following experts for their advice on the original version of the Pro-VC-Be questionnaire: C Betsch, M Deml, KB Habersaat, J Leask, and JK Ward. We also thank all partners within the JITSUVAX project (Jiu Jitsu with Misinformation in the Age of Covid) and the survey participants in all participating countries. We thank the various networks in each country from which participants were recruited for their involvement: the National Institute for Health Research’s (NIHR) Clinical Research Network (CRN) in the UK, the Schlesinger Group Germany provider in Germany; the Portuguese Society of Pediatricians and the Portuguese Association of General Practice and Family Medicine in Portugal; the National College of General Practice Teachers (CNGE acronym in French) in France; and The Finnish Medical Association in Finland. We thank Jo Ann Cahn for her English proofreading of the revised version of this article.

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/14760584.2023.2242479.

Additional information

Funding

This manuscript and the entire JITSUVAX project was funded by the European Union’s Horizon 2020 research and innovation program under Grant agreement number 964728 (JITSUVAX).