An instrument to measure psychosocial determinants of health care professionals’ vaccination behavior: Validation of the Pro-VC-Be questionnaire

ABSTRACT

Objectives

The lack of validated instruments assessing vaccine hesitancy/confidence among health care professionals (HCPs) for themselves, and their patients led us to develop and validate the Pro-VC-Be instrument to measure vaccine confidence and other psychosocial determinants of HCPs’ vaccination behavior among diverse HCPs in different countries.

Methods

Cross-sectional survey in October-November 2020 among 1,249 GPs in France, 432 GPs in French-speaking parts of Belgium, and 1,055 nurses in Quebec (Canada), all participating in general population immunization. Exploratory and confirmatory factor analyses evaluated the instrument’s construct validity. We used HCPs’ self-reported vaccine recommendations to patients, general immunization activity, self-vaccination, and future COVID-19 vaccine acceptance to test criterion validity.

Results

The final results indicated a 6-factor structure with good fit: vaccine confidence (combining complacency, perceived vaccine risks, perceived benefit-risk balance, perceived collective responsibility), trust in authorities, perceived constraints, proactive efficacy (combining commitment to vaccination and self-efficacy), reluctant trust, and openness to patients. The instrument showed good convergent and criterion validity and adequate discriminant validity.

Conclusions

This study found that the Pro-VC-Be is a valid instrument for measuring psychosocial determinants of HCPs’ vaccination behaviors in different settings. Its validation is currently underway in Europe among various HCPs in different languages.

KEYWORDS:

• Health care professionals
• questionnaire validation
• vaccination behaviors
• vaccine confidence
• vaccine hesitancy

1. Introduction

In 2019, the WHO ranked vaccine hesitancy, then defined as delay in acceptance or refusal of vaccination despite availability of vaccination services [1], as one of the 10 most important health threats in the world [2]. Since then, the WHO Behavioral and Social Drivers of Vaccinations working group has proposed an updated definition of hesitancy: ‘[m]otivational state of being conflicted about, or opposed to, getting vaccinated; including intentions and willingness’ [3]. Vaccine hesitancy depends on context and vaccine situations (both the specific vaccine and the target population). In the current COVID-19 pandemic, vaccine hesitancy remains the main barrier to achieving sufficiently high vaccine coverage against this infection and its emerging variants to allow collective protection, at least in countries with a large vaccine supply [4].

Vaccine hesitancy also affects health care professionals (HCPs) in many countries [5,6], including for the new COVID-19 vaccines [7]. For example, HCPs’ intention to accept future COVID-19 vaccines in 2020 ranged from 28% in China to 77% in France with strong heterogeneity depending on HCP type, setting, and country [8]. Vaccine hesitancy among HCPs is of particular concern for several reasons. First, it influences their acceptance of recommended vaccines for themselves and results in higher risks of transmission of vaccine-preventable diseases to patients as well as higher HCP absenteeism in an epidemic. Second, HCPs’ vaccine hesitancy reduces their willingness to recommend vaccines to their patients [9–12]. Third, patients strongly trust their HCPs about vaccination. Accordingly, these professionals are role models for patients and play a major role in the vaccination of the general population and at-risk groups, by addressing their patients’ concerns about vaccines and motivating them to be vaccinated [13]. HCPs’ vaccine hesitancy may negatively affect their ability to perform these tasks and appropriately address their patients’ vaccine hesitancy.

Identifying which HCPs are vaccine hesitant, quantifying the prevalence of vaccine hesitancy according to different socio-professional characteristics, understanding its nature and determinants, as well as its consequences on HCPs’ behaviors for themselves and their patients, are essential public health and research aims. Instruments have been developed to measure general vaccine hesitancy [14,15] and, during the pandemic, to measure COVID-19 vaccine hesitancy [16,17] in the general population, in particular concerning vaccines for children [3]. However, a bibliographic search conducted by our team in July 2021 on vaccine hesitancy among HCPs (see Table S1 Appendix A1 for more details) showed a lack of instruments specifically validated to assess vaccine hesitancy and other potential psychosocial determinants of vaccine behaviors for various vaccine situations and for different types of HCPs, settings (hospital, community, etc.), and countries.

We developed a new instrument, the Pro-VC-Be (Health Professionals Vaccine Confidence and Behaviors) questionnaire (henceforth the Pro-VC-Be). We have chosen to focus on vaccine confidence rather than on its related concept vaccine hesitancy because we feel that, from an intervention perspective, this approach would both facilitate dialogue with HCPs for whom vaccine hesitancy may be a sensitive issue and encourage vaccine confidence among them. Different researchers have used the concept of vaccine confidence in different ways [9,12,14,18]; we will discuss similarities of the Pro-VC-Be and previous definitions as well as theoretical concepts from health behavior theories in the next section.

The objective of the Pro-VC-Be is thus to measure various psychosocial factors that may play a role in vaccine confidence and vaccination behavior among different types of HCPs. This questionnaire was developed in French, so that it could be used in high-income French-speaking countries. The aims of this article are to present its development, its theoretical basis, and the results of its construct and criterion validation studies.

2. Material and method

2.1. Theoretical framework and models on which the Pro-VC-Be is based

The Pro-VC-Be is based on three theoretical frameworks or models (Table S2 Appendix A2). We relied on the Theoretical Domain Framework, a consensus approach for the development of a theoretical framework of constructs that may be relevant in vaccine-related intervention studies of HCPs focused on evidence-based practices. This framework is composed of a synthesis of theories of behavior and behavior change, clustered into 14 domains [19–21].

We also applied the Health Belief Model (HBM) hypotheses, in particular, the postulate that the adoption of preventive behaviors depends on their perceived benefits and risks, considered in light of the disease’s perceived severity [22]. Previous findings about vaccination behavior among both the general public and HCPs show that the HBM is strongly predictive of both vaccine acceptance and uptake [5,13,23,24].

Finally, we based some of the dimensions and items of the Pro-VC-Be on the 5C model for vaccine hesitancy: it is an extension of the 3C model [1], which postulates that this hesitancy results from a combination, varying with the context and the individual, of complacency (not perceiving diseases to be at high risk), (lack of) trust (in vaccines and the system that delivers them), and/or (lack of) convenience (anything that facilitates access to vaccines). The 5C focuses on confidence, complacency, constraints (perceived structural and psychological barriers to vaccination), calculation (engagement in extensive information searching), and collective responsibility (willingness to protect others) [14].

Although these frameworks/models differ in their definitions of concepts, they also overlap. For instance, some constructs of the 5C are similar to concepts of the HBM, while certain constructs in the Theoretical Domain Framework have counterparts in the 5C (see paragraph 2.2.2). Moreover, each of them sheds additional light on empirical results from our previous quantitative and qualitative studies investigating vaccine hesitancy among HCPs [12,25–27].

2.2. Description of the Pro-VC-Be’s domains

The Pro-VC-Be contains categories of questions intended to (Figure S1 Appendix A3): (a) probe vaccination behavior, (b) measure the core determinants of vaccine confidence and of vaccination behaviors identified in previous studies [9,12,14,18], and (c) examine other psychosocial factors (to be referred to hereafter as ‘intermediary factors’) that are expected to predict vaccine confidence and vaccination behavior, or to moderate or mediate relations between vaccine confidence and vaccination behavior.

2.2.1. Vaccination behavior

HCPs’ vaccination behavior has two principal elements (Table 1): their recommendations (or lack thereof) to their patients and their self-vaccination behavior (personal vaccine uptake). Recommendation-related behavior was measured in two ways: by items concerning immunization activity in general and by items concerning specific vaccines, populations, and vaccine contexts. The questions about immunization activity (raising the subject with patients, recommending and prescribing vaccines) can be used without reference to a specific vaccine situation. Specific questions are used to examine specific vaccine situations, by focusing for example on vaccines with suboptimal uptake in some populations and contexts. Self-vaccination behavior is measured separately for different vaccines that are recommended to HCPs in their work environment. Finally, the Pro-VC-Be addressed HCPs’ attitudes toward COVID-19 vaccines, before they were marketed.

Table 1. Set of items used in the pro-VC-Be questionnaire

VARIABLES
OUTCOMES
VACCINATION BEHAVIORS	General vaccination behavior	B1. With the patients you treat:
		a. How often do you bring up the subject of vaccination?
		b. How often do you recommend the vaccines that are indicated for them?
		c. How often do you prescribe indicated vaccines to them?
	Vaccine recommendation frequency	B3. How often do you recommend the following vaccines?
		A. Catch-up MMR for adolescents
		B. Pertussis vaccine in pregnancy (Quebec)/pertussis vaccine in mothers who have just given birth, if not vaccinated before pregnancy (France)
		C. Meningitis C vaccine at 12 months of age
		D. Human papilloma virus vaccine in young girls and boys aged 11 to 14 years old
		E. Catch-up hepatitis B vaccine in adolescents
		F. Seasonal flu vaccine in adults under 65 years old with chronic illness
	Stated willingness to accept future COVID-19 vaccines	I. If a Covid-19 vaccine were available (Oct-Nov 2020):
		1. Would you agree to recommend it to your patients?
		2. Would you agree to vaccinate yourself?
	Self-vaccination behavior	VP1. Were you vaccinated against seasonal influenza for the winter 2019–2020 season?
		VP11. For this coming winter (2020–2021), do you intend to be vaccinated against seasonal influenza? (France and Belgium only)
		VP2. Have you had a pertussis vaccination booster dose during your adult life in the past 20 years? (France only)
EXPLANATORY VARIABLES
CORE DETERMINANTS OF VACCINE CONFIDENCE AND OF VACCINATION BEHAVIORS	Perceived risks of vaccines	R1. Some vaccines can cause autoimmune diseases
		R2 The measles vaccine can cause autism in children
		R3 Some vaccines can cause multiple sclerosis
		R4 A vaccine developed in an emergency situation [for example, during an epidemic like that of COVID-19) does not offer a sufficient guarantee of safety
	Complacency	U1 Today, some vaccines recommended by authorities are not useful, because the diseases they prevent are not serious
		U2 Children are vaccinated against too many diseases
		U3 Children are vaccinated at too young an age
		U4 It is preferable to acquire immunity against infectious diseases naturally (by having the disease) than by vaccination
	Perceived benefit/risk balance of vaccines	BRB1 The benefits of the vaccine against measles are much greater than its potential risks
		BRB2 The benefits of the vaccine against influenza in people with a chronic disease are much greater than its potential risks
		BRB3 The benefits of the vaccine against hepatitis B in infants (or as catch-up in adolescents) are much greater than its potential risks
		BRB4 The benefits of the vaccine against human papillomaviruses are much greater than its potential risks
	Perceived collective responsibility	CR1 I recommend the vaccines on the vaccination schedule to my patients because it’s essential to contribute to protection of the population (community immunity)
		CR2 I recommend the vaccines in the official schedule to my hesitant patients, explaining to them the importance of community immunity
	Trust in authorities to inform about vaccines	C1 I trust the ministry of health to provide reliable information about the risks and benefits of vaccines
		C2 I trust the bodies that develop these guidelines (French national authority for health)
		C3 I trust science to develop safe effective new vaccines
		C4 I trust the ministry of health to establish the vaccination strategy
		C5 I trust the ministry of health to ensure that vaccines are safe
		C6 I am convinced that the ministry of health is not influenced by the pharmaceutical industry
	Perceived constraints to vaccinating patients	PC1 The cost of some vaccines is a problem for some patients and can keep me from prescribing them
		PC2 The lack of availability of certain vaccines is often a problem that can keep me from prescribing them to my patients.
		PC3 Not having vaccines in my office is a problem in my practice
		PC4 The legal and professional responsibilities associated with immunization practices are excessive
VARIABLES
INTERMEDIARY FACTORS BEHIND HCPS VACCINE CONFIDENCE AND VACCINATION BEHAVIORS	Openness to patients’ concerns regarding vaccination	OP1 Patients who are hesitant about the benefits and risks of vaccines have legitimate questions
		OP2 I inform my patients about the benefits and risks of vaccines but I let them make their decision without trying to influence them
		OP3 It is counterproductive to try to convince vaccine-hesitant patients to be vaccinated
		OP4 I think that patients have a duty to be vaccinated
		OP5 I am willing to let parents delay immunizing their children
	Professional norm	SN It is my professional responsibility to ensure that all of my patients without contraindications are vaccinated
	Commitment to the vaccination of their patients	CV1 I am actively involved in ensuring that my patients are vaccinated
		CV2 I do my best to motivate patients who are hesitant to get vaccinated
		CV3 I am committed to keeping my knowledge about vaccination up-to-date (CME, conferences, reading)
		CV4 I am committed to developing the skills needed to communicate better with my patients about vaccination
	Self-efficacy in addressing hesitancy	SE1 I feel comfortable advising my patients about the risks and benefits of vaccines
		SE2 I feel comfortable discussing vaccines with my patients who are highly hesitant about vaccination
		SE3 I feel sufficiently trained and informed to discuss vaccines with all patients
		SE4 I feel sufficiently trained on how to approach the question of vaccines with hesitant patients
	Reluctant trust	RT1 I recommend the vaccines in the official schedule even though I sometimes feel that I am not sufficiently informed about some of them
		RT2 I recommend the vaccines in the official schedule even though I feel that the objectives of the vaccination policy are not clear enough
		RT3 I recommend vaccines on the official schedule although I sometimes have doubts about their safety

2.2.2. Core determinants of vaccine confidence and of vaccination behavior

The Pro-VC-Be questionnaire probes core determinants (or dimensions) of vaccine confidence and of vaccination behavior among HCPs. Although we have previously investigated the role that some of these determinants play for some HCPs [9,12], little information about the applicability of others to them is available. Based on behavioral theories applied in the field of immunization (with their correspondences summarized in Appendix A2), the dimensions described below are included in the Pro-VC-Be (Table 1).

Perceived risks of vaccines: This dimension measures the risks that HCPs perceive in vaccination in general and the risks of specific vaccines that have been controversial (a sample item is ‘Some vaccines can cause autoimmune diseases’). It echoes the ‘preventive action risk perception’ dimension in the HBM as well as the ‘confidence in vaccine’ dimension of the 5C [14]. Previous studies suggest that safety concerns are an important dimension of vaccine hesitancy among the general public [1,14,18,24,28] as well as among HCPs [9,12]. Among the latter, it is related to a lower likelihood of recommending vaccines to patients [9,12,29].

Complacency: This dimension measures HCPs’ perception of the usefulness of vaccines on official immunization schedules (e.g. ‘Today, some vaccines recommended by authorities are not useful, because the diseases they prevent are not serious’). It reflects the perceived lack of risk of vaccine-preventable diseases that comprises the ‘complacency’ dimension of the 5C model [14] and the ‘preventive action benefits perception’ dimension in the HBM as well. A study among French GPs showed that 25% perceived some officially recommended vaccines as not useful; this perception was the strongest variable associated with less frequent vaccine recommendations [12,26]. Negative attitudes about the benefits of vaccines have also been associated with less frequent vaccine recommendations by Finnish HCPs [9].

Perceived benefit/risk balance: We included this dimension in the Pro-VC-Be because, according to the HBM model, it is central to understanding the adoption of prevention behaviors (e.g. ‘The benefits of the vaccine against measles are much greater than its potential risks’). This dimension also echoes the ‘calculation’ dimension from the 5C model. Studies showed it is strongly associated with GPs’ personal vaccination against A/H1N1 influenza [30]. Few studies, however, have evaluated its association with HCPs’ vaccine recommendation behavior [5].

Perceived collective responsibility: This dimension probes the extent to which HCPs recommend vaccines from the official schedule to their patients because they want to contribute to community immunity (e.g. ‘I recommend the vaccines on the vaccination schedule to my patients because it’s essential to contribute to protection of the population (community immunity)’). Some research also defines social benefits of vaccination as an expression of ‘benefits’ within the HBM [31]. Collective responsibility has been identified as an important factor behind vaccine confidence [14], but its role in HCPs’ vaccination behavior is unknown.

We expect that the dimensions of perceived vaccine risks, benefit/risk balance of vaccines, and collective responsibility, as well as complacency, can be used as direct measures of HCPs’ vaccine confidence (Figure S1 Appendix A3).

Trust in authorities: This dimension encompasses trust in institutions and health authorities as reliable sources of information on the benefits and safety of vaccines, and trust in them to define the vaccine strategy (e.g. ‘I trust the ministry of health to establish the vaccination strategy’). This dimension reflects HCPs’ perception of their practice environment – in particular the extent to which it is supportive – and resembles the 5C confidence dimension [14]. The latter assesses the extent to which people have confidence in the authorities to make decisions that are in the best interests of the population. Studies showed that trust in authorities is a central determinant of vaccine confidence, among both the general population [1,14,18,32] and HCPs [26,33]. Nonetheless, the extent to which it directly affects HCPs’ vaccine recommendation behaviors independently of vaccine confidence is not clear.

Perceived constraints: External constraints (e.g. difficulties in access to vaccines) may result in public vaccine hesitancy [1,14,18]. Transposed to HCPs, these constraints could arise from their working conditions and limit their commitment to vaccination of their patients (e.g. ‘Not having vaccines in my office/surgery is a problem in my practice’). Perceived constraints thus echo the 5C model’s dimension of ‘convenience’ [14]. We currently know too little about the role of perceived barriers in determining vaccine confidence among HCPs and inflecting their vaccine recommendation behavior, in particular, whether these constraints go beyond posing pragmatic barriers to affect HCPs’ attitudinal vaccine confidence.

We expect that trust in authorities and perceived constraints may reflect more contextual dimensions than the previous dimensions and have thus considered them separately among the core determinants (Appendix A3).

2.2.3. Intermediary factors behind HCPs’ vaccine confidence and vaccination behavior

The Pro-VC-Be also includes questions aimed at intermediary factors underlying HCPs’ vaccine confidence and vaccination behavior (Table 1 and Appendix A2). It remains to be seen whether these factors influence vaccination behavior directly, independently of vaccine confidence, or whether they moderate or mediate the links between vaccine confidence and vaccination behavior.

Self-efficacy: this dimension examines how well-equipped and prepared HCPs feel, in terms of knowledge and skills, to address vaccination with their patients (e.g. ‘I feel comfortable advising my patients about the risks and benefits of vaccines’). Self-efficacy is an essential domain of the Theoretical Domain Framework [19] and of other models of behavior change [18] because it may be amenable to change by various interventions. Based on the results from our previous research among GPs [12,26], higher self-efficacy can be expected to increase the likelihood of recommending vaccines to patients.

Commitment to the vaccination of their patients: this dimension probes the extent to which HCPs are devoted to and proactive in motivating their patients to accept vaccinations (e.g. ‘I am actively involved in ensuring that my patients are vaccinated’). This dimension resembles the empowerment domain of the Theoretical Domain Framework [19], a dimension that may be amenable to change by interventions. Its role in HCPs’ attitudes and behaviors has not often been investigated [34]. Commitment to vaccination may be a strong predictor of higher HCP acceptance of seasonal influenza vaccination and vaccination advocacy to patients [34]. But this remains to be confirmed for various types of health professionals, vaccine situations, and recommendation behavior.

Professional norm: This dimension measures HCPs’ sensitivity to both professional and social norms regarding vaccination, because professionals may be influenced by their peers (e.g. ‘Most of my colleagues recommend all vaccines to their patients’) as well as by guidelines [5]. It matches the social/group norms of the Theoretical Domain Framework [19]. Previous results regarding this dimension suggest that more pro-vaccine professional norms are related to higher vaccine confidence and a higher likelihood of HCPs recommending vaccines to patients [25].

Reluctant trust: Reluctant trust is conceptualized to describe the ‘leap of faith’ that laypeople make about expert systems and technologies that are not under their direct scrutiny [32]. This construct is intended to measure the extent to which HCPs might trust the vaccination system despite the concerns they have about some vaccines or the system itself (e.g. ‘I recommend vaccines on the vaccination schedule even though I sometimes have some doubts’). This phenomenon has been observed among French GPs [33]. Reluctant trust is important to assess, because it may allow HCPs who feel uncertain about some aspects of vaccines to recommend them nonetheless to their patients, but may also restrain them from systematically recommending them (ceiling effect). Uncertainties about vaccines or the system could be frequent among HCPs, because they are not scientific experts in vaccination [18,35]. Reluctant trust is thus hypothesized to act as a moderator between vaccine confidence and vaccination behavior.

Openness to patients’ concerns: this dimension measures HCPs’ attitudes toward vaccine-hesitant patients (e.g. ‘Patients who are hesitant about the benefits and risks of vaccines have legitimate questions’). We included this dimension because our previous qualitative surveys of physicians in France suggest that some physicians are more open to patients’ concerns and accept patients’ vaccine reluctance or requests (such as spreading vaccines out over time) with less difficulty than others do [25]. This is an important point because the international literature, in particular the literature on motivational interviewing, indicates that HCPs’ attitudes toward their patients (listening, empathetic, nonjudgmental, etc.) matter for the occurrence of behavior change [36]. Based on our empirical results above, HCPs with less vaccine confidence may be expected to be more open to patients’ concerns, and this openness might be related to a lower likelihood of recommending vaccines to vaccine-hesitant patients.

2.3. Stages of the instrument’s development and validation

After the development of the initial version of the instrument in French, two professional translators translated it into English (translation conflicts were solved by discussion). Then we submitted the English version to 5 international experts in the field of vaccine hesitancy (CB, JL, MD, KBH, JRW, see acknowledgments) for an evaluation of its content validity. In their feedback, the experts 1) asked for clarifications of the theoretical and empirical bases for the construction of the instrument; 2) proposed several additional items to include in the collective responsibility dimension as well as extensions of the measurement of HCPs’ perceived constraints to those related to daily stress (5C short version [14]), and to perceived legal and professional constraints in their vaccination activity, and finally extending the openness to patients dimension (e.g. willingness to let parents delay the vaccination of their children); 3) suggested the rewording of some items to make them more comprehensible (reluctant trust); and 4) advised harmonization of the response scales for all the dimensions (excluding the behavioral variables requiring ad hoc response scales) to facilitate score calculation.

We discussed and took these remarks and recommendations into account. For the collection of responses, we adopted a 5-level Likert scale (from ‘strongly disagree’ to ‘strongly agree’ with a ‘don’t know’ option) for all dimensions (excluding behavioral variables). All changes were made in the English version of the questionnaire. A second round of feedback from the experts validated the basis and content of the instrument and the changes made. At the end of this stage, two professionals back-translated the Pro-VC-Be into French.

We then proceeded in three stages: cognitive validation, pilot test with 144 HCPs, and survey with a large sample for validation.

Cognitive validation. We conducted a cognitive validation in March-April 2020 to verify that the way the questions were understood corresponded to the meaning we intended them to have. Four nurses in Quebec and six GPs in France participated. Two professional interviewers (one in France and the other in Quebec) conducted the interviews face to face or by telephone. Some nurses completed them in writing. In the interviews, respondents were asked for information about how they understood each question in the questionnaire and why.

The results showed the need to make the following modifications: 1) add a ‘does not apply to my practice’ option to answer behavioral questions (nurses in Quebec); 2) improve the wording of some questions or reword them (e.g. in a question on social norms, replace the expression ‘requirement of authorities’ by ‘professional responsibility’); and 3) adapt some questions to the country context (National Institute of Public Health responsible for vaccination in Quebec, Ministry of Health in France). Five questions were deleted because they were not understood, or not appropriate (e.g. ‘daily stress prevents me from vaccinating,’ from the 5C), not adapted to a country (no mandatory vaccination in Quebec), or could not be answered (‘most of my colleagues recommend all vaccines in the official vaccination schedule to their patients’ in the professional norm dimension).

Pilot test. The questionnaire was pilot-tested online in May and July 2020 with 144 HCPs (53 French GPs, 59 Belgian GPs, and 32 Quebec nurses). Given the epidemic then underway, and the already high prevalence of reluctance toward future COVID-19 vaccines in France [37], we added, before pilot-testing, 7 questions investigating: attitudes toward future COVID-19 vaccines (willingness to be vaccinated and to recommend the vaccines to patients); perceived safety of vaccines developed during an epidemic; trust in science to develop new, safe, and effective vaccines; and, in France only, history of influenza vaccination in 2019–2020; intention to be vaccinated against influenza in 2020–2021; and pertussis booster vaccination within the past 20 years. At the end of the cognitive validation, the Pro-VC-Be included 72 questions, while the version that underwent pilot testing had 79 items and took on average 15 minutes to complete.

We then computed a principal component analysis (PCA) to explore the dimensionality of each theoretical construct and Cronbach’s alpha coefficients to measure their internal consistency. After we excluded 24 items that were found to be multidimensional and/or lowered Cronbach’s alpha coefficient, the questionnaire comprised 55 questions: 41 for the core and intermediary factors and 14 for vaccination behaviors (Table 1).

2.4. Survey of HCPs

2.4.1. Design and population

A cross-sectional survey collected data from GPs in France and French-speaking parts of Belgium (Brussels and Wallonia), and nurses in Quebec (Canada) with the Pro-VC-Be in October-November 2020. These professionals were chosen because they are involved in general population immunization. In Quebec, nurses prescribe and administer almost all vaccines, without GP supervision. Both groups are also targeted as a priority group for COVID-19 vaccination [38–41]. In France, the survey took place among a nationally representative panel of 2,815 private (i.e. nonsalaried) GPs, set up in 2018, constructed by random selection of GPs from an exhaustive database of health professionals (national directory of health professionals), and representative for age, gender, region, workload, and HCP density in their practice zone [10]. In Quebec, we randomly selected 4,000 nurses from the list of the Quebec Order of Nurses: all those practicing in Quebec with an e-mail address available at the time of the survey were eligible (n = 3,973). In Belgium, we invited all GPs practicing in the regions of Brussels and Wallonia (n = 8,412), identified through the databases of several organizations (such as the order of GPs and various learned societies).

2.4.2. Data collection procedure

In France, we used a sequential mixed-mode design to collect the data: participants were invited to take part online, received 5 emailed reminders over 4 weeks on different days of the week and at different times of day, and were then contacted by telephone if they had not completed the survey. In Belgium and Quebec, the survey took place exclusively online with a similar reminder protocol.

The ethics boards of the University-Hospital-Center Saint-Pierre (Belgium, CE/20-10-14), the University of Aix-Marseille (France, 2020–12-03-010), and the University-Hospital Center of Québec–Laval University (Québec, #2021-5286) approved the study protocol and questionnaire.

2.4.3. Sample size

With a total sample size of 3,000 participants, we would obtain a 73:1 subject to item ratio regarding core and intermediary factors, which is well above the recommended minimum 20:1 ratio for computing exploratory or confirmatory factor analysis [42].

2.5. Statistical analyses

We weighted the data from the cross-sectional survey to match the sample to the national French GP, Belgian GP, and Quebec nurse populations for age, gender, and region; weighting for French GPs also matched them to the national population for workload and GP density in their practice area [10,43].

We coded the Likert scale answers for the items belonging to the vaccine confidence core and intermediary factors as follows: ‘strongly disagree’ = 1 to ‘strongly agree’ = 4. As the ‘don’t know’ option was separate from the Likert scale responses, we performed, for each dimension of the Pro-VC-Be, multiple correspondence analyses on the corresponding items to assess the correlations between ‘don’t know’ answers and the other response categories. We recoded the ‘don’t know’ answers between 1 and 4, according to the results of this multiple correspondence analysis (see Appendix B1).

We also conducted a sensitivity analysis by coding ‘don’t know’ answers as missing values (see Appendix B1.1).

For some of the behavioral factor items (vaccine recommendation frequency, collected with a Likert scale from 1 = ‘never’ to 4 = ‘always’), an option ‘does not apply to my practice’ was proposed, given the differences in HCPs’ roles between the participating countries. These answers, mainly observed among Quebec nurses (6 to 25% according to the vaccine situation), were coded as missing values.

2.5.1. Exploratory factor analysis

To evaluate the construct validity of the Pro-VC-Be, we randomly divided the sample into two halves to obtain a test and a validation sample.

We conducted an exploratory factor analysis (EFA) in the test sample, on the entire set of core determinants of vaccine confidence and of intermediary factors, following the recommendations of [42,44,45] (see Appendix B2.1 for more details, in particular, on the rotations used). We considered multiple criteria to evaluate the number of factors to retain: eigenvalues greater than 1, goodness of fit of the EFA model, and the interpretability of the factors, given the theoretical bases presented above [42,44]. Model fit was assessed with the root mean square error of approximation (RMSEA), the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the standardized root mean square residual (SRMR). Models with CFI ≥0.95, TLI ≥0.95, RMSEA <0.06, and SRMR <0.08 were considered to fit reasonably or well [46].

2.5.2. Item reduction analysis

Then, to ensure that only parsimonious, functional, and internally consistent items were included in each factor, we used two successive methods to reduce the number of items. First, based on the EFA results, we excluded items with loadings <0.40; we found no item that should have been excluded because of cross loading on two or more factors (loadings ≥0.32) [42]. Second, considering the remaining items for each factor, we computed adjusted item-total correlations to explore the correlations between each item and the sum score of the rest of the items loading on the same factor [47]: items with the lowest adjusted item-total correlation were deleted until, when possible, factors had three to four items. A second EFA was then run to assess the impact of the item deletions on the factors affected [44].

2.5.3. Tests of dimensionality: confirmatory factor analysis and bifactor EFA

Finally, to determine if the dimensionality of the factor structure identified by the EFA on the test sample was the same across a new independent sample, we conducted a confirmatory factor analysis (CFA) [47] on the validation sample, with factors allowed to correlate (see Appendix B3.1 for detailed methodology). As the examination of intercorrelations between factors suggested a hierarchical structure, we conducted a second-order CFA to test it: constructs correlating at 0.4 or above were considered to be measured by a common ‘higher-order dimension’. We assessed the model fit of the first- and second-order CFAs with the same criteria as those used in the EFA. To further test whether the factor structure identified from the EFA might reflect a unidimensional construct with multidimensional subconstructs, we implemented a bifactor EFA [47] (See Appendix B3.1 for more details).

2.5.4. Tests of construct validity: convergent and discriminant validity

We assessed convergent and discriminant validity of the factor solution based on the CFA parameter estimates: we used factor loadings as indicators of convergent validity (cutoff criteria: ≥0.71 (excellent), 0.63–0.70 (very good), 0.55–0.62 (good), 0.45–0.54 (fair), 0.32–0.44 (poor) [48]), and Pearson correlations between the factors as indicators of discriminant validity (≥0.80 indicating poor discriminant validity) [44]. We were unable to compare the Pro-VC-Be with other instruments, as no other instrument validated among HCPs exists to measure their vaccine confidence and determinants of their vaccination behavior.

2.5.5. Test of criterion validity: predictive validity of behavioral outcomes

We considered four behavioral (or attitudinal) outcomes as criteria to test the extent to which the factors measured in Pro-VC-Be predict HCPs’ different kinds of vaccination behaviors and attitudes (related to COVID-19 vaccines) [47]: 1) score of vaccine recommendation frequency for 6 specific vaccine situations; 2) general vaccination behavior score (based on items on vaccination in general and used as a generic measure of immunization activity, applicable independently of specific vaccine situations); 3) score of stated willingness to accept future COVID-19 vaccines (data collected when COVID-19 vaccines were not yet authorized); and 4) self-vaccination behavior score, among French GPs only (see Table 1 for the items used in these scores). Because the four scores were not normally distributed, we dichotomized them (see Appendix B4.1 for the steps we used). To distinguish HCPs most likely to recommend vaccines, get vaccinated, or accept future COVID-19 vaccination from those less likely to do these things, we set the following thresholds: 1) very frequent vaccine recommendation: score >75%; 2) very frequent immunization activity: score >75%; 3) strong acceptance of COVID-19 vaccine: score >4/6; 4) French GPs up-to-date with influenza vaccinations, and intending to get vaccinated against it, and up-to-date with their pertussis vaccination (score = 3/3). For the vaccine recommendation and general immunization scores, we also performed sensitivity analyses with a 90% dichotomization threshold.

After the previous steps rejected a unidimensional structure of the Pro-VC-Be (see Appendix B3.2.3), we tested the associations between our criteria and each of the factors obtained in the second-order CFA. We used the final set of items to calculate scores for each identified factor [47] (see Appendix B4.1 for more details).

We conducted the analyses separately for GPs and nurses because the target populations they care for may differ substantially; and the practice contexts in Quebec, on the one hand, and France and Belgium, on the other, differ markedly. The scores of each factor of the Pro-VC-Be were dichotomized at the mean of the population considered, to assess the extent to which HCPs with an above-average score reported better vaccine behaviors than those with below-average attitudes.

We performed weighted multiple robust Poisson regressions with robust error variances to correct the error overestimation of estimated relative risks that can occur when Poisson regression is applied to binomial data [49]. All regressions were adjusted for gender and age to estimate the relative risks between each of the four outcomes and the factors of the Pro-VC-Be questionnaire. We first tested each factor separately and then tested them all together in a global model, because some were moderately correlated. We computed the variance inflation factor (VIF) to test for multicollinearity in equivalent linear models and interpreted VIF values <5 as presenting no multicollinearity issues [50].

All analyses were based on two-sided P-values, with P < 0.05 indicating statistical significance. They were conducted with Mplus, version 7.2 for factor analyses, and SAS, version 9.4 for the others.

3. Results

The study population included 2,696 HCPs: 1,209 French GPs (45%), 432 Belgian GPs (16%), and 1,055 Quebec nurses (39%). One third were men (61% of French GPs, 57% of Belgian GPs, 12% of nurses); 34% (12% in France, 20% in Belgium, 48% in Quebec) were aged under 40, while 19% (37% in France, 49% in Belgium, 6% in Quebec) were 60 years or older. The two subsamples forming the test and validation samples (n = 1,348 each) did not differ significantly for country, gender, or age. Completing the online questionnaire took 10 minutes on average.

3.1. EFA and item reduction analysis

The first EFA conducted for the test sample on the entire Pro-VC-Be set of psychosocial dimensions (41 items) found nine factors with eigenvalues >1. We retained the 10-factor solution for two reasons, however: its fit was better (RMSEA = 0.024 [0.021; 0.027]; CFI = 0.97; TLI = 0.95; SRMR = 0.02) and the solution was closer to the theoretical constructs described above (with the exception of ‘professional norm’, which included only 1 item, loading on ‘commitment to vaccination’, see Table S3 Appendix B2.2.1).

The item reduction resulted in the exclusion of 10 items (including the ‘professional norm’ item) (Tables S3 and S4, Appendix B2.2.1). Running a second EFA on the remaining 31 items resulted in the same 10-factor solution with good fit (RMSEA = 0.014 [0.008; 0.019] CFI = 0.99; TLI = 0.99; SRMR = 0.01; Table 2). Sensitivity analyses with ‘don’t know’ answers coded as missing values and using full-information maximum likelihood to compute EFAs produced results almost identical to the main analysis (Appendix B2.2.2).

Table 2. Factor loadings of the final set of items used in the exploratory factor analysis^a on the test sample (n = 1,348)

		Factors extracted
		1	2	3	4	5	6	7	8	9	10
Perceived risks of vaccines	R1. Some vaccines can cause autoimmune diseases	0.72*	0.01	0.02	−0.05*	0.03	−0.02	−0.01	0.03	−0.03	−0.01
	R2 The measles vaccine can cause autism in children	0.47*	0.17*	−0.05	−0.05	0.03	0.04	−0.06	−0.02	−0.04	0.06
	R3 Some vaccines can cause multiple sclerosis	0.82*	0.00	−0.03	0.02	−0.03	0.02	0.00	0.00	0.01	0.01
Complacency	U1 Today, some vaccines recommended by authorities are not useful, because the diseases they prevent are not serious	0.04	0.57*	−0.02	−0.10*	0.01	0.01	−0.04	0.02	−0.08	−0.04
	U2 Children are vaccinated against too many diseases	0.03	0.84*	0.01	0.02	−0.05*	0.00	0.03	0.02	−0.03	−0.01
	U3 Children are vaccinated at too young an age	0.01	0.70*	−0.07*	−0.01	0.03	0.03	−0.04	−0.02	0.00	0.07*
Perceived benefit/risk balance	BRB1 The benefits of the vaccine against measles are much greater than its potential risks	−0.06*	−0.02	0.79*	−0.05*	−0.02	0.00	0.03	−0.01	0.05	−0.02
	BRB2 The benefits of the vaccine against influenza in people with a chronic disease are much greater than its potential risks	−0.01	0.02	0.61*	0.05	0.04	−0.03	−0.03	0.01	0.07	−0.14*
	BRB3 The benefits of the vaccine against hepatitis B in infants (or as catch-up in adolescents) are much greater than its potential risks	−0.02	−0.09*	0.70*	0.02	0.04	−0.02	0.00	−0.03	0.02	0.04
	BRB4 The benefits of the vaccine against human papillomaviruses are much greater than its potential risks	0.00	−0.03	0.72*	0.08*	0.04	−0.01	0.03	0.00	0.00	0.04
Perceived collective responsibility	CR1 I recommend the vaccines on the vaccination schedule to my patients because it’s essential to contribute to protection of the population (community immunity)	0.00	−0.02	0.02	−0.01	−0.04	0.01	0.01	−0.02	0.93*	0.01
	CR2 I recommend the vaccines in the official schedule to my hesitant patients, explaining to them the importance of community immunity	−0.01	0.01	−0.02	0.06	0.10	−0.04	0.02	0.06	0.65*	−0.02
Trust in authorities	C1 I trust the ministry of health to provide reliable information about the risks and benefits of vaccines	−0.05	−0.01	0.05	0.78*	−0.02	−0.05*	0.00	−0.03	0.05	−0.01
	C4 I trust the ministry of health to establish the vaccination strategy	−0.01	−0.02	−0.02	0.83*	0.00	−0.05*	0.02	−0.02	0.01	0.01
	C5 I trust the ministry of health to ensure that vaccines are safe	−0.01	−0.02	0.02	0.87*	0.04	0.04*	0.00	0.00	0.02	−0.01
Perceived constraints	PC1 The cost of some vaccines is a problem for some patients and can keep me from prescribing them	0.01	0.00	0.01	0.06	−0.04	−0.04	−0.03	0.55*	−0.05	0.08
	PC2 The lack of availability of certain vaccines is often a problem that can keep me from prescribing them to my patients.	0.01	0.04	0.00	−0.09*	0.01	0.01	0.01	0.69*	0.01	−0.02
	PC3 Not having vaccines in my office is a problem in my practice	−0.01	−0.06	−0.04	0.04	0.05	0.08*	0.03	0.49*	0.04	−0.08
Openness to patients	OP1 Patients who are hesitant about the benefits and risks of vaccines have legitimate questions	0.07	−0.01	0.01	−0.01	0.02	0.01	0.06	−0.05	0.01	0.47*
	OP2 I inform my patients about the benefits and risks of vaccines but I let them make their decision without trying to influence them	0.05	−0.06	−0.03	0.12*	−0.07	0.06	0.05	0.04	0.03	0.49*
	OP5 I am willing to let parents delay immunizing their children	−0.07*	0.12*	−0.02	−0.11*	0.13*	−0.04	−0.10*	−0.02	0.00	0.49*
Commitment to vaccination	CV1 I am actively involved in ensuring that my patients are vaccinated	0.00	−0.06	0.02	0.02	0.14*	0.04	0.62*	−0.05	0.09*	−0.12*
	CV3 I am committed to keeping my knowledge about vaccination up-to-date (CME, conferences, reading)	−0.01	0.03	−0.03	−0.04	0.07	−0.05*	0.77*	−0.02	0.02	−0.03
	CV4 I am committed to developing the skills needed to communicate better with my patients about vaccination	−0.06*	−0.02	0.07*	0.06*	−0.02	0.01	0.71*	0.05*	0.03	0.12*
Self-efficacy	SE1 I feel comfortable advising my patients about the risks and benefits of vaccines	0.00	−0.07*	0.00	0.02	0.74*	−0.03	0.10*	−0.04	0.05*	0.01
	SE2 I feel comfortable discussing vaccines with my patients who are highly hesitant about vaccination	0.00	−0.04	0.02	0.06*	0.79*	0.02	0.04	0.00	0.03	0.00
	SE3 I feel sufficiently trained and informed to discuss vaccines with all patients	−0.08*	−0.02	0.01	−0.03	0.78*	−0.08*	0.10*	0.01	0.00	0.00
	SE4 I feel sufficiently trained on how to approach the question of vaccines with hesitant patients	−0.02	0.03	0.04	0.01	0.84*	0.01	0.05*	−0.03	0.03	−0.01
Reluctant trust	RT1 I recommend the vaccines in the official schedule even though I sometimes feel that I am not sufficiently informed about some of them	0.04	−0.05*	0.05	0.03	−0.11*	0.71*	−0.07*	0.06*	−0.01	0.01
	RT2 I recommend the vaccines in the official schedule even though I feel that the objectives of the vaccination policy are not clear enough	−0.03	0.05*	−0.07*	−0.01	0.09*	0.75*	−0.05	0.02	0.04	−0.04
	RT3 I recommend vaccines on the official schedule although I sometimes have doubts about their safety	0.06	0.05	0.01	−0.08*	−0.03	0.64*	0.14*	−0.05	−0.07	0.09*

^aFactors estimated using maximum likelihood with robust standard errors, and CF-VARIMAX oblique rotation. Likert scales were treated as continuous variables. *P ≤ 0.05

3.2. Dimensionality

The CFA conducted on the validation sample confirmed the 10-factor structure identified with the last EFA. Its fit was good (RMSEA = 0.027 [0.024;0.030]; CFI = 0.96; TLI = 0.95; SRMR = 0.04, Figure 1). As the analysis of factor intercorrelations (Table 3) showed a very high correlation between self-efficacy and commitment to vaccination (Pearson ρ = 0.78, P < 0.001), we combined these two factors into a second-order factor that we called ‘proactive efficacy’ (see the discussion section). We also found moderate correlations (absolute ρ ranging from 0.40 to 0.52, P < 0.001) between perceived risks of vaccines, perceived benefit-risk balance, complacency, and perceived collective responsibility, and combined them into a second-order factor that we called ‘vaccine confidence’ because theory (HBM and 5C) postulates that these constructs contribute to vaccine confidence. Although trust in institutions and perceived constraints have been shown to contribute to vaccine confidence, we did not include them in this second-order factor for two reasons: 1) their correlations with these four factors were weaker than those between these four factors; 2) the four factors can be considered direct measures of vaccine confidence, while trust in institutions and perceived constraints are rather contextual dimensions. The second-order CFA run on the validation sample fitted the data very well (RMSEA = 0.027 [0.025; 0.030]; CFI = 0.96; TLI = 0.95; SRMR = 0.04, Figure 2) without localized strain (see Appendix B3.2.1). Correlations between the factors are presented in Table S5, Appendix B3.2.2. The bifactor analysis confirmed the multidimensional structure of the Pro-VC-Be (Appendix B3.2.3). Considering ‘don’t know’ answers as missing values and using full-information maximum likelihood to compute CFAs led to only slight changes (see Appendix B3.2.4 for more details).

Figure 1. Confirmatory factor analysis^a on the validation sample: loadings of items on factors (n = 1,348).

^a Factors were allowed to correlate. All factor loadings were set to be free, and the metrics of the factors were defined by fixing their variances to one. * P ≤ 0.05 ** P ≤ 0.01 *** P ≤ 0.001 Abbreviations. resp. = responsibility. RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis index; SRMR = standardized root mean square residual

Figure 2. Second-order confirmatory factor analysis^a on the validation sample: standardized loadings on first- and second-order factors (n = 1,348).

^a Factors were allowed to correlate. All factor loadings were set to be free, and the metrics of the factors were defined by fixing their variances to one.         * P ≤ 0.05 ** P ≤ 0.01 *** P ≤ 0.001 Abbreviations. resp. = responsibility. RMSEA = root mean square error of approximation; CFI = comparative fit index; TLI = Tucker-Lewis index; SRMR = standardized root mean square residual

Table 3. Analysis of the correlations^a between the factors of the confirmatory factor analysis on the validation sample (n = 1,348)

	R	U	BRB	CR	TA	PC	OP	CV	SE	RT
R	1.00	0.52	−0.51	−0.40	−0.29	0.13	0.14	−0.36	−0.42	0.26
U		1.00	−0.43	−0.44	−0.37	0.13	0.10 (NS)	−0.28	−0.30	0.27
BRB			1.00	0.48	0.31	−0.14	−0.09 (NS)	0.35	0.35	−0.20
CR				1.00	0.42	−0.13	−0.08 (NS)	0.42	0.40	−0.18
TA					1.00	−0.18	0.10 (NS)	0.18	0.23	−0.24
PC						1.00	−0.04 (NS)	−0.10	−0.16	0.22
OP							1.00	−0.02 (NS)	−0.04 (NS)	0.18
CV								1.00	0.78	−0.11
SE									1.00	−0.21
RT										1.00

Abbreviations: R = Perceived risks of vaccines; U = Complacency; BRB = Perceived benefit/risk balance; CR = Perceived collective responsibility; TA = Trust in authorities; PC = Perceived constraints; OP = openness to patients; CV = Commitment to vaccination; SE = Self-efficacy; RT = Reluctant trust

^aAll correlations significant at P < 0.05 except when NS stated. Colored cells indicate moderate to high correlations. The darker the cell, the higher the absolute correlation between the corresponding items

3.3. Construct validity: convergent and discriminant validity

In the second-order CFA, 8 of the 10 first-order factors and the 2 second-order factors had very good (loadings >0.63) to excellent (>0.71) convergent validity (P < 0.001). Convergent validity of the perceived constraints construct was fair (loadings from 0.45 to 0.71, P < 0.001). Convergent validity of openness to patients was slightly lower: two items had loadings >0.5 (P < 0.001) but the loading of the final item (‘OP5 I am willing to let parents delay immunizing their children’) was lower (0.33, P < 0.001).

Correlations between the factors of the second-order CFA were weak (Table S5, Appendix B3.2.2), except for vaccine confidence, which was moderately correlated with proactive efficacy (ρ = 0.60, P < 0.001) and trust in authorities (ρ = 0.52, P < 0.001). Despite these correlations, we chose for two reasons to decide against a third-order CFA: 1) keeping these factors separate was important, given the assumption of the theoretical model that intermediary factors (one of them proactive efficacy) may mediate or moderate the relations between vaccine confidence and vaccination behaviors; 2) trust in the health authorities may be a contextual factor influencing vaccine confidence. Overall, we considered that the discriminant validity of the constructs was adequate since correlations between the factors of the second order solution were not strong.

3.4. Criterion validity

Poisson regression models adjusted for gender and age were computed to assess the relations between each of the four behavioral or attitudinal criteria (dependent variables) and, separately, each of the six factors of the Pro-VC-Be resulting from the second order CFA (explanatory variables).

We first present the results of the models including the Pro-VC-Be factors separately. GPs with an above-average vaccine confidence score were significantly more likely to recommend vaccines to their patients very frequently than the other GPs (Table 4): the adjusted relative risk of reporting very frequent vaccine recommendations (score higher than 75%) was 1.4 (95% confidence interval, 1.2, 1.6), that is, the probability of very frequent recommendations was 40% higher for GPs with an above-average vaccine confidence score than for the others.

Table 4. Associations between self-reported vaccine recommendation frequency, general immunization activity, and the Pro-VC-Be factors among HCPs: results from multiple robust Poisson regressions (n = 2,696)

	Very frequent vaccine recommendations (score >75%, ref. No)				Very frequent immunization activity (score >75%, ref. No)
	GPs (n = 1,641)		Nurses (n = 1,055)		GPs (n = 1,641)		Nurses (n = 1,055)
Pro-VC-Be factors	Separately^a	Global	Separately^a	Global	Separately^a	Global	Separately^a	Global
	aRR [95% CI]
Vaccine confidence > mean (ref. No)	1.4 [1.2;1.6]	1.2 [1.03;1.4]	1.7 [1.4;2.1]	1.5 [1.2;1.8]	1.3 [1.2;1.5]	1.2 [1.1;1.4]	1.7 [1.3;2.2]	1.2 [0.9;1.5]
Proactive efficacy > mean (ref. No)	1.6 [1.4;1.8]	1.4 [1.3;1.6]	2.8 [2.3;3.5]	2.7 [2.1;3.3]	1.4 [1.2;1.6]	1.3 [1.1;1.4]	4.2 [3.0;5.9]	3.8 [2.7;5.4]
Trust in authorities > mean (ref. No)	1.3 [1.1;1.5]	1.1 [1.0;1.3]	1.3 [1.1;1.6]	1.0 [0.8;1.2]	1.0 [0.9;1.2]	0.9 [0.8;1.0]	1.7 [1.3;2.2]	1.2 [0.9;1.6]
Openness to patients > mean (ref. No)	1.0 [0.9;1.1]	1.0 [0.9;1.2]	1.2 [1.0;1.4]	1.0 [0.9;1.2]	1.0 [0.9;1.1]	1.0 [0.9;1.1]	1.3 [1.03;1.7]	1.1 [0.9;1.4]
Reluctant trust > mean (ref. No)	0.7 [0.7;0.8]	0.9 [0.8;0.98]	1.0 [0.8;1.2]	1.2 [1.0;1.4]	0.8 [0.7;0.9]	0.9 [0.8;1.0]	0.7 [0.6;0.95]	0.9 [0.7;1.1]
Perceived constraints > mean (ref. No)	0.9 [0.8;1.1]	1.0 [0.9;1.1]	1.0 [0.8;1.2]	1.0 [0.9;1.2]	0.9 [0.8;1.1]	1.0 [0.9;1.1]	0.8 [0.7;1.1]	1.0 [0.8;1.2]

Abbreviations. Pro-VC-Be = Health Professionals Vaccine Confidence and Behaviors; GPs = general practitioners; aRR [95% CI] = adjusted relative risk and its 95% confidence interval.

^aSeparately: Pro-VC-Be factors introduced separately as explanatory variables in models adjusted for gender and age; global: all Pro-VC-Be factors introduced in the same model.

GPs with higher vaccine confidence scores also reported more frequent general immunization activity: raising the subject of vaccination with patients, recommending and/or prescribing vaccines (Table 4; P < 0.001), higher acceptance of future COVID-19 vaccines (Table 5; P < 0.001), and, among French GPs, a higher score of self-vaccination for influenza and pertussis (Table 6; P < 0.001). Nurses with an above-average vaccine confidence score were also significantly more likely (+70%, P < 0.001) to recommend vaccines to their patients very frequently, to report higher immunization activity (+70%, P < 0.001; Table 4), and to be more accepting of future COVID-19 vaccines (+ 100%, P < 0.001, Table 5).

Table 5. Associations between acceptance of potential COVID-19 vaccines (in October-November 2020) and Pro-VC-Be factors: results from multiple robust Poisson regressions (n = 2,696)

	Strong acceptance of COVID-19 vaccines (score >4/6, ref. No)
Pro-VC-Be factors	GPs (n = 1,641)		Nurses (n = 1,055)
	Separately^a	Global	Separately^a	Global
	aRR [95% CI]
Vaccine confidence > mean (ref. No)	1.6 [1.3;1.9]	1.4 [1.1;1.6]	2.0 [1.7;2.3]	1.6 [1.3;1.9]
Proactive efficacy > mean (ref. No)	1.3 [1.1;1.5]	1.1 [0.9;1.3]	1.3 [1.2;1.5]	1.1 [1.0;1.3]
Trust in authorities > mean (ref. No)	2.0 [1.6;2.4]	1.8 [1.5;2.2]	1.9 [1.7;2.2]	1.6 [1.3;1.8]
Openness to patients > mean (ref. No)	0.8 [0.7;0.9]	0.9 [0.7;1.0]	1.0 [0.8;1.1]	0.9 [0.8;1.1]
Reluctant trust > mean (ref. No)	0.9 [0.7;0.99]	1.1 [0.9;1.2]	0.9 [0.7;0.96]	1.0 [0.9;1.2]
Perceived constraints > mean (ref. No)	0.9 [0.8;1.1]	1.0 [0.9;1.2]	0.9 [0.8;0.97]	0.9 [0.8;1.1]

Abbreviations. Pro-VC-Be = Health Professionals Vaccine Confidence and Behaviors; GPs = general practitioners; aRR [95% CI] = adjusted relative risk and its 95% confidence interval.

^aSeparately: Pro-VC-Be factors introduced separately as explanatory variables in models adjusted for gender and age; global: all Pro-VC-Be factors introduced in the same model.

Table 6. Associations between self-vaccination behaviors reported by French GPs and Pro-VC-Be factors^a: results from multiple robust Poisson regressions (n = 1,209)

	Up to date with influenza and pertussis vaccinations (score = 3/3, ref. No)
Pro-VC-Be factors	Separately^b	Global
	aRR [95% CI]
Vaccine confidence > mean (ref. No)	1.3 [1.1;1.4]	1.2 [1.02;1.3]
Proactive efficacy > mean (ref. No)	1.1 [1.02;1.3]	1.1 [0.9;1.2]
Trust in authorities > mean (ref. No)	1.2 [1.1;1.4]	1.1 [1.0;1.3]
Openness to patients > mean (ref. No)	0.9 [0.8;0.95]	0.9 [0.8;1.0]
Reluctant trust > mean (ref. No)	0.9 [0.8;0.97]	1.0 [0.9;1.1]

Abbreviations. Pro-VC-Be = Health Professionals Vaccine Confidence and Behaviors; GPs = general practitioners; aRR [95% CI] = adjusted relative risk and its 95% confidence interval.

^aSelf-vaccination behaviors were collected in French GPs only. We did not consider perceived constraints in the models because these constraints concerned HCPs’ vaccination practices for patients only.

^bSeparately: Pro-VC-Be factors introduced separately as explanatory variables in models adjusted for gender and age; global: all Pro-VC-Be factors introduced in the same model.

Proactive efficacy was the factor most strongly associated with high vaccine recommendation frequency and general immunization activity, among both GPs and nurses (Table 4): GPs with an above-average proactive efficacy score were more likely to report high scores of recommendation behavior (+60%, P < 0.001) and general immunization activity (+40%, P < 0.001); this was also observed in nurses (probability of recommendation behavior and immunization activity rose by 180% and 320%, respectively; P < 0.001). To a lesser extent, proactive efficacy was also significantly associated with COVID-19 vaccine acceptance in GPs and nurses (Table 5; P = 0.003) and with higher self-vaccination scores for influenza and pertussis (French GPs only, Table 6; P = 0.02).

An above-average score of trust in health authorities was significantly associated with a higher likelihood of recommending vaccines to patients (+30%) among both GPs (P < 0.001) and nurses (P = 0.003), and, among nurses only, of reporting very frequent general immunization activity (+70%, P < 0.001, Table 4). It was also associated with a higher COVID-19 vaccine acceptance score among both GPs and nurses (+100% and +90% respectively, P < 0.001, Table 5) and, among French GPs, with a higher self-vaccination score (P = 0.01, Table 6).

Openness to patients was associated with a higher likelihood of very frequent immunization activity among nurses (+30%, P = 0.03, Table 4) and with a lower likelihood of strong acceptance of COVID-19 vaccines (−20%, P < 0.001) among GPs overall (Table 5) and of a high self-vaccination score among French GPs (−10%, P = 0.01) (Table 6).

Among GPs, an above-average score of reluctant trust was significantly associated with lower likelihoods of very frequent vaccine recommendation (−30%, P < 0.001; Table 4), very frequent immunization activity (−20%, P < 0.001; Table 4), strong COVID-19 vaccine acceptance (−10%, P = 0.003, Table 5), and self-vaccination (−10%, P = 0.02; Table 6). This was also the case among nurses, for very frequent immunization activity (−30%, P = 0.02, Table 4) and strong COVID-19 vaccine acceptance (−10%, P = 0.004; Table 5).

Nurses perceiving high constraints to vaccination were less likely to report acceptance of potential COVID-19 vaccines (−10%, P = 0.01; Table 5). High perceived constraints were not significantly associated with other criteria, among either nurses or GPs.

In the global models, including all of the six Pro-VC-Be factors simultaneously and adjusted for gender and age, the relative risks were lower than in the separate analyses, and some associations that were significant in the separate analyses became nonsignificant (Tables 4–6). Vaccine confidence was no longer associated with very frequent immunization activity among nurses (Table 4), and proactive efficacy no longer a factor of either strong COVID-19 vaccine acceptance by GPs and nurses (Table 5), or of self-vaccination against influenza and pertussis by French GPs (Table 6). A significant association with trust in authorities persisted only for strong COVID-19 vaccine acceptance, among both GPs (P < 0.001) and nurses (P < 0.001); higher reluctant trust remained associated only with frequent vaccine recommendations among GPs (P = 0.02), while neither openness to patients nor perceived constraints remained associated with any of the four criteria.

Sensitivity analyses for vaccine recommendations and immunization activity scores using 90% dichotomization thresholds produced similar results, with larger RRs and only a few differences (see Tables S7 and S8, Appendix B4.2).

We found no issues of multicollinearity in the linear models and, as multicollinearity is a property of explanatory rather than dependent variables [51], we can conclude that there was no issue of multicollinearity in the robust Poisson models either.

4. Discussion

The Pro-VC-Be is a new instrument developed from three main theoretical frames/models (Theoretical Domain Framework, Health Belief Model, and 5C) and empirical research to measure psychosocial determinants – including vaccine confidence – of HCPs’ vaccination behavior. This instrument also measures various vaccination behaviors, including HCPs’ own vaccinations and their recommendation behavior toward their patients. The instrument was validated among 2,696 HCPs from different professions working in three French-speaking countries. The final results indicated a 6-factor structure with good fit: vaccine confidence (a second-order factor combining perceived vaccine risks, their perceived benefit-risk balance, and perceived importance of collective protection, as well as complacency), trust in authorities, proactive efficacy (a second-order factor combining commitment to vaccination and self-efficacy), reluctant trust, openness to patients, and perceived constraints. The instrument also showed good convergent validity, adequate discriminant validity, and good criterion validity. Finally, the results of our criterion validity tests indicated that proactive efficacy was the strongest predictor of high frequent vaccine recommendation and immunization activity among GPs and nurses. Vaccine confidence was also a significant predictor of vaccine recommendation and immunization activity for GPs, of vaccine recommendation among nurses, and of COVID-19 vaccine acceptance among both groups.

The Pro-VC-Be is not unidimensional, as we expected given its theoretical basis: a combination of three theoretical models and the inclusion of 10 dimensions. The rationale for this approach was to capture different types of psychosocial determinants that may influence/explain HCPs’ vaccination behaviors. The results of the criterion validation analysis (Tables 4–6) showed that using separate factors for different determinants enabled us to take into account the diversity of contexts and professions: the associations observed between the second-order CFA factors and the criterion variables varied in significance and magnitude according to the type of outcome variable (general immunization activity, vaccine recommendation, self-vaccination, COVID-19 vaccine acceptance) and type of HCP.

The second-order CFA, however, allowed the validation of two higher order factors: vaccine confidence and proactive efficacy. The vaccine confidence dimension resulted from the combination of four first-order factors: perceived vaccine risks, perceived benefit-risk balance of different vaccines, and perceived collective responsibility, together with complacency. These four factors were moderately correlated pairwise (Table 3) and contributed to the measurement of the same phenomenon: vaccine confidence. It has already been shown that HCPs’ perceptions of the safety and usefulness of vaccines contribute to their confidence in (or hesitancy about) vaccines and may explain their vaccination behaviors [5,10–12,28,29,33,52]. The significant contribution by the perceived benefit-risk balance and perceived collective responsibility to the vaccine confidence factor is a new finding [5,29,53,54]. The confirmation of the criterion validity of this dimension for all four criteria studied, except in the global model among nurses, suggests that the vaccine confidence factor is a good predictor of HCPs’ vaccination behaviors, for themselves and their patients.

The proactive efficacy factor resulted from the combination of the commitment to vaccination and self-efficacy factors, given their high correlation (Table 3). This factor measures the extent to which HCPs feel prepared to face vaccination tasks and invest in them. This dimension proved to be the most important driver of immunization activity and vaccine recommendations to patients. This was more pronounced for nurses than GPs, probably due to the greater heterogeneity of attitudes and behaviors among the former than the latter. These results are consistent with the literature showing the importance of self-efficacy, commitment, and empowerment in the adoption of different health behaviors in various population groups, including the adoption of evidence-based medicine practices by HCPs [19,34,55]. Moreover, measuring this dimension in HCPs could be a useful addition to more specific tools for evaluating the impact of interventions aimed at improving their vaccine-related knowledge and skills (e.g. refutational interview, motivational interview, etc.) [56,57]. The correlation (ρ = 0.6, P < 0.001; Table S5 Appendix B3.2.2) between vaccine confidence and proactive efficacy requires further investigation to better disentangle the links between these two dimensions and their respective impacts on recommendation behavior. The absence of any association between proactive efficacy and both self-vaccination and COVID-19 vaccine acceptance, once all other Pro-VC-Be factors have been taken into account, is consistent with the wording of the proactive efficacy items, which were oriented toward HCPs’ immunization activity with their patients, and with the fact that this survey preceded the availability of COVID-19 vaccines.

Trust is generally considered to be a crucial component of attitudes toward vaccination, including vaccine hesitancy [58,59]. Trust in authorities is a highly contextual dimension and varies greatly from country to country [60]. For example, a study at the time of the 2009 A/H1N1 pandemic showed marked differences in institutional trust between France (where it was low) and Quebec (where it was high) and linked these differences to the vaccine coverage rates in each country: 8% in France and 56% in Quebec [61]. In our study, the criterion validity of trust in authorities was high for a priori acceptance of COVID-19 vaccines: trust in health authorities has been put to test since the pandemic began by numerous controversies about their decisions and ability to control the pandemic [10,62]. Among both GPs and nurses, trust in authorities was associated with very frequent vaccination recommendations only when the other factors were not included in the model. This suggests that this trust in authorities is associated with some but not all of these factors, especially with vaccine confidence, as previously shown in French GPs [26,33].

Our study is the first, to our knowledge, to include and validate a quantitative measure of reluctant trust among HCPs. This concept describes the ‘leap of faith’ laypeople must make about expert systems and technologies not under their direct scrutiny [32,63]. Giddens (a sociologist) has theorized that true trust can be built only in a face-to-face relationship (for example, between a patient and his/her doctor). In contemporary societies, technologies such as vaccines and expert systems are developed and manufactured by individuals at a distance, whom the users do not know and never meet. Because trust in these technologies cannot be entirely satisfactory for users, Giddens describes it as ‘reluctant.’ The results of our validation study confirmed our previous findings that GPs’ trust in vaccination may be reluctant, as defined by Giddens [33], as is that of other HCPs (nurses) routinely involved in vaccinating the general population. The results of the criterion validity analyses (global models) suggest that reluctant trust dampens GPs’ vaccine recommendation behaviors, in line with our hypotheses. Further investigations, to be done for another paper, are necessary to understand why this was not observed for nurses and the nature of the links between reluctant trust in vaccines and the other factors studied here.

Our study provides construct validity for openness to patients, measuring HCPs’ attitudes toward vaccine-hesitant patients, which is another new dimension useful for addressing the determinants of HCPs’ behaviors. Its convergent validity, however, was somewhat weaker than that of the other Pro-VC-Be factors (Figure 2). When taken separately (without these other factors), its associations with the criteria differed according to the criterion and/or population considered and were no longer present in the global models (Table 4–6), a finding that suggests the complexity of these links. A better understanding of the links between HCPs’ openness to patients, perceptions of vaccine risks, and their recommendation behavior is again important as patient-HCP interaction patterns influence the outcome of the consultation [36]. It is also important to assess the impact of interventions on this dimension.

In another study among GPs, perceived constraints on them have been found to be negatively associated with self-vaccination behavior, independently of other dimensions but not with their recommendations of influenza or MMR vaccines to patients [29]. In this validation study, we found no association between perceived constraints and our criteria in the global models (adjusted for the other factors) while, in the models not so adjusted, a negative association was observed among GPs for nearly routine vaccine recommendations (score >90%) and among nurses for nearly routine immunization activity (score >90%) and strong COVID-19 vaccine acceptance (Tables 4–6 and Appendix B4.2). This suggests that other factors might be more relevant and that perceived constraints may not have any unique effect when the other predictors are taken into account.

5. Strengths

The strengths of this study are its large sample size and its representativeness for age, gender, and HCPs’ practice area. This was made possible by our strategy of repeated reminders to the HCPs contacted and by data weighting [43]. The inclusion of GPs and nurses in three different countries is another strength of our study. This examination of the Pro-VC-Be provides validation and insight into some dimensions not yet or rarely studied in HCPs in the field of vaccine hesitancy. The questionnaire can be completed online in 10 minutes, an acceptable time for most HCPs. However, in view of their time constraints, we are in the process of validating a short version of the Pro-VC-Be.

6. Limitations

Our study nonetheless has some limitations that should be kept in mind in interpreting its results. A test-retest of the Pro-VC-Be has not yet been carried out, as the consequences of the pandemic since March 2020 on primary health care have made the organization of such a test difficult. The convergent and discriminant validity of the Pro-VC-Be could not be studied in relation to other validated instruments because there are none that measure the determinants of HCP vaccination behavior. Moreover, the Pro-VC-Be does not include some areas that are important for understanding HCPs’ behaviors [19], notably their knowledge about vaccination. It was primarily designed to collect data about psychosocial factors that may explain HCPs’ vaccination behavior; but other tools specifically designed to collect data on other topics could be added to it [5]. For example, demoralization (which can affect extensive and heterogeneous populations confronted with unexpected dramatic events) may offer a possible explanation for some ‘avoidance behaviors,’ such as reluctance toward vaccines or poorer adherence to prevention behaviors in general [64]. Some approaches to unmasking and dealing with demoralization (by interventions ranging from the support of HCPs to targeted psychotherapeutic approaches) may be necessary in the context of a pandemic; they require specific measurement tools [65]. Finally, the Pro-VC-Be was developed and validated in French; it should also be validated in other languages.

7. Conclusion, prospects

In the era of COVID-19, there is a greater need than ever for validated instruments to study – comparably and reliably – the determinants of HCPs’ vaccination behavior in various vaccine situations and settings. The French version of the Pro-VC-Be constitutes a step in this direction as it allows the valid measurement of several important psychosocial determinants of HCPs’ vaccine behaviors [5] in different types of HCPs, settings, and contexts. In a crisis period, it could be adapted to measure the determinants of attitudes toward COVID-19 vaccination. Its design would also enable the study of the impact of interventions on its different factors: vaccine confidence, proactive efficacy, and openness to patients.

The validation of the Pro-VC-Be should now be extended to other countries and is currently underway in Europe as part of an H2020 European project (‘Jitsuvax’). The use of this tool in cultural and development contexts different from those of Western countries should be based on prior qualitative research with HCPs to verify whether the Pro-VC-Be factors are adapted to the reality of these countries. Finally, a short version will be prepared for use in interventional and public health settings.

Declaration of interests

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Authors contributions

PV, ED, and AG were involved in the conception and design of the Pro-VC-Be. PV, ED, and ND were involved in the organization and implementation of data collection. LF and PV were involved in the statistical analysis, interpretation of the data and drafting of the paper. AG, ED, ND, AF, LK, SL, AS, and PS were involved in the interpretation of the data and revising the paper critically for intellectual content; they approved it for publication. All authors agree to be accountable for all aspects of the work.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Ethical approval

Our work was conducted with the approval of an institutional ethics committee in each of the participating countries: the details of this have been included in the manuscript (paragraph 2.4.2 data collection procedure).

As the survey was conducted exclusively via the Internet in Quebec and Belgium, and via the Internet or telephone in France, the ethics committees approved that the completion of a questionnaire constituted consent by the health professionals and that written consent was not required.

Acknowledgments

We warmly thank the following experts for their invaluable advice on the first version of the Pro-VC-Be questionnaire: C Betsch, M Deml, KB Habersaat, J Leask, and JK Ward. We also thank the survey participants, as well as Jo Ann Cahn who revised the English of the manuscript. We have these experts’ permission to mention their names in the manuscript for publication.

Supplemental data

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was funded by the Direction de la Recherche, des Etudes, de l’Evaluation et des Statistiques, French Ministry of Health, under Grant number 2102173353; and by the European Union’s Horizon 2020 research and innovation program under Grant agreement number 964728 (JITSUVAX).