Interventions to increase seasonal influenza vaccine coverage in healthcare workers: A systematic review and meta-regression analysis

ABSTRACT

Influenza vaccination is recommended for healthcare workers (HCWs), but coverage is often low. We reviewed studies evaluating interventions to increase seasonal influenza vaccination coverage in HCWs, including a meta-regression analysis to quantify the effect of each component. Fourty-six eligible studies were identified. Domains conferring a high risk of bias were identified in most studies. Mandatory vaccination was the most effective intervention component (Risk Ratio of being unvaccinated [RR_unvacc] = 0.18, 95% CI: 0.08–0.45), followed by “soft” mandates such as declination statements (RR_unvacc = 0.64, 95% CI: 0.45–0.92), increased awareness (RR_unvacc = 0.83, 95% CI: 0.71–0.97) and increased access (RR_unvacc = 0.88, 95% CI: 0.78–1.00). For incentives the difference was not significant, while for education no effect was observed. Heterogeneity was substantial (τ² = 0.083). These results indicate that effective alternatives to mandatory HCWs influenza vaccination do exist, and need to be further explored in future studies.

Keywords:

• influenza
• vaccination
• influenza vaccine
• vaccine coverage
• healthcare workers
• healthcare
• systematic review
• meta-analysis
• meta-regression
• epidemiology

Introduction

Healthcare workers (HCWs) are at increased risk for influenza,¹ and may transmit the virus to their patients²; therefore annual influenza vaccination has been recommended for HCWs by public health authorities worldwide for over 2 decades.^3-5 Despite the vaccine's variable year-on-year efficacy,⁶ there is evidence that influenza vaccination of HCWs can reduce patient morbidity and mortality in long term care and in hospital settings,^7-9 as well as prevent sickness absence among healthcare staff.^10,11 As a result, influenza vaccination is regarded both as an occupational health measure and as an important element in infection control, commensurate with HCWs' duty to protect patients.¹²

Despite long-standing recommendations, influenza vaccine uptake remains low in most healthcare settings and in most countries,^5,13 falling far short of specified targets that usually reach 80% or 90%. As a result, various interventions to increase coverage of HCWs have been explored,^14,15 with a particular focus on mandatory influenza vaccination, whose merits and ethics remain strongly debated.^16-21 To a large extent this debate is driven by the fact that the evidence on influenza vaccination of HCWs is not unequivocally conclusive^22,23; the benefit, in terms of both personal protection and protection of patients, is smaller and much less clear⁶ than in the case of diseases for which vaccination is universally accepted and/or mandated, such as measles²⁴ and Hepatitis B²⁵.

Setting aside the “ethics” part of the question, for which there are arguments on both sides, we tried to address the “merits” part and find out how effective mandatory vaccination is, compared to other, less controversial alternatives. To that end, we undertook a systematic review of epidemiologic studies evaluating interventions to increase influenza vaccination coverage in HCWs, including a meta-regression analysis to quantify the benefit of each intervention component. We tried to include the widest possible range of studies in our review, but focused only in seasonal (i.e. not pandemic) influenza vaccination campaigns, in order to maximize generalizability of the results.

Results

Our literature search yielded 4,925 unique (non-duplicate) citations. After screening the title and abstract, we selected 146 full-text articles for retrieval. Thirty-seven studies, plus another 9 identified from reference lists of relevant articles, met our eligibility criteria and were included in the analysis (Fig. 1).^26-71 Publication years ranged from 1992 to 2015. Most studies were BnA without a control group,^{27-33,38-40,42-46,48,50,52,54-57,59-63,65-67,69,70} and there were also 3 BnA studies with a control group,^36,68,71 9 cRCTs^{26,34,37,41,47,49,51,58,64} and 2 RCTs.^35,53 Two uncontrolled BnA studies compared the intervention year (2005 in both cases) with the second-to-last preceding year (2003), because of a reported vaccine shortage during the immediately preceding one (2004).^43,56

Figure 1. Summary of literature search and selection.

Five studies^{35,41,54,58,71} evaluated separately more than one intervention on the same population or using the same controls, and 2 studies^27,28 were also performed on the same population. In two studies,^34,42 the same intervention was evaluated in independent populations or settings. As a result, the 46 studies contributed a total of 53 comparisons, nested within 47 clusters (Table 1).

Table 1. Characteristics of included studies.

Study	Place	Study type	Setting	Study year(s)	Population
Abramson et al, 2010^26	Jerusalem, Israel	cRCT	Primary care clinics	2007	All HCWs with direct patient contact
Ajenjo et al, 2010^27^*	USA (midwest)	BnA without control	Health provider	2006, 2007	All HCWs
Babcock et al, 2010^28^*	USA (midwest)	BnA without control	Health provider	2007, 2008	All HCWs
Awali et al, 2014^29	Detroit, USA	BnA without control	Hospital	2010, 2011	All HCWs
Camargo-Angeles et al, 2014^30	Alicante, Spain	BnA without control	Hospital	2010, 2011	All HCWs
Chamoux et al, 2006^31	Clermont Ferrand, France	BnA without control	Hospital	2002, 2003	All HCWs
Chittaro et al, 2009^32	Udine, Italy	BnA without control	Hospital	2004, 2005	All HCWs
De Juanes et al, 2007^33	Madrid, Spain	BnA without control	Hospital	2002, 2003	All HCWs
Dey et al, 2001^34^‡ (a)	Bury, UK	cRCT	Primary care teams	1999	All HCWs
Dey et al, 2001^34^‡ (b)	Bury, UK	cRCT	Nursing homes	1999	All HCWs
Doratotaj et al, 2008^35^†	Ohio, USA	RCT	Hospital	2004	All HCWs with direct patient contact
Harbarth et al, 1998^36	Geneva, Switzerland	Controlled BnA	Hospital	1995, 1996	All HCWs
Hayward et al, 2006^37	England, UK	cRCT	Nursing homes	2003	All HCWs
Heinrich-Morrison et al, 2015^38	Melbourne, Australia	BnA without control	Hospital	2013, 2014	All HCWs
Honda et al, 2013^39	Sapporo, Japan	BnA without control	Hospital	2011, 2012	All HCWs
Hood et al, 2009^40	Texas, USA	BnA without control	Health provider (pediatric)	2006, 2007	All HCWs
Kimura et al, 2007^41^†	California, USA	cRCT	Nursing homes	2001, 2002	All HCWs
Ksienski et al, 2014^42^‡ (a)	British Columbia, Canada	BnA without control	Hospitals	2011, 2012	All HCWs
Ksienski et al, 2014^42^‡ (b)	British Columbia, Canada	BnA without control	Residential care	2011, 2012	All HCWs
Kuntz et al, 2008^43^§	Iowa, USA	BnA without control	Hospital	2003, 2005	All HCWs
LaVela et al, 2015^44	USA	BnA without control	Chronic care center	2012, 2013	All HCWs
Lee et al, 2007^45	Singapore	BnA without control	Hospital	2004, 2005	All HCWs
Leitmeyer et al, 2006^46	Germany	BnA without control	Hospitals	2002, 2003	Physicians, nurses
Lemaitre et al, 2009^47	Paris, France	cRCT	Nursing homes	2005	All HCWs
Llupia et al, 2010^48	Barcelona, Spain	BnA without control	Hospital	2007, 2008	All HCWs
Looijmans-van den Akker et al, 2010^49	Netherlands	cRCT	Nursing homes	2005, 2006	All HCWs
Lopes et al, 2008^50	Sao Paolo, Brasil	BnA without control	Hospital	2005, 2006	All HCWs
Nace et al, 2011^51	Pennsylvania, USA	cRCT	Nursing homes	2002, 2003	All HCWs
Nicholson et al, 2009^52	Ohio, USA	BnA without control	Hospital	2006, 2007	All HCWs
Ohrt et al, 1992^53	Dallas, USA	RCT	Hospital	1990	Medical residents and students
Podczervinski et al, 2015^54^†	Seattle, USA	BnA without control	Ambulatory care	2010, 2011	All HCWs
Quan et al, 2012^55	California, USA	BnA without control	Hospital	2006, 2007	All HCWs
Rakita et al, 2010^56^§	Seattle, USA	BnA without control	Hospital	2003, 2005	All HCWs
Ribner et al, 2008^57	Atlanta, USA	BnA without control	Hospital	2005, 2006	All HCWs except physicians
Rothan-Tondeur et al, 2011^58^†	France	cRCT	Nursing homes	2005	All HCWs with direct patient contact
Samms et al, 2004^59	Charleston, USA	BnA without control	Hospital	2002, 2003	All HCWs
Sartor et al, 2004^60	Marseille, France	BnA without control	Hospital	1999, 2000	All HCWs
Seale et al, 2011^61	Sydney, Australia	BnA without control	Hospital	2007, 2008	All HCWs
Shah et al, 2008^62	New York, USA	BnA without control	Hospital (neonatal ICU)	2004, 2005	All HCWs
Shannon et al, 1993^63	Massachusetts, USA	BnA without control	Hospital	1990, 1991	All HCWs
Slaunwhite et al, 2009^64	Halifax, Canada	cRCT	Hospital	2004, 2005	All HCWs
Smedley et al, 2002^65	Southampton, UK	BnA without control	Hospital	1998, 1999	All HCWs
Smith et al, 2012^66	Wisconsin & Illinois, USA	BnA without control	Health provider	2010, 2011	All HCWs
Stuart et al, 2014^67	Monash, Australia	BnA without control	Hospital (Dept of nephrology)	2012, 2013	All HCWs
Tannenbaum et al, 1993^68	Montreal, Canada	Controlled BnA	Nursing home	1989	All HCWs
Tapiainen et al, 2005^69	Basel, Switzerland	BnA without control	Hospital (pediatric)	2003, 2004	All HCWs
Thomas et al, 1993^70	North Carolina, USA	BnA without control	Nursing home	1990	All HCWs
Zimmerman et al, 2009^71^§	Pittsburgh, USA	Controlled BnA	Hospital	2005, 2006	All HCWs except physicians

RCT: Randomized Controlled Trial; cRCT: Cluster Randomized Controlled Trial; BnA: Before-and-after study; HCW: Healthcare worker.

* Studies performed on the same population

^† Study contributing 2 correlated comparisons (different interventions, in the same population or with the same control group)

^‡ Study contributing 2 independent comparisons (same intervention, in different populations)

^§ Comparison with second-to-last preceding year

Most comparisons were performed in a hospital or nursing home setting, 32 and 11 respectively. Seven studies were limited to particular types of HCWs (Table 1). The majority of comparisons examined an intervention with multiple simultaneous components; the exception was “hard” mandates, which were assessed in 8 comparisons (7 studies, all uncontrolled BnA^{28,29,42,54,56,66,67}) with no other simultaneous component. In one study the mandatory policy included termination of employment for unvaccinated workers, but due to reactions this was ultimately put in abeyance⁴²; we considered this intervention as a “hard” mandate nonetheless, in an “intention-to-treat” manner. Regarding the remaining intervention components, increased access (mostly involving mobile carts) was assessed in 23 comparisons, increased awareness in 27, education in 18, incentives in 11, and “soft” mandates in 7. Of the 7 comparisons assessing “soft” mandates, all but one⁶¹ involved the use of a declination form, i.e., a mandatory written statement that the HCW refuses vaccination and provides the reasons for doing so.

Regarding the risk of bias, out of the 11 RCTs or cRCTs, in 7 the method of randomization was unclear,^{26,34,35,51,53,58,64} and one study employed a factorial design with partial randomization.⁴¹ Allocation concealment in the 2 RCTs was unclear,^35,53 5 studies did not report vaccination coverage at baseline for the intervention and control groups,^{34,35,37,47,53} and 9 studies (including the 3 controlled BnA studies) did not sufficiently report on potential baseline imbalances.^{34-36,47,51,53,58,68,71} Completeness of outcome data was not clarified in all but 5 studies,^{26,41,47,49,58} and in 3 of them the response rate was low enough to potentially bias the results.^41,47,58 Most studies offered vaccination in-house and could objectively ascertain vaccine coverage^{26,35,36,49,53,64,68,71}; one study used an external occupational health service,³⁷ another used general practitioner claim forms,³⁴ 3 studies used self-report via quesionnaires to ascertain vaccination,^41,47,58 and one study did not report the method of ascertainment.⁵¹

Of the uncontrolled BnA studies, none described particular concurrent events that could influence the post-intervention vaccination coverage, except in the 2 studies that reported a vaccine shortage during the previous year^43,56; although in these the comparison was made with the second-to-last year, the shortage might still have biased the post-intervention vaccination coverage. In five studies ascertainment of vaccination was via self-report,^{29,44,46,61,62} and in 2 the method was unclear^59,67; the remaining studies used objective means to ascertain vaccination. Completeness or near-completeness of outcome data could be established for 6 studies,^{27,28,39,45,62,69} while 3 studies used questionnaires and had a low (<50%) response rate, raising the possibility of selection bias. No study (including the controlled studies) had a reliable method of tracking participants that were vaccinated off-site or outside study arrangements.

Regarding calculation of effect measures, in 9 studies^{26,35,36,41,51,58,64,68,71} we employed the modified Poisson regression using GEE⁷² to calculate RR_unvacc using the available study data, while for the remaining studies we reconstructed contingency tables. In another 9 studies^{26,34,37,41,47,51,58,64,71} we applied a correction for clustering to the RR_unvacc by calculating the design effect; in six cases^{26,34,41,51,64,71} the ICC was provided or could be indirectly estimated from the study data, and in 3 cases^37,47,58 we had to resort to an external ICC estimate of 0.1 (an approximate mean of the 6 known/estimable ICC coefficients).

Entering all comparisons into the meta-regression model (Fig. 2), we found “hard” mandates to be very effective for increasing influenza vaccine coverage (RR_unvacc = 0.18, 95% CI: 0.08–0.45, p = 0.003). “Soft” mandates were also effective, but to a smaller extent (RR_unvacc = 0.64, 95% CI: 0.45–0.92, p = 0.022). Statistically significant effectiveness was found for increased access (RR_unvacc = 0.88, 95% CI: 0.78–1.00, p = 0.044) and increased awareness (RR_unvacc = 0.83, 95% CI: 0.71–0.97, p=0.02), for incentives the difference did not reach statistical significance (RR_unvacc = 0.89, 95% CI: 0.77–1.03, p = 0.12), while for education no effect was observed (RR_unvacc = 0.96, 95% CI: 0.84–1.10, p=0.57).

Figure 2. Forest plot: results from individual studies and random-effects meta-regression model (logarithmic scale). Vertical bars before study names indicate comparisons that are clustered together.

Substantial heterogeneity was identified⁷³; between-cluster variance τ² was 0.083, while within-cluster variance ω² was zero, indicating no clustering of effects between studies performed on the same population or using the same control group. The I² statistic⁷⁴ was 99.5%, meaning that almost the entire variance was due to differences between studies and not due to sampling; the large I² value is not unexpected though, given the large number of studies and the small standard errors for most of effect estimates.⁷⁵ Compared to an intercept-only model (τ² = 0.116, ω² = 0), a quarter of the variance was found to be explained by the moderators (pseudo-R² = 28.9%).

Visual inspection of the funnel plot of meta-regression residuals against their standard error identified a slight asymmetry (Fig. 3), while the Egger regression asymmetry test was statistically significant (p = 0.012), indicating possible publication bias or small-study effects. The funnel plot also highlights the studies contibuting most on the observed heterogeneity (those lying either side of the funnel).

Figure 3. Funnel plot of meta-regression residuals (observed–fitted log Relative Risk) against their standard errors.

The test of interaction with study type was not statistically significant for the first 4 intervention components, indicating that uncontrolled BnA studies did not produce different results compared to other designs; studies evaluating “soft” or “hard” mandates were all uncontrolled BnA, thus no interaction terms could be fitted. Regarding post- vs pre-pandemic studies, we did not observe statistically significant interaction for any intervention component but the term for “hard” mandates came very close to significance (p = 0.0506); indeed, among the studies assessing “hard” mandates, the 2 that had been conducted before the 2009 H1N1 pandemic^28,56 showed greater effectiveness than those conducted afterwards.^{29,42,54,66,67} Finally, in the model that included an intercept and the intervention year, no temporal trend for the effectiveness of the reviewed interventions was identified (yearly change ratio of RR_unvacc = 1.00, 95% CI: 0.98–1.03, p = 0.62).

Discussion

Our study indicates that it is possible to increase influenza vaccine coverage in HCWs, and there are various ways of doing so. We have found that mandating influenza vaccination, with consequences such as termination of employment for those refusing, is by far the most effective single intervention. There may be practical problems in its implementation, however, as it may cause resentment or opposition among HCWs⁷⁶; in one case, unions launched a successful legal challenge to the mandatory vaccination policy.⁴² As a result, despite the high effectiveness of “hard” mandates, exploring alternative options is worthwhile and useful.

Declination statements (which formed the bulk of the “soft” mandates component) can be considered a form of mild pressure, a way to make hesitant or indifferent HCWs accept the vaccine, or think seriously about their reasons for refusing. In our review, declination statements were found to be highly effective, indicating that they can be an important element in any campaign to vaccinate HCWs against influenza. Other intervention components, such as increased access, increased awareness and incentives (for which the difference did not reach statistical significance) were found to be less effective; in combination though, their cumulative effect could match that of declination statements. Notably, educational interventions were not found to be effective on average; however, as different interventions may have variable impact on the various categories of HCWs,^46,77 the role of education should still be examined in more detail for particular HCW groups.

Based on the above findings, an “all of the above” approach can be suggested to increase influenza vaccine coverage of HCWs, as an alternative to mandatory vaccination. Declination statements should be the key intervention in such an approach, complemented by better on-site access to vaccination, active promotion of the campaign and possibly other targeted interventions such as incentives and education. Prerequisites for all that include managerial commitment, provision of the necessary resources, and engagement of all stakeholders in the goal of raising influenza vaccine coverage.

In our review we did not include studies assessing pandemic vaccination coverage of HCWs. However, the 2009 H1N1 pandemic could potentially also affect the willingness of HCWs to get vaccinated against seasonal influenza, modifying the effect of interventions to increase coverage. Our analysis did identify a trend toward lower effectiveness of “hard” mandates after 2009, indicating that HCWs may be more reluctant, or even suspicious against being pushed too hard on influenza vaccination. This needs to be considered by planners of vaccination campaigns, and investigated in future studies.

The risk of bias is thought to be greater for BnA studies without a control group; indeed the factors that have prompted the study and intervention may affect comparability with the pre-intervention year. Nevertheless we did include these studies in our review, knowing that they form the bulk of the available literature on the subject at hand (and for certain intervention components, i.e. mandates, the only available literature). In addition, systemwide interventions on entire populations of HCWs leave less potential for selection bias, and taking only the last previous season maximizes comparability of the population. The use of a control group in a BnA study may partly guard against hypothetical temporal trends (which we did not find in our particular analysis) or unexpected events, but raises the issue of comparability between groups. Even the randomized studies in our review had a high risk of bias in several domains. Thus we pooled all available evidence and a test of interaction was performed to assess whether uncontrolled BnA studies gave materially different estimates than other study types; no statistically significant interaction by study type was observed for any intervention component.

Our review has several advantages: a clear objective, a comprehensive systematic literature search with specific eligibility criteria, a risk-of-bias assessment for the included studies and a meta-analytic synthesis of results. Moreover, our methodology allows us to disentangle the effects of individual components across studies very heterogenous in terms of interventions implemented; this is a particular strength of our study. However, substantial heterogeneity still remains, as indicated by the large τ² and modest R² (the inflated I² is rather misleading in our case⁷⁵). Various study-specific factors can well account for this heterogeneity, namely: populations studied (all HCWs, HCWs with patient contact, or particular types of HCWs); clinical setting (nursing homes, hospitals, other); country (different cultures may impact intervention effectiveness); the specific details of each intervention (the 6 components are by definition not completely homogenous) and the way these were implemented in each study. Also, some studies used subjective methods (self-report) to ascertain vaccination, and no study could reliably track participants vaccinated off-site, thus contributing to overall heterogeneity. As a result, the pooled effect sizes for each intervention component should not be interpreted as a fixed and absolute truth, valid to all settings; rather they should be viewed as average effects across all reviewed studies, reflecting the sum of current knowledge on the subject. Furthermore, the funnel plot and Egger test result potentially indicate the existence of publication bias, a finding that also needs to be taken into account.

Another limitation in the analysis is the fact that we did not examine potential interaction effects between intervention components; doing so with sufficient statistical power would require far more studies than the 45 available. The assumption of additive effects in our meta-regression model is not unreasonable though; indeed the 3 studies in our review that directly evaluated the combination of 2 intervention components did not demonstrate a statistically significant interaction effect.^35,41,71 In addition, the data were not sufficient to calculate subgroup pooled effects for different clinical settings (hospitals, nursing homes, primary care) or different groups of HCWs (physicians, nurses, other HCWs); therefore we cannot conclude on whether certain interventions may be more effective in some settings or for particular groups of HCWs.

In conclusion, we quantified the average effectiveness of the different components that can be employed in a campaign to increase influenza vaccination coverage in HCWs. We demonstrated that mandates, either “hard” or “soft” (declination statements), are most effective, followed by other interventions such as increased access and increased awareness. Educational interventions did not appear effective on average, although they could still be useful for particular groups of HCWs. Further research is warranted on the issue, along with commitment by all stakeholders to promote influenza vaccination of HCWs as a patient safety and quality of care issue. In our opinion, however, increasing coverage is best done in a cooperative spirit, as part of a safety culture and without punitive measures,⁷⁸ especially not before having exhausted all available options first.

Materials and methods

Literature search and selection

We searched MEDLINE and Scopus databases for published articles using the following combination of keywords: vaccin* AND (influenza OR flu) AND (“healthcare worker(s)” OR “health worker(s)” OR “health personnel” OR “health staff” OR “physician(s)” OR “doctor(s)” OR “nurse(s)” OR “practitioner(s)”). We applied no date or language restrictions, using an automated translation service (Google Translate) when necessary. In addition, we searched the reference lists of relevant papers to identify additional studies. We did not consider conference abstracts or other data not published in the peer-reviewed literature.

Literature search and selection was performed by 4 reviewers (TL, FK, EM, DP) working in pairs, i.e., each abstract and full text was reviewed twice. Discrepancies were resolved by consensus. We selected studies evaluating a campaign or otherwise defined intervention to increase vaccination coverage in HCWs; interventions should be clearly defined and include one or more components, either separate or concurrent. To be eligible, a study had to compare as outcome the vaccination coverage between the intervention and control groups, and provide sufficient numerical data to calculate an appropriate effect measure for the intervention, as described below. We did not include studies assessing “intention to be vaccinated” as outcome, only studies comparing actual vaccination rates. Eligible study types were Randomized Controlled Trials (RCTs), Cluster Randomized Controlled Trials (cRCTs) and Before-and-After studies (BnA) both controlled and uncontrolled. For uncontrolled BnA studies, the comparison should be made between the intervention year and the immediately preceding year, unless the study authors provided a clear and specific reason to suggest non-comparability (e.g. vaccine shortage) in that particular year; in that case comparison was made with the next preceding year. For each selected study, we used the criteria suggested by the Cochrane Effective Practice and Organization of Care group⁷⁹ to assess the risk of bias. Studies performed during the pandemic season 2009–2010 were excluded from our review.

Definition of interventions and effect measure

Two reviewers (TL, SB) abstracted the data and performed the analysis. For each study, intervention components were identified and codified into 6 broad categories (Table 2): increased access, increased awareness, education, incentives, “soft” mandates and “hard” mandates. Each intervention could consist of one or more of these components, evaluated concurrently (one intervention group, one comparison) or separately, as for example in a factorial design (multiple intervention groups, multiple comparisons).

Table 2. Intervention components to increase influenza vaccine coverage in HCWs.

Component	Description / Examples
1. Increased access	Any measure to make vaccination easier and more convenient, such as: free vaccine (without cost); extended vaccination hours; vaccination at the workplace using mobile carts; peer vaccination; vaccination fair.
2. Increased awareness	Non-educational measures to advertise any aspect of vaccination: posters, pamphlets, flyers, letters, reminders, newsletters, badges, etc. Also: personal advocacy (vaccination “champions”), provision of feedback regarding vaccination goals.
3. Education	Formal educational interventions such as: presentations, lectures, video projections, meetings, questionnaires, etc.
4. Incentives	At the individual level (gifts, perks, raffles, etc) or at the group level (vaccination fair with free drinks, bonus/reward for meeting vaccination targets, etc).
5. “Soft” mandate	Declination forms; vaccination mandates with no severe consequences for unvaccinated HCWs, or without enforcement.
6. “Hard” mandate	Mandatory vaccination as a condition for employment, or with severe restrictions for unvaccinated HCWs (such as forbidding patient contact or having to wear a mask).

Different relative effect measures can be used to compare vaccination coverage between the intervention and control group: the Risk Difference, the Odds Ratio and the Risk Ratio. For rare outcomes the choice is inconsequential; it becomes important, however, when the probability of the outcome can vary widely, because different relative effect measures imply different absolute effects over the range of baseline risks.⁸⁰ In the case of influenza vaccination, a given intervention to increase coverage is expected to have a progressively smaller absolute benefit with increasing baseline coverage (Fig. 4). For this reason, we chose the Relative Risk of being unvaccinated (RR_unvacc) as the most appropriate measure to consistently quantify the effect of an intervention across the entire range of baseline coverages; values of RR_unvacc < 1 suggest that the intervention is effective in reducing the number of unvaccinated HCWs, i.e. improving vaccine coverage.

Figure 4. Relationship between absolute risk difference and baseline risks for different effect measures. Using OR as an intervention effect measure implies that, for a given intervention effectiveness, the absolute benefit (i.e., absolute risk difference) is maximal when baseline risks are near 50%. Using RR_inv implies that the absolute benefit gets smaller as baseline risks increase. The latter is a much more reasonable assumption in the case of vaccination coverage, since if more HCWs are already vaccinated at baseline, fewer additional HCWs will get vaccinated for a given intervention effectiveness.

Statistical analysis

For every comparison in our review, we calculated the RR_unvacc and its log standard error; for simpler designs (such as uncontrolled BnA) we used the available study data to reconstruct contingency tables, while for more complex designs (such as controlled BnA or factorial designs) we employed a modified Poisson regression using Generalized Estimating Equations (GEE).⁷² In case of a clustered design, we also used the available study data to calculate the Intraclass Correlation Coefficient (ICC) and the design effect, in order to appropriately inflate the standard errors.⁸¹

All effect sizes and standard errors were subsequently entered into a random-effects meta-regression model with 6 binary predictors, one for each intervention component (Table 2), and no intercept. The aim was not to derive a pooled effect estimate for all studies, but to assess the effectiveness of each intervention component in improving vaccination coverage. Some comparisons were made on the same population or with the same controls, within the same study or even across different studies. In order to account for this clustering, we used a robust meta-regression approach with hierarchical dependence structure as described by Hedges et al.⁷³ A big advantage of this method is that it does not require meta-regression covariates to be fixed, unlike in other meta-regression approaches⁷³; assuming fixed covariates makes little sense within a random-effects framework, particularly when the focus is on the regression coefficients and not on a pooled effect estimate.

Publication bias was assessed using a funnel plot of study meta-regression residuals against their standard errors, and with the Egger regression asymmetry test.⁸² The risk of bias in uncontrolled BnA studies is considered to be higher than controlled BnA or cRCTs. To assess whether the results of uncontrolled BnA studies gave different results in our review, we did a test of interaction by re-fitting our model and including, for each predictor, an interaction term with study type (expressed as a binary variable: 1 = uncontrolled BnA, 0 = other study types). In similar fashion, in order to check whether the experience of the 2009 H1N1 influenza pandemic modified the effectiveness of each intervention component, we included interaction terms for post-pandemic vs pre-pandemic studies. In addition, we sought to identify a temporal trend in the effectiveness of the reviewed interventions by including in the model an intercept and the year that the intervention took place.

This review was conducted in accordance with the MOOSE guidelines.⁸³ All statistical analyses were performed with version 3.2 of the R software environment⁸⁴ using the packages “robumeta,” “metafor”⁸⁵ and “geepack”.⁸⁶

Disclosure of potential conflicts of interest

The authors declare that they have no conflicts of interest.

Authors' contributions

TL conceived the idea for this review and designed the search strategy, inclusion criteria and analysis protocol. TL, FK, EM and DP searched the literature and selected the studies. TL and SB extracted and analyzed the data, interpreted the findings and drafted the initial version of the manuscript. All authors participated in the further development of the manuscript and approved the final version for publication.