https://orcid.org/0000-0002-0239-7297
?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.ABSTRACT
Seasonal influenza continues to be a major cause of illness and death. Vaccination is the most cost-effective prophylaxis to prevent the disease and it is particularly important for people who are at high risk of serious complications derived from influenza, especially for people ≥65 years. In Italy, the influenza vaccination program has been unsuccessful with low rates of uptake in people ≥65 years. We analyzed all the community ≥65 years of the Health Promoting Agency (HPA) of Brescia (northern Italy) to evaluate the propensity attitudes toward influenza vaccination among people ≥65 years in four consecutive seasonal influenza campaigns (from 2014/2015 to 2017/2018). Information about subjects were retrieved from administrative databases. Data from 952,822 records were analyzed. The prevalence of vaccinated subjects in the four campaigns was 38.6%, 33.7%, 37.7%, and 40.1%, respectively. Among vaccinated people, the frequencies of individuals aged 65–69.9 years were lower than the frequencies of those in the other age classes, with highest frequencies of vaccinated people in the 75–79.9 years age-class. Overall, males showed a slightly higher propensity to be vaccinated and the propensity toward vaccination increased with age in both genders. Suffering from a chronic disease increased the propensity to vaccination; hypertension had the highest impact on the propensity whereas suffering from vasculopathy has the opposite effect.
The value of this study is the possibility to know the factors that might indicate a propensity to get an influenza vaccination and to consider a different approach to people ≥65 years with the characteristics indicating a lower propensity to vaccination.
1. Introduction
Seasonal influenza continues to be a major cause of illness and death, especially among the elderly and those with underlying serious comorbidities. Citation1 The World Health Organization (WHO) estimates that influenza affects 5–15% of the population and is responsible for 3 to 5 million cases of severe illness and about 290,000 to 650,000 deaths per year. Citation2 In the United States (US), it has been estimated that the annual influenza-associated mortality rates per 100,000 person/year is 19.6 and this rate increase up to 132.5 in the elderly (≥65 years).Citation3 The impact of influenza on health-care costs and productivity are substantial; in the US, influenza epidemics have been estimated to cost USD 87 billion (EUR 77 billion) per year,Citation4 and a recent review has estimated the direct and indirect cost of seasonal influenza ranging from USD 215 million (EUR 190 million) to USD 3.9 billion (EUR 3.5 billion) per year for European countries.Citation5
Vaccination is the most cost-effective prophylaxis to combat the disease and it is particularly important for people at high risk of influenza complications – pregnant women, the very young and very old people, immunocompromised people, and people with chronic underlying medical conditions – and for people who live with or care for the people at high risk.Citation6–Citation8 The World Health Organization (WHO) recommends annual vaccination for high-risk groups.Citation2,Citation9
People over 65 years are included among high-priority groups targeted for annual vaccination since the highest rates of deaths associated with influenza are recorded in this age groupCitation10 where it has been estimated that vaccination can reduce influenza-related morbidity by 60% and influenza-related mortality by 70–80%.Citation2 In Italy, according to the disposition of the Ministry of Health,Citation11 the influenza vaccination campaign starts at the beginning of November and stops at the end of December every year. Influenza vaccination is offered free of charge for groups at increased risk of influenza complications, such as the elderly ≥65 years of age and patients affected by chronic diseases. General practitioners (GPs) are in charge of the vaccination for population over 65 years and of patients with chronic diseases. The influenza vaccination program in Italy has been unsuccessful with low rates of uptake in elderly (≥65 years) (coverage rates under the minimum target level of 75%, with no differences between regions) and in individuals at high risk of developing influenza-related complications.Citation12 Among factors producing the low rate of influenza vaccination uptake in elderly, the vaccine hesitancy is a complex and multifactor-related one.Citation13 Over the last years, researchers have tried to identify potential barriers driving the influenza vaccine skepticism to better comprehend the phenomenon.Citation13 Understanding clinical and demographical factors associated with influenza vaccination can help identify targets for communication policies to develop strategies for maximizing influenza vaccine uptake.
In order to investigate the factors associated with attitudes toward influenza vaccination among elderly people, the population of the Health Protection Agency (HPA) of Brescia in northern Italy was considered as reference. The HPA is a unit of the public Italian National Health Service (INHS). To tailor the services on the need of the population, INHS is organized at regional level and it is delivered by HPA, which is the interface of the INHS with the population. In this study, we evaluated, through Generalized Linear Models (GLMs), how the prevalence of influenza vaccination varies among subjects according to demographic and health-related variables recorded in administrative databases. We studied all the community aged 65 years and over of the HPA of Brescia in the northern part of Italy in the four most recent influenza vaccination campaigns (from 2014/2015 to 2017/2018). Since in this period there were no changes in health policies for promoting influenza vaccination among elderly people, a comprehensive evaluation of the impact of the putative factors was performed by separate analyses for each campaign.
2. Methods
2.1. Samples and variables of interest
The study is based on the data included in the information system of the HPA of Brescia. This information system collects all the health data of the population generated by general practitioners, pharmaceutical consumption, hospital records, ambulatory care, vaccinations and that is linked with the office of vital statistics. In particular, information about all subjects 65 years and over referring to this HPA from 2014 to 2018 were retrieved from administrative databases, namely population database and immunization registryCitation14 with reference to four influenza vaccination campaigns (from 1 October to 31 December in the years 2014, 2015, 2016, and 2017). The four influenza vaccination campaigns (from 2014/2015 to 2017/2018) were selected because they are the first four seasons complete of all the variables of interest.
The variables of interest were:
Influenza vaccination status (binary variable, equal to 1 for each subject who underwent vaccination and 0 otherwise);
Demographic characteristics: gender and age (codified as categorical variable, using the following age classes: 65.0–69.9; 70.0–74.9; 75.0–79.9; 80.0–84.9, 85.0–89.9; 90.0–94.9; 95.0–99.9; 100.0–114.9 years); as a linear relationship between age and vaccination propensity is not realistic, the variable was classified to obtain a simpler representation and interpretation of model results. The number of age classes considered was large enough to obtain a sufficiently detailed evaluation of the association.
Presence of comorbidities: (15 binary variables: transplant, renal insufficiency, HIV/AIDS, neoplasms, diabetes mellitus, hypertension, cardio-vascular disorders, bronco-pneumatic, hepatic diseases, gastro-intestinal disease, neurological disorders, autoimmune diseases, endocrine pathologies, dysplasia, and rare diseases). The presence of comorbidities was validated by the presence of a certification of diseases registered in the database.
ealth events: admission to emergency unit, hospital admission, and death.
We selected four groups, each one including all subjects with age 65 or over at the beginning of one of the four influenza campaigns (1 October). Subjects with the following characteristics were excluded:
Subjects with missing date of influenza vaccination or vaccinated outside the campaign period (from 1 October to 31 December) (N = 2097);
Subjects deceased before the end of influenza vaccination campaign (by 31 December) (N = 1102), and subjects with inconsistent event recordings (e.g. date of hospital admission after date of decease) (N = 182).
The final data consisted of 952,822 records.
2.2. Data analysis
The analyses described in the following were performed for each of the four groups. The characteristics of the groups were summarized using counts and percentages. The association between vaccination status (response variable) and the other variables of interest (independent variables) was evaluated through GLMs, which use the binomial distribution and the link log. This was applied to estimate the prevalence ratios of vaccination.
To interpret the strength of association, several proposals in epidemiological literature are available. Different cutoffs have been reported to define the classification of the strength of association. Among the proposals, we decided to use the classification given by Schoenbach and Rosamond.Citation15 Thus, to interpret the strength of the association between vaccination and putative factors, the prevalence ratio was interpreted according to the following criteria:
- equal to 1: no association;
- greater than 1 and less than or equal to 1.3: weak positive association (greater than or equal to 0.77 and less than 1: weak negative association);
- greater than 1.3 and less than or equal to 1.7: modest positive association (greater than or equal to 0.59 and less than 0.77: modest negative association);
- greater than 1.7 and less than or equal to 3.0: moderate positive association (greater than or equal to 0.33 and less than 0.77: weak negative association);
- greater than 3.0: strong positive association (less than 0.33: weak negative association).
Demographic and health variables were included in the model as independent variables, using dummy variable coding. Estimates of the adjusted prevalence ratios of vaccination for each independent variable were reported. To summarize model explained variability we used the index tau-p for logistic regression models described by Menard.Citation16 This is a coefficient of determination that, in our context, gives a measure of predictive efficiency of the propensity scores in the task of identifying subjects undergoing and not undergoing vaccination. The index tau-p ranges from −1 to 1, corresponding to perfect and worst prediction, respectively. The value 0 is obtained when there is no association between the observed and the predicted classification, and thus positive values of the index quantify the usefulness of the score.
To describe in more detail how the propensity to undergo influenza vaccination varies according to the characteristics of interest, a normalized score that “ranks” the propensity of subjects was calculated. To such end, we used the formula below, which rescales the values of the linear predictors of the fitted models within the range [−1, 1]:
where indicates the estimated model linear predictor, and the suffix i indicates the i-th subject in the group. Due to the properties of the linear predictors, the score is a monotone function of the conditional probability of vaccination (conditional to the covariates included in the model) of each subject. Histograms showing the distribution of the score in the four groups were reported. Also, values of the score were reported for subjects with selected patterns of covariates, with respective cumulated frequencies, which give indications about the rank of subjects in terms of propensity to vaccination. The analyses were performed using the software R (release 3.5.1) and KNIME analytics platform (release 3.6.0).Citation17,Citation18
3. Results
3.1. Characteristics of the four groups
The four groups consist of 232,173, 236,146, 240,065, and 244,809 subjects, for the campaigns 2014/15, 2015/16, 2016/17, and 2017/18, respectively. The amount of excluded records was lower than 0.50% in each case (maximum: 0.46%).
The unadjusted prevalence of vaccinated subjects was 38.6% in the 2014/2015 campaign, 33.7% in 2015/2016, 37.7% in 2016/2017, and 40.1% in 2017/2018. The main characteristics of the subjects are summarized in .
Table 1. Demographic variables (gender and age) and the presence of comorbidities of the elderly people in four seasonal influenza vaccination campaigns (2014/2015–2017/2018).Reported in the table there are numbers and percentages of subjects for each category of gender, age group and for each comorbidity. The percentages refer to the number of subjects for vaccinated, not vaccinated and total, for each campaign, respectively. For each variable with only two categories (gender, comorbidity) results were reported for one category only. Panel 1: 2014/2015 and 2015/2016 seasonal influenza campaigns. Panel 2: 2016/2017 and 2017/2018 seasonal influenza campaigns.
The distributions of demographic variables (gender and age) are comparable across the four groups. Of note, the differences of relative distributions of age between vaccinated people and the overall population of people with more than 65 years, in particular for the first age classes, were: in vaccinated subjects, the frequencies of the first age class (65.0–69.9 years) were lower than the frequencies of the following age classes, whereas in the overall population such frequencies were the highest ones. For vaccinated subjects, the frequencies had a peak in the 75.0–79.9 years age class and in the overall population the frequencies showed a decreasing trend.
As concern, the presence of comorbidities, the prevalence of hypertension, diabetes, dysplasia and neoplastic diseases (i.e. the most common morbidities) are rather constant across the four campaigns, except for an increase of hypertension (roughly 5%), dysplasia (roughly 5%) and neoplastic diseases (roughly 1.5%) in the fourth campaign.
reports information about the cohorts in the four vaccination campaigns. Each year, roughly 5% of subjects “left” the cohort before the next campaign, while the remaining 95% was stable.
Table 2. Variation in the composition of the population across seasonal influenza campaigns (2014/2015–2016/2017) and estimation of the number of the vaccinated vs. not vaccinated in two consecutive campaigns. In the first row, percentages refer to the number of subjects common to the consecutive vaccination campaigns. In the second row, are reported, for VACCINATED: number of subjects vaccinated in two consecutive campaigns/number of subjects vaccinated in the first one, and respective percentage; for NOT VACCINATED: number of subjects not vaccinated in two consecutive campaigns/number of subjects not vaccinated in the first one, and respective percentage.
The frequencies of subjects who underwent influenza vaccination in two consecutive campaigns ranged from 75.9% to 85.1%, while the frequencies of subjects who do not underwent vaccination for two consecutive seasons were higher (from 84.8% to 90.6%).
3.2. Adjusted prevalence ratios
As a concern the predictive efficiency of the fitted models, the determination index tau-p values were 0.26, 0.22, 0.24 and 0.25, for campaigns I-IV, respectively. The estimated adjusted prevalence ratios of vaccination are reported in . For each campaign, males showed a slightly higher prevalence of vaccination than females (holding constant all the other covariates included in the model). With reference to age, the prevalence of vaccination was lower among subjects in the first age class (65.0–69.9 years, used as reference class) than in older ones. The highest prevalences concerned the age classes 75.0–79.9 years and the following, in which the prevalence often resulted greater than 2-fold with respect to the reference class. Thus, a moderate positive association was found with respect to the reference class.
Table 3. Estimated adjusted prevalence ratios of Seasonal influenza vaccination (PR) with respective 95% Confidence Intervals (CI) according to gender, age groups and comorbidity in the four seasonal influenza campaign (2014/2015–2017/2018).
For each campaign, transplants, neoplasms, diabetes, hypertension, bronco-pneumatic, hepatic diseases, gastric diseases, endocrine diseases, HIV, and dysplasia showed a weak positive association with vaccination. The highest impact was shown for hypertensive subjects, for whom the adjusted prevalence ratio was greater than 1.2 in each campaign. Vasculopathies had a weak negative association, as shown by estimates lower than 1 (except for the fourth campaign, when the prevalence ratio is practically negligible): so, it may be deduced that the prevalence of vaccination is higher in subjects not affected by vasculopathies. Finally, weak associations were shown by renal insufficiency, neuropathic diseases, autoimmune and rare diseases, but the “direction” is not the same across the four campaigns.
3.3. Score of “propensity” to be vaccinated based on health and demographic characteristics
The histograms in show the distribution of the normalized score: negative values of the score pertain to subjects with lower propensity to be vaccinated, whereas positive values pertain to more “inclined” ones. The value zero is attained when the log-propensity (i.e. the model linear predictor) is exactly halfway between its minimum and maximum values.
Figure 1. Distribution of normalized scores for vaccinated and not vaccinated subjects in each of the four campaigns.
In the “ideal” situation, where vaccinated and not vaccinated subjects have completely distinct characteristics, there should be no overlap between their scores. This is not the case of the examined groups. As shown in , in each campaign, vaccinated subjects had higher frequencies of positive scores than not vaccinated ones. However, in agreement with the values of the determination index previously reported, the distributions of the score for vaccinated and not vaccinated subjects are not well separated. This result may be attributed to the fact that no change of vaccination policies was promoted throughout the period considered in this analysis.
In the coefficients that allowed calculating the normalized scores as a function of the characteristics of subjects are reported. and allows to calculate the probability to undergo vaccination as a function of subject characteristics. It is shown that for each value of the score, the corresponding probability of undergoing vaccination is lower for the 2015/16 campaign as compared to the probabilitites for the other campaigns.
Table 4. Coefficients for calculating the normalized scores of “propensity” to be vaccinated as a function of the characteristics of subjects in study.
Table 5. Normalized scores of “propensity” to be vaccinated based on health and demographic characteristics for each influenza campaign, and respective cumulated frequencies, for subjects with selected characteristics.
Figure 2. Relationship between normalized scores and the probability of undergoing vaccination.
For illustrating the use of and we report some example in the followings. A women aged between 65 and 69 years without any comorbidity in the 2017/18 campaign has a score of −0.973; from the probability to undergo vaccination is about 0.20 (or 20%). A male with the same characteristics has a score of −0.886, corresponding to a probability of about 0.21. A woman aged between 80 and 84 years and hypertension as the only comorbidity has a score of 0.101, corresponding to about 0.48. A male with the same characteristics has a score of 0.188, corresponding to a probability of 0.51. Details for calculating the normalized score for each combination of characteristics are illustrated in the legend of .
The greatest contribution to the score was provided by age classes (higher than the reference class) followed by hypertension, transplant, bronco-pneumatic disease and dysplasia. For simple situations, such as males and females without comorbidities for every age class, or males and females with only one comorbidity and selected age classes, the scores were reported in along with the respective cumulated frequencies. For each vaccination campaign, the scores were near the minimum value for males and females aged 65.0–69.9 years (from −0.89 to −0.66 for males, and from −0.97 to −0.77 for females). Females in that age class are close to the minimum level of propensity, in fact, in each campaign the cumulated percentage of subjects with lower scores varies from 0.1% to 0.2%. Males in the same age group had a slightly higher propensity: in fact, the cumulated frequencies in the four groups varied from 5.6% to 6.9%. For each campaign and for each age class the prevalence of vaccination was higher in males than in females (in agreement with the values of the prevalence ratios by gender).
Overall, the score values tended to increase in age classes following the first one: for example, in the first campaign, for females in the 70.0–74.9 age class the cumulated frequency was slightly higher than 25%, so the score was close to the first quartile in the group. The highest score values pertained to the oldest males and females in each campaign except the second one.
For subjects with comorbidities, the most common comorbidities were considered (i.e. neoplastic diseases, diabetes, hypertension, and dysplasia). In , the respective scores are reported for males and females and for two age classes (65.0–69.9 and 80.0–84.9 years). Such scores were greater than scores of the same gender and same age subjects with no comorbidities. Overall, hypertension and dysplasia had a higher impact on the propensity to be vaccinated than neoplastic diseases and diabetes. When the presence of one of this comorbidities was “combined” with a higher age, the score reached higher levels. In , the maximum values were reported for hypertension, with cumulated frequencies ranging from 67.2% to 75.1% for females and from 76.1% to 87.4% for males.
4. Discussion
Influenza is a potentially serious disease: every season millions of people get influenza infection, hundreds of thousands of people are hospitalized and thousands or tens of thousands of people die from influenza-related complications every year.Citation19 Vaccination to prevent influenza is particularly important for people who are at high risk of serious complications derived from influenza and especially for older people.Citation1 To reach the best level of protection everyone who is at risk should be vaccinated annually.Citation2
In Italy, since its implementation, the influenza vaccination program has been unsuccessful, since it reached low rates of uptake in people aged over 65 years by far under the minimum target level of 75%.Citation8 Among factors producing the low rate of influenza vaccination uptake in elderly, there is vaccine skepticism.Citation9
We analyzed all the community aged over 65 years of the HPH of Brescia to evaluate the “propensity attitudes” toward influenza vaccination among the elderly people of the population of the HPA of Brescia.
This community was followed for four consecutive seasonal influenza campaigns (from 2014 to 2017) to find out if there are any particular factors which might help to increase the adherence to influenza vaccination and to implement the program with different approaches according to different characteristics.
Data from 952,822 records were analyzed. The prevalence of vaccinated subjects in the four campaigns was 38.6%, 33.7%, 37.7%, and 40.1%, respectively. Of note, among vaccinated people, the frequencies of individuals aged 65.0–69.9 years were lower than the frequencies of those in the other age classes, with the highest frequencies of vaccinated people in the 75–79.9 years age class.
It is well-known that influenza vaccine efficacy changes according to age, decreasing significantly with the increasing of age.Citation20 However, even if the vaccine efficacy in elderly is lower than that in other age groups, influenza vaccination remains the best cost-effective measure to reduce the burden of disease in terms of hospitalization and deaths in this population.
The most prevalent comorbidity in the study population were hypertension, diabetes, and cancer, and increased in the last campaign because of the rising age of the study population.
Across the four campaigns the community was stable and only 5% of subjects “left” the cohort before the next campaign. The percentage of subjects who did not undergo influenza vaccination in two consecutive campaigns ranged from 85% to 91% of the cohort, on the contrary, only 75-85% underwent vaccination in two consecutive campaigns.
In the four campaigns, males showed a slightly higher propensity to be vaccinated than females and the propensity toward vaccination increased with age in both genders. Suffering from a chronic disease (for example diabetes and hypertension) increased the propensity to vaccination.
Among comorbidities, hypertension had the highest impact on the propensity to vaccination whereas suffering from vasculopathy has the opposite effect.
One of the limit of this study is due to the system of classification of waiver for chronic conditions: if a patient has more than one waiver, the most critical was selected and it was not possible to know all the others. Moreover, this study does not include information on the extent of exposure to the vaccination campaigns or whether people received doctor recommendation to take the vaccine. The value of this study is the possibility to know the factors that might indicate a propensity to get an influenza vaccination and to consider a different approach to people aged over 65 years with the characteristics indicating a lower propensity to vaccination. The evaluation of the propensity score can help to inform the development of tailor-made communication strategies according to specific characteristics of the population (age and specific comorbidities). Also, profiling those who are not eager to get the influenza vaccination can drive specific interventions to sustain the vaccination programmes. As reported by others,Citation21 data assessing confidence in vaccines (such as the propensity score) can help policymakers to recognize the effects of their interventions on immunization attitudes and more effectively allocate resources to build confidence.
References
- CDC. Influenza (Flu) viruses. [ accessed 2019 Nov 13]. https://www.cdc.gov/flu/about/viruses/index.htm.
- WHO. WHO. Influenza. Fact Sheet N. 221. 2003 [accessed 2019 Nov 13]. https://www.who.int/mediacentre/factsheets/2003/fs211/en/.
- Thompson WW, Shay DK, Weintraub E, Brammer L, Cox N, Anderson LJ, Fukuda K. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA. 2003 Jan;289(2):179–86. doi:10.1001/jama.289.2.179.
- Molinari NA, Ortega-Sanchez IR, Messonnier ML, Thompson WW, Wortley PM, Weintraub E, Bridges CB. The annual impact of seasonal influenza in the US: measuring disease burden and costs. Vaccine. 2007 Jun;25(27), 5086–96.
- Peasah SK, Azziz-Baumgartner E, Breese J, Meltzer MI, Widdowson MA. Influenza cost and cost-effectiveness studies globally - a review. Vaccine. 2013 Nov;31(46):5339–48. doi:10.1016/j.vaccine.2013.09.013.
- Shahrabani S, Benzion U, Yom Din G. Factors affecting nurses’ decision to get the flu vaccine. Eur J Health Econ. 2009;10:227–31. doi:10.1007/s10198-008-0124-3.
- Shahrabani S, Benzion U. How experience shapes health beliefs: the case of influenza vaccination. Health Educ Behav. 2012;39:612–19. doi:10.1177/1090198111427411.
- Tsutsuia Y, Benzion U, Shahrabanic S. Economic and behavioral factors in an individual’s decision to take the influenza vaccination in Japan. J Socio Econ. 2012;41(5):594–602. doi:10.1016/j.socec.2012.05.001.
- Grohskopf LA, Sokolow LZ, Broder KR, Walter EB, Fry AM, Jernigan DB. Prevention and control of seasonal influenza with vaccines: recommendations of the advisory committee on immunization practices-United States, 2018-19 influenza season. MMWR Recomm Rep. 2018 Aug;67(3):1–20. doi:10.15585/mmwr.rr6703a1.
- CDC. CDC (Centers for Disease Control and Prevention). Estimating seasonal influenza-associated deaths in the United States: CDC study confirms variability of flu. 2016b [accessed 2019 Nov 13]. http://www.cdc.gov/flu/about/disease/us_flu-related_deaths.htm.
- Ministero della Salute. Prevenzione e controllo dell’influenza: raccomandazione per la stagione 219-2020. [ accessed 2019 Nov 13]. http://www.trovanorme.salute.gov.it/norme/renderNormsanPdf?anno=2019&codLeg=70621&parte=1%20&serie=null.
- Bonanni P, Ferro A, Guerra R, Iannazzo S, Odone A, Pompa MG, Rizzuto E, Signorelli C. Vaccine coverage in Italy and assessment of the 2012-2014 national immunization prevention plan. Epidemiol Prev. 2015;39(4 Suppl 1):146–58. Jul-Aug.
- Schmid P, Rauber D, Betsch C, Lidolt G, Denker ML. Barriers of influenza vaccination intention and behavior - a systematic review of influenza vaccine hesitancy, 2005-2016. PLoS One. 2017;12:e0170550.
- Lonati F, Scarcella C, Indelicato A, Brioschi A, Magoni M, Medea G, Saleri N, Orizio G, Donato F. Brescia local health autority population database: a method based on current data for monitoring chronic diseases and management. Epidemiol Prev. 2008 May–Jun;32(3):137–44.
- Schoenbach VJ, Rosamond WD. Understanding the fundamentals of epidemiology: an evolving text. Chapel Hill, NC: University of North Carolina; 2000 [accessed 2019 Nov 13]. www.epidemiolog.net/evolving/RelatingRiskFactorstoHealth.pdf.
- Menard S. Coefficients of determination for multiple logistic regression analysis. Am Stat. 2000;54:17–24.
- R Core Team. R: A language and environment for statistical computing. Vienna (Austria): R Foundation for Statistical Computing; 2018 [accessed 2019 Nov 13]. https://www.R-project.org/.
- Berthold MR, Cebron N, Dill F, Gabriel TR, Kötter T, Meinl T, Ohl P, Thiel K, Wiswedel B. KNIME-the Konstanz information miner: version 2.0 and beyond. AcM SIGKDD Explorations Newsl. 2009;11(1):26–31. doi:10.1145/1656274.1656280.
- CDC. CDC (Centers for Disease Control and Prevention). Past seasons estimated influenza disease burden. 2018 [accessed 2019 Nov 13]. https://www.cdc.gov/flu/about/burden/past-seasons.html.
- Dhakal S, Klein SL. Host factors impact vaccine efficacy: implications for seasonal and universal influenza vaccine programs. J Virol. 2019;93:e00797–19.
- Larson HJ, de Figueiredo A, Xiahong Z, Schulz WS, Verger P, Johnston IG, Cook AR, Jones NS. The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine. 2016;12:295–301. doi:10.1016/j.ebiom.2016.08.042.