1,680
Views
0
CrossRef citations to date
0
Altmetric
Research Paper

Human papillomavirus (HPV) vaccination across a cascade of knowledge, willingness, and uptake among gay, bisexual, and other men who have sex with men in Canada’s three largest cities

ORCID Icon, , ORCID Icon, ORCID Icon, ORCID Icon, , , , , ORCID Icon, , ORCID Icon, , ORCID Icon, , , ORCID Icon, , ORCID Icon & show all
Pages 5413-5425 | Received 18 Jun 2021, Accepted 06 Sep 2021, Published online: 02 Dec 2021

ABSTRACT

Background

Some Canadian jurisdictions offer publicly funded HPV vaccine to gay, bisexual, and other men who have sex with men (GBM) aged ≤26 years. We characterized factors associated with being in different stages of HPV vaccination.

Methods

Engage is a sexual health study of GBM in the three largest Canadian cities recruited via respondent driven sampling (RDS). We categorized participants as: (1) unaware of HPV vaccine, (2) undecided/unwilling to get vaccinated, (3) willing to get vaccinated, (4) vaccinated with one or more doses. Our RDS-II weighted analyses used multinomial logistic regression to identify factors associated with being in earlier stages of the cascade compared to Stage 4.

Results

Across the cities, 26–40%, 7–14%, 33–39%, and 13–28% were in Stages 1 to 4, respectively. Compared to Stage 4, being in earlier stages of the cascade was associated with bisexual-identification (Stage 1: adjusted odds ratio[aOR] = 2.84, 95% confidence interval[CI] = 1.06–7.62; Stage 2: aOR = 3.09, 95%CI = 1.19–8.05), having immigrated to Canada (Stage 1: aOR = 1.79, 95%CI 1.07–2.99), preference to keep same-sex romantic relationships private (Stage 1: aOR = 1.25, 95% CI = 1.05–1.48; Stage 2: aOR = 1.24, 95%CI = 1.05–1.46), not receiving sexual health information (Stage 1: aOR = 0.31, 95% CI = 0.13–0.71; Stage 2: aOR = 0.27, 95%CI = 0.12–0.64), not accessing a health-care provider (Stage 2: aOR = 0.36, 95%CI = 0.15–0.83), and no past hepatitis A/B vaccination (Stage 1: aOR = 0.16, 95% CI = 0.09–0.30; Stage 2: aOR = 0.18, 95%CI = 0.09–0.35; Stage 3: aOR = 0.38, 95%CI = 0.21–0.61).

Discussion

Interventions are needed to reduce social and financial barriers, increase sexual health knowledge, and improve GBM-competent health-care access to increase vaccine uptake among GBM.

Introduction

Gay, bisexual, and other men who have sex with men (GBM) are a priority population for human papillomavirus (HPV) vaccination due to a higher risk of HPV and HPV-associated disease compared to men who have sex with women.Citation1–5 Canada’s National Advisory Committee on Immunization (NACI) recommends all GBM get vaccinated regardless of age.Citation1 Canada has a publicly funded universal health-care system that is implemented at the provincial/territorial level such that vaccination programs, including the type of vaccine and who is eligible to receive them, may differ across the country. In September 2015, September 2016, and January 2016, respectively, the provinces of British Columbia, Ontario, and Québec, implemented targeted HPV vaccination programs allowing GBM ≤26 years old to receive publicly funded HPV vaccine.Citation6–8 Older men ineligible for programs must have vaccine costs covered by private insurance or pay out-of-pocket for a vaccine that costs up to $560.Citation9 Men ≥15 years old in Canada are considered to have completed their HPV vaccination series and be fully protected against HPV after three doses.Citation10 Québec is the only province where men 15–18 years old require only two doses for completion.Citation7 All provinces and territories in Canada also have school-based HPV vaccination programs where young girls, and as of 2018, boys can get vaccinated for free in the school setting.

Most existing literature on HPV vaccination among GBM has focused on vaccine acceptability and highlighted the influence of factors such as perceived disease severity, understanding the benefits of vaccination, positive attitudes toward the vaccine, self-efficacy, and cues to action such as HPV vaccine effectiveness, most of which are cognition-based features.Citation11–14 Focusing on these cognitive factors places most of the responsibility of getting vaccinated on the individuals themselves rather than addressing the broader social (e.g. socio-economic status and health-care quality) and programmatic barriers (e.g. health-care access and publicly funded program age and sexual orientation restrictions), which may also play a significant role. These social and programmatic factors may be comparable or have a greater impact on vaccination than cognitive features.Citation15,Citation16 Qualitative interviews conducted among Canadian GBM uncovered interrelated “structurally-produced vaccine hesitancies,” such as socio-economic factors and differences in healthcare recommendations, that must be addressed to increase HPV vaccine uptake.Citation9,Citation17 The World Health Organization’s (WHO) Strategic Advisory Group of Experts (SAGE) on Vaccine Hesitancy also emphasized that only looking at cognitive determinants when studying vaccine hesitancy may limit potential strategies to increase vaccine uptake at individual and population levels.Citation18 A recent systematic review and meta-analysis on acceptability, uptake, and completion of vaccines, including HPV, among GBM highlighted the importance of determining barriers and facilitators of vaccination by country given differences in socio-cultural factors and structure of healthcare systems.Citation19 Only one HPV vaccine acceptance study in the review was from Canada and it was conducted prior to the implementation of targeted publicly funded HPV vaccination programs. There were no Canadian studies on HPV vaccine uptake.

Additionally, among the existing studies on HPV vaccination among GBM, vaccine knowledge, willingness to get vaccinated, and uptake have been explored as separate, dichotomous outcomes, suggesting these outcomes occur in isolation from one another. This has resulted in a lack of understanding of how men unaware of the vaccine or those unwilling or willing to get vaccinated differ from those that have been vaccinated. A stage-based framework across these outcomes may provide a more thorough understanding of the vaccination process. Past studies have used stage-based approaches to explore predominantly cognitive determinants (e.g. attitudes and beliefs) of HPV vaccine decision-making among parents of boys as well as young women.Citation20–23 These stages have been informed by theoretical models such as the Precaution Adoption Process Model and Transtheoretical Model.Citation24,Citation25 The influence of the factors examined differed significantly depending on which stage people were in. In this study, we hypothesized that the same may be true for the influence of broader social and programmatic factors across stages of vaccination among adult GBM. To do this, we used a stage-based framework of HPV vaccine awareness, willingness, and uptake, which we will refer to as the HPV vaccination cascade.

Methods

Design and participants

We analyzed data from the Engage Study, a community-recruited sexual health cohort study of GBM in Vancouver, Toronto, and Montreal. Our study is cross-sectional in design since baseline data were used for these analyses; follow-up for Engage is still ongoing. Eligible men were ≥16 years of age, had sex with another man in the past 6 months, were able to read English or French, and provided written informed consent. Each city recruited men using respondent driven sampling (RDS) from February 2017–August 2019. RDS is used to recruit samples who are not feasible to recruit using random sampling methods.Citation26 Initial participants or “seeds” were recruited into the study and each seed was given six coupons to recruit members from their social networks. Participants were compensated $50 for participating in Engage and an additional $15 for each additional participant they recruited into the study. Recruitment details have been described in more detail elsewhere.Citation27–29 The study received ethical approval from Ryerson University, University of Toronto, St. Michael’s Hospital, University of Windsor, University of British Columbia, University of Victoria, Simon Fraser University, and McGill University Health Center.

Data collection

All participants self-completed a comprehensive questionnaire on demographics, health services use/access, vaccination, community and societal context, partner relationships, and sexual behaviors at enrollment using computer assisted self-interview (CASI). The questionnaire was developed based on several frameworksCitation30–32 (e.g. access to health care frameworkCitation33) and from field-tested instruments such as Canadian M-TrackCitation34 and the US Centers for Disease Control and Prevention behavioral surveys.Citation35 Each city also has a Community Engagement Committee, which was consulted in the creation of the questionnaire.

HPV vaccination cascade

For the HPV vaccination cascade, we characterized men in terms of their position along the cascade, which included the following mutually exclusive stages: (1) unaware of the HPV vaccine, (2) aware of the vaccine yet undecided/unwilling to get vaccinated, (3) aware of the vaccine and willing to get vaccinated, and (4) aware of the vaccine and vaccinated with one or more doses ().

Figure 1. Mutually exclusive categories of the HPV vaccination cascade, which were broken down by vaccine knowledge, vaccine initiation, and willingness to get vaccinated. Questions used to create stages included above each stage.

Figure 1. Mutually exclusive categories of the HPV vaccination cascade, which were broken down by vaccine knowledge, vaccine initiation, and willingness to get vaccinated. Questions used to create stages included above each stage.

Stages of the cascade were created using HPV vaccine-related questions from the Engage questionnaire (). Prior to asking the questions on vaccine willingness, a short preamble was provided explaining eligibility for publicly funded programs or the cost of the vaccine, if ineligible. Since men ≥27 years old would be ineligible for publicly funded vaccine in the three provinces, to align with vaccine willingness with men eligible for publicly-funded programs, we asked men ≥27 years old whether they would get vaccinated if the vaccine was free and they had to disclose their sexual activity to a healthcare provider (). For questions on vaccine awareness, men stating ‘no’ or ‘unsure’ to questions were grouped because they were not asked additional questions on HPV vaccination. Men stating ‘no’ or ‘unsure’ to the question on vaccine initiation were considered unvaccinated. Self-reported vaccination status has a 89–96% sensitivity, 76–97% specificity, and 73–84% accuracy.Citation36–39 Since 79% (982/1247) of men were willing to get vaccinated, there were not enough men in the ‘undecided’ or ‘unlikely’ categories to warrant their own stages for modeling purposes. Thus, willingness to get vaccinated was dichotomized as unlikely/undecided versus likely.

Variables of interest

The WHO SAGE Working Group Determinants of Vaccine Hesitancy Matrix guided independent variable selection.Citation18 The Matrix includes contextual influences, individual and group influences, and vaccine/vaccination-specific issues. We selected measures from the Engage questionnaire that aligned with the determinants from the Matrix ().

Table 1. Alignment of measures with the determinants of vaccine hesitancy matrixCitation18 for exploration of stage in the HPV vaccination cascade among gay, bisexual, and other men who have sex with men, Engage Study, 2017–2019

Statistical analysis

In studies using RDS recruitment, men with larger social networks may have a greater probability of being recruited into the study.Citation41 To account for this unequal probability of recruitment, all proportions, unless otherwise specified, and multivariable multinomial regression models were weighted using RDS-II Volz-Heckathorn weights.Citation41 These weights are inversely proportional to a participant’s self-reported social network size within each city.Citation41 Network size was measured using the question, “How many men who have sex with men aged 16 years or older, including trans men, do you know who live or work in the [City] area (whether they identify as gay or otherwise)? This includes gay/bi guys you see or speak to regularly; e.g., close friends, boyfriends, spouses, regular sex partners, roommates, relatives, people you regularly hang out with, etc.”

We described characteristics of the sample overall (unweighted), and by city (weighted). We estimated the proportion of men that fell into each category of the HPV vaccination cascade and tested for differences in proportions across cities using the chi-squared test. Seeds were included in all analyses.

Univariable and multivariable analysis

We explored social and programmatic factors associated with being in earlier stages of the cascade compared to having initiated vaccination (Stage 4). Vaccine initiation was selected as the reference category so we could determine the barriers men in earlier stages must overcome to reach the vaccination stage. Data were combined across cities. Since RDS relies on social network ties, recruiters may be more likely to recruit individuals more like themselves.Citation26,Citation42 Nonetheless, through multiple simulations, Avery, et al.Citation42 found that clustering had little impact on regression models using RDS data and simplified models without clustering adjustment were recommended. To explore this further in our data, clustering was explored at the seed (clustering originates from initial seed) and recruiter (direct recruiter-recruit relationship) level using multinomial logistic random effects models. Small covariance parameter estimates confirmed that clustering had little impact and so it was not considered in the final model (data not shown). Multinomial logistic regression with the SAS software procedure PROC SURVEYLOGISTIC was used to estimate unadjusted and adjusted odds ratios (ORs) and 95% confidence intervals (CIs).Citation43 RDS-II weights were calculated for each participant and incorporated in the PROC SURVEYLOGISTIC weight statement.Citation43 Our multivariable model was a hypothesis-generating model that was guided by the Determinants of Vaccine Hesitancy Matrix;Citation18 results should be interpreted accordingly. We were unable to thoroughly explore ethnicity/race in the multivariable model due to small cell counts for some ethnicities/races. We considered both age group (≤26 years old and ≥27 years old) and city as potential a priori effect modifiers. We tested these hypotheses separately by creating models with an interaction term between age group and each variable of interest (education, financial strain, sexual orientation, country of birth, receipt of sexual health information, preference to keep same-sex romantic relationships private, past hepatitis A or B vaccination, poor perceived healthcare quality, and any healthcare access) and an interaction term between city and each variable of interest. If the joint test of the interaction coefficients was significant, we then used linear combinations of model coefficients to calculate odds ratios within strata of the effect modifier to quantify the heterogeneity of these effects.Citation44 We used a complete case strategy since data were only missing for 2.3% in the multivariable model.

For further exploratory purposes, we fit models where the vaccine initiation category was further categorized as vaccine initiation (1–2 doses) and vaccine completion (3 doses), creating a five-stage vaccination cascade, to determine if men in earlier stages differed from those completing their HPV vaccination series. All analyses were conducted using SAS 9.4 (SAS Institute, Inc., Cary, North Carolina, USA). P-values were two-sided and statistical significance was determined using a p-value of <.05.

Results

A total of 2449 men were recruited in Engage through 240 seeds (Supplementary Materials, ). Sixteen men (0.7%) had missing HPV vaccination cascade outcome data and thus were excluded from the analysis resulting in a total sample of 2433 men (Vancouver = 750, Toronto = 516, Montreal = 1167). Across the three cities, age ranged between 16 and 80 years old (median = 31 years); 22–32% were ≤26 years old, 12–19% were living with HIV, 55–70% were white, 57–65% were born in Canada, 88–93% identified as a cisgender man, 72–80% identified as gay, and 64–77% had a post-secondary education (). Unweighted prevalence estimates by city are provided in Supplementary Materials, Table 2.

Table 2. Unweighted overall proportions and means/standard deviation and city-specific weighted proportions and means/standard deviation for baseline characteristics of Engage participants included in analysis (n = 2433)

Proportion of men across stages of the HPV vaccination cascade

The proportion of men who fell into each category of the cascade differed significantly by city (p = .001). Across the three cities, 26–40% were unaware of the HPV vaccine, 7–14% were undecided or unwilling to get vaccinated, 33–39% were willing to get vaccinated, and 13–28% had initiated vaccination ().

Figure 2. RDS-weighted proportions and 95% confidence intervals of men in each category of the HPV vaccination cascade, by city.

Figure 2. RDS-weighted proportions and 95% confidence intervals of men in each category of the HPV vaccination cascade, by city.

Characteristics associated with stages of the HPV vaccination cascade

We did not detect any significant effect modification by age group (data not shown). Age group was still included in the model since we were interested in its association with the cascade. There was significant effect modification by city for the Financial Strain Index score (p = .03), age group (p = .01), and education (p = .01). The final, adjusted model included sexual orientation, country of birth, receipt of sexual health information, preference to keep same-sex romantic relationships private, past hepatitis A or B vaccination, poor perceived healthcare quality, and any healthcare access and an interaction term for city and age group, city and Financial Strain Index score, and city and education.

All modeling comparisons were with men who had initiated vaccination as the outcome reference group (Stage 4). According to the multivariable weighted model findings (), having immigrated to Canada compared to having been born in Canada (aOR = 1.79, 95%CI 1.07–2.99), identifying as bisexual versus gay (aOR = 2.84, 95%CI 1.06–7.62), and preferring to keep same-sex romantic relationships private (aOR = 1.25, 95%CI 1.05–1.48) were significantly associated with increased odds of being unaware of the HPV vaccine (Stage 1). Identifying as queer versus gay (aOR = 0.19, 95%CI 0.08–0.45), having received sexual health information in the past six months versus not (aOR = 0.31, 95%CI 0.13–0.71), and past hepatitis A or B vaccination (as compared to not having received a past hepatitis A or B vaccine) (aOR = 0.16, 95%CI 0.09–0.30) were associated with decreased odds of being unaware of the vaccine.

Table 3. RDS-weighted multinomial logistic regression showing factors associated with being in earlier stages of the cascade compared to having initiated human papillomavirus (HPV) vaccination (received at least one dose of the vaccine) among men from the Engage Study (n = 2433)

Identifying as bisexual versus gay (aOR = 3.09, 95%CI 1.19–8.05) and preferring to keep same-sex romantic relationships private (aOR = 1.24, 95%CI 1.05–1.46) were significantly associated with increased odds of being undecided/unwilling to get vaccinated (Stage 2) (). Having received sexual health information in the past 6 months versus not (aOR = 0.27, 95%CI 0.12–0.64), past hepatitis A or B vaccination (as compared to not having received a past hepatitis A or B vaccine) (aOR = 0.18, 95%CI 0.09–0.35), and accessing a healthcare provider versus not (aOR = 0.36, 95%CI 0.15–0.83) were significantly associated with decreased odds of being undecided/unwilling to get vaccinated. Having received a past hepatitis A or B vaccination (aOR = 0.38, 95%CI 0.21–0.67) was also significantly associated with decreased odds of being in the willing to get vaccinated stage ().

In Vancouver, increasing financial strain (aOR = 1.22, 95%CI 1.07–1.40) was significantly associated with increased odds of being unaware of the HPV vaccine while being ≤26 years old versus ≥27 years old (aOR = 0.40, 95%CI 0.18–0.88) was associated with a decreased odds of being unaware of the vaccine (). Increasing financial strain (aOR = 1.40, 95%CI 1.16–1.69) was also associated with an increased odds of being undecided/unwilling to get vaccinated. In Montreal, being ≤26 years old versus ≥27 years old (aOR = 0.08, 95%CI 0.04–0.18) and having a post-secondary versus high school or less education (aOR = 0.21, 95 CI% 0.07–0.61) were significantly associated with a decreased odds of being unaware of the vaccine (). Both factors were also associated with a decreased odds of being undecided/unwilling to get vaccinated (≤26 years: aOR = 0.20, 95%CI 0.09–0.47; post-secondary education: aOR = 0.35, 95%CI 0.12–0.98). Being ≤26 years old (aOR = 0.13, 95%CI 0.06–0.28) was also associated with a decreased odds of willing to get vaccinated (). Overall, unweighted and weighted multivariable model findings had a relatively similar qualitative interpretation but with noticeably more variability for some variables in weighted findings (Supplementary Materials, Table 3).

Vaccine completion as stage 5

In the analysis for which Stage 4, vaccine initiation, was further divided into initiation versus completion of all required doses, the proportions initiated (Stage 4) and completed (Stage 5), respectively, were 9% (95%CI 6.3–12.7%) and 11% (95%CI 7.5–14.0%) in Vancouver, 10% (95%CI 5.9–13.6%) and 18% (95%CI 11.3–25.5%) in Toronto, and 8% (95%CI 5.1–10.8%) and 5% (95%CI 3.4–6.9%) in Montreal. When we used Stage 5 as the outcome reference category for modeling, similar findings were observed compared to when the reference category was vaccine initiation ().

Table 4. RDS-weighted multinomial logistic regression showing factors associated with being in earlier stages of the cascade compared to having completed human papillomavirus (HPV) vaccination (received all three doses of vaccine) among men from the Engage Study (n = 2433)

Discussion

By applying our HPV vaccination cascade approach, we found that soon after publicly funded HPV vaccination programs were introduced in Canada for GBM ≤26 years old, 26–40% of such men in the three largest cities in Canada were still unaware of the HPV vaccine. A minority (7–14%) of men were undecided or unwilling to get vaccinated against HPV, whereas 33–39% were willing to get vaccinated, and 13–28% had received at least one dose of the vaccine series. Men unaware of the HPV vaccine or undecided/unwilling to get vaccinated had the most barriers to vaccination such as lower socio-economic status, sexual orientation non-disclosure, lack of sexual health knowledge, and not accessing healthcare and other vaccines. Other characteristics such as bisexual identity and older age were also significantly associated with being in the early stages of the cascade. Conversely, fewer differences were seen when vaccinated GBM were compared with men willing to get vaccinated, suggesting that once men have accepted vaccination these characteristics may play a smaller role in understanding why men willing to get vaccinated have not yet received it. Similar associations were seen when earlier stages were compared to vaccine completion.

There are clear challenges in optimizing vaccine uptake among GBM using the current structure of GBM-targeted HPV vaccination programs in the three Canadian provinces. To initiate vaccination against HPV within these publicly-funded programs, GBM must be ≤26 years old, educated about the HPV vaccine, have access to a provider, able/feel comfortable disclosing their sexual orientation to that provider, and have that provider respond appropriately. Our findings indicate there are barriers to this process for GBM. We saw clear differences in age where GBM ≥27 years old were in earlier stages of the cascade, which was most evident in Montreal. Although the reason behind this may be multifaceted, one explanation is that these older men are not getting vaccinated since they are ineligible for publicly funded programs and the high cost of the vaccine.Citation9,Citation45 GBM ≥27 years old are still recommended to get vaccinated in Canada and, if available, could have vaccine costs covered by private insurance, but this divergence between public programs and NACI recommendations makes it difficult for physicians to make clear HPV vaccination recommendations to this group.Citation17 We also found healthcare access and sexual orientation disclosure, other requirements of publicly funded vaccine uptake, were barriers to vaccine initiation. Hesitancy to disclose one’s sexual orientation has been recognized as a barrier to vaccine acceptance and uptake across GBM populations.Citation46–48 In qualitative interviews among young GBM in Florida, US, men stated feeling ashamed, awkward, or judged when discussing sexual behaviors with healthcare providers who appeared uncomfortable discussing sexuality.Citation47 Men felt more comfortable going to a sexual health clinic to get vaccinated even if they already had established relationships with their primary care provider.Citation47

Compared with vaccinated men, men identifying as queer versus gay had lower odds of being unaware of the vaccine, which may partly be explained by higher educational attainment among men identifying as queer and being more actively connected to a community where sexual health knowledge is shared.Citation49 Compared with vaccinated men, men identifying as bisexual versus gay had higher odds of being in the first two stages of the cascade. Lower uptake among bisexual versus gay men has also been seen for the hepatitis B vaccine and several other healthcare services such as HIV testing.Citation50,Citation51 Lower uptake may be attributed to bisexual men having less access to care or a usual place of care compared with gay men,Citation51 both of which may lead to reduced HPV awareness and willingness to get vaccinated. Bisexual adults may also be less likely to disclose their sexual orientation to healthcare providers.Citation50,Citation52

Our findings suggest that social and economic inequities in financial strain, immigration, and education are associated with GBM being in earlier stages of the HPV vaccination cascade. Men immigrating to Canada had higher odds of being unaware of the vaccine but had lower odds of being undecided/unwilling to get vaccinated. Studies have found that Canadian immigrants would be willing to get vaccinated if a doctor recommended it, yet healthcare providers may be less likely to recommend and have discussions around vaccination with immigrants, leading to reduced awareness.Citation53,Citation54 City-specific differences were seen for financial strain and education, highlighting that constraints to vaccination may also differ by jurisdiction. Although we could not effectively explore ethnicity/race in multivariable models, univariable analyses indicated ethnicity/race may influence where men fall along the cascade, highlighting the value of conducting sub-population specific studies in the future.

The influence of social and economic factors, such as lower income, education, and unemployment have been significantly associated with lower acceptability and uptake of vaccines, including HPV, among GBM in other countries, highlighting this as an international issue.Citation19 The high cost of the vaccine among men ineligible for publicly funded vaccine likely contributes to these social inequities but these barriers may also exist among those that can receive publicly funded vaccine meaning cost is not the only issue. A natural experiment in Calgary, Canada, found that significantly fewer young females had been vaccinated against HPV in community vaccination clinics if they lived in the most materially deprived neighborhoods whereas these differences were not apparent in school-based vaccination programs.Citation55 Issues around access, such as transportation, time away from work, and added inconvenience for parents instead of vaccine cost may have contributed to low uptake given the vaccine was publicly funded.Citation55 We hypothesize that GBM experiencing inequities in our study may face similar barriers to HPV vaccination.

These barriers to HPV vaccination must be addressed among GBM, particularly for birth cohorts that cannot benefit from universal school-based HPV vaccination programs. Interventions at the provider and healthcare system level are needed to address these issues in Canada. It is imperative to educate providers on HPV, availability of publicly funded vaccination programs, NACI recommendations, and the importance of recommending vaccine to their patients even when not publicly funded since eligibility requirements for GBM-targeted programs do not align with vaccination recommendations in Canada. Tailored messaging around recommendations, HPV risk, and insurance coverage is necessary for men ≥27 years old. To increase uptake of current publicly funded targeted programs in Canada, interventions to increase GBM’s perceived safety to disclose their sexual orientation to health care providers are needed. Provider reminders to offer the HPV vaccine and remaining doses during routine healthcare visits may also help increase uptake.Citation56–58 In our study, we found men previously vaccinated against hepatitis A or B had a higher odds of initiating HPV vaccination, as has been seen in other studies.Citation19 Bundling vaccines for GBM may be a convenient, time-saving option.Citation59 Other system level opportunities include offering vaccine-only or “express-lane” vaccination services and extending office hours to reduce wait times and increase convenience; strategies that have been successfully used to increase influenza vaccination rates in the past.Citation60 In Canada and other jurisdictions with gender-neutral school-based programs, increasing efforts to ensure optimal uptake of these programs may alleviate these barriers for some GBM in the future. Potential methods include increasing HPV education among nurses and parents and adopting the ‘once eligible, always eligible’ policy nationally where those qualifying for a publicly funded HPV vaccine in school remain eligible in the future.Citation61,Citation62

Developing tools to increase awareness around sexual health, HPV, and the vaccine is also essential in increasing HPV vaccine uptake. This need was further highlighted in our study where more vaccinated men had recently received information on sexual health (e.g. through social media, community, and providers) compared to men in the first two stages of the cascade. Increasing awareness among immigrants or sexual minorities may require more tailored interventions such as educational videos in multiple languages or using relatable characters in campaigns whom the target community can identify with to increase perceptions around risk.Citation63,Citation64

This study had several strengths. Few studies have used a stage-based approach to explore HPV vaccination, and to the best of our knowledge, this is the first conducted in a GBM population and after implementation of GBM-targeted vaccination programs. Our sample was also community-recruited and conducted in the three largest Canadian cities, providing a ‘snapshot’ of the uptake of HPV vaccination programs, and vaccine knowledge and willingness to get vaccinated, within these large urban centers in the early phases.

There were also limitations. HPV vaccine uptake was self-reported. Despite high sensitivity, specificity, and accuracy of self-report HPV vaccine uptake among adults, misclassification may still exist in our outcome. Participant’s network size when correcting for sample selection bias could not be validated; assessment of both unweighted and weighted findings are recommended.Citation65 The cross-sectional design of the study means temporality of time-variant variables cannot be confirmed. In studies on HPV vaccination using stage-based approaches in other populations, people who were undecided about getting vaccinated were distinct from those that were unwilling or not intending to get vaccinated.Citation20–23 We agree that these two groups may differ but unfortunately were unable to explore them separately due to the small proportion of men in each group in our sample. Within most of the characteristics explored, we observed similar patterns among men that were undecided or that were unwilling to get vaccinated, and this categorization still allowed us to explore the differences between this more hesitant group versus men willing to get vaccinated.

Using a stage-based framework revealed that exploring vaccine uptake through a dichotomous lens may be too simplistic of an approach to understanding the complexities of HPV vaccination. For example, as mentioned previously, compared to vaccinated men, men that immigrated to Canada had an increased odds of being unaware of the HPV vaccine but appeared to have lower odds of being undecided/unwilling to get vaccinated. Grouping all unvaccinated men together may have diluted these effects. Using this HPV vaccination cascade, we also determined that three to four years after HPV vaccination programs for GBM were introduced in Canada, clear gaps in knowledge around the HPV vaccine remain, and men unaware of the vaccine or undecided/unwilling to get vaccinated faced the most inequities and constraints to vaccination. Reducing the barriers men are facing in earlier stages of the cascade by implementing tailored educational and physician and health system-oriented interventions is crucial to encourage men along the path toward vaccine uptake. Future research directions should include implementing and evaluating the effectiveness of interventions and continued monitoring of HPV vaccine uptake among GBM in Canada. Moreover, the factors explored in this study are not inclusive of all potential social and programmatic barriers or facilitators to vaccination GBM may experience. Future studies should explore the impact of other factors from the Determinants of Vaccine Hesitancy Matrix such as the influence of the GBM community and romantic partners, health system and providers-trust, and language or religion. Stage-based frameworks can also be used for other vaccines and populations and may help fill knowledge gaps around vaccine uptake among adult populations.

Disclosure of potential conflicts of interest

CS has research grants paid to the organization (INSPQ or CRCHU de Québec-Université Laval) for clinical trials and epidemiological studies funded by non-profit organizations: MSSS, Bill & Melinda Gates Foundation and Michael Smith Foundation). CS is an active member of the Comité sur l’immunisation du Québec and the National Advisory Committee on Immunization HPV Vaccination and Herpes Zoster Vaccination Working Group. FC received grants for research projects through the research centre from Roche Diagnostics, Becton Dickinson and Merck Sharp and Dome, honorariums for presentations from Merck Sharp and Dome and Roche diagnostics, and has participated in an expert group for Merck Sharp and Dome.SD is a member and Chair of the National Advisory Committee on Immunization. DHST’s institution has received research grants for investigator-initiated research from Abbvie, Gilead and Viiv Healthcare; DHST’s institution has also received support for industry-sponsored clinical trials from Glaxo Smith Kline. JC has research funding from ViiV Healthcare and Gilead Sciences, and reports remuneration for advisory work (ViiV Healthcare, Gilead Sciences and Merck Canada).

Statement of contribution

The Engage Cohort Study is led by Principal Investigators in Toronto by Trevor A. Hart & Daniel Grace, in Montreal by Joseph Cox and Gilles Lambert; and in Vancouver by Jody Jollimore, Nathan Lachowsky, and David Moore. More information about the Engage Cohort Study can be found here: https://www.engage-men.ca/

Supplemental material

Supplemental Material

Download ()

Acknowledgments

The authors would like to thank the Engage/Momentum II study participants, office staff, and community engagement committee members, as well as our community partner agencies. The authors also wish to acknowledge the support of Catharine Chambers, Ashley Mah, and Jennifer Gillis and their contribution(s) to the work presented here.

Supplemental data

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2021.1979379

Additional information

Funding

Engage-HPV is funded by the Canadian Institutes for Health Research (CIHR) Canadian Immunization Research Network [CIRN, 151944] and a CIHR Foundation Grant awarded to ANB [148432]. Engage/Momentum II is funded by CIHR [#TE2-138299, FDN-143342, PJT-153139], the Canadian Association for HIV/AIDS Research [CANFAR, #Engage], the Ontario HIV Treatment Network [OHTN, #1051], and the Public Health Agency of Canada [#4500370314], and Ryerson University. RG is supported by a CIRN Trainee Scholarship. ANB is a Canada Research Chair in Sexually Transmitted Infection Prevention and a recipient of a Department of Family and Community Medicine Non-Clinician Research Scientist Award, University of Toronto. DM and NJL are supported with scholar awards from the Michael Smith Foundation for Health Research [#5209, #16863]. TAH is supported by a Chair in Gay and Bisexual Men’s Health from the OHTN. DG is a Canada Research Chair in Sexual and Gender Minority Health. GO is a Canada Research Chair in Global Control of HPV-Related Disease and Cancer. DHST is a Canada Research Chair in HIV Prevention and Sexually Transmitted Infection Research.

References

  • National Advisory Committee on Immunization (NACI). Canadian immunization guide: part 4 - active vaccines - human papillomavirus vaccine. Government of Canada; 2016. [Accessed December 3 2020]. https://www.canada.ca/en/public-health/services/publications/healthy-living/canadian-immunization-guide-part-4-active-vaccines/page-9-human-papillomavirus-vaccine.html.
  • CDC. Recommendations on the use of quadrivalent HPV vaccine in males — ACIP. MMWR. 2011. 60(50):1705–08.
  • JCVI. JCVI statement on HPV vaccination of MSM. JCVI; 2015. [Accessed December 3 2020]. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/477954/JCVI_HPV.pdf
  • Australian Government Department of Health. Men who have sex with men are recommended to receive HPV vaccine. In: The Australian Immunisation Handbook. Published 2018. [accessed 2020 Nov 15]. https://immunisationhandbook.health.gov.au/recommendations/men-who-have-sex-with-men-are-recommended-to-receive-hpv-vaccine.
  • Nyitray AG, Carvalho da Silva RJ, Baggio ML, Lu B, Smith D, Abrahamsen M, Papenfuss M, Villa LL, Lazcano-Ponce E, Giuliano AR, et al. Age-specific prevalence of and risk factors for anal human papillomavirus (HPV) among men who have sex with women and men who have sex with men: the HPV in men (HIM) study. J Infect Dis. 2011;203(1):49–57. doi:10.1093/infdis/jiq021.
  • Government of Ontario M of H and L-TC. Vaccines and immunization. [accessed 2020 Jan 15]. http://www.health.gov.on.ca/en/pro/programs/immunization/.
  • Santé et Services Sociaux Québec. VPH: vaccin contre les virus du papillome humain. [accessed 2020 Jan 3]. https://www.msss.gouv.qc.ca/professionnels/vaccination/piq-vaccins/vph-vaccin-contre-les-virus-du-papillome-humain/.
  • Immunize BC. HPV (Human Papillomavirus). Evidence-based immunization information and tools for B.C. residents. Quest HPV study. Published March 21, 2011. [accessed 2020 Jan 3]. https://immunizebc.ca/hpv.
  • Grace D, Gaspar M, Paquette R, Rosenes R, Burchell AN, Grennan T, Salit IE. HIV-positive gay men’s knowledge and perceptions of human papillomavirus (HPV) and HPV vaccination: a qualitative study. PLoS ONE. 2018;13(11):11. doi:10.1371/journal.pone.0207953.
  • National Advisory Committee on Immunization (NACI). Updated recommendations on human papillomavirus (HPV) vaccines: 9-valent HPV vaccine and clarification of minimum intervals between doses in the HPV immunization schedule. Ottawa, Canada: Public Health Agency of Canada; 2016.
  • Lau JTF, Wang Z, Kim JH, Lau M, Lai CHY, Mo PKH. Acceptability of HPV vaccines and associations with perceptions related to HPV and HPV vaccines among men who have sex with men in Hong Kong. PLOS ONE. 2013;8(2):e57204. doi:10.1371/journal.pone.0057204.
  • Nadarzynski T, Smith H, Richardson D, Jones CJ, Llewellyn CD. Human papillomavirus and vaccine-related perceptions among men who have sex with men: a systematic review. Sex Transm Infect. 2014;90(7):515–23. doi:10.1136/sextrans-2013-051357.
  • Wheldon CW, Daley EM, Buhi ER, Nyitray AG, Giuliano AR. Health beliefs and attitudes associated with HPV vaccine intention among young gay and bisexual men in the Southeastern United States. Vaccine. 2011;29(45):8060–65. doi:10.1016/j.vaccine.2011.08.045.
  • Gerend MA, Madkins K, Phillips G, Mustanski B. Predictors of human papillomavirus vaccination among young men who have sex with men. Sex Transm Dis. 2016;43(3):185–91. doi:10.1097/OLQ.0000000000000408.
  • Taylor D, Bury M, Campling N, Carter S, Garfied S, Newbould J, Rennie T. A review of the use of the Health Belief Model (HBM), the Theory of Reasoned Action (TRA), the Theory of Planned Behaviour (TPB) and the Trans-Theoretical Model (TTM) to study and predict health related behaviour change. London, United Kingdom: NHS - National Institute for Health and Clinical Excellence; 2007.
  • Betsch C, Schmid P, Heinemeier D, Korn L, Holtmann C, Böhm R, Angelillo IF. Beyond confidence: development of a measure assessing the 5C psychological antecedents of vaccination. PLoS ONE. 2018;13(12):12. doi:10.1371/journal.pone.0208601.
  • Grace D, Gaspar M, Rosenes R, Grewal R, Burchell AN, Grennan T, Salit IE. Economic barriers, evidentiary gaps, and ethical conundrums: a qualitative study of physicians’ challenges recommending HPV vaccination to older gay, bisexual, and other men who have sex with men. Int J Equity Health. 2019;18(1). doi:10.1186/s12939-019-1067-2.
  • SAGE Working Group. Report of the SAGE working group on vaccine hesitancy. Geneva, Switzerland: World Health Organization; 2014. [Accessed  January 14 2021]. https://www.who.int/immunization/sage/meetings/2014/october/1_Report_WORKING_GROUP_vaccine_hesitancy_final.pdf
  • Nadarzynski T, Frost M, Miller D, Wheldon CW, Wiernik BM, Zou H, Richardson D, Marlow LAV, Smith H, Jones CJ, et al. Vaccine acceptability, uptake and completion amongst men who have sex with men: a systematic review, meta-analysis and theoretical framework. Vaccine. 2021;39(27):3565–81. doi:10.1016/j.vaccine.2021.05.013.
  • Perez S, Tatar O, Gilca V, Shapiro GK, Ogilvie G, Guichon J, Naz A, Rosberger Z. Untangling the psychosocial predictors of HPV vaccination decision-making among parents of boys. Vaccine. 2017;35(36):4713–21. doi:10.1016/j.vaccine.2017.07.043.
  • Perez S, Shapiro GK, Brown CA, Dube E, Ogilvie G, Rosberger Z. “I didn’t even know boys could get the vaccine”: parents’ reasons for human papillomavirus (HPV) vaccination decision making for their sons. Psycho-Oncology. 2015;24(10):1316–23. doi:10.1002/pon.3894.
  • Reiter PL, McRee A-L, Kadis JA, Brewer NT. HPV vaccine and adolescent males. Vaccine. 2011;29(34):5595–602. doi:10.1016/j.vaccine.2011.06.020.
  • Allen JD, Mohllajee AP, Shelton RC, Othus MKD, Fontenot HB, Hanna R. Stage of adoption of the human papillomavirus vaccine among college women. Prev Med. 2009;48(5):420–25. doi:10.1016/j.ypmed.2008.12.005.
  • Prochaska J, DiClemente C. The transtheoretical approach: crossing traditional boundaries of therapy. Florida, USA: Krieger Publishing Company; 1984.
  • Weinstein N, Sandman P, Blalock S. The precaution adoption process model. In: Karen Glanz, Barbara K. Rimer, K. Viswanath, (Eds.), Health behaviour and health education: theory, research, and practice. San Francisco, USA: Jossey-Bass; 2008. p. 123–47.
  • Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44(2):174–99. doi:10.2307/3096941.
  • Cox J, Apelian H, Moodie E, Messier-Peet M, Hart TA, Grace D, Moore DM, Lachowsky NJ, Armstrong HL, Jollimore J, et al. HIV pre-exposure prophylaxis (PrEP) use among Urban Canadian gay, bisexual and other men who have sex with men for whom PrEP is clinically recommended: baseline results from the Engage cohort study. CMAJ Open. 2021;9(2):E529–E538. Published online. doi:10.9778/cmajo.20200198.
  • Moore DM, Cui Z, Skakoon‐Sparling S, Sang J, Barath J, Wang L, Lachowsky N, Cox J, Lambert G, Noor SW, et al. Characteristics of the HIV cascade of care and unsuppressed viral load among gay, bisexual and other men who have sex with men living with HIV across Canada’s three largest cities. J Int AIDS Soc. 2021;24(4):e25699. doi:10.1002/jia2.25699.
  • Hart TA, Moore DM, Noor SW, Lachowsky N, Grace D, Cox J, Skakoon-Sparling S, Jollimore J, Parlette A, Lal A, Apelian H, Sang JM, Tan DHS, Lambert G, and the Engage Study Team. Prevalence of HIV and sexually transmitted and bloodborne infections (STBBI), and related preventive and risk behaviours, among gay, bisexual and other men who have sex with men in Montreal, Toronto and Vancouver. Results from the Engage Study. Can J Public Health. 2021. doi:10.17269/s41997-021-00546-z. Online ahead of print.
  • Ivankovich MB, Fenton KA, Douglas JM. Considerations for national public health leadership in advancing sexual health. Public Health Rep. 2013;128(Suppl 1):102–10. doi:10.1177/00333549131282S112.
  • Santos G-M, Do T, Beck J, Makofane K, Arreola S, Pyun T, Hebert P, Wilson PA, Ayala G. Syndemic conditions associated with increased HIV risk in a global sample of men who have sex with men: table 1. Sex Transm Infect. 2014;90(3):250–53. doi:10.1136/sextrans-2013-051318.
  • Singer M. Introduction to syndemics: a critical systems approach to public and community health, 1st ed. San Francisco, USA: Jossey-Bass; 2009.
  • Levesque J-F, Harris MF, Russell G. Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013;12(1):18. doi:10.1186/1475-9276-12-18.
  • Centre for Communicable Diseases and Infection Control. M-track: enhanced surveillance of HIV, sexually transmitted and blood-borne infections, and associated risk behaviours among men who have sex with men in Canada. Phase 1 report. Ottawa, Canada: Centre for Communicable Diseases and Infection Control; 2011. [Accessed July 9, 2021]. https://www.deslibris.ca/ID/232461
  • Gallagher KM, Sullivan PS, Lansky A, Onorato IM. Behavioral surveillance among people at risk for HIV infection in the U.S.: the national HIV behavioral surveillance system. Public Health Rep. 2007;122(Suppl 1_suppl):32–38. doi:10.1177/00333549071220S106.
  • Rolnick SJ, Parker ED, Nordin JD, Hedblom BD, Wei F, Kerby T, Jackson JM, Crain AL, Euler G. Self-report compared to electronic medical record across eight adult vaccines: do results vary by demographic factors? Vaccine. 2013;31(37):3928–35. doi:10.1016/j.vaccine.2013.06.041.
  • Niccolai LM, McBride V, Julian PR. Sources of information for assessing human papillomavirus vaccination history among young women. Vaccine. 2014;32(25):2945–47. doi:10.1016/j.vaccine.2014.03.059.
  • Thomas R, Higgins L, Ding L, Widdice LE, Chandler E, Kahn JA. Factors associated with HPV vaccine initiation, vaccine completion, and accuracy of self-reported vaccination status among 13- to 26-year-old men. Am J Mens Health. 2018;12(4):819–27. doi:10.1177/1557988316645155.
  • Oliveira CR, Avni-Singer L, Badaro G, Sullivan EL, Sheth SS, Shapiro ED, Niccolai LM. Feasibility and accuracy of a computer-assisted self-interviewing instrument to ascertain prior immunization with human papillomavirus vaccine by self-report: cross-sectional analysis. JMIR Med Inform. 2020;8(1):e16487. doi:10.2196/16487.
  • Hamby S, Turner H, Finkelhor D. Financial strain index. Crimes Against Children Research Center; 2011. doi:10.13140/RG.2.1.2368.0161.
  • Volz EM, Heckathorn DD. Probability based estimation theory for respondent driven sampling. J Off Stat. 2008;24(1):79–97.
  • Avery L, Rotondi N, McKnight C, Firestone M, Smylie J, Rotondi M. Unweighted regression models perform better than weighted regression techniques for respondent-driven sampling data: results from a simulation study. BMC Med Res Methodol. 2019;19(1):202. doi:10.1186/s12874-019-0842-5.
  • SAS Institute Inc. SAS/STAT 14.2 user’s guide the surveylogistic procedure. Cary, North Carolina: SAS Institute Inc; 2016. [Accessed January 14 2021]. https://support.sas.com/documentation/onlinedoc/stat/142/surveylogistic.pdf
  • Szklo M, Nieto J. Epidemiology: beyond the basics, 4th ed. Burlington, USA: Jones & Bartlett Learning; 2018.
  • Grewal R, Deeks SL, Hart TA, Cox J, De Pokomandy A, Grennan T, Lambert G, Moore D, Brisson M, Coutlée F, et al. Human papillomavirus (HPV) vaccine uptake among a community-recruited sample of gay, bisexual, and other men who have sex with men in the three largest cities in Canada from 2017 to 2019. Vaccine. 2021;39(28):3756–66. doi:10.1016/j.vaccine.2021.05.031.
  • Oliver SE, Hoots BE, Paz-Bailey G, Markowitz LE, Meites E. Increasing human papillomavirus vaccine coverage among men who have sex with men—National HIV behavioral surveillance, United States, 2014. J Acquir Immune Defic Syndr. 2017;75(Suppl3):S370–S374. doi:10.1097/QAI.0000000000001413.
  • Wheldon CW, Daley EM, Buhi ER, Baldwin JA, Nyitray AG, Giuliano AR. HPV vaccine decision-making among young men who have sex with men. Health Educ J. 2017;76(1):52–65. doi:10.1177/0017896916647988.
  • Rank C, Gilbert M, Ogilvie G, Jayaraman GC, Marchand R, Trussler T, Hogg RS, Gustafson R, Wong T, and the ManCount Study Team. Acceptability of HPV vaccination and sexual experience prior to disclosure to health care providers among MSM in Vancouver, Canada. Vaccine. 2012;30(39):5755–60. doi:10.1016/j.vaccine.2012.07.001.
  • Goldberg SK, Rothblum ED, Russell ST, Meyer IH. Exploring the Q in LGBTQ: demographic characteristic and sexuality of queer people in a U.S. representative sample of sexual minorities. Psychol Sex Orientat Gend Divers. 2019;7(1). accessed 2021 Jan 20. https://escholarship.org/uc/item/094524n2.
  • Reiter PL, Brewer NT. Hepatitis B vaccination among a national sample of gay and bisexual men. Sex Transm Dis. 2011;38(3):235–38. doi:10.1097/OLQ.0b013e3181f422bb.
  • Ward BW, Dahlhamer JM, Galinsky AM, Joestl SS. Sexual orientation and health among U.S. adults: national health interview survey, 2013. Natl Health Stat Rep. 2014;77:1–10.
  • Brooks H, Llewellyn CD, Nadarzynski T, Pelloso FC, De Souza Guilherme F, Pollard A, Jones CJ. Sexual orientation disclosure in health care: a systematic review. Br J Gen Pract. 2018;68(668):e187–e196. doi:10.3399/bjgp18X694841.
  • McComb E, Ramsden V, Olatunbosun O, Williams-Roberts H. Knowledge, attitudes and barriers to human papillomavirus (HPV) vaccine uptake among an immigrant and refugee catch-up group in a western Canadian province. J Immigr Minor Health. 2018;20(6):1424–28. doi:10.1007/s10903-018-0709-6.
  • Wilson L, Rubens-Augustson T, Murphy M, Jardine C, Crowcroft N, Hui C, Wilson K. Barriers to immunization among newcomers: a systematic review. Vaccine. 2018;36(8):1055–62. doi:10.1016/j.vaccine.2018.01.025.
  • Musto R, Siever JE, Johnston JC, Seidel J, Rose MS, McNeil DA. Social equity in human papillomavirus vaccination: a natural experiment in Calgary Canada. BMC Public Health. 2013;13(1):640. doi:10.1186/1471-2458-13-640.
  • Edelstein M, Iyanger N, Hennessy N, Mesher D, Checchi M, Soldan K, McCall M, Nugent J, Crofts J, Lo J, et al. Implementation and evaluation of the human papillomavirus (HPV) vaccination pilot for men who have sex with men (MSM), England, April 2016 to March 2017. Euro Surveill. 2019;24(8):8. doi:10.2807/1560-7917.ES.2019.24.8.1800055.
  • Leung SOA, Akinwunmi B, Elias KM, Feldman S. Educating healthcare providers to increase human papillomavirus (HPV) vaccination rates: a qualitative systematic review. Vaccine X. 2019;3:100037. doi:10.1016/j.jvacx.2019.100037.
  • Ruffin MT, Plegue MA, Rockwell PG, Young AP, Patel DA, Yeazel MW. Impact of an electronic health record (EHR) reminder on human papillomavirus (HPV) vaccine initiation and timely completion. J Am Board Fam Med JABFM. 2015;28(3):324–33. doi:10.3122/jabfm.2015.03.140082.
  • Apaydin KZ, Fontenot HB, Borba CPC, Shtasel DL, Ulery S, Mayer KH, Keuroghlian AS. Three-dose HPV vaccine completion among sexual and gender minority young adults at a Boston community health center. Vaccine. 2018;36(32):4897–903. doi:10.1016/j.vaccine.2018.06.057.
  • Stinchfield PK. Practice-proven interventions to increase vaccination rates and broaden the immunization season. Am J Med. 2008;121(7):S11–S21. doi:10.1016/j.amjmed.2008.05.003.
  • Dubé E, Gagnon D, Clément P, Bettinger JA, Comeau JL, Deeks S, Guay M, MacDonald S, MacDonald NE, Mijovic H, et al. Challenges and opportunities of school-based HPV vaccination in Canada. Hum Vaccines Immunother. 2019;15(7–8):1650–55. doi:10.1080/21645515.2018.1564440.
  • Salvadori MI. Human papillomavirus vaccine for children and adolescents. Paediatr Child Health. 2018;23(4):262–65. doi:10.1093/pch/pxx179.
  • Frank LB, Murphy ST, Chatterjee JS, Moran MB. Baezconde-Garbanati L. telling stories, saving lives: creating narrative health messages. Health Commun. 2015;30(2):154–63. doi:10.1080/10410236.2014.974126.
  • McEwen M, Farren E. Actions and beliefs related to hepatitis B and influenza immunization among registered nurses in Texas. Public Health Nurs. 2005;22(3):230–39. doi:10.1111/j.0737-1209.2005.220306.x.
  • Johnson TP ed. Handbook of health survey methods. 1 ed. Hoboken, USA: Wiley; 2014.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.