Interventions to increase HPV vaccination coverage: A systematic review

ABSTRACT

We reviewed intervention studies designed to increase human papillomavirus (HPV) vaccination coverage to further understand the impact interventions can have on HPV vaccination coverage. We searched 5 databases for intervention studies published from June 2006 to May 2015. Studies were included if they quantitatively measured HPV vaccination coverage as an outcome and were conducted in the United States. We abstracted outcomes, methods, and results from each study and classified by type of intervention conducted. Findings from 34 studies suggest many types of intervention strategies can increase HPV vaccination coverage in different settings, and with modest cost. Interventions were effective especially when implemented in combination at both provider and community levels. However, not all interventions showed significant effects on coverage. More research is needed to identify the best methods for widespread implementation of effective strategies.

KEYWORDS:

• coverage
• human papillomavirus
• HPV
• vaccination
• interventions

Introduction

Human papillomavirus (HPV) is the most common sexually transmitted infection among men and women in the United States.¹ Approximately 79 million Americans are infected with HPV, and every year, an estimated 14 million people become newly infected.¹ There are more than 100 types of HPV, and though not all lead to disease, some types of HPV can cause genital warts and others cause cancer, including cervical, vulvar, vaginal, penile, anal, and oropharyngeal cancers.^1,2 In the United States, approximately 27,000 men and women develop new HPV associated cancers each year.³ Approximately 360,000 people develop genital warts each year.¹

HPV vaccination is a means of primary prevention against HPV associated cancers and diseases. There are 3 HPV vaccines licensed for use in the United States. The quadrivalent vaccine (4vHPV) was licensed by the Food and Drug Administration (FDA) in 2006,⁴ the bivalent vaccine (2vHPV) was licensed in 2009,⁵ and the 9-valent vaccine (9vHPV) was licensed in 2014.⁶ The 4vHPV and 2vHPV vaccines protect against virus types 16 and 18, which cause an estimated 66% of cervical cancers, and the 4vHPV vaccine also protects against types 6 and 11, which cause about 90% of genital warts and recurrent respiratory papillomatosis.^4-7 The 9vHPV vaccine protects against types 6, 11, 16, 18, 31, 33, 45, 52, and 58 and has the potential to prevent approximately 81% of cervical cancers.² Clinical trial data have shown that the vaccines are safe and effective, and post-licensure studies indicate that vaccination dramatically reduces the incidence and prevalence of HPV, genital warts, and cervical and anal dysplasias.^8-13 All three HPV vaccines that are licensed for use in the United States are recommended by the Advisory Committee on Immunization Practices (ACIP).⁶

Though safe and effective HPV vaccines have been available since 2006, HPV vaccination coverage in the United States remains low.¹⁴ The Healthy People 2020 target for completion of the three-dose HPV vaccination series is 80% for adolescent boys and girls aged 13 to 15 years;¹⁵ however, current coverage estimates fall considerably short of this goal. In 2014, only 60.0% of girls aged 13 to 17 years had initiated the series with at least one HPV vaccine dose, and 39.7% had received three doses.¹⁴ Coverage for adolescent boys in 2014 was even lower, with only 41.7% of boys aged 13 to 17 years receiving at least one dose and 21.6% receiving three doses.¹⁴ Though 2014 estimates demonstrate significant increases from previous years' assessments of HPV vaccination coverage, given comparatively low coverage and slow uptake, it is critical for healthcare providers and public health organizations to increase efforts to improve HPV vaccination coverage and reduce the burden of HPV associated cancers and diseases.^14,16

Previously published systematic reviews have compiled some of the available evidence for intervention strategies to increase HPV vaccination coverage, but no comprehensive review has been published to date. Many HPV vaccination systematic reviews have only addressed factors associated with HPV vaccination, such as demographics or perceived barriers, or have reviewed interventions that target intermediate outcomes, such as vaccination knowledge or intention to vaccinate.^17-21 One recently published review summarized 14 interventions with HPV vaccination coverage as a study outcome and concluded that most of the interventions reviewed significantly increased HPV vaccination coverage, in contrast to findings from a previous review of educational interventions.^21,22 This recent review was an important contribution to the literature on HPV vaccination interventions; however, it only examined non-individual level system or community-based interventions.²²

To help healthcare professionals continue to evaluate the entire spectrum of published interventions to improve HPV vaccination coverage, we conducted a comprehensive review of the literature that falls within the Community Guide's categories of interventions to increase appropriate vaccination (Table 1).²³ The Community Guide is a centralized resource developed by an independent Task Force, the Centers for Disease Control and Prevention (CDC), and other partner organizations.²⁴ Collaborators aggregate and systematically review literature about public health interventions.²⁴ As it relates to vaccination, the Community Guide broadly reviews and evaluates interventions that can increase general vaccination coverage, but the Guide does not present information for specific vaccines or account for the unique challenges that adolescent vaccines such as the HPV vaccine face.²³ The present review offers a look at the intervention literature related to HPV vaccination in this framework to understand where strategies categorized by the Community Guide may or may not be promising in the context of HPV vaccination.

Results

Intervention study characteristics

Two thousand five hundred and sixty nine studies were identified in the primary search, resulting in 34 studies eligible for inclusion in the review (Fig. 1). Table 2 illustrates the characteristics of the studies included for review. A majority of the studies focused exclusively on girls (n = 24, 70.6%) ^{25-32,36-40,42,44-48,50,55-58} and were implemented prior to the 2011 ACIP recommendation that boys receive HPV vaccination (n = 22, 64.7%).^{25-27,29-32,36-40,42,44,46-48,50,55-58} Studies addressed vaccination for many different age groups, including adolescents and young adults. Sixteen studies (47.1%) assessed intervention effects on more than one vaccine in the adolescent platform, including HPV vaccine,^{25-27,31-34,38,42,45,46,50,52,53,55,56} while eighteen (52.9%) assessed intervention impact only on HPV vaccination coverage.^{28-30,35-37,39-41,43,44,47-49,51,54,57,58}

Figure 1. Flow chart of review process and study selection.

Table 1. Intervention categories (n = 34).

Category name	Definition adapted from Community Guide^23	N (%)
Increasing Community Demand for HPV Vaccination
Vaccination requirements for school attendance	HPV vaccination requirements are laws or policies requiring HPV vaccinations as a condition of school attendance. Laws are created by states, with requirements established by the legislature and embodied in statutes or adopted as administrative rules by health or education departments. Institutions, such as colleges and private schools, may establish additional policies for attendance.	3 (8.8%)
Patient reminder and recall systems	Patient reminder and recall interventions involve reminding a target population that HPV vaccinations are due (reminder) or late (recall). Reminders and recalls differ in content and are delivered by various methods, such as by telephone, letters, or postcards.	8 (23.5%)
Patient education	Patient education provides information to target populations. Information is disseminated with the goal of informing and motivating individuals to seek HPV vaccination.	2 (5.9%)
Community-based interventions implemented in combination	Community-based interventions implemented in combination involve two or more interventions to increase HPV vaccination rates within a targeted population. Efforts can involve partnerships between organizations, government, and HPV vaccination providers to implement interventions to increase community demand or enhance access to HPV vaccination services.	6 (17.6%)
Provider- or System-Based Interventions to Increase HPV Vaccination
Provider assessment and feedback	Provider assessment and feedback involves retrospectively evaluating the performance of providers in delivering HPV vaccinations to a patient population and giving them feedback on their performance. Assessment and feedback can also involve incentives or benchmarking.	1 (2.9%)
Provider reminders	Provider reminders inform those who administer vaccinations that patients are due for specific vaccinations. Techniques by which reminders are delivered vary, but can include notes prepared in advance and posted in patient charts, alerts in electronic medical records, or letters sent by mail.	2 (5.9%)
Healthcare system-based interventions implemented in combination	Healthcare system-based interventions implemented in combination involve the use of two or more interventions to increase HPV vaccination rates within a targeted client population. Interventions are primarily implemented in healthcare settings, although efforts may include additional activities within the community.	9 (26.5%)
Enhancing Access to HPV Vaccination Services
Vaccination programs in schools	HPV vaccination programs in schools are interventions delivered on-site to improve immunization rates. HPV vaccination programs are often collaborations between schools and local health departments, private healthcare providers, or community healthcare services.	2 (5.9%)
Reducing out-of-pocket costs	Reducing out-of-pocket costs to families for HPV vaccinations or administration of HPV vaccinations can be implemented by paying for vaccinations or administration, providing insurance coverage, or reducing copayments for HPV vaccinations at the point-of-service.	1 (2.9%)

Table 2. Study characteristics (n = 34).

Total studies (n = 34)	N (%)
Studies that focused on females only Adolescents^* Young adults^** Adolescents and young adults^***	24 (70.6%) 16 (47.1%) 4 (11.8%) 4 (11.8%)
Studies that focused on males only Adolescents^*	1 (2.9%) 1 (2.9%)
Studies that focused on both females and males Adolescents^* Adolescents and young adults^***	9 (26.5%) 7 (20.6%) 2 (5.9%)
Randomized controlled trials (RCTs)^****	16 (47.1%)
Quasi-experimental or observational studies^****	19 (55.9%)
Studies that were conducted after 2011 recommendation for HPV vaccination for boys	12 (35.3%)
Studies that assessed only HPV vaccine	18 (52.9%)

* Study populations with age range of 9-17

** Study populations with age range of 18-26

*** Study populations with age range of 9-26

**** One study included a randomized trial and observational element; therefore, it is classified in two categories and percentages will not add up to 100.0%.

Intervention study results

A forest plot of selected results of intervention studies measuring series initiation (≥1 dose of HPV vaccine) is shown in Figure 2. A forest plot with additional results can be seen in Figure S1. Table 3 presents the study design, methods, outcomes, and selected results for all of the studies included in the systematic review.

Figure 2. Forest plot of selected HPV vaccination intervention results: series initiation. ▪ : Effect estimates from original source. ▴ : Effect estimates calculated by review authors based on given data in source. *Results presented if study included a measure of series initiation (one or more doses of HPV vaccine). Results were excluded if study did not include a measure of series initiation or if results were not presented in the original paper such that an effect size could be calculated. If studies measured series initiation in more than one population or using different methods, the specifics are indicated in parentheses following the study author's name. F denotes studies that focused on females and M denotes studies that focused on males. Full study characteristics and results can be found in Table 3. **Results were determined to be significant by the original paper. ^†Effect estimate's confidence intervals could not fit in the plot scale. Cassidy et al. found an OR of 9.43 (2.69–33.10) for series initiation. ^‡Effect estimate's confidence intervals could not fit in the plot scale. Perkins et al. found an OR of 11 (6.9–17) for series initiation among males.

Interventions to increase community demand for HPV vaccination

Nineteen studies (55.9%) utilized interventions to increase community demand for HPV vaccination.^25-43 Three studies examined the effect of vaccination requirements for school attendance on HPV vaccination coverage.^25-27 All three studies were ecological studies examining how vaccination policies for school attendance affected series initiation as a primary outcome.^25-27 Vaccination requirements consisted of educational requirements for parents, requirements for HPV vaccination for school attendance, or requirements for another adolescent vaccination for school attendance; however, requirements for school attendance typically had broad opt-out provisions, with waivers available for religious, medical, and/or philosophical objections.^25-27 Findings from the school vaccination policy studies consistently showed that while school requirements increased other adolescent vaccinations' coverage, HPV vaccination coverage only increased by a small amount or not at all.^25-27 One of the studies demonstrated that in the five states and the District of Columbia that had school entry or educational requirements for HPV vaccination in the 2008–2009 school year, no significant difference was found in 2008–2009 NIS-Teen HPV vaccination coverage for jurisdictions that had requirements vs. those that did not, though requirements for other adolescent vaccinations resulted in significant increases in coverage for those vaccines.²⁵ The two other school policy studies examined whether other adolescent vaccination requirements had any spillover effects on HPV vaccine series initiation.^26,27 One study looked at tetanus-diphtheria-acellular pertussis (Tdap) vaccination requirements enacted during or before the 2009–2010 school year and found that a greater, though still small, percentage of females had initiated the HPV vaccination series in states that had Tdap vaccination mandates (percentage point difference: 4.4, p = .004).²⁶ The other study observed the effect that mandates enacted in Michigan in 2010 for Tdap, meningococcal, and varicella vaccination had on HPV vaccination coverage in two cohorts of 6^th graders; one cohort of 6^th graders in 2009, the other 6^th graders in 2010.²⁷ The study found small significant increases in HPV series initiation by age 13 for girls (hazard ratio [HR] 1.18, p < .001), especially if the HPV vaccine was co-administered at the first adolescent visit (HR 1.22, p < .001).²⁷ However, proportionally the increases were much smaller compared to the coverage increases for the other adolescent vaccines; HPV coverage increased approximately 5 percentage points, while Tdap, meningococcal, and varicella coverage increased approximately 20 percentage points.²⁷

Table 3. HPV vaccination intervention study summaries (n = 34). OR: odds ratio, AOR: adjusted odds ratio, RR: risk ratio, ARR: adjusted risk ratio, HR: hazard ratio. Confidence intervals are presented for ratios only if a p value was not presented in the original paper.

Author, year, title	Outcome (O) and methods (M)	Intervention (I) and control (C) conditions	Target population, sample size, and setting	Selected results measured in percentages	Selected results measured in ratios
Increasing Community Demand for HPV Vaccination
Vaccination requirements for school attendance
Bugenske E, et al. 2012. Middle school vaccination requirements and adolescent vaccination coverage.^25	O: Up to date for HPV vaccinations during one year M: Ecological study. Reviewed states' middle school vaccination for school entry and vaccination education requirements and categorized states by status based on Immunization Action Coalition data. State requirement status was then correlated with estimated coverage from NIS-Teen data.	I: State had policy mandating HPV vaccination for school entry or state had policy mandating education on HPV vaccination C: State had no policy on HPV vaccination or education	20,066 13–17 year old males and females from the 2009 NIS Teen dataset, but restricted to females only (number of females not reported) 50 states + DC: I: 5 states + DC C: 45 states	Series initiation (I vs. C, p = not significant): I: 45.0% C: 44.2%	—
Dempsey A, et al. 2011. Human papillomavirus vaccination rates and state mandates for Tetanus-containing vaccines.^26	O: Series initiation during one year M: Ecological study. Each state categorized by whether adolescent Td and/or Tdap vaccines were mandated for school entry based on Immunization Action Coalition data. Mean HPV vaccine series initiation levels among adolescent females were compared between each mandate category.	I: State has Td or Tdap school mandate C: State has no Td or Tdap school mandate	13–17 year old males and females from the 2009 NIS Teen data set, but restricted to females only (number of females not reported) 50 states + DC: I: 30 states + DC C: 20 states	Series initiation (I vs. C, p = .004): I: 47.3% C: 42.9%	—
Potter R, et al. 2014. Adolescent immunization coverage and implementation of new school requirements in Michigan, 2010.^27	O: Series initiation following implementation of new school mandates for Tdap/MCV4/varicella M: Ecological study. Data were derived from the Michigan statewide immunization information system. The study assessed the immunization status by age 13 of Michigan children enrolled in sixth grade in 2009 or 2010.	I: Enrolled in 6th grade post 2010 adolescent vaccination requirements C: Historical control	128,627 6th grade females: I: 63,448 C: 65,179 Michigan	Series initiation by age 13: I: 34.3% C: 29.6% Series initiation by age 13 when another adolescent vaccination was administered at the first adolescent visit: I: 40.6% C: 36.7% Average required vaccine (Tdap/MCV4) coverage change after 2010 requirements: Difference of +20% between I and C	Series initiation by age 13, all females (I vs. C, p<.001): HR 1.18 Series initiation by age 13, females who had HPV co-administered at the first adolescent visit (I vs. C, p<.001): HR 1.22 Hazard ratios were proportionally greater for required vaccines' coverage (Tdap, MCV4) than HPV vaccine coverage.
Patient reminder and recall systems
Chao C, et al. 2015. A randomized intervention of reminder letter for human papillomavirus vaccine series completion.^28	O: Series completion after 12 months M: Randomized controlled trial. Eligible females were members of the health plan who had at least one but no greater than two doses of the series.	I: Females or parents of females received quarterly reminder letters with date of first dose receipt, adolescent immunization schedule, a message to encourage next dose receipt, and a phone number for the patient's home medical center. C: Received standard of care with no reminder letter.	12,205 9–26 year old females: I: 9,760 C: 2,445 Kaiser health plan members in Southern California	Series completion, 9–26 year olds (I vs. C, p<.01): I: 56.4% C: 46.6% Series completion, 9–17 years olds (I vs. C, p<.01): I: 66.2% C: 53.5% Series completion, 18–26 year olds (I vs. C, p<.01): I: 43.5% C: 37.0%	—
Kharbanda E, et al. 2011. Text message reminders to promote human papillomavirus vaccination.^29	O: On-time receipt of next dose (dose 2 after 6 months or dose 3 after 8 months) M: Prospective cohort study. Eligible subjects were identified through billing records and hospital-based immunization registries and enrolled through purposive sampling. Charts were reviewed by researchers to determine demographic data and receipt of doses.	I: Text message reminder, up to 3 weekly texts that the participant was due for her next dose C1: Controls who were offered intervention but opted-out C2: Historical controls	1,512 parents of 9–20 year old females (some of sample may be 18–20 females themselves): I: 124 C1: 208 C2: 1,080 9 pediatric sites (5 academic, 4 private) in New York City	Received next dose on-time (I vs. C1, p = .001; I vs. C2, p = .003): I: 51.6% C1: 35.0% C2: 38.1% Intervention effect results were sustained at 4 months post-intervention.	Received next dose on-time (I vs. C1, p = .003; I vs. C2, p = .002): I vs. C1: AOR 2.03 I vs. C2: AOR 1.83
Patel A, et al. 2014. Staying on track: A cluster randomized controlled trial of automated reminders aimed at increasing human papillomavirus vaccine completion.^30	O: On-time series completion within 8 months of initiation M: Randomized controlled trial. Health centers were matched and randomized prior to participant recruitment but neither participants nor providers were blinded to study arm assignment.	I: Received automated reminders (text, e-mail, phone, private Facebook message, or standard mail). Each intervention participant received four messages (one if she selected standard mail), sent three days apart prior to doses two and three. C: Standard of care	365 18–26 year old females: I: 180 C: 185 10 outpatient reproductive health centers in NC, UT, AZ, WA, CO, and CA: I: 5 centers C: 5 centers	On-time series completion (I vs. C, p = not significant): I: 17.2% C: 18.9%	—
Suh C, et al. 2012. Effectiveness and net cost of reminder/recall for adolescent immunizations.^31	O: Series initiation after 6 months, and up to date completion after 12 months M: Randomized controlled trial. In each practice, adolescents who had not received 1 or more targeted vaccinations (including HPV vaccine) were randomly selected and randomized to intervention or control conditions.	I: Families were sent a first letter and autodialer telephone call. Adolescents still in need of targeted immunizations 1 month later received a second autodialer telephone call. A final letter was sent to adolescents still needing immunizations 2 months after the initial R/R. C: Standard of care	972 11–18 year old females: I: 503 C: 469 4 private pediatric practices in metropolitan Denver	Overall series initiation (I vs. C, p < .05): I: 26.5% C: 15.3% Up to date completion from 0 baseline doses (I vs. C, p < .05): I: 11.7% C: 4.4% Up to date completion from ≥1 baseline doses (I vs. C, p = not significant): I: 74.7% C: 73.9%	—
Szilagyi P, et al. 2013. A randomized trial of the effect of centralized reminder/recall on immunizations and preventive visits for adolescents.^32	O: Series initiation, dose 2, and completion after 1 year M: Randomized controlled trial. Publicly insured adolescent girls who had not received any dose of HPV vaccine were identified in participating primary care practices. The study randomly assigned adolescents to one of two intervention conditions or control condition.	I1: Reminder letters advised parents to call their adolescent's primary care practice to schedule an appointment. The letters provided the practice's telephone number. I2: Telephone reminders were sent at the same frequency as letters by an autodialer service. C: Standard of care	2,037 10.5–17 year old females: I1: 682 I2: 699 C: 656 37 primary care practices in a non-profit managed care organization in upstate New York	Series initiation: I1: 27% I2: 27% C: 21% Dose 2: I1: 26% I2: 26% C: 18% Series completion: I1: 18% I2: 19% C: 14%	Series initiation (I1 vs. C, p = not significant; I2 vs. C, p = not significant): I1: HR 1.3 I2: HR 1.1 Dose 2 (I1 vs. C, p < .05; I2 vs. C, p < .01): I1: HR 1.5 I2: HR 1.6 Series completion (I1 vs. C, p = not significant; I2 vs. C, p < .05): I1: HR 1.4 I2: HR 1.5
Bar-Shain D, et al. 2015. Direct messaging to parents/guardians to improve adolescent immunizations. ^33	O: Receipt of next dose after 6 months. M: Prospective cohort study. Eligible participants due/overdue for an HPV vaccine dose were identified through EHRs.	I: A stepwise messaging cascade was used. E-mail sent first, if provided. If no e-mail provided, then texted. If not able to receive a text, an automated recorded voice message was delivered. If no working phone numbers were available, a postcard was sent.	2,897 11–18 year old males and females Academic tertiary public health system in Northeast Ohio	22.9% (n = 745 doses delivered) of adolescents received a needed HPV vaccine dose Patients needing the third dose of HPV vaccine were significantly more likely to get vaccinated (31.0%) than patients needing the first (16.9%) or second doses (26.3%) in the series (p < .0001).	—
Morris J, et al. 2015. Comparison of reminder methods in selected adolescents with records in an immunization registry.^34	O: Series initiation and completion after 6 months. M: Prospective cohort study. Participants who were not up to date on at least one of the recommended adolescent vaccines were selected randomly from San Diego County Immunization Registry records. Once contacted, parents of participants were given a choice of reminder method.	I: Parents chose to receive at least one reminder: mail, e-mail, or text reminder. C1: Received an enrollment phone call, but declined to participate. C2: No contact control group (eligible, but not sampled).	121,409 parents/guardians of 11–17 year old males and females: I: 1,797 (282 mail, 552 text message, 963 e-mail) C1: 3,253 C2: 116,356 San Diego County	Series initiation percent increase from baseline, females (compared from baseline, all p < .05): I1 e-mail: +9.7% I1 mail: +10.3% I1 text: +11.5% C1: +6.7% C2: +4.9% Series initiation percent increase from baseline, males (compared from baseline, all p < .05): I1 e-mail: +20.6% I1 mail: +17.6% I1 text: +29.1% C1: +16.3% C2: +10.9% Series completion percent increase from baseline, females (compared from baseline, all p < .05): I1 e-mail: +13.6% I1 mail: +15.0% I1 text: +21.6% C1: +8.7% C2: +6.2% Series completion percent increase from baseline, males (compared from baseline, all p < .05): I1 e-mail: +10.0% I1 mail: +9.5% I1 text: +16.1% C1: +8.6% C2: +6.4%	—
Rand C, et al. 2015. Effectiveness of centralized text message reminders on human papillomavirus coverage for publicly insured adolescents.^Citation35	O: Series initiation, dose 2, and completion after 8 months. M: Randomized controlled trial. Parents of adolescents who had no previous HPV vaccination were eligible to receive the intervention.	I: Up to 4 text messages were sent to parent/guardians with a reminder that their child was due to receive an HPV vaccine dose C: Sent 1 text with a general adolescent health message	3,812 11–16 year old males and females: I: 1,893 C: 1,919 Non-profit managed care organization in upstate New York	Series initiation: I: 16% C: 13% Dose 2: I: 7% C: 6% Series completion: I: 2% C: 2%	Series initiation (I vs. C, p = 0.04): HR 1.3 Dose 2 (I vs. C, p = not significant): HR 1.2 Series completion (I vs. C, p = not significant): HR 1.3
Patient education
Hopfer S. 2011. Effects of a narrative HPV vaccination intervention aimed at reaching college women: A randomized controlled trial.^36	O: Series initiation after 2 months M: Randomized controlled trial. Participants were randomly sampled from the university clinic database. Two months after receiving intervention or control, participants were emailed asking whether they received vaccine or not.	I1: Video of vaccine decision narratives delivered by peers and experts/providers I2: Video of vaccine decision narratives delivered by peers I3: Video of vaccine decision narratives delivered by experts/providers C: Informational video with no narrative, informational website, or no message	404 18 to 26 year old females: I1: 101 I2: 101 I3: 50 C: 152 1 university health clinic	Series initiation (I1 vs. C, p = .035; I2 vs. C, p = not significant; I3 vs. C, p = not significant): I1: 21.8% I2: 17.8% I3: 6.0% C: 11.8%	Series initiation (I1 vs. C, p = .036; I2 vs. C, p = not significant, I3 vs. C, p = not significant): I1: OR 2.07 I2: OR 1.61 I3: OR 0.48
Vanderpool R, et al. 2013. “1-2-3 Pap” intervention improves HPV vaccine series completion among Appalachian women.^37	O: Series completion after 12 months M: Randomized controlled trial. Nurses provided dose 1 free of charge and offered study enrollment following dose 1. Enrolled women were randomized to intervention or control condition.	I: Watched 13 minute DVD video grounded in information, motivation, and behavioral skills theory. Participants also received follow up reminder calls for doses 2 and 3, similar to the control group. C: Standard of care (educational pamphlet and telephone reminders for doses 2 and 3)	344 18 to 26 year old females: I: 178 C: 166 Recruited from multiple locations (local health departments, medical clinics, community colleges, outdoor festivals, Wal-Mart stores, businesses, and women's homes) in Appalachian Kentucky	Series completion (I vs. C, p = .03): I: 43.3% C: 31.9%	Series completion (I vs. C, p = .001): AOR 2.44
Community-based interventions implemented in combination
Kempe A, et al. 2012. Effectiveness and cost of immunization recall at school-based health centers.^38	O: Received any dose of HPV vaccine, and dose 2/completion among those who initiated at the school after six months M: Prospective cohort study. 6th grade girls with parental consent who needed at least one HPV vaccine dose were recalled at a school-based recall location. Immunization rates were examined from the state immunization information system.	I: Recalled to receive 1+ HPV dose(s) at a school-based recall location. Students were recalled up to 2 times by 1 of 3 methods: a pass sent to the student in their classroom, a phone call to the classroom, or a staff member of the health center walking into their classroom to escort them to the clinic.	265 6th grade females 4 public middle school school-based health centers in Denver	Received any dose of HPV vaccine: 59% Females who needed a first HPV dose at baseline: Dose 2: 25% Series completion: 0% Parents were less likely to consent for their child to get HPV vaccine (62%) than other adolescent vaccines (no p values reported; Tdap: 78%; MCV4: 69%).	—
Patel D, et al. 2012. Human Papillomavirus vaccine intent and uptake among female college students^39	O: Series initiation after 6 months M: Randomized controlled trial. Participants were randomized to receive either intervention or control condition. HPV vaccine uptake at six months following enrollment was identified.	I: Received mailed educational fact sheet and mailed reminder C: Standard of care	256 18–26 year old females: I: 128 C: 128 1 university health clinic at the University of Michigan	Series initiation (I vs. C, p = not significant): I: 5.5% C: not reported	Series initiation (I vs. C, p = not significant): RR .84
Cates J, et al. 2011. Evaluating a county-sponsored social marketing campaign to increase mothers' initiation of HPV vaccine for their pre-teen daughters in a primarily rural area.^40	O: Series initiation after 6 months M: Prospective cohort study. Counties were selected by a convenience sample to represent a broader 13 county district. Coverage data were abstracted in the study counties via the North Carolina Immunization Registry.	I: HPV vaccination educational information was placed in intervention healthcare provider offices and community locations, such as pharmacies, salons, and grocery stores. Materials included posters, brochures, a website, a radio PSA, media releases, hotline, “starting the conversation” guides for providers, buttons for providers, and sticky notes to provide provider reminders. C1: Received no campaign, but shared regional similarities to intervention counties C2: Received no campaign, rest of state's counties	251,398 9–13 year old females: I: 19,963 C1: 13,716 C2: 218,259 109 counties in North Carolina: I: 4 counties C1: 9 counties (geographically similar to intervention counties) C2: 96 counties (geographically distinct from intervention counties)	Series initiation (all intervention counties vs. C1, p < .01; all intervention counties vs. C2, p < .01) I county 1: 7.08% I county 2: 6.76% I county 3: 3.23% I county 4: 1.86% C1: 5.18% C2: 5.04%
Cates J, et al. 2014. Intervention effects from a social marketing campaign to promote HPV vaccination in preteen boys.^41	O: Series initiation after 3 months and 6 months post-intervention M: Prospective cohort study. Intervention counties were located in a 13-county region. Coverage data were abstracted in the study counties via the North Carolina Immunization Registry.	I: HPV vaccination posters and brochures were distributed to intervention county health departments and 194 providers; 2 radio PSAs aired in the counties; providers were given a free CME online training; providers were given a communication tip sheet; and a website was launched with links to credible resources about HPV vaccine C: Received no campaign	25,869 9–13 year old males: I: 19,842 C: 6,027 28 counties in North Carolina: I: 13 counties C: 15 counties (geographically distinct from intervention counties)	Series initiation: I: 7.3% C: 5.2% Series initiation 6 months post-intervention: I: 6.8% C: 6.6%	Series initiation (I vs. C, p = .002): HR 1.34 Series initiation 6 months post-intervention (I vs. C, p = not significant): HR .99
Szilagyi P, et al. 2011. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. ^42	O: Series initiation, dose 2, and series completion after 12 months M: Randomized controlled trial. Adolescents in eight urban primary care practices were randomly assigned to study group. Data were abstracted from medical records and the state registry following the conclusion of the intervention period.	I: Trained immunization navigators at practices implemented immunization tracking, telephone and/or mail reminder/recall, and home visits if participants remained unimmunized or behind on preventive care visits. C: Standard of care	3,752 11–15 year old females: I: 1,903 C: 1,849 Eight primary care practices in Rochester, New York	Series initiation following intervention: I: 58.5% (+15.4 percentage point change from baseline) C: 42.9% (0 percentage point change from baseline) Dose 2 following intervention: I: 52.0% (+35.6 percentage point change from baseline) C: 36.2% (+19.8 percentage point change from baseline) Series completion following intervention: I: 36.5% (+33 percentage point change from baseline) C: 24.1% (+20.9 percentage point change from baseline)	Series initiation following intervention (I vs. C, p < .001): ARR 1.4 Dose 2 following intervention (I vs. C, p < .001): ARR 1.4 Series completion following intervention (I vs. C, p < .001): ARR 1.5
Rickert V, et al. 2015. School-based HPV immunization of young adolescents: Effects of two brief health interventions.^43	O: Series initiation and series completion after three years and four months M: Randomized controlled trial with no control condition. Parents of young adolescents receiving care at a school-based health center were randomly assigned to 4 potential intervention conditions. Asking the rhetorical question “do you want to protect your child from cancer?” was intended as a “foot-in-the-door” technique to begin a conversation, and one or two sided messages were tested to evaluate message persuasiveness.	I1: No rhetorical question, 1 sided message I2: Rhetorical question, 1 sided message I3: No rhetorical question, 2 sided message I4: Rhetorical question, 2 sided message	445 parents of 11–15 year old adolescents: I1: 116 I2: 109 I3: 106 I4: 114 5 school-based health clinics in Galveston, Texas: 3 in middle schools, 2 in high schools	—	Rhetorical questions were not significantly associated with series initiation (RR 1.15: CI 0.89, 1.50), dose 2 receipt, or series completion (ratios not reported). Message sidedness was not significantly associated with intention to vaccinate or series initiation, dose 2 receipt, or series completion (ratios not reported).
Provider- or System-Based Interventions to Increase HPV Vaccination
Provider reminders
Ruffin M, et al. 2015. Impact of an electronic health record reminder on human papillomavirus vaccine initiation and timely completion.^44	O: Series initiation and timely series completion during a 3 year period M: Retrospective cohort study. Eligible females with at least one doctor appointment in the study period were eligible for inclusion.	I: Practices used an institution-created electronic health record system that provided electronic prompting for HPV vaccine doses. C: Practices used an electronic health record system that did not provide electronic prompting.	15,021 9–26 year old females: I: 5,994 C: 9,027 9 community-based family medicine practices at 2 academic institutions in the Midwest: I: 5 practices C: 4 practices	Series initiation (I vs. C, p < .001): I: 35.0% C: 21.3% Timely series completion (I vs. C, p < .001): The intervention group had significantly greater on-time series completion. Percentiles presented graphically, see original paper for details	The intervention group had significantly higher odds of initiation at each level of covariates (AORs presented graphically, see original paper for detail).
Szilagyi P, et al. 2015. Effect of provider prompts on adolescent immunization rates: A randomized trial.^45	O: Series initiation, dose 2, and series completion after 12 months M: Randomized controlled trial. Primary care clinics at a local (GR-PBRN) and national (CORNET) level were randomized to receive intervention or standard-of-care control conditions. Intervention practices that lacked EHRs that could be programmed to deliver prompts utilized nurse/staff driven prompts. Following the intervention, participants within each site were randomly selected to have their charts reviewed and immunization data abstracted.	I: Practices used provider prompts (either through electronic health records or through nurse/staff) C: Standard of care	1,771 11–17 year old females: GR-PBRN sites in New York: I: 397 C: 420 CORNET sites in New York: I: 478 C: 476	GR-PBRN sites: Percentage point change in series initiation from baseline: I: 0 C: +1 Percentage point change in dose 2 from baseline: I: +1 C: 0 Percentage point change in series completion from baseline: I: 0 C: +2 CORNET sites: Percentage point change in series initiation from baseline: I: +5 C: +7 Percentage point change in dose 2 from baseline: I: +5 C: +3 Percentage point change in series completion from baseline: I: +2 C: −-2	GR-PBRN sites: Change in series initiation from baseline (I vs. C, p = not significant): AOR 0.92 Change in dose 2 from baseline (I vs. C, p = not significant): AOR 1.01 Change in series completion from baseline (I vs. C, p = not significant): AOR 0.93 CORNET sites: Change in series initiation from baseline (I vs. C, p = not significant): AOR 0.96 Change in dose 2 from baseline (I vs. C, p = not significant): AOR 1.06 Change in series completion from baseline (I vs. C, p = not significant): AOR 1.13
Provider assessment and feedback
Gilkey M, et al. 2014. Increasing provision of adolescent vaccines in primary care: A randomized controlled trial.^46	O: Series initiation after 5 months and 1 year M: Randomized controlled trial. Primary care clinics randomized to receive one of two intervention conditions or control condition. Post-intervention, the state's immunization registry provided HPV vaccine coverage data for younger patients (ages 11–12 years) and older patients (ages 13–18 years) in the clinics.	I1: In-person AFIX consultation I2: Webinar AFIX consultation C: Standard of practice	50,369 11–18 year old females at intervention sites: 11–12 year olds: 14,994 13–18 year olds 35,375 91 primary care clinics in North Carolina: I1: 30 clinics I2: 31 clinics C: 30 clinics	Overall difference in series initiation at 5 months compared to control, 11–12 year olds (I1 vs. C, p = .02; I2 vs. C, p < .01): I1: 1.5% I2: 1.9% Overall difference in series initiation at 5 months compared to control, 13–18 year olds (I1 vs. C, p = not significant; I2 vs. C, p = not significant) I1: 0.4% I2: -0.1% Overall difference in series initiation at 1 year compared to control, 11–18 year olds: All, p = not significant	—
Healthcare system-based interventions implemented in combination
Cassidy B, et al. 2014. A quality improvement initiative to increase HPV vaccine rates using an educational and reminder strategy with parents of preteen girls.^47	O: Series initiation and series completion after 13 months M: Quasi-experimental design. Intervention was designed based on predictors of parental acceptance and Health Belief Model. Prior to the intervention, physicians and office staff received education on HPV and HPV vaccine. A historical control group of patients in the same practice provided a comparison.	I: Parents received a FAQ brochure and clinical protocol/script from physician. Also received practice-based telephone reminders for dose completion. C: Historical control group	53 parents of 11–12 year old females: I: 24 C: 29 1 private pediatric practice in an urban location	Series initiation (I vs. C, p = .001): I: 75.0% C: 24.1% Series completion (I vs. C, p < .001): I: 62.5% C: 6.9%	Series initiation (I vs. C, p = significant): OR 9.43 (2.69–33.10) Series completion (I vs. C, p = significant): OR 22.50 (4.29–117.99)
Fiks A, et al. 2013. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt.^48	O: Series initiation, dose 2, and series completion after one year M: Randomized controlled trial. Practices cluster-randomized to receive clinician-intervention condition or none. Girls due for HPV dose 1, 2, or 3 were randomly assigned within each practice to receive family- focused reminders and educational telephone calls or none. Final vaccination rates were assessed at the end of the study period.	I1: Family-focused intervention (reminder and educational telephone calls) I2: Clinician-focused intervention (clinician education, electronic health record prompts, and audit/feedback) I3: Combined family-focused and clinician-focused interventions C: Standard of practice	22,486 11–17 year old females: I1: 5,680 I2: 5,557 I3: 5,561 C: 5,688 22 primary care practices in New Jersey and Pennsylvania	Series initiation: I1: 18% I2: 24% I3: 25% C: 16% Dose 2: I1: 71% I2: 64% I3: 73% C: 65% Series completion: I1: 73% I2:67% I3: 76% C: 63%	Series initiation: I1 vs. C: HR 1.1, p = .03 I2 vs. C: HR 1.5, p = .003 I3 vs. C: HR 1.6, p = .001 Dose 2: I1 vs. C: HR 1.2, p < .001 I2 vs. C: HR 1.0, p = not significant I3 vs. C: HR 1.3, p = .008 Series completion: I1 vs. C: HR 1.4, p < .001 I2 vs. C: HR 1.1, p = not significant I3 vs. C: HR 1.5, p < .001
Matheson E, et al. 2014. Increasing HPV vaccination series completion rates via text message reminders.^49	O: Receipt of next dose (dose 2 or series completion) and on-time receipt of next dose after 8 months M: Prospective cohort study. Participants were recruited via convenience sampling. Prior to implementation of the intervention, providers and clinical staff underwent education explaining HPV disease, the role of the vaccine in disease prevention, and current HPV vaccination coverage at the state, national, and practice level. Coverage data were extracted from the state immunization registry.	I: Enrolled in text message reminder system. Participant received three text message reminders per dose: one message 7 days prior to each HPV vaccine due date, one on the vaccine due date, and one 7 days after the due date C1: Indicated interest in participating but never enrolled in text message reminder system C2: Standard of care	312 11–22 year old males and females and/or their parents: I: 37 C1: 43 C2: 232 1 pediatric practice in North Carolina	Dose 2 (I vs. C1, p < .001; I vs. C2, p < .001): I: 73% C1: 33% C2: 27% On-time dose 2 (I vs. C1, p = not significant; I vs. C2, p = .035): I: 38% C1: 21% C2: 21% Series completion (I vs. C1, p = .008; I vs. C2, p = .018): I: 16% C1: 0% C2: 5% On-time series completion (I vs. C1, p = .018; I vs. C2, p = .007): I: 14% C1: 0% C2: 3%	—
Moss J, et al. 2012. Increasing adolescent immunization by webinar: A brief provider intervention at federally qualified health centers.^50	O: Series initiation, dose 2, or series completion after 1 month M: Prospective cohort study. Clinical coordinators from 17 federally qualified health centers participated in a competition to increase uptake of recommended adolescent vaccines. Participating FQHCs were based on a convenience sample. Data on vaccine uptake came from the state immunization registry.	I: Vaccination coordinators attended a webinar that reviewed provider-based changes recommended by the CDC's AFIX program and received weekly follow-up emails. C: Pre-AFIX webinar conditions	7,337 12–17 year old females: I: 3,673 C: 3,664 17 FQHCs in North Carolina	Series initiation (I vs. C, p = .029): I: 54.0% C: 52.4% Dose 2 (I vs. C, p = .001): I: 36.1% C: 35.0% Series completion (I vs. C, p = .001): I: 22.0% C: 21.0%	—
Perkins R, et al. 2015. Effectiveness of provider-focused intervention to improve HPV vaccination rates in boys and girls.^51	O: Series initiation and receipt of next dose during the intervention period and after 6 months M: Randomized controlled trial. Rates were compared at baseline and two follow-up periods in the intervention and control health centers.	I: Clinics received provider-focused intervention consisting of: (1) repeated contacts (i.e. meetings every 4–6 weeks during the project period) to establish trust and accountability and to support providers to make practice-wide changes, (2) focused education on the morbidity and mortality from HPV, vaccine safety, and vaccine efficacy, (3) individualized feedback on vaccination rates relative to the practice-wide, state, and national rates, and (4) incentives in the form of maintenance of board certification requirements. Intervention practices then came up with their own intervention plans, which included utilizing reminder/recall, strongly recommending the vaccine, standing orders, provider reminders, and school based health centers. C: Standard of care	13,118 11–21 year old males and females: I: 4,093 (1,749 females, 2,344 males) C: 9,025 (4,037 females, 4,988 males) 8 FQHCs: I: 2 FQHCs C: 6 FQHCs	—	Series initiation during intervention, females (I vs. C, p < .001): OR 1.6 (not sustained at 6 months post-intervention) Series initiation during intervention, males (I vs. C, p < .001): OR 11 (sustained at 6 months post-intervention) Next dose completed, females (I vs. C, p < .05): OR 1.4 (sustained at 6 months post-intervention) Next dose completed, males (I vs. C, p < .05): OR 23 (sustained at 6 months post-intervention)
Pahud B, et al. 2015. A pilot program to improve vaccination status for hospitalized children.^52	O: Series initiation after 1 month M: Observational study. Randomly selected hospitalized children/adolescents were enrolled in the study. Immunization records of participants were assessed to determine vaccination status at baseline and whether they needed to be caught up on the schedule.	I: For those determined to be not up-to-date, a personalized vaccine plan was created and shared with the parents/guardians of the child. If parents requested the vaccine dose while the child was hospitalized, the care team delivered it. If not, the doses needed were sent to the child's PCP and also sent home with the family.	72 11+ year old males and females: I: 34 1 children's hospital in Kansas City, MO	Series initiation: I: 20.6%	—
Chung R, et al. 2015. Keen on teen vaccines: Improvement of adolescent vaccine coverage in rural North Carolina.^53	O: Series initiation and completion after one year M: Prospective cohort study. All practices within the intervention county that provided immunizations to adolescents and that were using the North Carolina Immunization Registry (NCIR) were eligible for inclusion. Intervention conditions were introduced in two phases and outcomes were compared to 4 matched comparison counties.	I: Multicomponent intervention consisted of provider education, immunization registry based mailed reminder and recall, patient education, provider incentives, and school-based parent education. C: 4 comparison counties	11–18 year old adolescents: Sample size not reported 5 counties in North Carolina: I: 7 practices within one county C: 4 non-intervention counties.	Overall series initiation, 11–12 year old females Range of percentage increases: 27.4%–43.4% Overall series initiation, 11–12 year old males: Range of percentage increases: 14.2%–32.1% Overall series initiation, 13–18 year old females and males: Data not presented Overall series completion, 13–18 year old females: Data not presented Overall series completion, 13–18 year old males: Range of percentage increases: 1.6%–4.2%	Overall series initiation, 11–12 year old females (I vs. C, p = significant, not reported): AOR 1.19 (1.02–1.40) Overall series initiation, 11–12 year old males (I vs. C, p = significant, not reported): AOR 1.23 (1.02–1.51) Overall series initiation, 13–18 year old females and males: Data not presented, not significant Overall series completion, 13–18 year old females: Data not presented, not significant Overall series completion, 13–18 year old males (I vs. C, p = significant, not reported): AOR 1.46 (1.07–1.99)
Staras S, et al. 2015. Increasing human papillomavirus vaccine initiation among publicly insured Florida adolescents.^54	O: Series initiation after 3 months. M: Quasi-experimental. Two interventions were tested: a gender-specific postcard campaign was developed to address the gender diversity in vaccine series initiation and differential parental concerns, and an in-clinic health information technology system designed to assess interest in learning about the vaccine. Adolescents were randomly assigned to one of three intervention conditions, or standard of care.	I1: Received postcards sent in August and October. I2: Used health information technology (HIT) system to verify vaccination history and indicate interest in learning about vaccine. HIT system summarized responses for providers. I3: Received postcard and HIT system. C1, C2, C3: Standard of care.	14,148 11–17 year old adolescent males and females: I1: 2,839 I2: 1,774 I3: 886 C1: 2,824 C2: 3,889 C3: 1,936 Primary care clinics serving Medicaid and Children's Health Insurance Program beneficiaries in Florida	Series initiation, females: I1: 5.5% I2: 6.0% I3: 7.5% C1: 3.6% C2: 4.0% C3: 3.1% Series initiation, males: I1: 5.7% I2: 7.0% I3: 7.2% C1: 5.4% C2: 4.8% C3: 4.7% Overall, 288 of 5,663 in the intervention group (5%) initiated the HPV vaccine series.	Series initiation, females (I1 vs. C, p = significant, not reported; I2 vs. C, p = not significant, not reported; I3 vs. C, p = significant, not reported): I1: AOR 1.6 (1.1–2.4) I2: AOR 1.3 (0.9–2.0) I3: AOR 2.0 (1.1–3.7) Series initiation, males (I1 vs. C, p = not significant, not reported; I2 vs. C, p = significant, not reported; I3 vs. C, p = significant, not reported): I1: AOR 1.1 (0.8–1.5) I2: AOR 1.4 (1.0–2.0) I3: AOR 1.6 (1.0–2.5)
Bundy D, et al. 2013. The ImmProve Project: leveraging electronic health record data to promote immunization delivery.^55	O: Up-to-date status for HPV series and time to receipt of next dose after 9 months M: Interrupted time series cohort study for intervention 1, and randomized controlled trial for intervention 2. All enrolled providers received education and training on the CDC schedule and how to utilize the intervention materials. For intervention 1, all patients in this cohort received the intervention condition; for intervention 2, providers were randomized to control condition or intervention.	I1: Electronic health record derived immunization prompts, in the form of clinical decision support when overdue children presented for care I2: Quarterly provider-specific bulletins listing overdue patients C1: Historical control for comparison with I1 C2: Standard of care for comparison with I2, patients were exposed to I1	12.5–14.5 year old females: Sample size not reported 1 hospital-based pediatric primary care clinic	—	Up-to-date status (I1 vs. C1): No significant increase (estimates not reported) Time to receipt of next dose (I2 vs. C2, p = not reported): HR 1.27 (0.91–1.77)
Enhancing Access to HPV Vaccination Services
Vaccination programs in schools
Daley M, et al. 2014. School-located vaccination of adolescents with insurance billing: cost, reimbursement, and vaccination outcomes.^56	O: Series initiation after one school year M: Randomized controlled trial. Students at schools that hosted school-located vaccination clinics that billed health insurance were randomly sampled. Randomly sampled students at randomly selected schools provided a comparison group. Immunization records for the samples were thoroughly updated using school, parent, and registry records.	I: Schools hosted 3 days of vaccination clinics, timed by the dosing schedule of HPV vaccine. Clinics billed insurance for vaccination services provided. C: No intervention	2,000 6–8^th grade females: I: 1,000 (n = 600 6^th graders, n = 400 7^th and 8^th graders) C: 1,000 14 Denver, Colorado non-charter public schools: I: 7 schools C: 7 schools	Series initiation, 6^th graders (p = not reported): I: 34% C: 18% Series initiation, 7^th and 8^th graders (p = not reported): I: 20% C: 7% Series completion, 6^th graders (I vs. C, p < .001): I: 13.2% C: 2.0% Series completion, 7^th and 8^th graders (I vs. C, p = not significant): I: 9.3% C: 5.8%	Series initiation, 6^th graders (p = significant, not reported): ARR 1.69 (1.21, 2.36) Series initiation, 7^th and 8^th graders (p = significant, not reported): ARR: 2.56 (1.34, 4.88)
Stubbs B, et al. 2014. Evaluation of an intervention providing HPV vaccine in schools.^57	O: Number of doses received and site at which doses were received after 9 months M: Prospective cohort study. School officials identified 6 regions in the county, and for each they selected a middle school to host clinics that provided HPV vaccine. Other schools in the county served as satellite schools whose students could receive HPV vaccine at the host school clinics in their region.	I1: Host schools had several one-day, on-site HPV vaccination clinics for students in partnership with local health department I2: Satellite school students had access to host school clinics in their region	7,916 10–17 year old females: I1: 1,781 I2: 6,135 28 schools in Guilford County, North Carolina: I1: 6 schools I2: 22 schools	Number of doses received at any location: No doses: 97.9% One dose: 0.2% Two doses: 0.3% Series completion: 1.7% (high completion if initiated: 80% of those who initiated received all 3 doses) Any dose received, by location: I1: 6.2% I2: 1.0%	Any dose received, by location (I1 vs. I2, p < .05): OR 6.56
Reducing out-of-pocket costs
Harper D, et al. 2014. The influence of free quadrivalent human papillomavirus vaccine (HPV4) on the timely completion of the three dose series.^58	O: Effect of economic incentive (free vaccine) on number of doses received and on-time series completion. M: Nested retrospective cohort study. The medical center's vaccine program distributed free HPV vaccine provided by a local foundation for females who otherwise had no source of payment for the vaccine. Patient characteristics, vaccination history, and payer source for each of the three HPV doses were abstracted from electronic records.	I: Exclusively received free vaccine doses sponsored by foundation C1: Exclusively received vaccine doses paid for by public insurance C2: Exclusively received vaccine doses paid for by private insurance	494 10–26 year old females: 136 10–18 year olds 358 19–26 year olds I: 47 C1: 447 C2: 447 1 safety net health care system in Kansas City, Missouri	On-time series completion, 10–18 year olds: I: 0% C1: 14% C2: 32% On-time series completion, 19–26 year olds: I: 21% C1: 12% C2: 32% Adolescents (10–18) did not receive sufficient numbers of grant-sponsored doses to conclude if free HPV4 affected on-time completion rates. No significant differences were found in adult (19–26) on-time completion rates among adults receiving 3 doses by payer source of the vaccine.	Adolescents (10–18) did not receive sufficient numbers of grant-sponsored doses to conclude if free HPV4 affected on-time completion rates. On-time series completion with at least one dose paid for by grant sponsorship, 19–26 year olds: OR 1.56, p = not significant

Eight studies examined patient reminder and recall systems to increase community demand for HPV vaccination.^28-35 Patient reminder and recall systems utilized many methods of reminding or recalling patients; one study used letters,²⁸ one used letters or phone calls,³² two used text messages,^29,35 two provided multiple types of reminders,^31,33 and two allowed a choice (such as telephone call, letter, text message, Facebook message, or email).^30,34 The eight studies addressed a variety of measures, including series initiation, series completion, and receipt of next vaccine dose. The mailed letter intervention found significantly greater HPV series completion in the entire study age range intervention group (percentage point difference: 9.8, p < .01).²⁸ Another study comparing a mailed letter intervention to a telephone reminder intervention found significantly greater dose 2 and series completion in both intervention groups; the mailed letter intervention resulted in an 8 percentage point increase in dose 2 coverage (HR 1.5, p < .05), and the telephone reminder intervention resulted in an 8 percentage point increase in dose 2 (HR 1.6, p < .01) and a 5 percentage point increase in series completion (HR 1.5, p < .05).³² Not all primary measures were found to be significant in this study, however. Series initiation did not differ significantly between intervention and control groups.³²

The studies assessing text message reminders showed increases in coverage, with significantly greater on-time receipt of next HPV vaccine dose in an intervention group against two different control groups (adjusted odds ratio [AOR] 2.03 and AOR 1.83, p = .002 and .003) in one study²⁹ and small differences in series initiation between an intervention and control group (percentage point difference: 3.0; HR 1.3, p = .04) in another study.³⁵ Studies including several types of reminder methods also demonstrated increases in coverage. One used a messaging cascade of multiple types of reminders and found that 22.9% of due or overdue patients received their next dose of HPV, and also showed that this cascade method was most effective at encouraging series completion (p < .0001).³³ Another intervention implemented more than one reminder method by using mailed letters and telephone reminders, and found significantly greater HPV series initiation and completion from zero baseline doses in the intervention group (percentage point difference in series initiation: 11.2, p < .05; percentage point difference in completion from zero baseline doses: 7.3, p < .05).³¹ However, up-to-date series completion from ≥1 baseline doses did not differ significantly between the intervention and control group.³¹ Studies allowing patient choice of reminder method had mixed results, with one intervention not showing any significant difference in series completion,³⁰ while another demonstrated a range of significant percent increases in initiation and completion (e-mail, mail, text, and telephone call, all p < .05).³⁴

Two randomized controlled trials in females aged 18–26 years utilized education using video technology to deliver messages about HPV vaccination.^36,37 One study assessed the effectiveness of a variety of video narrators on series initiation, and found significant greater series initiation in the intervention group that had both a provider expert and peer narrator in the video (percentage point difference: 11, p = .035; odds ratio [OR] 2.07, p = .036).³⁶ The other study looked at how a video-based intervention impacted series completion, and found significantly greater series completion for the group that viewed a short theory-driven video presentation (percentage point difference: 11.5, p = .03; AOR 2.44, p = .001).³⁷

The final six “increasing community demand” interventions utilized multiple intervention strategies in combination with each other.^38-43 One prospective cohort intervention implemented reminder and recall in a school-located vaccination program.³⁸ The results demonstrated 59% of participants getting any dose of HPV vaccine, but only 25% received dose 2 and 0% completed the full series.³⁸ Two other interventions aimed to increase HPV vaccination among adolescent females and males respectively, utilizing broad, multi-faceted public awareness campaigns targeted to each audience in conjunction with education for healthcare providers.^40,41 One of these interventions targeted females using a social marketing campaign, and though significant differences were seen between all intervention counties and each control group (all, p < .01), the 2 counties with higher initiation were only between 1.6 and 2.0 percentage points greater than the controls, and 2 counties were found to have significantly lower initiation rates.⁴⁰ The other social marketing campaign targeted adolescent males, and while those in the intervention group were more likely to initiate the HPV vaccination series (percentage point difference: 2.1; HR 1.34, p = .002), this was again a small increase, and results were not sustained at a 6 month follow up assessment.⁴¹

Another intervention using multiple methods utilized immunization navigators trained to track immunization records, lead reminder and recall efforts, and conduct home visits.⁴² This intervention resulted in significant positive changes from baseline in series initiation (adjusted risk ratio [ARR] 1.4, p < .001), dose 2 receipt (ARR 1.4, p < .001), and series completion (ARR 1.5, p < .001).⁴² The final two studies in this category assessed educational vaccination messaging in a school-located clinic⁴³ and a reminder and recall combined with patient education intervention,³⁹ but found no significant effects on measures of HPV vaccination coverage.^39,43

Provider or healthcare system based interventions to increase HPV vaccination

Twelve studies (35.3%) were identified as provider or healthcare system-based interventions.^44-55 Three studies assessed single method interventions targeted at providers: provider assessment and feedback⁴⁶ and provider prompts.^44,45 The intervention assessing provider assessment and feedback utilized CDC's AFIX (Assessment, Feedback, Incentives, and eXchange) program and targeted it specifically to HPV vaccination.⁴⁶ The randomized controlled trial had two intervention groups—one in-person, one web-based—and both found small but significant increases in coverage among 11–12 year olds at 5 months post-intervention (1.5% increase in initiation with in-person AFIX (p = .02) and 1.9% increase with web-based AFIX (p < .01)), but not for 13–18 year olds at 5 months, and results were not sustained for any group at 1 year post-intervention.⁴⁶ Another study implementing electronic health record-generated provider prompts found significantly greater series initiation in the intervention group (percentage point difference: 13.7, p < .001) and significantly greater timely series completion (point estimates not presented; p < .001).⁴⁴ However, another randomized controlled trial assessing the use of provider prompts via electronic health record or nurses/staff found no significant effect on series initiation, dose 2, or series completion, despite strong acceptance of the intervention among providers.⁴⁵

The majority of provider-targeted interventions utilized multiple intervention methods in combination with each other.^47-55 Many assessed interventions in more than one primary care practice,^48,53 in federally qualified health centers (FQHC),^50,51 or in large hospital clinic settings.^52,55 Three were focused primarily at the provider level.^50,52,55 One intervention presented webinars for FQHC providers based on the CDC's AFIX program principles, with follow-up weekly educational emails for the providers.⁵⁰ The results found significant, but small increases of 1 to 1.6 percentage points in HPV series initiation (p = .029), dose 2 (p = .001), and series completion (p = .001).⁵⁰ Another intervention created individualized vaccine catch-up schedules for hospitalized children not up-to-date for HPV vaccination, and increased series initiation in this group by 20.6% (p = not reported),⁵² while another using provider prompts, clinical decision support, and individualized provider feedback found no significant effect of the intervention on up-to-date status of the participants or timeliness of vaccine receipt.⁵⁵

The remaining multiple method provider-based interventions implemented complex interventions targeting both provider behavior and community demand.^{47-49,51,53,54} The first examined the effects of a family-focused intervention consisting of patient education and reminder and recall, and a provider intervention consisting of provider education, assessment and feedback, and provider reminders.⁴⁸ Both intervention strategies implemented together found the largest increases in HPV vaccination coverage across all measures, compared to when each strategy was implemented separately (series initiation: HR 1.6, p = .001; dose 2: HR 1.3, p = .008; series completion: HR 1.5, p < .001).⁴⁸ Two other trials first implemented provider education, followed by supplemental interventions such as school-based telephone reminder and recall⁵³ or targeted interventions for each individual practice based on mutually identified barriers to HPV vaccination.⁵¹ The intervention outcomes for both studies showed increases in coverage, indicating that series initiation was significantly more likely for females (adjusted odds ratio [AOR] 1.19, p = significant, but not reported;⁵³ OR 1.6, p < .001⁵¹) and males (AOR 1.23, p = significant, but not reported;⁵³ OR 11, p < .001⁵¹), and next dose receipt was also significantly more likely for females (OR 1.4, p < .05)⁵¹ and males (OR 23, p < .05).⁵¹ Many of the increases persisted for outcomes measured 6 months post-intervention, except the series initiation results for females.⁵¹ Another study demonstrated that patient reminder and recall combined with an in-clinic education and health information system summarizing patient intention data for providers resulted overall in 5% increases in HPV vaccination series initiation, with slightly greater increases for boys (p = significant, but not reported).⁵⁴

The two studies that observed the largest percentage point increases in HPV vaccination coverage in any of the intervention studies reviewed also had the smallest sample sizes, which is important to note.^47,49 These small observational studies took place in single pediatric practice settings and utilized a combination of patient reminder and recall, patient educational materials, provider education, and provider protocols. Large, significant differences were seen in outcomes between intervention and control groups in both studies.^47,49 One study demonstrated that series initiation was greater by 50.9 percentage points (p = .001; OR 9.43, p = significant, but not reported) and series completion was greater by 55.6 percentage points (p < .001; OR 22.5, p = significant, but not reported).⁴⁷ The other study showed that dose 2 completion was greater by 40 percentage points compared to a non-enrolled control group (p < .001) and 46 percentage points compared to a non-intervention group (p < .001), and series completion was greater by 16 percentage points compared to a non-enrolled group (p = .008) and 11 percentage points compared a non-intervention group (p = .018).⁴⁹

Interventions to enhance access to HPV vaccination services

Three intervention studies (8.8%) were designed to primarily enhance access to HPV vaccination services.^56-58 Two of these studies aimed to increase access by implementing vaccination programs located in schools.^56,57 The first study found significant increases in initiation in both a 6th grade (percentage point difference: 16; ARR 1.69, p = significant, but not reported) and 7–8^th grade population (percentage point difference: 13; ARR 2.56, p = significant, but not reported).⁵⁶ The results of the second study found minimal overall participation in the intervention, with only 2% of the population receiving any dose, though there was high series completion (80%) among those who initiated the series.⁵⁷ Students at host-school sites compared to satellite sites were also significantly more likely to have received any dose of the HPV vaccine (OR 6.56, p < .05).⁵⁷

The final study provided grants to reduce out-of-pocket costs to patients associated with receiving the HPV vaccine, but results did not indicate significant differences in number of doses received or on-time completion between those who received grant sponsored doses and those who did not.⁵⁸

Results from economic analyses

Ten studies assessed the economic effects of implementing HPV interventions.^{31-34,38,42,46,48,56,57} Table 4 presents summaries of the economic findings of these studies. Many of these studies implemented reminder and recall interventions.^31-34,38,42 One study found that the overall operating costs for implementing mail and telephone reminder and recall ranged from $1,087 to $1,349, with three of the practices demonstrating positive net additional revenues after the study period and one practice demonstrating a net revenue loss, calculated by subtracting total operating costs from total additional revenues for each practice.³¹ Another mail or telephone reminder and recall intervention demonstrated average costs of $18.78 for mail and/or $16.68 for telephone calls per child per year, with the cost per additional child fully vaccinated at $324.75 for the mail group and $487.03 for the telephone group.³² Additional reminder and recall interventions ranged from between $1.25 and $1.50 for each mailed postcard and < $0.10–$0.80 for each text message/e-mail/phone message.^33,34 Reminder and recall costs in a school-based health center setting ranged from $1.12 to $6.87 per child immunized.³⁸ The cost of a full “immunization navigator” program including reminder and recall was found to be $45.74 per child per year, with the cost per additional child fully vaccinated at $465.⁴²

Table 4. Selected results from economic analyses (n = 10).

Author	Intervention codes	Operating costs/costs per site	Cost per unit
Suh^31	Patient reminder and recall systems	Ranged from $1,087 to $1,349 per practice	—
Szilagyi 2013^32	Patient reminder and recall systems	—	$18.78 per child per year for mailed reminders; $16.68 per child per year for telephone reminders; $463.99 per additional child fully vaccinated for mailed reminders; $714.98 per additional child fully vaccinated for telephone reminders
Bar-Shain^33	Patient reminder and recall systems	Messaging cost was less than 1% of gross revenue for visits where needed doses were given	<$0.10 per automated message for e-mail, text, and phone messages; $1.50 per postcard; $1.77 per dose delivered
Morris^34	Patient reminder and recall systems	—	$1.25 for each postcard sent; $0.80 for each text message and e-mail sent
Kempe^38	Community based intervention implemented in combination	—	Ranged from $1.12 to $6.87 per recalled child immunized over four schools
Szilagyi 2011^42	Community based intervention implemented in combination	—	$45.74 per child per year; $465 per additional child fully vaccinated
Gilkey^46	Provider assessment and feedback	$152 per clinic for in-person consultations $100 per clinic for webinar consultations	—
Fiks^48	Healthcare-systems based intervention implemented in combination	—	Per additional dose delivered: $24.00 for dose 1, $42.00 for dose 2, and $189.00 for dose 3 for combined family/provider intervention
Daley^56	Vaccination programs located in schools	—	$23.98 per vaccine administered (does not include cost to purchase vaccine)
Stubbs^57	Vaccination programs located in schools	$135,397.44 for total program	—

Costs of other interventions implemented in school-located clinics and with providers/healthcare systems were also evaluated.^46,48,56,57 One school-located vaccination clinic found a cost of $23.98 per vaccine administered, not counting the cost to purchase the vaccine.⁵⁶ The other school-located vaccination clinic intervention used only 36% of their original budget, for a total operating cost of $135,397.44.⁵⁷ The largest percentage of the expenses went to personnel costs (67%).⁵⁷ In a study focusing on provider and family interventions, the cost per additional dose delivered for the most effective intervention for increasing series initiation (provider-focused) was $6; for increasing dose 2 and series completion, the most effective intervention (family-focused) was $10 for dose 2 and $6 for dose 3 per additional dose delivered; and for the most effective intervention overall (combined provider and family-focused), the cost was $24 for dose 1, $42 for dose 2, and $189 for dose 3.⁴⁸ Finally, AFIX webinars for providers showed promise in reducing travel costs; the cost of delivering the intervention was $152 per clinic for in-person consultations, and $100 per clinic for webinar consultations.⁴⁶

Discussion

Of the 34 HPV vaccination intervention studies identified, the majority of the studies were designed to increase community demand for HPV vaccination, but a substantial number of studies also targeted providers and healthcare systems. Only a few studies reviewed addressed access to vaccination services. Overall, while most of the intervention methods identified in the literature were based on evidence-based vaccination intervention categories recommended by the Community Guide,²³ there was considerable variation in the efficacy of these interventions in the context of HPV vaccination. A substantial number of studies did show significant increases in HPV vaccination coverage following the intervention. The only Community Guide category of intervention that we reviewed that is not currently recommended by the Community Guide is patient education when used alone, due to insufficient evidence.²³ The studies reviewed here should be examined both in context with their methods and with overall Healthy People 2020 targets, which will require large increases in coverage, to understand the potential impact that these reviewed interventions may have on HPV vaccination coverage.

The single-method intervention strategies that frequently produced statistically significant increases in HPV vaccination coverage were reminder and recall^28-35 and patient education.^36,37 Reminder and recall interventions were the most common type of intervention reviewed. Reminder and recall intervention methods ranged from improving series initiation with a reminder message that a child is due to start the HPV vaccine series, to recall efforts to get children overdue for dose 2 or 3 to get caught up and complete the series. Regardless of type of reminder or recall utilized (text, mail, phone, e-mail, etc.), many interventions resulted in a range of increases in a variety of measures of HPV vaccination coverage,^28,29,31-35 though not all resulted in increases.³⁰ The Community Guide does not recommend patient education when used alone as a strategy to increase vaccination coverage due to insufficient evidence;²³ however, the two patient education studies reviewed here demonstrate some potential promise of this type of intervention as it relates to HPV vaccination.^36,37 The interventions solely implementing patient education both used video as a means of delivering information about the HPV vaccine, but these interventions were directed at young adults aged 18–26 years, not the target age group for routine vaccination in the United States. Both trials demonstrated significant differences in coverage compared to non-intervention groups, though their sample sizes were moderate.^36,37

Provider assessment and feedback, when used alone, produced significant but small increases in HPV vaccination initiation, though these changes were not sustained one year after the intervention period.⁴⁶ Again, in considering the findings of these studies, it is important to keep in context that large increases in coverage are needed to reach the Healthy People 2020 targets.

Numerous effective multi-component interventions were implemented in the community^38-43 or in healthcare systems.^47-55 Methods varied across studies, though the most common strategy included in the multi-component interventions was some type of reminder and recall system. These efforts were frequently accompanied by education of parents and/or providers, enhanced practice-based IT systems, and/or provider incentives. The studies took place in a variety of settings and there was a wide range in the number of participants. Many of the largest increases in coverage were seen as a result of these multiple strategy interventions, particularly those that involved a provider-targeted strategy. While not all of the multi-component interventions found large or significant increases, many of the studies reviewed demonstrated that implementing multiple strategies to address barriers to vaccination can be effective in increasing HPV vaccination coverage.

The intervention methods that had inconsistent support for their effectiveness or did not find significant findings were school-located vaccination services, provider reminders used alone, vaccination requirements for school attendance, and programs to reduce out of pocket costs. Vaccination programs in schools and provider reminders used alone are both evidence-based strategies cited by the Community Guide's recommendations,²³ but results presented in the studies focused on HPV vaccination were inconsistent.^44,45,56,57 Both categories had one study that had significant results^44,56 and one that did not have significant results.^45,57 Vaccination requirements for school attendance have been shown to be effective in increasing vaccination coverage for other vaccines,²³ but no difference was seen in series initiation with a middle school entry requirement for HPV vaccination.²⁵ It is important to note that requirements for HPV vaccination for school attendance have broad opt-out provisions, with waivers available for religious, medical, and/or philosophical objections. There did appear to be small spill-over effects to HPV vaccination from other adolescent vaccination requirements,^26,27 though small spill-over effects may not help HPV vaccination coverage increase by the amount needed to reach the Healthy People 2020 target of 80% coverage. Programs to reduce out of pocket costs similarly are cited to increase vaccination per the Community Guide's recommendations,²³ but for the study examining HPV vaccination specifically, the strategy did not significantly influence coverage.⁵⁸ For those strategies with inconsistent evidence, more research may be needed to better understand the influence of those strategies on HPV vaccination coverage. For those strategies that did not demonstrate changes in coverage, many potential barriers to HPV vaccination have been identified in the literature,^20,21 and it may be that these particular intervention strategies do not address an adequate number of barriers to see a consistent change in vaccination delivery. Beyond addressing an adequate number of barriers, additional baseline differences in study conditions such as study design or community context could have affected study outcomes.

Some interventions by design may be more resource-intensive than others, but many studies' economic analyses demonstrated that interventions could be implemented with modest cost.^{31-34,38,42,46,48,56,57,59} Due to variations in measures of cost, it was difficult to directly compare economic measures across the studies. However, the data from the studies reviewed generally found that implementing an HPV vaccination intervention is not cost-prohibitive.

Though HPV vaccines have been available for more than 9 years, HPV vaccination intervention research specifically focused on increasing coverage is just emerging. The studies reviewed here indicate that many types of interventions may be promising. With this knowledge in mind, an important next step for researchers and practitioners to consider is how to translate evidence into practice. Many of the interventions reviewed here are targeted to specific populations, so additional efforts are needed to study how intervention models can be implemented on a wide scale and adapted to different settings. Additional research is needed to examine how strategies can be customized and commonly integrated into practice.

In addition to this, there is also a need to examine more closely the effects of interventions to improve provider communication and recommendations for vaccination. The literature shows that though provider recommendations for HPV vaccination are highly correlated with higher coverage, many providers do not routinely recommend HPV vaccination.^60-62 Though this fact is demonstrated strongly in the literature, few of the studies reviewed in this paper aimed to strengthen the provider recommendation through interventions. As this is one of the most highly correlated factors in the literature with higher HPV vaccination coverage, this suggests that there is a need for more intervention research in this area.

Another important takeaway from this review is the need for continued aggregation of evidence resulting from vaccination intervention research, especially as it relates to specific vaccinations. Given that barriers to HPV vaccination are often unique and increasing coverage may require different approaches, intervention research specific to HPV vaccination must be reviewed in context of other vaccination intervention research. Having more specificity in recommendations from the Community Guide regarding individual vaccinations and populations would be helpful. As more HPV vaccination intervention research continues to be published, reviews such as these should continue to be updated to highlight trends of what is working in the field.

Of the studies identified for inclusion in this review, there was a high amount of variability between intervention strategies, populations, and the measures used. This review demonstrates a clear need for more studies with population level samples, comparison groups, and randomized designs to help indicate which interventions may be most effective in increasing coverage. There is also a strong need to implement and evaluate HPV vaccination interventions that include coverage of males as an outcome. Finally, as several of the studies reviewed indicated that strategies implemented in combination show promise in their ability to increase vaccination coverage, particularly when the intervention includes a provider-focused element, more research should be conducted to assess these types of multi-component interventions.

One major limitation of this systematic review was that many studies had limited generalizability in their results, as many had small sample sizes or were observational studies. Without a comparison group in several of the studies, it is difficult to state conclusively whether the intervention was truly responsible for changes seen. Additionally, each study measured HPV vaccine uptake differently. This made it difficult to evaluate and compare effectiveness across studies, and precluded meta-analysis. It is important, when evaluating the impact of each intervention, to consult the original studies for a deeper understanding of study design and measures. Another limitation is that the ACIP recommendations only extended to include boys in 2011, which is a likely factor in the limited number of interventions found in the literature that targeted boys. This dearth in the literature impacts the knowledge about interventions to increase HPV vaccination coverage in boys. A final limitation is that interventions conducted outside of the United States were not included in this review. The decision to exclude studies conducted in international settings was based on differences in healthcare systems and school-based vaccination policies compared to the United States, but successful interventions could have been evaluated and then assessed for feasibility or adaption to United States settings. This decision could also be seen as a strength of this review, however, as differences in context may render comparisons invalid.

Conclusion

This review presented a thorough examination of the literature from licensure of the first HPV vaccine to the present, making information on HPV vaccination interventions easily accessible for practitioners working on this priority public health issue. Variable study populations and outcome measures precluded meta-analysis, but the literature overall indicates that there are several strategies that may be promising. While single strategies can be effective, interventions may be most successful when implemented in combination with each other to address multiple barriers at community and provider levels.

Most importantly, because the evidence suggests that several of the interventions reviewed here may be effective, more research is needed to translate the evidence into comprehensive strategies that can be implemented in a variety of settings. Additional efforts are needed to study how strategies can be adapted and integrated into practice on a wide scale.

Methods

Search strategy and selection criteria

Two authors searched PubMed, Web of Science, Wiley Online Library, Cumulative Index to Nursing and Allied Health Literature, and Google Scholar databases to capture a comprehensive list of peer-reviewed studies published between June 2006 and May 2015. Relevant MeSH (Medical Subject Headings) search term keywords were entered in each database (HPV, human papillomavirus, adolescent, vaccine or vaccination, immunization, intervention, coverage), and the search was limited to English language and peer-reviewed literature.

Studies were eligible for inclusion if they fulfilled the following criteria: interventions were designed to measure increases in HPV vaccination coverage as an outcome, provided quantitative data for HPV vaccination coverage specifically, and were conducted in the United States. Both randomized controlled trials and observational studies were eligible. Intervention studies that only provided qualitative results, were only published as conference abstracts, or only assessed HPV knowledge, attitudes, or intention to get vaccinated, were excluded. After review of article titles and abstracts, relevant full-text articles were obtained and independently reviewed by two authors to verify that they met the criteria. Finally, references from systematic reviews^17-22 as well as references from the studies identified were also assessed for inclusion in the review. Discrepancies related to inclusion criteria were resolved through discussions among all three authors.

Data extraction

Following the final round of selection of studies, two authors independently extracted information about intervention methods, study design, outcomes, participants, and results. Though outcome measures varied largely across studies, the principal summary measures extracted were percentage changes in coverage, risk ratios, hazard ratios, and/or odds ratios pertaining to receipt of dose 1, 2, and/or 3 of the series, up-to-date status, and timeliness of receipt. If study results were presented as percentage differences between intervention and control groups, one author calculated percentage point changes between groups to provide a common metric for comparison. If studies examined any economic dimensions of intervention implementation, this information was also extracted. The independent abstractions were compared and discrepancies were resolved through discussion.

Based on a review of each study's intervention methods, each study was classified in an intervention category to group similar types of interventions together for in-depth review. Intervention categories were adapted from categories used by the Community Guide's section on Increasing Appropriate Vaccination and were adapted to specifically include only HPV vaccination.²³ The broad categories of interventions identified in the literature were “increasing community demand for HPV vaccination,” “provider- or system-based interventions to increase HPV vaccination,” and “enhancing access to HPV vaccination services.” Interventions were further classified within each category by intervention method implemented. Definitions for methodological categories can be found in Table 1. It is important to note that though the classification of interventions in this review is based on the Community Guide's categorizations, this review is not the same as the reviews conducted by the Community Guide. The review design did not allow for abstracting data in the same detail as the Community Guide, there was no quality of evidence analysis to put forth a recommendation or non-recommendation, and no separate economic evaluation of interventions beyond what intervention study authors put forth themselves was conducted. Additionally, a risk of bias assessment was not performed, as that was beyond the scope of this review.

Abbreviations

CDC	=	Centers for Disease Control and Prevention
HPV	=	human papillomavirus
4vHPV	=	quadrivalent HPV vaccine
2vHPV	=	bivalent HPV vaccine
9vHPV	=	9-valent HPV vaccine
FDA	=	Food and Drug Administration
ACIP	=	Advisory Committee on Immunization Practices
MeSH	=	Medical Subject Headings
Tdap	=	tetanus-diphtheria-acellular-pertussis
HR	=	hazard ratio
OR	=	odds ratio
AOR	=	adjusted odds ratio
RR	=	risk ratio
ARR	=	adjusted risk ratio
AFIX	=	assessment, feedback, incentives, and eXchange
FQHC	=	federally qualified health centers

Disclosure of potential conflicts of interest

No potential conflicts of interest were disclosed.

Acknowledgments

This publication was supported by Cooperative Agreement Number 3U36OE000002 from the Centers for Disease Control and Prevention and the Association of Schools and Programs of Public Health. The findings and conclusions of this publication do not necessarily represent the official views of CDC or ASPPH. This publication was also supported by 2013 Prevention and Public Health Funds, Immunization Program Technical and Analytical Assistance in Support of HPV Vaccination, Contract #200-2009-28537 Task Order-091.