Multi-level quality improvement strategies to optimize HPV vaccination starting at the 9-year well child visit: Success stories from two private pediatric clinics

ABSTRACT

HPV vaccination rates remain far below goal, leaving many adolescents unprotected against future HPV-related cancers. Starting HPV vaccine at age 9 may improve timely preteen vaccination. The “HPV Vax at 9” Quality Improvement intervention paired HPV vaccination with 9- and 10-year well child visits and was piloted at two pediatric clinics (n = 9 sites) in Washington between 2018 and 2022. Supporting interventions included standardized immunization schedule posters in exam rooms, electronic medical record supports, provider and staff training, strong provider recommendations, printed educational resources, and peer-to-peer champion coaching. Provider and clinic acceptance was high with HPV vaccine administration occurring at 68–86% of the 9- and 10-year well child visits. During the first year, HPV initiation rates at age 9–10 increased by 30 percentage points or more at each clinic. Sustained improvements in initiation and series completion were seen with completion at age 11–12 rising as much as 40 percentage points from 22 to 62%. Downward pressure of the COVID-19 pandemic on HPV vaccination rates was mitigated. Pairing HPV vaccine with 9- and 10-year well child visits, posting the standardized immunization schedule, and instituting EMR supports for HPV at 9 may be effective and sustainable strategies to simplify clinic workflows and increase timely HPV vaccination.

KEYWORDS:

• Quality improvement
• HPV vaccination at age 9
• immunization schedule
• well child visit
• COVID-19

Introduction

Human papillomavirus (HPV) vaccination of preteens can prevent 90% of HPV-related cancers (oropharyngeal, cervical, anal, penile, vaginal, vulvar).^1,2 However, HPV vaccination lags in comparison to other adolescent vaccines, remaining below the Healthy People 2030 goal of 80%.³ The 2021 NIS-Teen Survey indicates 61.7% of 13–17 year olds in the United States have completed the HPV series, compared to 89.6% for tetanus, diphtheria, and acellular pertussis (Tdap) and 89.0% for first dose meningococcal conjugate (MenACWY).⁴ Unfortunately, during the COVID-19 pandemic, administration of adolescent vaccines fell between 20% and 24% in 2020 and modeling predicts 2 to 10 years for recovery.⁵ Medical providers report ongoing challenges related to the pandemic including staffing challenges and the redirection of limited resources.^6,7 Simple and effective strategies are critical to raising vaccination rates.

Routine recommendation at age 9 is a promising best practice to increase HPV vaccination rates, endorsed by the American Academy of Pediatrics, American Cancer Society (ACS), and the National HPV Vaccination Roundtable.^8–10 Notably, HPV vaccination starting at age 9 aligns with the guidelines established by Centers for Disease Control and Prevention’s Advisory Committee on Immunization Practices.^11,12 An optimized immune response, more opportunities for series completion before age 13, and easier conversations with parents are benefits to recommending and starting HPV vaccination at age 9.^8,12−14

Starting HPV vaccine at age 9 or 10 and administering the second dose at the subsequent annual well child visit may simplify workflows. Using a 12-month, instead of 6-month dosing interval, has been shown to result in a more robust immune response and avoids an extra “vaccine only” visit at 6 months.^13,15 Early recommendation at age 9–10 allows for more opportunities to complete the 2-dose series between ages 9 and 12 which is especially important during disruptions in patient care, like those experienced during the COVID-19 pandemic.^6,16 It may also reduce the need for robust reminder-recall systems for HPV vaccine, which are time intensive for staff.¹⁴

Uptake of age 9 recommendations has been slow despite identification of effective interventions to prompt providers to start HPV at age 9, such as provider education, a focus on cancer prevention, and electronic medical record (EMR) clinical decision supports.^16–18 A 2021 national provider survey found that only 21% of providers were routinely recommending at age 9 to 10.¹⁹

To facilitate HPV vaccination starting at age 9, we developed and piloted “HPV Vax at 9,” a multi-level quality improvement (QI) intervention that aims to address provider knowledge and improve clinic-level HPV vaccination practices at well child visits. We report our efforts in two Western Washington pediatric clinics whose QI implementation span the pandemic time-period. Our main outcomes were to assess HPV initiation starting at age 9 and series completion rates over time. Our secondary outcomes include assessment of provider- and clinic-level acceptance of “HPV Vax at 9”.

Materials and methods

Setting

We partnered with two private pediatric clinics in King County, Washington, between 2018 and 2022. Both were identified via a convenience sample, having expressed interest in HPV vaccine quality improvement, and both had an existing provider champion to lead the QI efforts. Clinic A is a single site pediatric clinic with 10 pediatricians, located within an urban multispecialty clinic. Clinic A’s patient population includes over 4,300 patients aged 9–17 years, of whom approximately half are 9–12 years. Clinic B is a multi-site, urban/suburban pediatric practice with 85 providers and 8 clinic sites. All sites participated in the QI pilot and are referred to collectively as “Clinic B”. Clinic B’s patient population includes over 44,500 patients aged 9–17 years; approximately half of these are 9–12 years. Commercial private insurance is the predominant payor for both clinics. Table 1 provides detailed clinic and patient population characteristics.

Table 1. Clinic characteristics, demographics, and interventions.

Clinic Characteristics			Clinic A	Clinic B
Number of pediatric providers			10	85
Types of pediatric providers			10 MD^a	73 MD, 1 DO,^b 11 ARNP^c
Number of clinic sites			1	8
Urban designation			Urban	Urban/suburban
EMR			Epic	Greenway: Intergy
Demographics of Active Patient Panel, Age 9–17 y (data source: EMR)
Insurance, n (%)		Commercial private	3,693 (84.5%)	41,253 (92.7%)
		Medicaid	401 (9.2%)	2,600 (5.8%)
		None identified	265 (6.1%)^d	669 (1.5%)
		Other	11 (0.3%)	0 (0.0%)
Gender, n (%)		Male	2,146 (49.1%)	22,553 (50.7%)
		Female	2,223 (50.9%)	21,967 (49.3%)
		Unknown	1 (0.0%)	2 (0.0%)
Age, n (%)		9–10 y	1,128 (25.8%)	11,436 (25.7%)
		11–12 y	1,128 (25.8%)	11,155 (25.1%)
		13–17 y	2,114 (48.4%)	21,931 (49.3%)
		Total 9–17 y	4,370	44,522
Well child visit during May 2021–April 2022, n (%)		9–10 y	859 (76.2%)	7,154 (62.6%)
		11–12 y	814 (72.2%)	7,006 (62.8%)
		13–17 y	1,386 (65.6%)	11,013 (50.2%)
Interventions
“HPV Vax at 9” training			YES	YES
Provider scripts for age 9–10			YES	YES
Standardized immunization schedule poster			YES	YES
Policy to start HPV at age 9			YES (May 2018)	YES (February 2021)
EMR clinical decision support, first dose prompt:			Age 11	Age 9 (February 2021)
EMR order sets: HPV vaccine preselected at age 9–10 well child visit			YES	NO
Printed resources: lobby poster and HPV informational pamphlets			YES	YES
HPV vaccine information cue card			YES (May 2021)	YES (July 2021)
Peer-to-peer coaching			NA	YES
Use of standard orders: allows medical assistants to recommend and administer HPV before provider visit			NO	YES
Reminder-recall for HPV at 12–17 years			YES	NO
Outreach for overdue well child visit, age 9–17			YES	NO
Patient newsletter and blog post			NO	YES
Quality metrics dashboard for providers			NO	NO

^aDoctor of Medicine.

^bDoctor of Osteopathic Medicine.

^cAdvanced Registered Nurse Practitioner.

^dIncludes n = 234 (5.4%) with incomplete payor data due to Clinic A’s EMR conversion in September 2020.

The QI intervention “HPV Vax at 9” was initially developed and implemented by Clinic A in May 2018 under the direction of Clinic A’s champion. Washington’s Comprehensive Cancer Control Program partnerships were leveraged to informally evaluate the intervention, to identify key components and streamline its delivery for further testing. These activities were conducted in collaboration with project leaders from Washington State Department of Health (WA DOH), the American Cancer Society, and University of Washington Health Promotion Research Center. Clinic B began implementation of “HPV Vax at 9” in February 2021.

During the initial implementation periods, both clinics were simultaneously participating in immunization QI collaboratives focused on improving HPV vaccination rates with routine recommendations at age 11–12 and an emphasis on high-quality provider recommendations, avoiding missed opportunities, and using reminder-recall. “HPV Vax at 9” activities and materials were delivered outside of these QI collaboratives. Both clinics additionally continued to recommend HPV vaccination at age 11–17, using a bundled recommendation when other adolescent vaccines were due. In the pre-pandemic period (2018–2019), Clinic A sent batched reminder-recall letters about 4–6 times per year for overdue 2^nd HPV vaccine at age 12–17, with each overdue patient receiving a maximum of 2 letters per year. Outreach for overdue well child visits at age 9–17 (more than 12 months since the most recent well child visit) was conducted approximately quarterly by e-mail or phone depending on patient provided contact information. During the pandemic (2020–2022), the frequency of both reminder-recall and well child visit outreach at Clinic A was substantially reduced to approximately 2 times a year due to limited staffing. Clinic B did not do any outreach for overdue well child visit, nor send reminder-recalls for HPV during the pilot due to limited staffing. In response to the pandemic and Washington State’s “Stay Home, Stay Healthy” Order, in-person care was significantly disrupted at both clinics for older children with no in-person well child visits or vaccinations for 9–17 year olds between mid-March through mid to end of May 2020.

Materials – “HPV Vax at 9” interventions

The initial “HPV Vax at 9” intervention at Clinic A included six components: 1) Provider and staff training, 2) Recommendation script, 3) Standardized immunization schedule poster, 4) Policy change, 5) EMR support, and 6) HPV vaccine messaging via printed resources including a lobby poster, pamphlets, and information cue card to increase awareness of HPV vaccination at 9–10. The components are described below, including any modifications for Clinic B.

“HPV Vax at 9” training for providers and staff

A one-hour training was conducted to increase provider and staff knowledge about the HPV vaccine as a cancer prevention tool and to introduce starting HPV vaccine at the 9- and 10-year well child visits as a strategy to allow more opportunities to increase timely completion of the HPV vaccine. Clinic A received the training in-person at the end of April 2018. Clinic B received the training virtually in May 2021. Both trainings were delivered by the Clinic A champion. Clinic A’s training was attended by all providers and most staff. Clinic B’s virtual training was attended by a wide range of providers and staff (40 MDs, 20 RNs, 15 MAs, and a few front desk staff) and was recorded for asynchronous learning. During the first 3 months, it was watched 14 times and slides viewed 69 times; by month 15, the recording was watched a total of 36 times.

Provider scripts for age 9–10 recommendation

“High quality” provider recommendations are well established as the strongest facilitator of HPV vaccination initiation and completion.^20,21 Based on the literature, we developed a sample script, and trained providers and staff on how to use it with their 9–10-year-old patients. The script included language such as, “Your child is due for the HPV vaccine today. It is an important vaccine to prevent HPV-related cancers. I recommend getting the first dose today and the final dose at your checkup next year.”

Standardized immunization schedule poster

We created large format posters (18 × 24 inch), with a simplified, standardized immunization schedule for display in every exam room. Clinic A’s schedule includes vaccines from birth to age 21, featuring HPV vaccination at age 9–10 with catch-up at 11–12. Clinic B’s schedule includes vaccines from birth to age 17, featuring HPV vaccination at age 9 and 10. It was also posted to their website and included in patient newsletters.

Policy in support of “HPV Vax at 9”

Clinic champions facilitated policy-level support for systematizing HPV vaccine starting at age 9. In May 2018, Clinic A adopted the policy to start HPV vaccine at age 9–10 well child visits, with subsequent dose 12 months later. With strong support from leadership, Clinic B officially adopted the policy to start HPV vaccine at age 9, communicating this to providers by e-mail in February 2021. The e-mail communication emphasized the rationale for starting at age 9, specifically noting the better immune response and more chances to get vaccinated on time. Clinic B also focused on pairing HPV vaccine with the well child visits.

EMR supports

Due to institutional barriers, Clinic A was unable to change the HPV EMR prompt to age 9. Alternatively, they created well child order sets with HPV preselected at age 9 and 10 and instituted an EMR prompt for second dose of HPV, regardless of starting age. Clinic B’s EMR prompt for HPV was changed to age 9 in February 2021 during their EMR conversion.

HPV messaging: Lobby poster, informational pamphlets, cue card

We emphasized cancer prevention messaging through printed resources, highlighting HPV vaccine starting at age 9. Clinic A used an internally created poster, while Clinic B displayed a poster from ACS in their lobbies.²² HPV informational pamphlets from WA DOH and ACS were made available to both clinics. An HPV vaccine information cue card was created to empower front desk and medical assistant staff to provide accurate answers to basic questions about the HPV vaccine. Clinic B developed an HPV dosing infographic starting at age 9 for inclusion in their patient newsletter and social media blogpost.

Peer-to-peer coaching – Clinic B only

The QI pilot facilitated peer-to-peer coaching between Clinic A and Clinic B Champions for implementation planning and support. Initial coaching, beginning in November 2020, focused on the importance of enacting policy change and instituting EMR prompts. Subsequent coaching sessions occurred at least quarterly to discuss progress, barriers, and facilitators, resolve challenges, and share lessons learned. Written progress reports summarized coverage rates, uptake at well child visits and coaching comments. Both Clinic B’s Champion and vaccine coordinator reviewed and contributed to their progress reports.

Data collection

The University of Washington institutional review board and The UnitedHealth Group’s Office of Human Research Affairs designated this work as exempt (UHG project 2022-0024).

Clinic characteristics

Data on payor mix, gender, panel size, well child visits, and HPV vaccine administration at well child visits were extracted from electronic medical records (EMR). A clinic designee validated the EMR data specifications, de-identified the data using HIPAA compliant safe harbor methods and provided clinic-level summaries to the project team. Active patients were defined as patients, age 9–17 as of 1 May 2021, with one or more clinic visits during the 3-year period from May 2019 through April 2022, and who have a current pediatric clinic provider. Patients who provided notification of transfer of care were not included.

Vaccine initiation and series completion rates

Coverage rate reports and panel sizes were obtained from the Washington State Immunization Information System (WAIIS), a lifetime immunization registry for Washington residents. The coverage rates are de-identified and population based, allowing clinics to measure their vaccination rates (number and percent of patients up to date with vaccines).²³ Comparison coverage rates for Washington State and King County were compiled and analyzed by WA DOH in a publicly available report.²⁴

HPV vaccines administered at 9- and 10-year old well child visits

The HPV vaccine administered at well child visit was based on same day coding for HPV vaccine and well child visit. Clinic A’s data was queried and saved for January 2018–April 2019 (baseline and year 1) and later collected for May 2021–April 2022 (year 4). Due to Clinic A’s EMR conversion, a full data set is not available for years 2 and 3. Clinic B’s data was collected for October–January 2021 (baseline), February–April 2021 (early intervention: policy change and EMR prompt) and May 2021-April 2022 (year 1, full intervention).

Analysis

Vaccine initiation and series completion

The WAIIS coverage rate reports for the HPV series were analyzed for ages 9–10, 11–12, 13–17, and age 13 which included vaccination through age 13, up until the 14^th birthday. The age 13 coverage rates were followed to examine the youngest subset of the 13–17 year group since the WAIIS reporting platform did not have the capability to measure series completion by the 13^th birthday. HPV series completion at age 9–10, 11–12 and 13 was defined as 2 doses. HPV series completion at age 13–17 was defined as 2 or 3 doses, determined by WAIIS based on age at initiation. The coverage rates for Clinic A were run by the champion at interval time periods using a systematic approach to capture data for each age group between 9 and 17. Data was then merged to allow for a more direct comparison to Clinic B. As example, the 9-year and 10-year coverage rates were merged to provide a 9–10-year rate. The coverage rates for Clinic B included data from all eight sites, without site identifiers. These rates were run by Clinic B vaccine coordinator at least quarterly and submitted for analysis to the project leader. Coverage rates were not measured by race, ethnicity, or payor, and additionally, rates were not tracked by gender during the period of this report.

The coverage rates were imported into Excel for analysis and visualization of HPV vaccination initiation and completion coverage rates over time. Notably, the denominator of WAIIS coverage rates fluctuates as patients age into and out of the age group or receive vaccinations at certain external locations (such as other clinics) that may retain WAIIS patient ownership. To facilitate ease of identification of trends in initiation and completion rates, we report the n, denominator, and (%) for each rate.

Provider- and clinic-level acceptance of “HPV Vax at 9”

HPV vaccine administration at the 9- and 10-year-old well child visits served as a proxy measure for provider acceptance of the “HPV Vax at 9” intervention and subsequent recommendation of the HPV vaccine to their 9–10-year-old patients. Rates of HPV vaccine administration at the 9- and 10-year well child visits were imported into Excel for analysis and visualization. Feedback shared in coaching sessions, as well as feedback shared directly with the clinic champion by provider and staff were documented via notes and served as a qualitative measure of clinic acceptance.

Results

Clinic characteristics

Although both clinics are in King County, WA, they differ regarding urban versus suburban designation, number, and types of pediatric providers and number of patients. Almost 85% or more of patients had commercial private insurance in both settings. Seventy-six percent of Clinic A’s and 63% of Clinic B’s 9–10-year-old patients had a well child visit between May 2021 and April 2022 (Table 1).

Both clinics experienced a 3–4-month temporary delay of vaccination records transfer into WAIIS after their respective EMR conversions (Clinic A: September–December 2020; Clinic B: February–June 2021). Both clinics experienced fluctuation in their patient panel size in WAIIS. Clinic A’s panel of 9–17 year-olds experienced expected fluctuation between May 2018 (3,480), December 2019 (3,998) and December 2020 (3,700), but fell abruptly in spring and fall of 2021 when patient ownership in WAIIS was reassigned depending on where patients received the COVID-19 vaccine. As of May 2022, Clinic A’s WAIIS panel of 9–17 year olds was 3,127. Clinic B’s WAIIS panel of 9-17-year-olds increased from 43,757 in May 2021 to 45,859 in May 2022.

Initiation and completion coverage rates

Clinic A

HPV initiation and completion coverage rates increased dramatically in all age groups between 2018 and 2022. Within the first year, initiation rates at age 9–10 increased by 30 percentage points from n = 106/922 (11%) in May 2018 to n = 445/1,077 (41%) in April 2019, with overall maximum increase of 45 percentage points in under 4 years of QI implementation to n = 601/1,071(56%) in September 2021 (Figure 1a). Initiation at age 11–12 increased 8 percentage points in the first year from n = 655/876 (75%) in May 2018 to n = 845/1,020 (83%) in April 2019, with maximum increase of 14 percentage points to n = 917/1,036 (89%) in December 2019, then decreasing slightly to n = 750/869 (86%) in December 2021 (Figure 1b). Initiation at age 13 increased approximately 6 percentage points in the first year from n = 359/403 (89%) in April 2018 (May 2018 datapoint not available) to n = 403/422 (95%) in April 2019, reaching a high of n = 426/434 (98%) by December 2019 (data not shown). Initiation rates at age 13–17 were already high when the QI intervention began but still increased from n = 1,594/1,682 (95%) in May 2018 to n = 1,772/1,827 (97%) in December 2019 (data not shown).

Figure 1. Clinic A: HPV vaccine initiation and completion coverage rates by age, 2018–2022. a = initiation rates at age 9–10, b = initiation rates at age 11–12, c = completion rates at age 11–12, d = completion rates at age 13. The shaded time periods for Clinic A are “Year 1” (May 2018–April 2019), “COVID-19 pandemic starts” (March–May 2020), “Year 4” (May 2021–April 2022). In response to the pandemic and Washington State’s “Stay Home, Stay Healthy” Order, in-person care was significantly disrupted for older children between mid-March and mid-May 2020. As a result, there were no in-person well child visits or vaccinations for 9–17 year olds during that period. Data source: Washington State Immunization Information Registry (WAIIS).

Completion rates at age 11–12 increased 9 percentage points in the first year from n = 195/876 (22%) in May 2018 to n = 316/1,020 (31%) in April 2019, reaching n = 548/1,036 (53%) by December 2019, and achieving an overall maximum increase of 40 percentage points to n = 582/942 (62%) in December 2020. There was notably no decline in completion rates among the 11–12 age group during this time, which included the first 9 months of the COVID-19 pandemic (Figure 1c). Completion at age 13 increased by approximately 12 percentage points in the first year from n = 270/403 (67%) in April 2018 (May 2018 datapoint not available) to n = 335/422 (79%) in April 2019 with a total increase of 19 percentage points to n = 374/434 (86%) by December 2019 before leveling off with rates fluctuating between 83% and 88% (Figure 1d). Completion at age 13–17 increased 9 percentage points from n = 1,392/1,669 (83%) in February 2018 (May 2018 datapoint not available) to a maximum of n = 1,622/1,767 (92%) in December 2020 (data not shown). For all age groups, there was some leveling of coverage rates between December 2020 and May 2022.

Clinic B

Twelve months after the full implementation of “HPV Vax at 9”, Clinic B’s HPV initiation and completion coverage rates increased in all age groups, except at age 13, between May 2021 and May 2022. Initiation rates at age 9–10 increased 33 percentage points from n = 98/12,229 (1%) to n = 3,975/11,636 (34%) (Figure 2a). Initiation rates at age 11–12 increased 18 percentage points from n = 4,321/11,406 (38%) to n = 6,398/11,483 (56%) (Figure 2b). Initiation rates at age 13 decreased 11 percentage points from n = 3,062/4,099 (75%) to n = 3,433/5,334 (64%) (data not shown), whereas initiation rates at age 13–17 increased 4 percentage points overall from n = 13,071/20,121 (65%) to n = 15,794/22,740 (69%) (data not shown). Completion rates at age 11–12 increased 8 percentage points from n = 953/11,406 (8%) to n = 1,872/11,483 (16%) (Figure 1c). Completion rates at age 13 decreased 5 percentage points from n = 1,909/4,134 (46%) to n = 2,210/5,334 (41%) (Figure 2d), whereas completion rates at age 13–17 increased 6 percentage points overall from n = 9,833/20,127 (49%) to n = 12,397/22,740 (55%) (data not shown).

Figure 2. Clinic B: HPV vaccine initiation and completion coverage rates by age, during the first 12 months of full implementation of “HPV Vax at 9” at Clinic B, 2021-2022. Coverage rates were not collected prior to May 2021. a = initiation rates at age 9-10, b = initiation rates at ages 11-12, c = completion rates at age 11-12, d = completion rates at age 13. Data source: Washington State Immunization Information Registry (WAIIS).

Vaccine administration at 9- and 10-year old well child visits

Clinic A

In Clinic A’s typical workflow, providers recommend and order the HPV vaccine without utilizing standing orders. Prior to 2018, two of 10 providers were already recommending at age 9. Beginning May 2018, four of the remaining providers self-selected to start at 9, and four self-selected to start primarily at 10. Within the first year, HPV vaccine was administered at n = 179 (39%) of 9-year and n = 379 (91%) of 10-year well child visits, up from 10% and 25% at baseline. Clinic A champion shared provider-level summary data with providers during the first year of implementation. By mid-2020, all providers adopted 9 as the starting age. Despite provider and staff turnover during the pandemic, starting HPV at age 9 was reinforced as the clinic norm by EMR order sets and the immunization schedule posters. New providers and staff assimilated into the clinic’s vaccine culture without HPV vaccine-specific training. By year 4, HPV vaccine administration at 9- and 10-year well child visits reached n = 382 (85%) for age 9 and n = 373 (86%) for age 10 (Figure 3).

Figure 3. Percent of 9- and 10-year well child visits with same day HPV vaccine administration at Clinic A and Clinic B. The following time periods were evaluated for Clinic A: baseline, January–April 2018; year 1, May 2018–April 2019 (full intervention of “HPV Vax at 9” with policy change, EMR supports, provider training, immunization schedule poster and printed resources); year 4, May 2021–April 2022. The following time periods were evaluated for Clinic B: baseline, October 2020- January 2021; early intervention, February-April 2021 (policy change and EMR prompt); year 1, May 2021-April 2022 (full intervention with policy change, EMR prompt, provider training, immunization schedule poster and printed resources) Data source: electronic medical records based on submitted codes for well child visit and same day HPV vaccine.

Clinic B

Clinic B already had standing order workflows in place to allow medical assistants to recommend and administer routine vaccinations prior to the provider seeing the patient. They leveraged standing orders and the new age 9 EMR prompt to facilitate HPV vaccination starting at age 9. Providers were notified if patient or parent was hesitant, so the provider could give a strong recommendation and address concerns. Early implementation of policy change and EMR prompt in February 2021 facilitated same day administration of HPV vaccine at n = 328 (48%) of 9-year and n = 370 (62%) of 10-year well child visits between February and April 2021, up from 2% and 3% at baseline. During the first year of full implementation of “HPV Vax at 9”, HPV vaccine administration further increased to n = 2,501 (68%) of 9-year and n = 2,437 (70%) of 10-year well child visits (Figure 3). Provider-level summary data was shared monthly with providers. New staff and providers who joined after the initial training were oriented to the clinic’s standard immunization schedule, EMR prompt, and cue card, but did not receive other HPV vaccine-specific training. The recorded training was available for viewing but not required.

Provider and clinic feedback

Preliminary feedback about the “HPV Vax at 9” QI pilot was largely positive. Themes addressed appreciation for materials, ease of implementing the QI components, benefits of streamlining clinical practices, and improved self-efficacy to provide a strong HPV vaccine recommendation, especially to start the HPV vaccine series at age 9 using a cancer prevention message. The poster of the immunization schedule was reported as the most important printed resource. Anecdotally, the posters provided a strong visual cue to recommend at 9, reduced confusion and errors by staff, provided accurate information to parents, and added authority to the clinics’ strong recommendation for HPV vaccination at age 9. The lobby poster was identified as helpful but less important, and pamphlets were not routinely distributed unless a patient or parent presented as hesitant to the vaccine. Champion B reported that the cue cards were empowering and especially helpful during initial staff training.

Exemplar quotes from providers are provided below.

• “The giant poster of the immunization schedule is magic”

• “Being able to spread out the vaccines instead of bundling is an added benefit”

• “Giving a simple strong recommendation focused on cancer prevention is easier and more effective than getting into an awkward discussion about sex”

• “I now have a better way to talk about the [HPV] vaccine and parents can see that I’m excited about it”

• “I wish we had started vaccinating at age nine sooner. It is so important, and it’s made it so simple”

Clinic B champion shared several reflections on implementation of “HPV Vax at 9”:

• “The more energy you put into the medical assistants to use the practical pitch [script], the better it works”

• “Touch everybody with this message, involve your whole staff”

• “Best when we can build systems that automate it [HPV Vax at 9]”

• “This intervention will continue to make a major impact on cancer prevention because the changes to the system will remain in place indefinitely with minimal ongoing effort.”

Discussion

Both clinics in the “HPV Vax at 9” pilot observed remarkable HPV vaccination success over a short period of time. They demonstrated that they could successfully recommend and initiate HPV vaccine starting at the 9-year well child visit, observing a 30 percentage points or more increase in HPV initiation coverage rates at age 9–10 within the first year of implementation. The clinics were able to implement “HPV Vax at 9” with relative ease. The multi-component intervention facilitated a wide range of beneficial changes, including policy level (HPV vaccination starting routinely at age 9), clinic level (infrastructure improvements via standard immunization schedule and EMR supports), provider/staff level (increased knowledge about HPV vaccine, improved self-efficacy to provide a strong recommendation) and parent/patient level (increased awareness of HPV vaccine, and vaccine initiation and/or completion). Importantly, posting the clinic-specific standardized immunization schedule (supported by EMR) allowed for greater synchrony between providers and staff, reducing unnecessary variation.

High HPV vaccine administration rates at the 9- and 10-year-old well child visits suggest that providers were receptive to the “HPV Vax at 9” intervention and that parents were responsive to their clinic’s messaging to initiate vaccination at age 9 or 10. Notably, neither clinic provided advance pre-appointment notice to parents about their clinic’s updated HPV immunization schedule for 9–10 year-old children, although Clinic B notified parents generally via their newsletter and blogpost.

Peer-reviewed research suggests that initiation before age 11 coupled with interventions including EMR prompts and consistent, strong messaging from providers and staff, will likely contribute to improvements in series completion rates over time.^16–18 Clinic A’s sustained increase in HPV initiation and completion over 4 years demonstrates sustainability of both the QI components and the success of their HPV vaccination program. Of note, Clinic A’s HPV completion rates for 13–17-year olds consistently exceeded the 80% Healthy People 2030 goal prior to 2018. However, while participating in “HPV Vax at 9”, the age 13 completion rates increased, surpassing 80% continuously since October 2019. We anticipate that Clinic B’s completion rates, especially at 11–12, will continue to rise during years 2–3 as patients age up into the next age group and return for the second HPV dose at their annual well child visit. Regular and systematic outreach for overdue annual well child visits, coupled with regular use of reminder-recall for overdue HPV vaccine at age 12–17 would likely further enhance vaccination rates.

The attention on HPV vaccination broadly (i.e., messaging and tools that benefit all HPV vaccine eligible age groups) combined with explicit focus on starting at age 9, seemingly helped mitigate the negative effect of the COVID-19 pandemic on HPV vaccination rates. Despite the pandemic, Clinic A continued to experience a modest 5 percentage points increase in initiation coverage rates (from 83% to 88%) among 11–12 year-olds between April 2019 and June 2021. In contrast, during a similar time-period (June 2019-June 2021), Washington State reported a 5.1 percentage points statewide decrease in HPV initiation coverage rate (from 39.6% to 34.5%) for the same age group.²⁴ The downturn in age 13 completion rates at Clinic A from 88% in December 2021 to 83% in May 2022 may reflect pandemic related disruptions in care between March and May 2020, for those that were 11 years old during that time (Figure 1d).

Clinic B observed a less noticeable coverage benefit at age 13; however, it might be expected that vaccination at age 9–10 would take a minimum of 3–4 years to fully influence age 13 rates as patients transition into older age groups. Although Clinic B experienced declines in HPV coverage rates at age 13, initiation (−11 percentage points) (data not shown) and completion (−5 percentage points) (Figure 2d), it was less than expected given their declines in coverage rates for Tdap (−18 percentage points) and MenACWY (−14 percentage points) between May 2021-May 2022 (data not shown). Overall, the age 13 rates may suggest age-specific barriers to vaccine access or completion through at least May 2022. It will be important to monitor vaccination rates, to see if trends emerge across vaccines or if they are vaccine specific; this may inform intervention efforts targeting patients that were age 11–12 at the beginning of the COVID-19 pandemic.⁴

Overall, our results are consistent with other published studies evaluating HPV vaccination before age 11. In a retrospective study in Minnesota, patients who initiated HPV vaccination at 9–10 years old were more likely to have on-time series completion by age 13.5 compared to those initiating at 11 to 12 years (97.5% versus 78%).²⁵ After a three-part QI intervention (revised EMR alert, HPV QI team formation, and clinic incentive) in Ohio, the percentage of patients receiving an HPV vaccine before age 11, increased from 4.6% to 35.7% at 6 months and 60.8% at 18 months.¹⁷ A 2016 multi-level intervention trial in Boston, which included provider education, data feedback, tailored system changes and early initiation of HPV vaccine series at age 9 or 10, led to improvement in vaccine series initiation and completion in several clinics.¹⁸ The long-term impact of this multilevel intervention on HPV vaccination rates was evaluated at two Federally Qualified Health Centers who initiated at age 10, reinforced by an EMR prompt.¹⁶ HPV vaccination completion rates by age 13 increased from 62% to 88% through October 2020, 4 years after the completion of the intervention. Adopting an ‘earlier age’ recommendation may have increased vaccination opportunities, partially compensating for missed routine well child visits during the COVID-19 pandemic. When comparing our results to other studies, it is important to note that we measured population coverage rates instead of on-time completion rates for individual patients, and our age 13 rates measured vaccination through age 13 instead of by the 13^th birthday.

Comparable to other studies, providers in our QI intervention reported that recommending at age 9 using a cancer prevention message is easier and faster than recommending at age 11–12 and helps overcome many barriers, particularly reducing the perceived association of the HPV vaccine with sexual activity.¹⁴ Additionally, providers reported that even when parents decline the initial recommendation, starting the conversation at age 9 allows parents time to learn about the vaccine and often results in acceptance of the vaccine at age 10 or 11. This is consistent with a study reporting that most parents (69%) accept or intend to accept HPV vaccination after declination.²⁶

Limitations

Given the scope and limitations of our QI pilot, there were several things we were unable to evaluate including use and quality of provider recommendations, impact of clinic champions, and controlling for QI collaborative efforts. We have limited systematic feedback from providers about the utility of the specific QI components; however, mixed methods evaluation is underway to assess this. Both clinics in our QI pilot expressed a clear interest in HPV vaccine quality improvement, had an existing and committed vaccine champion, and participated in related immunization QI collaboratives concurrently during the early months of “HPV Vax at 9”. These factors may bias our results. It is likely that benefits gained from QI collaborative efforts spilled over into “HPV Vax at 9”, and vice versa. As reported in the literature, vaccine champions and a multi-level approach combining QI coaching with provider training are both facilitators of success for HPV vaccine QI programs.^27,28 A general clinical “push” to strengthen HPV vaccination strategies is very important, however, offering vaccination starting at age 9 via our QI project is arguably the innovation that energized the clinics’ HPV vaccination efforts.

There are additional limitations that affect generalizability of our findings including clinic characteristics, approach to well child visits, and data issues. Our clinics were urban/suburban with predominately private commercial payors and relatively high rates of 9- and 10-year well child visits. Even in these privately insured populations and with outreach at Clinic A, about 25–40% of the 9–12-year-old patients did not present for a well child visit between May 2021 and April 2022. The frequency and prioritization of annual well child visits may vary depending on individual clinics’ approach and outreach for well child visits. Patient populations and payor mix may additionally impact the rates of well child visits and overall success in other settings.²⁹ On a population level, this means that there are many children who miss opportunities to be offered HPV vaccine through well child visits.

Our analyses were constrained by the variability in our data sources. As noted earlier, data were compiled from multiple sources with limited data overlap. We were unable to examine demographics by race and ethnicity as well as coverage rates by gender, race, ethnicity, or payor. Datasets that allow for more exploration may help uncover disparities in certain populations (e.g., gender, race/ethnicity, English language proficiency) that may benefit from targeted and tailored interventions.

While WAIIS coverage rate reports were run and saved at interval time periods during the entire pilot for both clinics, clinic demographic data (payor, gender, EMR panel size) was accessed and validated from the respective EMRs on 30 November 2022 (Clinic A) and 2 December 2022 (Clinic B); patients who transferred care prior to those dates were not included. The payor mix was determined as of the data validation date. The extent to which individual payors varied between 2021 and late 2022 is unclear. Additionally, missing payor data was identified at Clinic A and was determined to be partially attributed to the EMR conversion (September 2020) which required insurance coverage to be reentered. We estimate that Clinic A’s payor data is partially incomplete, including n = 234 patients (5.4%) with missing updated insurance and no visits since the EMR conversion. Despite these issues with payor data, the overall payor mix as of late 2022 is provided to further describe each clinic’s population characteristics at a snapshot in time. Clinic champions estimate that the payor mix is roughly consistent with their historical trends (see Table 1).

Data transfer delays were also problematic. Clinic A experienced a temporary delay in transfer of vaccine records from the EMR to WAIIS after their EMR conversion in September 2020; coverage rate reports were paused until the issue was corrected. Coinciding with Clinic B’s EMR conversion in February 2021, they also experienced a similar data transfer delay, which prevented timely reporting into WAIIS of vaccinations administered between February 2021 and May 2021. Clinic B attempted manual entry of vaccines into WAIIS but aborted this process quickly due to staffing issues. The relevant effect of the data transfer delay is that the initial May 2021 coverage rates for Clinic B are more reflective of baseline rates before February 2021. After resolution of the issue in June 2021, the rates for August 2021 and beyond were deemed accurate reflections of the reporting periods. Clinic B’s coverage rates and well child-same day vaccination data were collapsed at the overall clinic level for all eight sites, so we were unable to conduct site-level analyses.

As noted previously, there was extensive fluctuation in WAIIS patient panel size, particularly in 2021, resulting from re-attribution of patient ownership based on receipt of COVID-19 vaccine at some external mass vaccination sites. Due to initial high demand for COVID-19 vaccine for ages 12–17 in mid-May 2021 and ages 5–11 in November 2021, Clinic A’s panels of 12–17 and 9-11-year-old patients in WAIIS were acutely reduced (over the span of 4–8 weeks) by 25% and 29%, respectively, as patients sought COVID-19 vaccine across the region. This inadvertent re-attribution of patients may artificially lower coverage rates at Clinic A for May 2021 and beyond since many of their highly vaccinated patients received COVID vaccine elsewhere. Patients will migrate back into Clinic A’s WAIIS panel as they return for flu vaccine, routine vaccines, or COVID-19 boosters. Similar to Clinic A, some Clinic B patients may have received COVID-19 vaccine elsewhere, especially during the initial high demand in Spring 2021, causing a potential unmeasured decrease in panel size prior to Clinic B’s first measured rates in May 2021. However, the WAIIS patient panel fluctuation was likely less pronounced at Clinic B because they offered their own COVID-19 mass vaccination clinic, restricting participation to their established patients. Between May 2021 and May 2022, Clinic B’s panel of 9–17-year olds increased by approximately 5% from 43,757 to 45,859, driven primarily by an increase in 13–17-year olds.

Additionally, the WAIIS database has its own inherent limitations as it may include patients who have moved out of state, or new patients with incomplete records. Depending on how clinics manage their patient panel in WAIIS, the data may result in an underestimate of actual vaccine coverage rates. Beginning in 2016 and throughout the intervention, Clinic A actively managed their WAIIS panel as part of outreach and reminder-recall processes, routinely identifying and inactivating patients who had moved out of state or had been lost to followup. Clinic B did not actively manage their WAIIS panel, and therefore, Clinic B’s measured coverage rates are likely an underestimate of actual coverage rates. Rates reported in the WAIIS database are also not directly comparable to the NIS-Teen data due to differences in how the data is collected and validated.^4,23,30

Conclusion

The COVID-19 pandemic presented several challenges and barriers to HPV vaccination in the United States, highlighting the need for simple and effective strategies to improve HPV vaccination efforts. Our multilevel quality improvement pilot initiated HPV vaccination at age 9, pairing it with well child visits and leveraging evidence-based strategies including provider focused interventions, clinical staff training, structural improvements, and system changes to raise HPV vaccination coverage rates. Overall, coverage rates indicate improved vaccination initiation and completion rates particularly at 9–12 years, despite the ongoing pandemic. Maintenance of QI intervention components at these two clinics shows strong promise to produce sustainable gains in HPV vaccination coverage and improvements in vaccine workflow and delivery starting at the 9-year well child visit. To maximize HPV vaccination completion by age 13, additional efforts may be needed to identify and reach children who miss the well child vaccination opportunities.

Acknowledgments

The contents of this paper are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by CDC/HHS, or the US Government. Special thanks to providers and staff members that played a pivotal role in implementation, data collection and provision of feedback, and to Washington’s Comprehensive Cancer Control Program which facilitated linkages and championed this project.

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

Funding for project implementation at Clinic B was provided by Amerigroup Washington, Inc, which supported two of the project leads and LiveSmyle Foundation 501(c)(3), which supported print resources utilized as part of this intervention. The participation of UW authors was supported by CDC cooperative agreements [U48DP006398]. The participation of the WA DOH author was supported in kind via [NU58DP006326-01-00].