Effect of immunization registry-based provider reminder to initiate HPV vaccination at age 9, Washington state

ABSTRACT

Human papillomavirus (HPV) vaccination rates are lower than Tetanus-diphtheria-pertussis (Tdap) and Meningococcal conjugate (MenACWY) rates, although the Advisory Committee on Immunization Practices recommends all three vaccines be given routinely at age 11–12. Evidence is mounting that children who initiate HPV vaccination starting at age 9 are more likely to complete the series on time. Washington state implemented a provider reminder through its immunization information system (WAIIS) in January 2023 to increase HPV vaccine initiation at 9-years-old by updating the forecasted recommended age for HPV from age 11 to 9. The effectiveness of provider reminders when implemented via an immunization information system (IIS) is poorly understood. We evaluated the impact of this forecast update using a seasonally adjusted interrupted time series regression of weekly HPV initiations at 9-years-old before and after implementation. We also examined time series trends of vaccine administration between 2018 and 2023 for HPV initiation at age 9, as well as Tdap, MenACWY and HPV initiation at age 11. The WAIIS forecast update doubled the weekly rate of HPV initiation among 9-year-olds in Washington state, although the weekly count of initiation at 9 remains far lower than initiations at 11. Jurisdictions wanting to increase HPV vaccine initiation at earlier ages should consider updating their forecast algorithm and investing in complementary evidence-based strategies such as provider and parent education, and clinic-based quality improvement efforts. The reach of IIS forecaster updates may be enhanced by working with administrators of electronic medical record systems to ensure parity of provider prompts with IIS.

KEYWORDS:

• Human papillomavirus
• immunization information system
• provider reminder
• HPV at 9
• vaccine forecast

Introduction

Human Papillomavirus (HPV) is transmitted through skin-to-skin and sexual contact and causes cancer of the oropharynx, cervix, vulva, vagina, anus, and penis. About 36,500 new cases of cancer are attributable to HPV in the United States each year, approximately 90% of which could be prevented by the 9-valent HPV vaccine.¹ The Advisory Committee on Immunization Practices (ACIP) recommends HPV vaccination routinely at ages 11–12 along with tetanus, diphtheria and pertussis (Tdap) and meningococcal conjugate (MenACWY) vaccines, but also indicates that HPV vaccination can be given starting at age 9 years.^2,3 Rates of HPV vaccination lag behind rates for Tdap and MenACWY vaccination.⁴ The American Academy of Pediatrics (AAP) and American Cancer Society (ACS) have released recommendations that emphasize earlier HPV vaccine initiation at age 9.^5,6 Observational studies have found that patients who initiate at age 9 have higher rates of on-time series completion by their 13th birthday.^7–9

In Washington state, Tdap is mandated for school attendance in 7th grade (typically ages 12–13), but MenACWY and HPV are not.¹⁰ In 2022, registry-based surveillance indicated vaccine population coverage rates of 48%, 41% and 36% for Tdap, MenACWY initiation and HPV initiation, respectively, among 11–12-year-old children in Washington state. And among 13–17-year-old children, the 2022 coverage rates for Tdap, MenACWY initiation and HPV up-to-date (series completion) were 73%, 65% and 43%.¹¹

In January of 2023, to improve HPV vaccination coverage, the Washington State Department of Health (DOH) updated the software algorithm that determines vaccine eligibility (i.e., “the forecaster”) to recommend HPV vaccination at age 9 instead of age 11 in the Washington State Immunization Information System (WAIIS), the state’s immunization registry. This establishes a permanent “provider reminder” to initiate HPV vaccination at age 9 for all healthcare providers who use WAIIS to assess patient vaccination history. Washington state is among the first jurisdictions to forecast HPV vaccination due at 9 years as a provider reminder in a state immunization registry.

The update to the WAIIS forecast algorithm is a major development, but it is not the first or last effort in the state to increase HPV initiation at 9 years old. To provide context and improve our ability to infer attribution of changes in vaccine uptake to the forecasting update, we engaged stakeholders to develop a timeline of activities in Washington state targeting HPV initiation among 9-year-old children. We evaluated trends in the weekly number of HPV initiations among 9-year-old children against this timeline of activities to draw conclusions about the impact of the forecaster update on HPV initiation at age 9. Specifically, we used seasonally adjusted interrupted time series (ITS) regression to quantify the extent to which the forecaster update increased initiation of HPV vaccination among 9-year-olds.

Methods

In this analysis, we used a seasonally adjusted ITS regression of weekly HPV initiations at 9-years-old to quantify the number of initiations attributable to and the effect size of the update to the WAIIS forecaster in January 2023. ITS is a quasi-experimental method that is used extensively to quantify the effect of interventions or policy changes in public health and medicine when randomization of subjects is not possible. Unlike simple before-versus-after comparisons, ITS accounts for preexisting trends in the outcome and can adjust for time-varying effects such as seasonality or prior interventions.^12,13 Results from an ITS are intuitive and can be displayed visually, which facilitates understanding.

For context, we also contrasted the 9-year-old HPV initiation time series with weekly vaccination counts for HPV initiation, Tdap, and MenACWY initiation among 11-year-olds between 2018 and 2023.

Intervention

On January 20, 2023, DOH updated the forecaster program to recommend HPV vaccination at age 9 instead of age 11 in WAIIS. Prior to this, the recommended date of initiation was the patient’s 11th birthday, the minimum valid date was the 9th birthday, and the overdue date was the 13th birthday plus 4 weeks. With the forecaster update to age 9, the HPV status of patients 9–10 years old changed from “optional” to “due now.” This change means that all healthcare providers who use WAIIS to check patient immunization status will receive a notification that patients aged 9 and older are due for HPV vaccination. This change in forecaster policy is permanent and systemic, potentially reaching most healthcare providers in the state. The update was communicated to more than 27,000 healthcare professionals, local health jurisdictions, school nurses, and child vaccine providers in Washington state by two e-mails sent one week prior to and the day of the update. Although dentists are not typically vaccinators in Washington state, the update was also communicated on February 23, 2023, to more than 33,500 dental professionals.

Data sources

The WAIIS is a mature lifetime registry that tracks immunization records for people of all ages in Washington state. Healthcare providers voluntarily report immunizations for their patients to WAIIS. Despite voluntary reporting, WAIIS captures the vast majority of routine childhood vaccine administrations in the state. The state provides free access to all childhood vaccines through its universal vaccine purchasing system.¹⁴ New patient records are initiated in the WAIIS when birth certificates for newborns in the state of Washington are loaded into the system, when a provider adds a new patient record to WAIIS or when the state receives an electronic immunization record for a new patient from providers, pharmacists or health plans. New patient records are those that do not match the identifying information associated with an existing WAIIS patient record. As of 2019, WAIIS contained records for 128% of the adolescent population compared to the US Census and 99% of children ages 4 months to 5-years-old had two or more immunizations on record. WAIIS overestimates the adolescent population of the state in part because not all adolescents who have moved out of the state have been marked as such by their healthcare providers.¹⁵

The Washington State Office of Financial Management curates Small Area Demographic Estimates of single-year-age Census estimates for internal use by state agencies, but updates using 2020 Census were not available.¹⁶ We pulled annual population estimates of 9-year-olds from 2011–2020 and predicted populations for 2021–2023 based on a simple linear regression of year on population (adj. r² = 0.998). The log of these population estimates were incorporated into the model as an offset, which changes the interpretation of the model results from incident rate ratios (IRR) of the weekly counts of vaccinations from a static population to IRRs of the weekly population-based rate of vaccination. We used Census-based population estimates which may not be comparable to registry-derived population estimates that are sometimes used to estimate vaccine coverage.¹⁷ Census-based estimates were selected because they seemed more appropriate than a WAIIS-based denominator given the goal of describing HPV initiation behavior in Washington state.¹⁸ We do not present results in terms of rates or coverage.

Members of the Washington State HPV Free Task Force^a and Vaccine Advisory Committee (VAC)¹⁹ were asked to describe any promotional and organizational activities in the state related to increasing HPV initiation at age 9. These activities date back to 2017 and range in scope from letters from the Secretary of Health to healthcare providers encouraging HPV initiation at age 9 to multifaceted quality improvement efforts implemented in specific healthcare clinics by community partners.²⁰

Measures

Weekly counts of children receiving valid vaccinations of four different vaccine types were pulled from WAIIS by date of vaccination to create weekly time series of vaccinations running from January 1, 2018, to August 14, 2023. Participants were limited to those who had a Washington state address on file at the time of vaccination. The four time series were HPV initiation (i.e., a participant received the first dose of a series) at 9 years old, HPV initiation at 11 years old, MenACWY initiation at 11 years old and the most recent Tdap vaccination administered at 11 years old. While the HPV and MenACWY time series describe initiation of these vaccines, the Tdap dose is more likely reflective of the recommended booster dose for this age group. Due to the way these time series were pulled, we refer to them interchangeably as both counts of vaccinations and counts of children because there is only one vaccination in each series for each child. Counts were aggregated by internationally standard (ISO 8601) week and year classifications, ending with week 31 of 2023. The last incomplete week was excluded.

The 11-year-old time series data is provided for context and comparison to the 9-year-old HPV initiation time series. The interrupted time series regression was limited to the 9-year-old HPV initiation time series. Predictors of the weekly rate of HPV initiation among 9-year-olds included an offset of the log of the annual population, pairs of sine and cosine functions of week-time with an underlying period reflecting the full seasonal cycle to account for seasonality and a linear week trend. We also incorporated a dummy variable indicating the mean increase in HPV initiations at week 9 of 2021 (February 28 thru March 6) due to the “Clinic B” quality improvement project described in Zorn et al.²⁰ and other limited “HPV @ 9” provider education events that occurred near this time. Another dummy variable for the WAIIS forecaster update was set for week 3 of 2023 (January 16 thru 27). If significant, these dummy variables indicate an immediate change in the mean weekly rate of HPV initiation. We also evaluated whether there was a change in the slope of initiations following the forecaster update using a “ramp” variable with a value of 0 until the forecaster update and which increments by one each week thereafter.

Statistical analysis

We evaluated trends in the weekly number of HPV initiations among 9-year-old children against the timeline of activities to draw conclusions about the impact of the forecaster update on HPV initiation at age 9. Trends in weekly counts of vaccinations for 9- and 11-year-olds since 2018 were plotted for comparison. For the regression analysis, data prior to 2021 were excluded because all vaccinations were inordinately impacted by the COVID-19 pandemic in 2020 and the seasonal patterns after 2020 are not comparable to prior years.

The 9-year-old time series was decomposed to evaluate underlying trends and seasonal effects. A negative binomial model was selected because it can accommodate over dispersed count data which would violate a key assumption of a Poisson model. A negative binomial generalized linear model of weekly initiation counts was fit with the ‘glm.nb’ function in the R (version 4.2.1) ‘MASS’ package using a log link. A dummy indicator of a step-change the week of the forecaster update is used to describe the mean change in the weekly rate of HPV initiations associated with the forecaster update. The model with the step-change for the forecaster update is referred to as the “effect” model. The step-change coefficient from the effect model is exponentiated to produce an IRR attributable to the forecaster update after adjusting for seasonal patterns, prior interventions that effected weekly vaccine initiation, and changes in the population size. The same final model without the intervention indicator and limited to the weeks prior to the intervention was used to predict the weekly number of HPV initiations absent the intervention (i.e., the counterfactual model). The difference between the observed time series and predicted counts from the counterfactual model in the weeks following the intervention was attributed to the forecaster update. Using these two approaches we can quantify the impact of the forecaster update in terms of an adjusted effect size (i.e., the ratio of weekly initiation rates with and without the intervention) and as the number of HPV initiations attributable to the update.

The model-building approach followed published steps for similar models used to evaluate policy changes with seasonally adjusted time series data^12,21,22 and prioritized parsimony (i.e., a model with a limited number of highly predictive covariates) and interpretability. The order of the harmonic sine and cosine seasonal terms was identified with likelihood ratio tests to select the lowest order after which the next highest order ceased to significantly improve model fit. Diagnostic residual, autocorrelation (ACF) and partial autocorrelation (PACF) plots were inspected for lingering patterns and autocorrelation.

Sensitivity analysis

The week of the forecaster update was shifted up to two weeks before and after the week of implementation to assess the presence of lagged effects and sensitivity of the intervention effect. Reductions in Akaike information criterion (AIC) greater than two in magnitude were considered evidence of improved model fit.

Results

Partners identified a long history of promotional activities in Washington state dating back to 2017 directed toward increasing initiation of HPV vaccination among 9-year-olds. Table 1 lists these activities. These include the publication of formal recommendations by national partners, communications from DOH to healthcare providers, presentations by DOH and state and local partners to healthcare providers, clinic-based quality improvement efforts focused on “HPV @ 9,” and organizing events where partners planned or disseminated promotional material. When possible, dates and estimates of reach were provided.

Table 1. Timeline of promotional and organizational activities conducted in Washington state related to increasing HPV initiation at age 9 as compiled by members of the Washington State HPV Free Task Force and Vaccine Advisory Committee (VAC).¹⁹.

National Guidance
December 2016	Advisory Committee on Immunization Practices (ACIP) recommends routine HPV vaccination at age 11 or 12 years. Vaccination can be given starting at age 9 years.^Citation3
May 2018	American Academy of Pediatrics (AAP Red Book: 2018 Report of the Committee on Infectious Diseases 31st edition) recommends starting HPV series between 9 and 12 years.^Citation23
July 2020	American Cancer Society updates guideline to recommend HPV vaccination starting at 9.^Citation6
Washington State Department of Health (DOH) Activities and Communications
April 2017	Letter to healthcare providers emphasizing HPV series can be initiated at age 9 (mailed).
October 2019	Letter to healthcare providers re-emphasizing HPV series can be initiated at age 9. E-newsletter − 3,728 recipients (24% unique opens); posted to website January 2020.
January 2020	DOH publications that included some HPV at 9 messaging: Adolescent Immunization At-A-Glance Poster (Pub 348–739);^a HPV vaccine educational flyer (Pub 348–612);^b HPV vaccine educational pamphlet (Pub 348–516).^c
December 2021	WAIIS update: HPV vaccination status is no longer suppressed in the Washington State Immunization Information System (WAIIS) patient portal. WAIIS forecasts HPV vaccine at age 11, but it will count as a valid dose starting at age 9. Email − 18,767 recipients (26% unique opens).
July 2022	Washington State Vaccine Advisory Committee (VAC) formally requests the Secretary of Health take action to encourage providers to routinely start HPV vaccination at age 9, to convey this information on all HPV immunization materials issued by DOH, to change the WAIIS forecasting to prompt at age 9 and to track and publish state and county data on HPV vaccination coverage rates for children ages 9–10.
January 2023	WAIIS forecasting algorithm changed on 1/20/2023 to indicate HPV vaccine due at age 9. An alert on the homepage of the WAIIS advised users of the change from 1/17/2023–2/3/2023.
January 2023	Two letters to healthcare providers, local health jurisdictions, school nurses, and childhood vaccine providers alerting them to the WAIIS HPV forecast change to age 9; included a list of HPV at 9 resources. Emails − 1/13/2023 (27,160 recipients, 49% unique opens) and 1/20/23 (27,160 recipients, 48% unique opens).
January 2023	Notification of WAIIS HPV forecast change sent via 2 E-newsletters to healthcare providers and vaccine coordinators (5,242 recipients, 52% open rate).
February 2023	HPV at age 9 offered as an optional IQIP strategy. IQIP is CDC’s immunization quality improvement program for providers, administered at the state level.
February 2023	Letter to dental professionals about WAIIS HPV forecast change. Email − 33,584 (23% unique opens).
March 2023	Letter to providers from WA Secretary of Health about HPV at age 9. Email − 27,190 recipients (52% unique opens).
March 2023	Publications (see above) updated to highlight age 9 messaging and translated into 5 languages.
Education Events
January 2021	Virtual HPV @ 9 presentation at health system medical conference (102 provider attendees).
September 2021	Virtual HPV @ 9 presentation at county level immunization coalition meeting.
September 2021	Virtual HPV @ 9 presentation at community clinic (47 staff and provider attendees).
May 2023	Washington State Immunization Summit: HPV @ 9 presentation (359 attendees).
May 2023	Webinar: WAIIS forecast change; clinic implementation strategies for HPV @ age 9 (252 attendees).
June 2023	Webinar: Identifying barriers to HPV vaccination, addressing HPV vaccine hesitancy, and promotion of HPV @ 9 (143 attendees).
Quality Improvement Projects with Focus on HPV @ 9
May 2018	HPV @ 9 implemented in 1 clinic (10 providers; 1,128 9–10 year olds). Noted as Clinic A in Zorn et al.^Citation20
February 2021	HPV @ 9 implemented in 1 multi-site clinic. EMR HPV prompt at age 9. (85 providers; 11,436 9–10 year olds) noted as “Clinic B” in Zorn et al.^Citation20
February 2021	HPV @ 9 implemented in 1 clinic (5 providers, 500 9–10 year olds).
April 2022	Washington Child Health Improvement Partnership (WACHIP) Immunization Learning Collaborative, cohort 4; 4 of 9 clinics adopted HPV @ 9 as an intervention.
February 2023	HPV @ 9 implemented in 5 clinics (36 providers: 3,782 9–10 year olds).
April 2023	WACHIP Immunization Learning Collaborative, cohort 5; 4 of 11 clinics adopted HPV @ 9 as an intervention.
June 2023	HPV @ 9 implemented in 8 clinics (38 providers: 2,538 9–10 year olds).
Washington State HPV Free Task Force Meetings, Letters, Materials
September 2021 - Ongoing	Virtual meetings with HPV @ 9 presentations including implementation strategies and resources, 3 times per year (40–70 attendees).
June 2022	Back to school letter sent to vaccine providers; included HPV @ 9 resources.
June - December 2022	Free printed HPV materials for providers. Protect Your Preteen/Teen with Vaccines (National HPV Vaccination Roundtable: 235 large posters/2,216 flyers);^d Protect Your Child (DOH Pub 348–612: 1,805 flyers).^b
July 2023	Back to school letter sent to vaccine providers and dental professionals; included HPV @ 9 resources.

^ahttps://doh.wa.gov/sites/default/files/legacy/Documents/Pubs/348–739-AdolescentImmmunizationSchedulePoster.pdf.

^bhttps://doh.wa.gov/sites/default/files/2022–07/348–612-ProtectChildAgainstViruses-en-L_0.pdf.

^chttps://doh.wa.gov/sites/default/files/legacy/Documents/8200//348–516-HPVshot-en-L.pdf.

^dhttps://hpvroundtable.org/wp-content/uploads/2022/06/Co-brand_HPV_Roundtable_2022_Flier_8-5x11_WEB-v1.pdf.

Following both the introduction of age 9 provider prompts in “Clinic B’s” electronic medical record (EMR) in 2021 and the WAIIS forecaster in 2023, HPV initiation at age 9 increased immediately. These increases are apparent in the state-level time series after removing seasonal patterns (Figure 1). The time series for weekly HPV initiations among 9-year-olds was decomposed and the seasonal component was deducted from the time series. Figure 1 presents this seasonally adjusted time series. Two major fluctuations in the time series are apparent after seasonal adjustment at the beginning of 2021 and 2023. These are denoted with vertical dashed lines. The 2021 fluctuation was evaluated against the timeline of promotional activities to identify possible explanations and corresponds well to the timing of “Clinic B”s EMR prompt deployment, discussed below. The 2023 fluctuation corresponds well with the forecaster update at week 3 of 2023.

Figure 1. Seasonally adjusted weekly counts of HPV initiation among 9-year-olds in Washington state, 2021 to mid-2023. The red vertical dashed lines labeled ‘“Clinic B” intervention implementation’ and “forecaster update intervention implementation” correspond to the implementation of public health interventions described in-text.

A synthesis of published literature, promotional activities (Table 1) and inspection of the age 9 HPV initiation time series was used to identify reasonable model covariates. In Table 1, two quality improvement projects were implemented by different clinics in 2018 and 2021. These are described in Zorn et al. as “Clinic A” and “Clinic B,” respectively. Excluding the forecaster update, the 2021 “Clinic B” project is the only activity with substantial reach and documented effectiveness. It is also the only activity in Table 1 that included a change to the clinic’s EMR prompt to remind providers to initiate vaccination at age 9. The EMR prompt implemented in 2021 by “Clinic B” is functionally comparable to the forecaster prompt implemented by the state in 2023. Zorn et al. report an active panel of 11,436 patients who are 9–10 years old at this multi-site clinic. If we assume the 9-year-old panel is approximately half of this, this constitutes nearly 6% of the state’s 9-year-old population. The authors go on to report that the percent of 9-year-old well child visits with same-day HPV vaccination increased from a baseline of 2% to 48% between February and April 2021 following the early implementation of a policy change and age 9 EMR prompt in February 2021. In terms of counts, this corresponded to an increase in HPV initiations at age 9 well child visits at “Clinic B” from about 4.75 per week to 32.6 per week in the months before and after the EMR prompt. In the Figure 2 inset and Figure 3, we can observe a sharp increase in statewide HPV initiations among 9-year-olds around week 9 of 2021. The mean weekly number of initiations in the four weeks before and after week 9 of this analysis increased from 46 per week to 96 per week. The corresponding timeframe and magnitude of the “Clinic B” effect with the 9-year-old time series and persistence of the effect after seasonal adjustment indicate the project at “Clinic B” contributed to the statewide changes in the time series around week 9. For these reasons and to improve model fit we chose to explicitly model “Clinic B” in this analysis. The “Clinic B” model parameter was significant.

Figure 2. Time series for HPV initiation (i.e., a participant received the first dose of a series) at 9 years old, HPV initiation at 11 years old, MenACWY initiation at 11 years old and the most recent Tdap vaccination administered at 11 years old in Washington state, 2018 to mid-2023.

HPV: Human Papillomavirus vaccine; Tdap: tetanus, diphtheria and pertussis vaccine; MenACWY: meningococcal conjugate vaccine.

Figure 3. The “observed” time series is the actual number of HPV vaccine initiations among 9-year-olds in Washington state, 2021 to mid-2023. The “effect model” time series includes data through the end of the observed period and quantifies the effect of the forecaster update as part of the model. The “counterfactual model” is limited to data prior to the forecaster update and does not model the effect of the forecaster update. Predicted weekly counts from the “counterfactual model” reflect what we could have expected had the forecaster not been updated. The red vertical dashed lines labeled ‘“Clinic B” intervention implementation’ and “forecaster update intervention implementation” correspond to the implementation of public health interventions described in-text.

Looking across all vaccination time series for age 9- and 11-year-olds there are fascinating fluctuations related to age, vaccine type, the COVID-19 pandemic and seasonality. Figure 2 presents the weekly count of HPV initiations among 9-year-olds along with HPV initiations, MenACWY initiations and Tdap administrations among 11-year-olds. The weekly count of HPV initiations among 11-year-olds ranges up to 3,312 while the 9-year-old time series only reaches a maximum of 327 in a single week. The peaks in initiation for all vaccines among 11-year-olds are centered on the beginning of the school year. For all four of the time series, there is a large drop in early 2020 when much of the state was shut down due to the COVID-19 pandemic. In 2021 and 2022, the sharp “back to school” peaks in vaccination among 11-year-olds are reduced for all three vaccines compared to pre-pandemic years (i.e., 2018 and 2019). HPV initiation at 9 increased modestly and began to take on a similar seasonal pattern centered around the school year in 2021. The increase we attribute to “Clinic B” in week 9 of 2021 and a sharp increase in initiations in the beginning of January 2023 are visible in the 9-year-old HPV time series, but not in any of the 11-year-old time series.

We plotted observed and model-predicted weekly counts of HPV initiation at age 9 to inspect model fit and visualize the meaning of each model. Figure 3 presents the observed and predicted weekly counts of HPV initiation among 9-year-olds for both the final effect and counterfactual models from 2021 to mid-2023. Both models included two pairs of sine and cosine terms, which were significant, indicating a good fit for seasonally repeating patterns. Shifting the forecaster indicator one to two weeks before and after did not improve fit. In the effect model, the weekly time trend was small (IRR: 1.002, 95% CI: 1.001–1.003), but also significant, indicating a small weekly increase in the rate of HPV initiations across the time period. The “Clinic B” term is also significant and corresponds to an IRR of 1.878 (95% CI: 1.591–2.218), meaning that, after controlling for seasonality, a weekly increasing trend and the later forecaster update, the “Clinic B” QI project and limited concurrent “HPV @ 9” education events increased the statewide weekly rate of HPV initiation 1.88 times the prior rate or an 88% increase. This corresponds to the approximate doubling of weekly HPV initiations in the four weeks before and after week 9 of 2021. Similarly, the WAIIS forecaster update in January 2023 increased the weekly rate of HPV initiations by 1.968 (95% CI: 1.760–2.200) times the weekly prior rate or a 97% increase after controlling for “Clinic B,” seasonality, and the underlying weekly trend. There was no significant change in the slope (i.e., the trajectory) of the time series following the forecaster update.

The difference between observed counts and predicted counts from the counterfactual model can be interpreted as the number of HPV initiations that were caused by the forecaster update. The counterfactual model is the same as the effect model but does not include the forecaster implementation variable and is limited to data from the weeks prior to the forecaster update. Following the implementation of the forecaster update (i.e., from week 3 to week 31 of 2023), there were 7,172 HPV initiations among 9-year-olds in Washington state. The counterfactual model predicted 3,608 initiations during this period, so 3,564 are attributable to the forecaster update – a 99% increase over what was expected.

From Figure 3, we can see that both models follow the observed time trend well. The residual deviance of the effect model was 138.3 with 127 degrees of freedom and the distribution of deviance residuals was approximately centered on 0 (median 0.1) indicating a good fit. Plots of deviance residuals did not indicate any lingering patterns. ACF and PACF plots of deviance residuals indicated an absence of lingering autocorrelation.

Discussion

Provider reminders to vaccinate is an evidence-based strategy for improving vaccine uptake, but a key knowledge gap of this strategy is a lack of published evaluation of this approach when deployed via a regional or state-level immunization information system (IIS).^20,24–27 Here, we describe increases in HPV initiation among 9-year-olds in Washington state following the implementation of a provider reminder through the state’s immunization registry system. After adjusting our model for other recent “HPV @ 9” projects, seasonal patterns, and underlying trends, we found that updating the forecast due date from age 11 to age 9 in the WAIIS registry doubled the rate of weekly HPV initiation among 9-year-olds.

There is an apparent decrease in “back to school” peaks of weekly vaccine administrations at age 11 in the years after 2020 (Figure 2). In addition to the major disruptions caused by the pandemic, 2020 was also when the Washington State Board of Health implemented a revised rule related to the previously mentioned Tdap mandate. Prior to 2020, Tdap was required for 6th grade attendance (typically ages 11–12), but beginning in 2020, Tdap was required for 7th grade attendance (typically ages 12–13). This change was planned in 2019 to mirror national guidance that Tdap doses administered to children ages 7–10 were invalid and needed to be repeated when 11–12 years old.²⁸ It was noted that many children started kindergarten early, resulting in them starting 6th grade at age 10 and this created an administrative burden on school staff trying to keep track of valid Tdap doses. National guidance was later revised to count Tdap doses administered at age 10 as valid, but the state rule has remained focused on 7th grade. We inspected a comparable weekly time series of 12-year-old HPV initiation (not published) and noted increased “back to school” peaks of HPV initiation counts in 2021 and later years compared to earlier years. This suggests that more children are delaying HPV initiation to correspond with the updated Tdap mandate. This underscores the importance of promoting earlier HPV initiation through other means.

We suspect the change to the Tdap mandate and changes in patterns of weekly administrations underpin declines in the coverage of Tdap, MenACWY initiation and HPV initiation among 11–12-year-olds in the years that followed 2020. For example, the 2019 coverage rate of having had one or more doses of Tdap, MenACWY and HPV among 11–12 year-olds was 60%, 51%, and 42%, respectively. This is noticeably higher than the previously described coverage rates of 48%, 41%, and 36% in 2022.¹¹ Of note, while the age of HPV initiation seems to have fluctuated since 2020, there have been no corresponding changes in up-to-date status of HPV coverage among 13–17-year-olds.¹¹

The coverage rates cited here may seem low compared to other data sources such as the Teen National Immunization Survey,⁴ but this is typical of long-lived administrative databases such as an immunization registry. Washington state has a mature immunization registry which has mostly complete data on state immunization administrations, but also has “fragmented” or “orphaned” patient records from duplicate records that are unable to be merged because of conflicting or incomplete identifying information and undocumented migration, where a patient leaves the state but an out-of-state address is never updated in the system. These issues lead to an inflated denominator and an underestimate of true coverage rates.¹⁸ Consequently, these coverage rates are difficult to compare to other data sources, but they are useful for assessing changes in coverage over time. This analysis sidesteps denominator inflation issues by focusing on counts of administrations and by using Census-based denominators to offset model-based IRRs.

This analysis points to the effectiveness of the forecaster update at increasing HPV initiation at 9 years old. But, compared to 11-year-olds, the HPV doses administered at age 9 are still low. There were, on average, 247 HPV initiations per week among 9-year-olds and 591 HPV initiations per week among 11-year-olds after the forecaster update (January 20 to August 14, 2023). To initiate HPV vaccination among all 9-year-olds in the state (approximately 101,000 youth), the mean number of weekly vaccinations in a year would have to approach 1,943. To achieve statewide coverage of earlier initiation, complementary evidence-based strategies are needed such as parent education, reminder/recall campaigns, and provider audits and training.

In practice, myriad factors can influence the effectiveness of forecaster schedule changes to drive vaccination uptake. Providers, for example, may need assistance updating EMR system prompts and policies to support earlier HPV initiation. EMR programming may supersede immunization information system (IIS) prompts or schedules, blocking a large swathe of providers from receiving IIS-based reminders. Patterns of IIS and EMR use by providers to deliver and document vaccination status are poorly understood and would benefit from further investigation.^29–31 Even if not immediately evident in the EMR, IIS-based provider reminders are effectively permanent and may drive slow shifts in standards and norms in vaccination practices. The WAIIS forecaster update also provides encouragement to healthcare providers to initiate HPV vaccination at age 9 who may otherwise feel restricted by the current ACIP recommendations. Further research should investigate how initiating HPV at 9 impacts completion rates by 13 years old, any changes in the distribution of HPV immunization timing across age groups and whether moving HPV to 9 years old affected MenACWY or Tdap coverage.

This paper uses interrupted time series regression techniques to adjust for seasonal patterns and infer a counterfactual estimate of how many HPV initiations would have occurred had the intervention not happened. A limitation of this approach is our inability to account for the effect of events that occur at the same time as the intervention and which also influence HPV initiation. Subject matter experts documented known interventions and we incorporated these into our model where it seemed appropriate and improved model fit, but there may be other undocumented concurrent events that influenced HPV initiations in January 2023. Another potential source of error is the use of row-level timestamps in the tables of WAIIS address history to determine historical addresses at the time of a vaccination. Timestamps may be updated for unrelated reasons, such as routine database management, which could contribute to erroneous historical address information.

Strengths of this paper include the use of a quasi-experimental design that provides a relatively rigorous effect estimate that accounts for seasonal effects and the engagement of partners to articulate a near-complete timeline of promotional and educational activities which may have also impacted the outcome.

The WAIIS forecaster change increased the weekly rate of HPV vaccinations at age 9 in Washington state. This increase has persisted in the 7 months since the forecast change. Jurisdictions that manage IIS registries should consider similar updates to their forecaster as part of a larger effort to increase HPV initiations at age 9. In addition to updating how patients are forecast for HPV vaccination, jurisdictions should implement complementary evidence-based strategies such as reminder/recall campaigns and clinic-based quality improvements to increase vaccine uptake. Ensuring parity of provider prompts between IIS and major EMR’s may additionally increase HPV vaccinations at age 9.

Acknowledgments

Jonathan Downs, Sahla Suman Thekke Edivettiyakath and Jeff Chorath, Washington State Department of Health.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Department of Health and Human Services, Centers for Disease Control and Prevention through the Immunization and Vaccines for Children grant and the Comprehensive Cancer Control grant.

Notes

a. The Washington HPV Free Task Force is a work group dedicated to increasing HPV vaccination rates and reducing the amount of HPV associated disease in Washington state by engaging and supporting diverse partners and increasing knowledge about HPV and cancer prevention.