Human Papillomavirus vaccination clinical decision support for young adults in an upper midwestern healthcare system: a clinic cluster-randomized control trial

ABSTRACT

Introduction

Human papillomavirus (HPV) vaccination rates are low in young adults. Clinical decision support (CDS) in primary care may increase HPV vaccination. We tested the treatment effect of algorithm-driven, web-based, and electronic health record-linked CDS with or without shared decision-making tools (SDMT) on HPV vaccination rates compared to usual care (UC).

Methods

In a clinic cluster-randomized control trial conducted in a healthcare system serving a largely rural population, we randomized 34 primary care clinic clusters (with three clinics sharing clinicians randomized together) to: CDS; CDS+SDMT; UC. The sample included young adults aged 18–26 due for HPV vaccination with a study index visit from 08/01/2018–03/15/2019 in a study clinic. Generalized linear mixed models tested differences in HPV vaccination status 12 months after index visits by study arm.

Results

Among 10,253 patients, 6,876 (65.2%) were due for HPV vaccination, and 5,054 met study eligibility criteria. In adjusted analyses, the HPV vaccination series was completed by 12 months in 2.3% (95% CI: 1.6%–3.2%) of CDS, 1.6% (95% CI: 1.1%–2.3%) of CDS+SDMT, and 2.2% (95% CI: 1.6%–3.0%) of UC patients, and at least one HPV vaccine was received by 12 months in 13.1% (95% CI: 10.6%–16.1%) of CDS, 9.2% (95% CI: 7.3%–11.6%) of CDS+SDMT, and 11.2% (95% CI: 9.1%–13.7%) of UC patients. Differences were not significant between arms. Females, those with prior HPV vaccinations, and those seen at urban clinics had significantly higher odds of HPV vaccination in adjusted models.

Discussion

CDS may require optimization for young adults to significantly impact HPV vaccination.

Trial Registration

clinicaltrials.gov NCT02986230, 12/6/2016.

KEYWORDS:

• Clinical decision support
• cluster randomized control trial
• human papillomavirus vaccination
• primary cancer prevention
• primary care
• shared decision making
• young adults

Introduction

Human papillomaviruses (HPV) contribute to 91% of cervical cancers and are the leading cause of anal, oropharynx, penile, vulvar, and vaginal cancers.^1,2 In the U.S., HPV-associated cancers cost an estimated $2–8 billion annually,^3,4 costs that have likely risen substantially.⁵ A 9-valent HPV vaccine is available in the U.S. for the strains that most frequently cause cancer and genital warts and is recommended by the Advisory Committee on Immunization Practices (ACIP) for all individuals aged 11–26, with initial doses available at age 9.² Two doses (0, 6–12 month schedule) of the HPV vaccine are recommended for those aged 9–14, and three doses (0, 1–2, 6 month schedule) for those aged 15–26 or for those with immunocompromising conditions.² Early vaccination against HPV is recommended because HPV infections are commonly acquired after adolescents and young adults become sexually active.⁶ While cases of HPV can clear naturally,⁶ failed attempts at vaccinating adolescents still leads to a large group of unvaccinated and unprotected young adults. Adults aged 27–45 are eligible for the three dose schedule and are encouraged to have a shared decision-making conversation with their provider about taking the HPV vaccine series.²

HPV vaccines are safe and adverse events are rare.^7–11 Yet only 49% of adolescents have completed an HPV vaccine series,¹² with lower rates in rural areas.^12,13 Males make up a small portion of the vaccinated population,^14–18 although HPV coverage rates have increased in men aged 19–26 in recent years from 8.2% in 2014 to 26.3% in 2018.^17,18 Still, young men aged 22–26 have lower rates (21.8%) than those aged 19–21 (34.4%),¹⁸ all of which are lower than the rates for women of comparable ages (19–26 = 52.8%, 19–21 = 53.3%, 22–26 = 52.5%).¹⁸ Vaccination rates are also lower in young adults than in adolescents.¹⁶

HPV vaccine uptake has been hampered by a number of barriers for both adolescents and young adults,¹⁹ such as knowledge gaps,^19–21 cost concerns,¹⁹ and negative perceptions of the HPV vaccine.¹⁹ Some individuals may still believe in common myths about HPV vaccines, such as: they are unsafe; evidence is lacking on their ability to prevent cancer; they have been inadequately tested or are too new; are not needed when Pap smears are available; or that the age range for vaccination starts too young.²² Young adults may experience additional barriers, such as recent findings that married men, and married or partnered young adults in general, are less likely to be vaccinated.^14,21 Education levels have also been shown to impact HPV vaccination series initiation in young adults, where those with college educations have significantly higher odds of starting the series than those with a high school diploma or less.¹⁶

Young adults aged 18–26 that visited the office of a doctor in the last year have been shown to have significantly greater odds of starting the HPV vaccination series compared to those without any visits.¹⁶ Yet others have reported that a major barrier to HPV vaccination uptake may be a lack of providers recommending HPV vaccination to eligible patients.²³ Furthermore, rural communities have been shown to lack resources and rural-focused interventions.²⁴ Interventions are needed,^24,25 like clinical decision support (CDS) for patient and provider shared decision-making, as well as shared decision-making tools (SDMT), also referred to as decision aids, that may educate young adults and counter misinformation.

In primary care clinics in the U.S., primary care clinicians (PCC) provide access to cancer preventive services, such as HPV vaccination. They do so within time-limited visits, navigating what may be conflicting prevention guidelines and competing demands.^26–36 As we described in a recent paper,³⁶ CDS systems may take some of the burden off PCCs by utilizing algorithms and electronic health record (EHR) data to identify patients that are due or overdue for HPV vaccination, then providing patient-tailored and intervention specific CDS that patients and PCCs can use during shared decision-making discussions regarding how to proceed.^36–40

Randomized control trials (RCT) on HPV vaccination CDS or EHR prompts for adolescents aged 11–17 present mixed results.^41–44 One study found no significant impact on vaccination rates in adjusted models.⁴¹ Others found that compared to usual care (UC), CDS modestly increased vaccination rates for those ages 11–13 and age 14,^42,43 and that clinician-focused CDS impacted HPV vaccination initiation and a family-focused intervention promoted series completion.⁴⁴

Research suggests that decision aids, like SDMT geared toward patients, may also assist with vaccine uptake. Scalia and colleagues recently conducted a systematic review of RCTs testing shared decision-making skills training and/or decision aid interventions on vaccination uptake and other patient outcomes.⁴⁵ This review included three studies related to HPV vaccination.⁴⁵ However, only two of these three studies used a decision aid, one aimed at mothers of adolescent females and another at parents of adolescents aged 9–18,^46,47 the latter of which included a supplemental file with screenshots of the decision aid that noted it was also for young adults up to age 26, although results for young adults were not included in the paper.⁴⁷

Research is needed on the impact of HPV vaccination CDS and SDMT for young adults aged 18–26. We designed web-based, algorithm-driven (following ACIP guidelines), and EHR-linked CDS for young adults due for HPV vaccination seen in primary care, a CDS that included other primary and secondary cancer prevention and screening items and cardiovascular risk reduction areas for eligible patients.³⁵ In addition, we developed a multi-page and HPV-specific SDMT for young adults aged 18-26.^31,35

The primary objective of this study was to test the treatment effect of HPV CDS with or without HPV SDMT on the percentage of young adult primary care patients aged 18–26 who completed the HPV vaccination series within 12 months after a study index visit in the CDS intervention arm clinics (one arm with and one arm without SDMT) compared to UC clinics. A secondary objective was to test the treatment effect of CDS with or without SDMT on additional HPV vaccinations received during the 12-month follow-up period for young adults without a complete course of HPV vaccinations at the study index visit. We hypothesized that compared to UC, by 12 months intervention arm patients would have significantly higher rates of: 1) completing the HPV vaccine series; and 2) having additional HPV vaccinations. We also tested the exploratory hypothesis that rates differed between intervention arms (CDS versus CDS+SDMT).

Methods

Design

We conducted a three-arm, parallel group, primary care clinic cluster-RCT at Essentia Health, an integrated healthcare system with clinics in Minnesota, North Dakota, and Wisconsin that serve a large rural Upper Midwestern population. Thirty-four clinics (clusters) (with three clinics sharing PCCs randomized together) were randomly allocated in a 1:1:1 ratio to one of three arms: CDS; CDS+SDMT; and UC (Figure 1).^{30,32,34–36}

Figure 1. Study CONSORT diagram.

† Ineligible for analysis includes patients with delayed documentation of screening (n = 460) and those whose age at follow-up was higher than ages included in algorithms (n = 694).

The original 18-month patient accrual period planned for this study was shortened to 7.5 months due to disruptions in healthcare, and vaccinations, from the SARS-CoV-2 pandemic. Shortening the accrual period to 7.5 months (8/01/2018-3/15/2019) allowed us to maintain the planned 12-month follow-up period for each index visit.

The Essentia Health Institutional Review Board reviewed, approved, and monitored this study, and granted a waiver of informed consent. An Independent Project Safety Officer monitored this study.

Study participants

The study population included patients receiving care from PCCs (physicians, nurse practitioners, and physician assistants) practicing in randomized primary care clinics. Eligibility criteria included being ages 18–26 and lacking a completed HPV vaccination series at the time of the index visit (the first visit at which a patient was eligible for HPV vaccination during the study accrual period) at a study clinic. The CDS algorithms determined patient eligibility for HPV vaccination from 8/01/2018-3/15/2019, and these patients served as the denominator for the analyses. Exclusion criteria included: not meeting eligibility criteria; pregnancy either at the time of the index visit or in the 12 months prior to the index visit; and having hospice, Alzheimer’s/cognitive impairment, or non-skin cancer codes in the EHR in the past 12 months.³⁵

Data sources and variable definitions

The CDS archived and stored EHR data for each visit in a protected data repository on or after the index visit. Stored data included patient age, sex, race, tobacco use, current medications, diagnosis and procedure codes, lab results, number of office visits, exclusion criteria, vaccinations completed, and vaccinations due. A separate EHR data pull provided patient ethnicity and insurance type. Regarding payment of HPV vaccinations in the study population, the Patient Protection and Affordable Care Act (ACA) requires that health insurance plans and issuers cover the entire cost of the HPV vaccination series for populations recommended for vaccination by the ACIP in the U.S.^2,48 However, some insurance types, like workers compensation and automobile accident insurance, may only cover costs associated with the covered injury and not preventive care. Patients lacking insurance would not have had coverage for preventive care like HPV vaccinations. The HPV vaccine was not provided for free as part of the RCT. Clinic descriptors used as control variables in adjusted analyses were obtained from healthcare system administrative records and clinic-level EHR summary data for patient visits occurring prior to randomization (10/2014-3/2016). We captured follow-up data for 12 months after each patient’s index visit between 8/1/2018-3/15/2020, ending the day before regional SARS-CoV-2 restrictions were announced.

Binary dependent endpoints (Yes/No) included: 1) HPV vaccination series completion; and 2) receipt of one or more HPV vaccine doses during the 12-month follow-up period. Receipt of three doses was considered vaccine series completion.² Outcomes were derived from manually triggered programmatic EHR data extractions capturing all EHR care recorded from 8/1/2018-3/15/2020. Because EHR data may be out of date, additional binary composite endpoints defined success as: HPV vaccination series completion (or receipt of one or more HPV vaccine doses) during the 12-month follow-up and/or new documentation in the EHR during follow-up that the HPV vaccination series was completed prior to the index visit, but was not documented at the time of the index visit. These binary composite endpoints were exploratory, rather than planned.

Interventions

The CDS was programed to provide primary (HPV vaccination, tobacco cessation, weight management) and secondary (breast, cervical, colorectal, and lung) cancer prevention recommendations for patients at average risk meeting eligibility criteria (e.g., not up to date on HPV vaccination or breast, cervical, colorectal, or lung cancer screening; tobacco use; obesity).³⁵ The cancer prevention and screening CDS areas, which followed United States Preventive Service Task Force (USPSTF)^49–52and ACIP guidelines,² were added to a cardiovascular risk reduction and glycemic control CDS that was already available in study intervention clinics as part of two other National Institutes of Health-funded studies.^30,31,35 For the patients in the present study, who were aged 18–26, cancer prevention and screening CDS components were limited to: HPV vaccination; tobacco cessation and weight management if either applied and patients were overdue for HPV vaccination; and cervical cancer screening CDS recommendations for women aged 18 to 26 who were due or overdue for cervical cancer screening.³⁵ However, there was no SDMT related to cervical cancer screening in this study.³⁵

The CDS included patient and PCC printed handouts and an electronic EHR interface for PCCs.^{30,31,34–36} Rooming staff triggered the CDS by entering patients’ vital signs into the EHR. CDS algorithms used EHR data to identify and display an alert for overdue patients. We instructed rooming staff to print the patient and PCC handout(s), give the patient handout to the patient, and place the PCC handout on the exam room door or hand it to the PCC. CDS materials were to be printed regardless of the reason for the visit. For patients eligible for HPV vaccination, patient versions of the CDS printout stated: “Talk to your doctor about whether you are due for an HPV vaccine,” and the PCC printout stated: “HPV vaccine may be due,” both of which were under a “Cancer Prevention” heading. Examples of the CDS printouts are included in a recent paper by Elliott et al.³⁵ The CDS included a two-week suppression window after each eligible visit to prevent alert fatigue.^31,35

In the CDS+SDMT intervention arm, patients also received abbreviated printed information on HPV vaccination for use in shared decision-making (see Appendix 1).^31,35 The electronic EHR interface gave PCCs access to a multi-page HPV vaccination SDMT in the CDS+SDMT intervention arm that PCCs could print for patients (see Appendix 2). The multi-page SDMT described basic information on HPV and HPV vaccines, HPV-associated genital warts and cancers, vaccine risks and benefits, and statements that patients could consider when deciding whether to take the vaccine. The multi-page SDMT was originally to be printed with the CDS for eligible patients; however, we found during pilot testing of the CDS and SDMT (including breast, colorectal, and lung cancer SDMT for patients eligible for those screenings) in two non-RCT healthcare system clinics that the amount of paper printed was a burden on clinic workflow, resulting in the CDS+SDMT intervention arm instead automatically printing the abbreviated HPV vaccination information along with the CDS patient and PCC printouts for eligible patients. UC clinics did not receive the CDS or SDMT but already had a PCC EHR alert for patients eligible for HPV vaccination. Unlike the current EHR alert, which only alerted the PCC that a patient was due for an HPV vaccination, the CDS alert provided printed material for both eligible patients and PCCs. Intervention clinic PCCs were also encouraged during training to use the CDS during a shared decision-making discussion with the patient about taking the vaccine series. In the CDS+SDMT arm, patients also received the abbreviated SDMT on HPV vaccination, and PCCs could choose to print the multi-page SDMT through the electronic CDS interface that they could then give to patients.

Rooming staff, PCC, and clinic management training was multi-pronged and ongoing.^31,35 We developed an e-learning module that was assigned to intervention clinic staff using the e-learning platform utilized by the healthcare system for disseminating system-wide training.³¹ We also conducted baseline and booster in person training sessions at clinics,³⁵ where members of the research team demonstrated the CDS (and the SDMT for the CDS+SDMT intervention arm clinics) and talked about how the CDS aligned with institutional aspirational aims, quality measures, team model of care, and was based on the latest recommendations from respected groups like the USPSTF and ACIP.³¹ Furthermore, we conducted two training webinars that were recorded and made available to intervention clinics.³¹ Research team project managers regularly communicated with clinic management and provided individualized monthly update reports to each intervention clinic showing CDS print rates for the clinic and individual PCCs.³¹ The team also initiated a lunch and learn incentive program for clinics that met print rate goals for sustained periods. A “suggested improvements” button was built into the CDS and users were encouraged to reach out to research team project managers when any issues were encountered with the CDS.³⁰

Sample size and power

We based power calculations on preliminary data from 11,970 patients ages 18–26 eligible for HPV vaccination with visits to study clinics over 18 months. Using original power calculation assumptions, but with an updated sample size of 5,054 reflecting patient accrual over 7.5 months, this study was powered at 80% (2-sided test p < .025) to detect a difference in the percentage of patients up-to-date for the composite screening endpoint of 20% in either intervention arm compared to 30% in UC. Power calculations assumed 10 clinics per study arm with equal patient counts (n = 168/clinic), ICC = .02 for the primary endpoint, and a generalized linear mixed model for analysis.

Statistical analysis

Analyses included descriptive statistics and unadjusted and adjusted generalized linear mixed models with a logit link and binomial error distribution in testing primary and secondary hypotheses. Adjusted models included two terms specifying the planned primary contrasts of CDS versus UC and CDS+SDMT versus UC, pre-specified covariates, and a random intercept for clinic. Clinic-level covariates included the percentage of patients up-to-date on breast cancer screening prior to intervention start (representing a measure of the attentiveness of the clinic to screening) and clinic rurality based on primary Rural-Urban Commuting Area codes (urban 1–6 [metropolitan-micropolitan], rural 7–10 [small town-rural]).⁵³ These two covariates were chosen due to their inclusion in the clinic randomization process.⁵⁴ Patient-level covariates included age at index visit, sex (male/female), race, number of HPV vaccinations as of the index date, and insurance type (private insurance or Medicare for those eligible for Medicare; public payors Medicaid, Federal, or Indian Health Services; unknown or other insurance types [workers compensation, government other, automobile accident insurance, program, other]). Exploratory analyses contrasted the CDS and CDS+SDMT study arms. Tests of the two planned contrasts were two-sided with p-values less than .025 considered statistically significant. Due to multiple comparisons, we report 97.5% confidence intervals (CIs) for odds ratio estimates for key study arm contrasts, and 95% CIs for all other estimates. Heterogeneity of treatment effects analyses tested differences in the magnitude of the intervention effect by patient sex with the inclusion of a study arm by patient sex interaction term in the main analytic models.

Results

In total, 10,543 patients aged 18–26 had index visits at study clinics over the 7.5-month accrual period and were assessed for eligibility for HPV vaccination by the CDS algorithms (Figure 1). Of these patients, 6,876 (65%) were due for HPV vaccination (e.g., were eligible for the HPV vaccination per ACIP guidelines for adults aged 18–26 and had either no prior HPV vaccination doses on record in the EHR or were due for another dose in the series). After exclusions, the analytic sample for the primary analysis concerning completion of an HPV vaccination series within the 12-month follow-up period consisted of 5,054 patients, of whom 5.6% completed the vaccine series (6.0% CDS, 4.7% CDS+SDMT, 6.1% UC). Of those 5,054 patients, 3,910 (77.4%) had no HPV vaccinations at the time of the index visit, while 604 (12.0%) had one HPV vaccination dose, and 540 (10.7%) had two HPV vaccination doses (Table 1). Those without any HPV vaccination doses at the index visit had the lowest rate (1%) of completing the vaccine series by 12 months, while those with one dose and two doses at the index visit had completion rates of 8% and 34%, respectively.

Table 1. The effect of clinical decision support with or without shared decision-making tools on HPV vaccination compared to usual care in young adults ages 18–26 in the 12 months following a study index visit from 08/01/2018–03/15/2019 in randomized primary care study clinics: patient characteristics (N = 5,054) and study clinic characteristics (N = 34) by study arm

Characteristic	CDS (n = 1,511) n (%)	CDS+SDMT (n = 1,470) n (%)	UC (n = 2,073) n (%)
Patient-level summaries
Age (years)^a	21.5 (2.3)	21.5 (2.4)	21.5 (2.4)
Female	665 (44.0)	631 (42.9)	879 (42.4)
Race
African American	54 (3.6)	48 (3.3)	71 (3.4)
Asian	13 (.9)	14 (1.0)	16 (.8)
American Indian/Alaska Native	47 (3.1)	23 (1.6)	20 (1.0)
Native Hawaiian/Other Pacific Islander	4 (.3)	2 (.1)	7 (.3)
White	1361 (90.1)	1348 (91.7)	1915 (92.4)
Unknown	32 (2.1)	35 (2.4)	44 (2.1)
Hispanic ethnicity	27 (1.8)	39 (2.7)	55 (2.7)
Insurance type at index visit
Private	1028 (68.0)	1020 (69.4)	1283 (61.9)
Medicare^b	9 (.6)	12 (.8)	13 (.6)
Public payor^c	336 (22.2)	298 (20.3)	546 (26.3)
Other^d	14 (.9)	19 (1.3)	28 (1.4)
Unknown	124 (8.2)	121 (8.2)	203 (9.8)
Any outpatient visits in prior year	345 (22.8)	314 (21.4)	457 (22.1)
Current smoker	251 (16.6)	176 (12.0)	344 (16.6)
Number of HPV vaccinations as of index date
0	1,172 (77.6)	1,155 (78.6)	1,583 (76.4)
1	183 (12.1)	159 (10.8)	262 (12.6)
2	156 (10.3)	156 (10.6)	228 (11.0)
Clinic-level summaries at the time of randomization	(n = 11 clinics)	(n = 11 clinics)	(n = 12 clinics)
Urban clinic location	3 (27.3)	3 (27.3)	3 (25.0)
Unique patient visits at randomization^a	5,530.3 (2,376.0)	5,295.1 (3,930.7)	6,651.9 (4,055.6)
Percentage of patients with Medicaid coverage^a	17.5 (7.6)	17.1 (4.9)	18.0 (5.7)
Percentage of patients who are smokers^a	14.0 (2.6)	15.2 (3.3)	15.3 (3.1)
Percentage of patients up-to-date on breast cancer screening^a	74.9 (8.5)	77.2 (6.2)	79.3 (5.1)

CDS = Clinical decision support. SDMT = Shared decision-making tool. UC = Usual care.

^aMean (SD).

^bFor those receiving Social Security Disability Insurance or otherwise eligible for Medicare.

^cMedicaid, Federal, and Indian Health Services.

^dOther, workers compensation, government other, automobile, program.

Patients included in the analysis had a mean age of 21.5 (standard deviation [SD] = 2.4) years, 43.0% were women, 91.5% were White, 3.4% were African American, 1.8% were American Indian/Alaska Native, .9% were Asian, .3% were Native Hawaiian/Other Pacific Islander, 2.4% were Hispanic, 15.3% were current smokers, and 22.1% had an outpatient visit in the year prior to the index visit (Table 1). The majority of patients (90%) had insurance that would have covered HPV vaccination costs under the ACA, and the other 10% may or may not have had HPV vaccination coverage under the ACA. The mean time from the index visit to the 12-month follow-up data pull was 382.2 days (SD = 8.1). Table 1 also presents clinic-level summary information for the randomized clinics. The clinics’ aggregated attributes were similar across the three study arms. More clinics (74%) were designated rural than urban (26%).

We documented CDS activation and printing by rooming staff and PCCs. They opened the electronic interface for 45.4% of eligible patients in the CDS arm and 58.6% in the CDS+SDMT arm, and printed materials for 43.8% of eligible patients in the CDS arm and for 58.2% in the CDS+SDMT arm.

Completion of an HPV vaccination series by 12 months

Among 5,054 patients due for HPV vaccination, 285 (5.6%) completed their HPV vaccination series by 12 months. The adjusted model-derived percentage of patients with a completed vaccination series was 2.3% (95% CI = 1.6%-3.2%) in the CDS arm, 1.6% (95% CI = 1.1%-2.3%) in the CDS+SDMT arm, and 2.2% (95% CI = 1.6%-3.0%) in the UC arm (Table 2). The adjusted odds ratio was 1.04 (97.5% CI = .68–1.59, p = .827) for the planned contrast comparing the CDS versus UC arms and was .71 (97.5% CI = .46–1.12, p = .086) for the CDS+SDMT versus UC arms. Neither met the pre-specified alpha of .025 in unadjusted or adjusted analyses. In the adjusted analysis of the composite endpoint examining a completed HPV vaccination series and/or new documentation of a completed HPV vaccination series prior to the index date, the model-derived percentage for the endpoint ranged from 9.2% in the CDS+SDMT arm to 11.3% in the CDS arm, with no statistically significant differences found. The adjusted odds ratio for the exploratory contrast comparing vaccination completion in the CDS versus CDS+SDMT arms was 1.46 (95% CI = .96–2.21, p = .077) (not shown in Table 2).

Table 2. The effect of clinical decision support with or without shared decision-making tools on HPV vaccination compared to usual care in young adults ages 18–26 in the 12 months following a study index visit from 08/01/2018–03/15/2019 in randomized primary care study clinics: patient predicted percentages and between arm comparisons of patients either completing a HPV vaccination series or receiving at least one new vaccination dose in the 12 months following a study index visit date by study arm

	Model-Derived Estimates by Study Arm^a			CDS vs. UC		CDS+SDMT vs. UC
Model	CDS (95% CI)	CDS+SDMT (95% CI)	UC (95% CI)	OR (97.5% CI)	p	OR (97.5% CI)	p
Completion of a HPV vaccination series by 12 months following a study index visit date
	(n = 1,511)	(n = 1,470)	(n = 2,073)
Unadjusted^b	5.7% (4.3%-7.5%)	4.4% (3.2%-6.0%)	5.8% (4.5%-7.4%)	0.99 (.62–1.57)	0.942	0.76 (.46–1.24)	0.195
Adjusted^c	2.3% (1.6%-3.2%)	1.6% (1.1%-2.3%)	2.2% (1.6%-3.0%)	1.04 (.68–1.59)	0.827	0.71 (.46–1.12)	0.086
	(n = 1,661)	(n = 1,600)	(n = 2,253)
Composite endpoint^d,f	11.3% (9.7%-13.2%)	9.2% (7.7%-10.9%)	10.4% (9.0%-12.0%)	1.10 (.85–1.43)	0.379	0.88 (.67–1.1)	0.245
At least one HPV vaccination in the 12 months following the study index visit date
	(n = 1,511)	(n = 1,470)	(n = 2,073)
Unadjusted^b	15.8% (12.8%-19.4%)	11.6% (9.1%-14.6%)	14.3% (11.6%-17.4%)	1.13 (.76–1.68)	0.477	0.79 (.52–1.19)	0.178
Adjusted^c	13.1 (10.6%-16.1%)	9.2% (7.3%-11.6%)	11.2% (9.1%-13.7%)	1.19 (.82–1.74)	0.282	0.80 (.55–1.18)	0.191
	(n = 1,661)	(n = 1,600)	(n = 2,253)
Composite endpoint^e,f	21.1% (18.2%-24.4%)	16.3% (13.7%-19.1%)	18.6% (16.1%-21.4%)	1.17 (.88–1.57)	0.206	0.85 (.63–1.15)	0.211

Note. Boldface indicates statistical significance (p < .025). Patients (N = 5,054) and Study Clinics (N = 34). CDS = Clinical decision support. HPV = Human papillomavirus. SDMT = Shared decision-making tool. UC = Usual care.

aPredicted percentages from a generalized linear mixed model.

bUnadjusted models include fixed effects for study arm and a random intercept for clinic.

cAdjusted models include fixed effects for study arm, clinic-level (urbanicity, percent of patients up-to-date on breast cancer screening at baseline), patient-level (age, sex, race, insurance type, number of completed vaccinations as of index date), and a random intercept for clinic.

dHPV vaccination series complete and/or new documentation of HPV vaccination series completed prior to the index date.

eAny HPV vaccination and/or new documentation of HPV vaccination series completed prior to the index date.

fAdjusted models include fixed effects for study arm, clinic-level (urbanicity, percent of patients up-to-date on breast cancer screening at baseline), patient-level (age, sex, race, insurance type), and a random intercept for clinic.

Patients in urban clinics had significantly higher odds of completing the HPV vaccine series than those in more rural clinics (aOR = 1.75, 95% CI = 1.28–2.39, p = .001), as did females versus males (aOR = 2.55, 95% CI = 1.92–3.40, p <.001) (Table 3). Compared to those with private insurance or Medicare, those with unknown or other insurance types had significantly lower odds of completing the series (aOR = .30, 95% CI = .15–.62, p = .001). Compared to patients with no vaccinations at the time of the index date, those with one vaccination dose were significantly more likely to complete the series (aOR = 5.70, 95% CI = 3.80–8.55, p < .001) as were those with two vaccination doses (aOR = 34.56, 95% CI = 24.74–48.29, p <.001).

Table 3. The effect of clinical decision support with or without shared decision-making tools on HPV vaccination compared to usual care in young adults ages 18–26 in the 12 months following a study index visit from 08/01/2018–03/15/2019 in randomized primary care study clinics: independent variables in the adjusted models presented in Table 2

Independent Variables	OR (95% CI)	p
Completion of the HPV vaccination series by 12 months following a study index visit date
Study arm^a
CDS	1.04 (.72–1.51)	0.827
CDS+SDMT	0.71 (.49–1.05)	0.086
Clinic urban status^b	1.75 (1.28–2.39)	0.001
Clinic percentage of women up-to-date on breast cancer screening	1.01 (.98–1.04)	0.604
Female patient	2.55 (1.92–3.40)	<0.001
Age at index visit	0.97 (.91–1.03)	0.265
Patient race^c
Nonwhite	1.64 (.96–2.81)	0.072
Unknown	1.96 (.85–4.50)	0.113
Insurance type^d
Public payor	0.72 (.52–1.00)	0.053
Unknown/other	0.30 (.15–.62)	0.001
Number of HPV vaccinations as of index date^e
1	5.70 (3.80–8.55)	<0.001
2	34.56(24.74–48.29)	<0.001
At least one HPV vaccination in the 12 months following the study index visit date
Study arm^a
CDS	1.19 (.86–1.65)	0.282
CDS+SDMT	0.80 (.58–1.12)	0.191
Clinic urban status^b	1.48 (1.12–1.96)	0.008
Clinic percentage of women up-to-date on breast cancer screening	1.00 (.97–1.03)	0.951
Female patient	2.27 (1.91–2.69)	<0.001
Age at index visit	0.86 (.83–.89)	<0.001
Patient race^c
Nonwhite	1.14 (.80–1.61)	0.476
Unknown	1.81 (1.09–3.00)	0.023
Insurance type^d
Public payor	0.82 (.67–1.01)	0.056
Unknown/other	0.44 (.30–.64)	<0.001
Number of HPV vaccinations as of index date^e
1	4.36 (3.53–5.37)	<0.001
2	3.97 (3.20–4.94)	<0.001

Note. Boldface indicates statistical significance (p < .05). Patients (N = 5,054) and Study Clinics (N = 34). CDS = Clinical decision support. Public payor = Medicaid, Federal, Indian Health Services. SDMT = Shared decision-making tool. Unknown/other = Blank/missing, workers compensation, government other, automobile, program, other.

aReference = Usual care.

bReference = Rural.

cReference = White.

dReference = Private insurance or Medicare.

eReference = 0.

A heterogeneity of treatment effects analysis assessing differences in the magnitude of the intervention effect on completion of the HPV vaccine series by patient sex indicated that the CDS arm versus UC arm comparison did not differ by patient sex (women: aOR = 1.02, 95% CI = .67–1.57; men: aOR = 1.07, 95% CI = .61–1.88; interaction p = .893) and that the CDS+SDMT arm versus UC arm comparison did not differ by patient sex (women: aOR = .71, 95% CI = .45–1.11; men: aOR = .73, 95% CI = .41–1.31; interaction p = .925).

At least one new HPV vaccination by 12 months

Among 5,054 patients due for HPV vaccination, 754 (14.9%) had received at least one HPV vaccination during the 12-month follow-up. As shown in Table 2, the adjusted model-derived percentage of patients with at least one HPV vaccination was 13.1% (95% CI = 10.6%-16.1%) in the CDS arm, 9.2% (95% CI = 7.3%-11.6%) in the CDS+SDMT arm, and 11.2% (95% CI = 9.1%-13.7%) in the UC arm. The adjusted odds ratio was 1.19 (97.5% CI = .82–1.74, p = .282) for the planned contrast comparing the CDS versus UC arms and was .80 (97.5% CI = .55–1.18, p = .191) for CDS+SDMT versus UC. Neither unadjusted nor adjusted analyses met the pre-specified alpha of .025. The adjusted odds ratio for the exploratory contrast comparing at least one HPV vaccination in the CDS versus CDS+SDMT arms was 1.48 (95% CI = 1.05–2.10, p = .028) (not shown in Table 2).

In the adjusted analysis of the composite endpoint examining any HPV vaccination and/or new documentation of a completed HPV vaccination series prior to the index date, the adjusted, model-derived percentages for the endpoint were as follows: 21.1% (95% CI = 18.2%-24.4%) in the CDS arm; 16.3% (95% CI = 13.7%-19.1%) in the CDS+SDMT arm; and 18.6% (95% CI = 16.1%-21.4%) in the UC arm (Table 2). There were no statistically significant differences seen by study arm.

Patients in urban clinics had significantly higher odds of at least one HPV vaccination during the 12-month follow-up period than those in rural clinics (aOR = 1.48, 95% CI = 1.12–1.96, p = .008) (Table 3). Females also had significantly higher odds than males (aOR = 2.27, 95% CI = 1.91–2.69, p <.001), and a one-year increase in age was associated with a 14% decrease in the odds that patients would have at least one new HPV vaccination (aOR = .86, 95% CI = .83–.89, p < .001). Compared to white patients, those with unknown race were more likely to have at least one HPV vaccination (aOR = 1.81, 95% CI = 1.09–3.00, p = .023). Patients with unknown or other insurance types had lower odds compared to those with private insurance or Medicare (aOR = .44, 95% CI = .30–.64, p < .001). Compared to patients with no vaccination doses at the time of the index date, those with one (aOR = 4.36, 95% CI = 3.53–5.37, p < .001) or two (aOR = 3.97, 95% CI = 3.20–4.94, p < .001) vaccination doses had higher odds of at least one additional vaccination during follow-up.

A heterogeneity of treatment effects analysis assessing differences in the magnitude of the intervention effect on receipt of at least one HPV vaccination by patient sex indicated that the CDS arm versus UC arm comparison did not differ by patient sex (women: aOR = 1.23, 95% CI = .86–1.75; men: aOR = 1.15, 95% CI = .77–1.70; interaction p = .739) and that the CDS+SDMT arm versus UC arm comparison did not differ by patient sex (women: aOR = .77, 95% CI = .53–1.11; men: aOR = .86, 95% CI = .57–1.29; interaction p = .595).

Discussion

In a clinic cluster-RCT of a patient-tailored, point-of-care CDS intervention for HPV vaccination in young adults aged 18–26, we found no statistically significant impact on HPV vaccination in either the CDS or the CDS+SDMT intervention arms compared to UC in the 12 months following a study index visit. HPV vaccination rates were low across all study arms. Patients had slightly higher predicted percentages of having at least one HPV vaccination compared to completing the full series. In adjusted models, we found that having one and two doses already completed at the study index visit significantly increased the odds that patients would complete the vaccine series during follow-up compared to those without any initial vaccine doses. We also found that females were more likely than males to complete the full series and have at least one HPV vaccination during follow-up. This is similar to other research comparing HPV vaccination uptake by sex.^{14–18,55,56} However, we found no difference in the magnitude of the intervention effect on HPV vaccination uptake after adding an interaction term between study arm and patient sex to adjusted models. As previously reported,^12,13 patients seen in rural clinics had lower likelihoods of vaccination than those in urban clinics. Younger patients in the study were also more likely to receive an additional vaccination dose, similar to prior research.^16,18 Our finding that patients with other types of health insurance had lower HPV vaccination odds is unsurprising, as some of these insurance types (e.g., workers compensation and automobile accident insurance) would not be required to cover unassociated visit costs by the ACA. Of note, only 1.2% of participants had other types of insurance in the present study.

We observed that the arm receiving the CDS+SDMT intervention had directionally lower rates of HPV vaccination than those receiving the CDS alone, and significantly lower rates of receipt of at least one additional dose, although intervention print rates were 14.8% higher in the CDS+SDMT arm than in the CDS arm. PCCs reported higher satisfaction with the CDS+SDMT intervention than the CDS intervention alone (CDS arm = 75% very/somewhat satisfied; CDS+SDMT arm = 83.6% very/somewhat satisfied) in a recently published study survey.³⁶ This suggests the need for a better understanding of how SDMT affect HPV vaccination decisions in patient-centered care.

Unfortunately, RCT on CDS and SDMT for HPV vaccination amongst young adults seen in a general primary care population are rare. While some prior research supported CDS and reminder prompts for HPV vaccination with adolescents,^42–44 other research showed similar results as the present study. One study presented conflicting results, with generally low HPV vaccination rates and lower rates in a CDS intervention group for children aged 13 compared to historical UC.⁴² Yet in the same study, the last year of the CDS intervention period included “4 of the 5 highest quarterly rates” (p. 2) of HPV vaccination series completion for children aged 14.⁴² Also, a recent RCT of an automated EHR reminder prompting clinicians when patients were due for a second or third HPV vaccination dose reported the intervention was associated with significantly higher vaccination rates than UC in unadjusted models for adolescents aged 11–17.⁴¹ Nevertheless, adjusted models showed no significant differences in vaccination rates.⁴¹ Factors other than CDS or EHR reminders likely impacted HPV vaccination uptake. For example, recent research suggests that renewed efforts at public health campaigns on HPV vaccination in the U.S. may be needed, as the awareness of HPV and HPV vaccination has dropped in recent years both overall and to a greater extent within some groups, such as adult males, individuals living rurally, and racial minority groups.⁵⁷

Limitations

Our sample, and the healthcare system population, was over 90% white, limiting generalizability to more diverse populations. We excluded patients aged 11–17, who received a separate EHR alert. We shortened the accrual period to remove the impact of the SARS-CoV-2 pandemic adversely impacting vaccinations, although power was maintained. Consequently, we were unable to examine how the CDS and the CDS with SDMT may have impacted vaccination rates over a longer accrual period (e.g., 18 months) or following the resumption of in person healthcare visits during the SARS-CoV-2 pandemic. We did not restart accrual after the end of regional healthcare restrictions, as telemedicine visits became common within the healthcare system and vaccination rates declined institutionally. We also lacked data on HPV vaccination completion outside of the healthcare system either before or after an index visit that was not updated in the healthcare system’s EHR in the 12-month follow-up period after an index visit.

Pragmatic clinical care trials provide evidence of how interventions function in real world practice, but have weaknesses that may affect robustness, like noncompliance.^58,59 In a recent survey of PCCs practicing in intervention arm clinics in this study, majorities of PPCs agreed that the CDS alerted them to patients they did not know were overdue for a cancer screening (57% in the CDS arm, 68% in the CDS+SDMT arm); however, PCCs gave low self-reported CDS use rates, with only 18% of CDS arm and 19% of CDS+SDMT arm PCCs responding that they always or usually used the CDS paper printouts, with even fewer (12% and 20%, respectively) agreeing that they always or usually used the electronic version of the CDS.³⁶ In addition, intervention clinics experienced a variety of printing issues during the study, primarily related to printer driver, firmware, and network connection errors, which may have adversely affected CDS use and fidelity to the intervention.^30,31,34 However, research team members and healthcare system information services troubleshot these issues.^30,31,34 The CDS alone may also have lacked the strength to alter vaccination noncompliance in young adults, as it only notified patients and PCCs that patients were due. Furthermore, pre-implementation interviews with PCCs suggested some PCCs had doubts about the HPV vaccine.³⁰ Yet qualitative interviews with 37 patients in the present RCT’s CDS intervention arms showed that patients found many benefits in the CDS for cancer prevention and screening generally.⁶⁰ The qualitative results from those interviews also suggested that patients were more likely to make a cancer prevention and screening decision when their PCC reviewed the CDS with them than when the PCC did not use the CDS in the appointment, presenting an area for future research.⁶⁰ Previous research has highlighted how providers recommending HPV vaccination can significantly increase HPV vaccination uptake.⁶¹

Lack of use of the CDS within intervention arm clinics may be the main factor impacting our non-significant findings, as suggested by our recently published PCC survey paper.³⁶ The results of this survey suggested that relying on print rates alone does not provide enough information on if and how the CDS and SDMT were used in practice. As we reported in a previous paper, of the four patients aged 18–26 due for an HPV vaccine and seen in intervention arm clinics who took part in qualitative interviews, one received an HPV vaccination, one did not make a choice, and two expressed opinions against HPV vaccination and chose not to receive the vaccine.⁶⁰ Three of the four also responded that they discussed HPV vaccination with their PCC during the visit.⁶⁰ However, capturing how PCCs and patients used the CDS and SDMT for HPV vaccination within intervention clinics was beyond the scope of the present study. Future research is needed to understand how often and in what ways PCCs and patients utilize these sorts of tools during clinic visits and how that use may impact HPV vaccination rates for young adults.

Another limitation to this study is that not all clinics met the target print rate, and one intervention clinic stopped printing the CDS at times during the accrual period. Workflows could also differ by clinic.³¹ For example, while a few clinics had printers within exam rooms, most healthcare system clinics in this study had centralized printers, and printing the CDS required rooming staff to leave the room in these cases, retrieve the printed materials, and bring them back to the patient’s exam room, a process that may not have been followed consistently in practice.

When reviewing internal CDS use tracking in our RCT, we found that the multi-page HPV vaccine SDMT available for PCCs to print electronically in the CDS+SDMT intervention arm was rarely accessed and printed. If patients had had direct access to the multi-page SDMT, the SDMT may have assisted with addressing previously identified knowledge gap barriers to HPV vaccination.^19,20 The SDMT may also have helped dispel myths regarding the HPV vaccine,²² as was suggested by healthcare system PCCs in key informant interviews conducted prior to HPV CDS and SDMT implementation.³⁰ Ensuring education on HPV and HPV vaccination is received by young adults, for example through brief educational interventions, may help improve vaccination uptake in this group.⁶² As more clinical care moves online, embedding CDS and SDMT within electronic patient portals would give patients more control of the receipt of these tools and the ability to review materials outside of appointments. Mobile health (mHealth) also has the potential to increase HPV vaccination in both adolescents and young adults.⁶³ However, more replication research is needed in broader populations to adapt and maximize mHealth and other electronic direct-to-patient approaches for disseminating HPV vaccination information to young adults.⁶³ Research could also include young adults and clinic staff in the development of electronic tools and decision aids aimed at increasing HPV vaccination uptake.

Conclusion

HPV vaccination, a clinically effective strategy for reducing cancer burden, ranks above average in terms of population health impact and cost-effectiveness among other preventive services.⁶⁴ We identified large gaps in the percentages of young adults aged 18–26 not up-to-date on HPV vaccination in the 12 months following a study index visit in primary clinics within one Upper Midwestern healthcare system. Unfortunately, a point-of-care CDS with or without SDMT did not significantly improve HPV vaccine uptake in young adults compared to UC. Future research could assess whether alternative CDS content, a more engaging format, or a different workflow strategy would be more effective.

Acknowledgements

The authors thank Essentia Health and HealthPartners Institute, and Austin Land, Essentia Health Research Informatics Analyst II, for assistance with data collection and transfer.

The research presented in this paper is that of the authors and does not reflect the official policy of the NIH.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2022.2040933.

Disclosure statement

No potential competing interest was reported by the authors.

Additional information

Funding

This work was supported by the National Cancer Institute under Grant R01CA193396.