Real-world impact and effectiveness assessment of the quadrivalent HPV vaccine: a systematic review of study designs and data sources

ABSTRACT

Introduction

Vaccine effectiveness and impact studies are typically observational, generating evidence after vaccine launch in a real-world setting. For human papillomavirus (HPV) vaccination studies, the variety of data sources and methods used is pronounced. Careful selection of study design, data capture and analytical methods can mitigate potential bias in such studies.

Areas covered

We systematically reviewed the different study designs, methods, and data sources in published evidence (1/2007–3/2020), which assessed the quadrivalent HPV vaccine effectiveness and impact on cervical/cervicovaginal, anal, and oral HPV infections, anogenital warts, lesions in anus, cervix, oropharynx, penis, vagina or vulva, and recurrent respiratory papillomatosis.

Expert opinion

The rapid growth in access to real-world data allows global monitoring of effects of different public health interventions, including HPV vaccination programs. But the use of data which are not collected or organized to support research also underscore a need to develop robust methodology that provides insight of vaccine effects and consequences of different health policy decisions. To achieve the WHO elimination goal, we foresee a growing need to evaluate HPV vaccination programs globally. A critical appraisal summary of methodology used will provide timely guidance to researchers who want to initiate research activities in various settings.

KEYWORDS:

• Human papillomavirus
• real-world evidence
• study design
• vaccine effectiveness
• vaccine impact

1. Introduction

Human papillomavirus (HPV) infections can resolve spontaneously without causing disease. However, in some individuals, persistent high-risk HPV infections can ultimately cause progressive disease, leading to cancers of the cervix, vagina, vulva, penis, anus, and cancerous lesions of the oropharynx [1,2]. Furthermore, low-risk HPV infection, primarily types 6 and 11, can cause anogenital warts (AGWs) and recurrent respiratory papillomatosis (RRP).

Four vaccines have been developed toward primary prevention of infection with certain HPV types and related diseases: two bivalent (2vHPV, HPV16/18, Cervarix®, GlaxoSmithKline, licensed in 2007; 2vHPV, HPV16/18, Cecolin®, Xiamen Innovax Biotech Co, approved by the Chinese Medical Products Administration in 2019), a quadrivalent (4vHPV, HPV6/11/16/18, Gardasil/Silgard®, Merck & Co. Inc., Kenilworth, Nj, USA, licensed in 2006), and a nonavalent vaccine (9vHPV, HPV6/11/16/18/31/33/45/52/58, Gardasil 9®, Merck & Co. Inc., Kenilworth, NJ, USA, licensed in 2014) [3,4].

Once a vaccine is approved and used in a vaccination program, monitoring of the vaccine’s benefits becomes an integral part of its life cycle [5–7]. The effects of vaccination can be evaluated and defined either as vaccine effectiveness or as vaccine impact. In vaccine effectiveness (VE) studies, the direct effect of vaccination is measured at the individual level by comparing the occurrence of the vaccine-preventable outcomes between vaccinated and unvaccinated individuals from the same population – this means that the individual vaccination status for each study participant has to be known within the study. In vaccine impact studies, the vaccine’s combined direct and indirect effects are measured at the population level by comparing the occurrence of the preventable outcomes before and after vaccine introduction, agnostic of the vaccination status of each individual [5].

In contrast to clinical trials, which are performed on highly selected, preferably uninfected populations, vaccine effectiveness and impact studies are observational, generating complementary evidence of vaccine effects in settings in which individuals are vaccinated as recommended by the health authorities – with some variation related to the degree of compliance with said authority recommendations. Monitoring the effect of a vaccine in a population requires methods to capture data and frequently relies on existing, real-world data sources. Consequently, study methods used for measuring vaccine effectiveness or impact will need to take into account/be tailored to the purpose and availability of actual real-world data. In the case of HPV vaccines, with many vaccine-preventable outcomes and various implementation approaches at hand, the variety of data sources and methods used is large. In addition, there is an increased risk for confounding when utilizing real-world data. Thus, careful selection of study design, data capture, and analytical methods can mitigate potential bias in vaccine effectiveness and impact studies.

This study aims to systematically review the different study designs, methods, and data sources in published evidence, which assessed the HPV vaccine effectiveness and impact on both cervical and non-cervical endpoints. Built on the 2016 Garland et al. [8], this review has expanded from both scope and time-frame perspectives. We also provided recommendations on how researchers from countries of different resource levels, with access to different sources of HPV data, can consider designing and initiating vaccine effectiveness or impact assessment.

2. Methods

2.1. Search strategy and selection criteria

This systematic review includes data extracted from studies of 4vHPV (and/or 9vHPV) vaccine effectiveness (VE) and impact on cervical/cervicovaginal, anal, and oral HPV infections, AGW, precancerous or cancerous lesions in anus, cervix, oropharynx, penis, vagina or vulva and recurrent respiratory papillomatosis (RRP). We included studies published between January 2007 and March 2020 (for an overview of the methodology applied, see Supplemental file 1). A PRISMA diagram displaying the literature search is in Figure 1.

Figure 1. PRISMA diagram with details of the literature search and extraction process for the period 01–01-2007 – 31-03-2020, on all outcomes.

Cervical abnormalities were identified based on cytology and histology (see Supplemental file 1). Anal abnormalities were identified based on histology. Detailed study inclusion and exclusion criteria are shown in Table 1. Juvenile-onset RRP (JO-RRP) was distinguished from adult-onset RRP (AO-RRP) based on the age at diagnosis: JO-RRP is found in children following HPV transmission from the mother during childbirth, compared to AO-RRP, which has adult onset (ages 18 years and older) [9]. As no published paper on AO-RRP was identified, below we report on JO-RRP studies only.

Table 1. Criteria for study inclusion/exclusion

Population(s)	4vHPV and/or 9vHPV vaccinated or non-vaccinated females and males of any age from any country.
Interventions	Vaccination with 4vHPV vaccine and/or 9vHPV vaccine
Comparator	Vaccine effectiveness: vaccinated vs. vaccine-naive subjects from the same or a similar population Vaccine impact: pre-vaccination period vs. post-vaccination period, or time trend analyses after the start of vaccination.
Outcomes	HPV genital, anal or oral infections Anogenital warts Recurrent respiratory papillomatosis Cervical or anal lesions Anal, cervical, oropharyngeal, penile, vaginal or vulvar cancer
Time	1 January 2007–31 March 2020
Study design	Observational studies aiming to evaluate ‘real-world benefit’ on: Vaccine effectiveness – the proportion of infection or disease prevented among vaccinated individuals, estimated by comparing the frequency of outcomes in vaccinated vs unvaccinated individuals within similar populations. Vaccine impact – the population-prevented fraction of infection or disease, assessed by comparing prevalence or incidence in the vaccine era to a comparable population from the pre-vaccine era or by measuring population-level trends over time. Included: Observational studies (including cohort, cross-sectional, case-control), on 4vHPV and/or 9vHPV vaccine effectiveness and impact in any language aiming to evaluate ‘real-world benefit’ Excluded: Burden of disease studies with no concurrent vaccine uptake, health-economic studies, modeling/simulation/extrapolation studies, awareness, knowledge, and/or acceptability studies, results of long-term follow-up studies of clinical trial populations, studies exclusively reporting data on 2vHPV vaccine, literature review articles, conference abstracts

2.2. Data extraction and analysis

Two researchers independently performed title/abstract selection of the publications using Rayyan [10], applying the predefined selection criteria. The full-text selection was performed by one reviewer, with 10% of non-selected publications being checked by a second reviewer as a quality control measure. Similarly, data extraction was performed by one reviewer with re-extraction of one publication per endpoint by a second reviewer as a quality control measure. We assessed the quality of identified literature by a modification of the Robins-I tool (method see Supplemental file 1). We extracted information on country, year, study population, key study features like study design (i.e. cohort, cross-sectional, repeated cross-sectional or case–control design), effectiveness and/or impact type measured, outcomes, and confounders (see Supplemental file 1 for the full list of variables and their definitions). VE and impact trends recorded in the publications over time and across countries were summarized. The study design, population features, outcome, and exposure data sources for each HPV-related endpoint were reviewed for both VE and impact studies.

Finally, observational studies are prone to several limitations, including biases and confounding. We examined and described the main strategic approach investigators attempted to mitigate against biases and confounding factors in these observational studies.

3. Results

3.1. Studies included

In total, 158 publications describing 178 studies of VE or impact of the quadrivalent HPV vaccine were included: 78 VE studies [11–78], 82 impact studies [79–158], and 18 studies reporting both VE and impact [62,159–174]. The number of studies (175) exceeded the number of publications (158) because some publications covered multiple countries (n = 2) or multiple outcomes (n = 16), where each country or outcome was counted as a separate study. As the length of time of use of the 9vHPV vaccine was relatively short, no published studies on 9vHPV were found. Hence, this review focuses on 4vHPV only.

The number of publications per calendar year are shown in Figure 2A. Sorting the evidence by year of publication shows a steady increase in the number of publications over time, with the highest number of publications (36 studies) occurring in 2019. The first impact study was published in 2009, which preceded effectiveness studies by three years [79]. In recent years, there has been an increase in combined publications, with 7/36 in 2019 reporting both effectiveness and impact.

Figure 2. Trend in publications of vaccine impact and effectiveness studies.

Studies were reported from 23 countries (Figure 2B). Applying the World Bank Development Indicators (www.worldbank.org), these publications were mainly from high-income countries, as those countries were the first to introduce national HPV vaccination programs and are also more likely to have outcome data sources, which facilitate impact studies. Nevertheless, we could note that a new set of reporting countries have recently emerged in the effectiveness/impact study field. Indeed, VE against infection [31,59,60], as well as cervical lesions [59], has been reported for some early-implementing low- and lower-middle-income countries, including Bhutan, India, Mongolia, and Rwanda. Franceschi et al. used urine testing in Bhutan and Rwanda to measure HPV prevalence at vaccine program introduction and observed a statistically significantly lower prevalence among vaccinated than unvaccinated students in Rwanda (vaccine type prevalence ratio = 0.12, 95% CI 0.03–0.51) and a similar tendency in Bhutan, although with wide CIs [31]. Basu et al. investigated protection conferred against incident HPV16/18 infection among a cohort of HPV-vaccinated, unmarried girls in India over a median follow-up of 7 years and could make observations relative to dose scheduling for 15–18-year-old girls [59]. Finally, Batmunkh et al. performed a retrospective cohort study on VE using self-administered vaginal swabs subjected to high-risk HPV testing, 5 years after an original vaccination campaign. The study observed a reduced prevalence of vaccine-type HPV among vaccinated women compared to the unvaccinated women [60].

The majority of studies included women only (118/178, 66%, Figure 2C), a smaller number of studies (43/178, 24%) included both women and unvaccinated men (impact studies with the possibility to assess herd protection [AGW, n = 30; infection, n = 13]), while 10 studies focused on unvaccinated males only (10/178, 6%), mostly related to AGW or anal infection, to assess herd protection. With increased gender-neutral vaccination, a set of recent studies (7/178, 4%) have been performed in vaccinated men (effectiveness studies [AGW, n = 3; infection, n = 4]).

Six outcomes were examined: AGW, cervical lesions, genital, anal and oral infection, and finally JO-RRP. Figure 2D shows the number of studies for each of these outcomes per year. The first publication was a pre-post impact study of a female-only 4vHPV program on AGW, which is an initial marker of vaccine impact because AGW can be diagnosed shortly following HPV infection. The study used the electronic database of the Melbourne Sexual Health Center to investigate a time trend in the diagnosis of genital warts, without linkage to vaccination status [79]. The number of studies on AGW appears stable, while publications on cervical lesions (first publication in 2011) and genital infections (first publications in 2012) steadily increased. As for the other outcomes, publications on anal infection are intermittent, without a clear trend. Interest in oral infection is increasing, with four publications in 2019. Finally, for JO-RRP only a single publication was available through our search and for AO-RRP we found no publications.

Data extracted from the studies are provided in Supplements 2–5, with separate tables for the major outcomes, AGW, cervical lesions, and genital infections, and a single table for the remaining outcomes, anal and oral infection, and JO-RRP. A compilation of study details is shown in Table 2, by outcome and by study focus.

Table 2. Study characteristics

Study details	Outcomes
	Anogenital warts		Cervical lesions		Genital infection		Anal infection or lesion		Oral infection		JO-RRP		Total
	I	VE	I	VE	I	VE	I	VE	I	VE	I	VE	I	VE
	37	13	28	31	31	37	1	4	1	10	1		99*	95*
Study design
Cohort	22 ^A1	12 ^B1	16 ^C1	24 ^D1	-	4 ^F1	-	3 ^H1	-	2 ^J1	-	-	38	45
Cross-sectional	2 ^A2	1 ^B2	2 ^C2	1 ^D2	3 ^E2	8 ^F2	-	-	1 ^I1	4 ^J2	-	-	8	14
Repeated Cross-sectional	13 ^A3	-	10 ^C3	2 ^D3	28 ^E3	24 ^F3	1^G3	1 ^H3	-	4 ^J3	1 ^K3	-	53	31
Case-control	-	-	-	4 ^D4	-	1 ^F4	-	-	-	-	-	-	-	5
Population
Female	6 ^A5	10 ^B5	28 ^C5	31 ^D5	24 ^E5	33 ^F5	-	2 ^H5	-	3 ^J5	-	-	58	79
Male	-	1 ^B6	-	-	5 ^E6	3 ^F6	1 ^G6	2 ^H6	-	7 ^J6	-	-	6	13
Female and male	31 ^A7	2 ^B7	-	-	2 ^E7	1 ^F7	-	-	1 ^I7	-	1 ^K7	-	35	3
Outcome data source
Claims database	12 ^A8	5 ^B8	3 ^C8	3 ^D8	-	-	-	-	-	-	-	-	15	8
Electronic health record	11 ^A9	3 ^B9	8 ^C9	6 ^D9	1 ^E9	1 ^F9	-	-	-	-	1 ^K9	-	21	10
Registry	13 ^A10	4 ^B10	13 ^C10	18 ^D10	4 ^E10	3 ^F10	-	-	-	-	-	-	30	25
Survey	1 ^A11	1 ^B11	4 ^C11	4 ^D11	26 ^E11	33 ^F11	1 ^G11	4 ^H11	1 ^I11	10 ^J11	-	-	33	52
Exposure data source
Claims database	3 ^A12	3 ^B12	-	1 ^D12	-	-	-	1 ^H12	-	-	-	-	3	5
Electronic health record	-	2 ^B13	1 ^C13	2 ^D13	1 ^E13	-	-	-	-	-	-	-	2	4
Registry	3 ^A14	7 ^B14	4 ^C14	16 ^D14	8 ^E14	7 ^F14	-	-	-	1 ^J14	-	-	15	31
Survey	-	1 ^B15	2 ^C15	8 ^D15	15 ^E15	22 ^F15	1 ^G15	1 ^H15	-	6 ^J15	1 ^K15	-	19	38
Not described	31 ^A16	-	21 ^C16	2 ^D16	7 ^E16	5 ^F16	-	-	1 ^I16	1 ^J16	-	-	60	8
Other^#	-	-	-	2 ^D17	-	3 ^F17	-	2 ^H17	-	2 ^J17	-	-	-	9

^aIncluding 18 studies that performed both impact and effectiveness. I – impact; JO-RRP – juvenile-onset recurrent respiratory papillomatosis; VE – vaccine effectiveness

^#Other includes combinations of listed data sources, or a study-specific database.

Number as in reference list:

A1: 86, 89, 90, 93, 95, 100, 110, 112, 115, 117, 122, 125, 130, 132, 134, 139, 140, 148, 152, 153, 159, 161; A2: 84, 97; A3: 79, 81, 82, 83, 87, 88, 98, 102, 106, 116, 119, 129, 143; A5: 86, 93, 98, 116, 159, 161; A7: 79, 81, 82, 83, 84, 87, 88, 89, 90, 95, 97, 100, 102, 106, 110, 112, 115, 117, 119, 122, 125, 129, 130, 132, 134, 139, 140, 143, 148, 152, 153; A8: 84, 90, 93, 95, 115, 125, 130, 132, 139, 148, 159, 161; A9: 79, 81, 82, 83, 87, 106, 116, 122, 129, 140, 143; A10: 86, 88, 89, 97, 100, 102, 110, 112, 117, 119, 134, 152, 153; A11: 98; A12: 125, 132, 159; A14: 152, 153, 161; A16: 79, 81, 82, 83, 84, 86, 87, 88, 89, 90, 93, 95, 97, 98, 100, 102, 106, 110, 112, 115, 116, 117, 119, 122, 129, 130, 134, 139, 140, 143, 148; B1: 16, 18, 22, 24, 26, 41, 43, 52, 57, 58; B2: 8; B5: 16, 18, 22, 26, 41, 43, 52, 57, 159, 161; B6: 24; B7: 48, 58; B8: 43, 57, 58, 159, 161; B9: 24, 41, 52; B10: 16, 18, 22, 26; B11: 48; B12: 43, 58, 159; B13: 24, 52; B14: 16, 18, 22, 26, 41, 57, 161; B15: 48; C1: 80, 96, 104, 105, 113, 114, 120, 127, 142, 155, 156, 158, 161, 166, 167, 172; C2: 91, 133; C3: 94, 109, 123, 128, 145, 146, 150, 151, 154, 157; C5: 80, 91, 94, 96, 104, 105, 109, 113, 114, 120, 123, 127, 128, 133, 142, 145, 146, 150, 151, 154, 155, 156, 157, 158, 161, 166, 167, 172; C8: 114, 161, 167; C9: 109, 141, 145, 146, 150, 151, 157, 172; C10: 80, 91, 96, 104, 105, 113, 120, 123, 128, 133, 154, 155, 156, 158; C11: 94, 127, 142, 166; C13: 145; C14: 151, 155, 161, 167; C15: 166, 172; C16: 80, 91, 94, 96, 104, 105, 109, 113, 114, 120, 123, 127, 128, 133, 142, 146, 150, 154, 156, 157, 158; D1: 17, 19, 23, 27, 32, 33, 36, 37, 42, 44, 49, 59, 61, 64, 69, 72, 74, 75, 76, 78, 161, 166, 167, 172; D2: 30; D3: 28, 68; D4: 20, 34, 55, 67; D5: 17, 19, 20, 23, 27, 28, 30, 32, 33, 34, 36, 37, 42, 44, 49, 55, 59, 61, 64, 67, 68, 69, 72, 74, 75, 76, 78, 161, 166, 167, 172; D8: 75, 161, 167; D9: 28, 33, 36, 67, 76, 172; D10: 17, 19, 20, 23, 27, 32, 34, 37, 42, 44, 49, 55, 61, 64, 69, 72, 74, 78; D11: 30, 59, 68, 166; D12: 75; D13: 55, 64; D14: 161, 167; D15: 30, 36, 42, 44, 68, 76, 166, 172; D16: 28, 37; D17: 59, 67; E2: 131, 165, 173; E3: 62, 85, 92, 99, 101, 103, 107, 108, 111, 118, 121, 123, 124, 126, 135, 136, 137, 144, 147, 149, 162, 163, 164, 168, 169, 170, 171, 174; E5: 85, 92, 99, 103, 107, 108, 111, 118, 121, 124, 131, 135, 136, 137, 144, 147, 162, 164, 165, 168, 170, 171, 173, 174; E6: 123, 126, 149, 163, 169; E7: 62, 101; E9: 124; E10: 103, 108, 131, 144; E11: 62, 85, 92, 99, 101, 107, 111, 118, 121, 123, 126, 135, 136, 137, 147, 149, 162, 163, 164, 165, 168, 169, 170, 171, 173, 174; E13: 108; E14: 85, 101, 131, 137, 144, 162, 165, 174; E15: 62, 118, 121, 123, 124, 135, 136, 147, 149, 164, 168, 169, 170, 171, 173; E16: 92, 99, 103, 107, 111, 126, 163; F1: 13, 36, 59, 60; F2: 30, 31, 39, 51, 53, 65, 165, 173; F3: 12, 15, 21, 25, 35, 38, 45, 50, 54, 56, 62, 66, 70, 71, 73, 77, 162, 163, 164, 168, 169, 170, 171, 174; F4: 11; F5: 11, 12, 13, 15, 21, 25, 30, 31, 35, 36, 39, 45, 50, 51, 53, 54, 56, 59, 60, 65, 66, 70, 71, 73, 77, 162, 164, 165, 168, 170, 171, 173, 174; F6: 38, 163, 169; F7: 62; F9: 36; F10: 12, 45, 73; F11: 11, 13, 15, 21, 25, 30, 31, 35, 38, 39, 50, 51, 53, 54, 56, 59, 60, 62, 65, 66, 70, 71, 77, 162, 163, 164, 165, 168, 169, 170, 171, 173, 174; F14: 45, 56, 65, 77, 162, 165, 174; F15: 11, 15, 25, 30, 31, 36, 38, 39, 45, 50, 51, 53, 54, 56, 62, 65, 66, 70, 71, 77, 162, 164, 165, 168, 169, 170, 171, 173, 174; F16: 12, 13, 21, 35, 163; F17: 59, 60, 73; G3: 149; G6: 149; G11: 149; G15: 149; H1: 13, 24, 36; H3: 46; H5: 13, 36; H6: 24, 46; H11: 13, 24, 36, 46; H12: 24; H15: 46; H17: 13, 36; I1: 160; I7: 160; I11: 160; I16: 160; J1: 13, 36; J2: 29, 62, 65, 160; J3: 40, 47, 63, 195; J5: 13, 36, 65; J6: 29, 40, 47, 62, 63, 160, 195; J11: 13, 29, 36, 40, 47, 62, 63, 65, 160, 195; J14: 65; J15: 29, 40, 47, 62, 63, 195; J16: 160; J17: 13, 36; K3: 138; K7: 138; K9: 138; K15: 138;

3.2. Study design

The two most commonly utilized study designs were the repeated cross-sectional design (85, 43.3%) and the cohort design (83, 42.3%). Repeated cross-sectional studies were conducted slightly more frequently to evaluate impact (53, 62.4%) than VE (32, 37.6%), and were most frequently used to study genital infection (53, 62.4%). Cohort studies, on the other hand, were conducted slightly more frequently to evaluate VE (45, 54.2%) than impact (38, 45.8%), and were most frequently used to study cervical lesions (40, 48.2%) and AGW (34, 40.1%). Single timepoint cross-sectional studies were conducted more frequently to evaluate VE (14, 60.9%) than impact (9, 39.1%), and were most frequently used to study genital infection (12, 52.2%) and oral infection (5, 21.7%). Only a limited number of case–control studies (5, 3.1%) were performed, exclusively in VE studies (as expected, given that impact studies would not need to know the vaccine exposure for cases/controls) and mainly for cervical lesion endpoints (4, 80%), generally with a limited study sample size [20,34,55,59].

3.3. Population features

We identified a number of different types of study populations chosen to reflect the specific outcome of interest, mode of transmission (horizontal vs vertical) and whether particular risk groups (such as MSM and anal cancer) are known. As a consequence, AGW impact studies were predominantly performed in both females and males (31/37, 83.8%, Table 2), whereas AGW VE studies were more frequently focused on females only (10/13, 76.9%). Anal infection studies were relatively frequently performed in men (3/5, 60%), and oral infection studies were often performed in both females and males (8/11, 72.7%). In the JO-RRP study, children (both male and female) born to vaccinated and unvaccinated women were investigated. Several studies focused on higher-risk populations (patients in sexual health clinics, n = 13; MSM, n = 4; HIV-infected population, n = 1).

3.4. Outcome data sources

The most frequently used sources of outcome data for both VE and impact studies were special studies specifically designed to assess outcomes for reporting effectiveness or impact (n = 86, 44%), especially when studying HPV infection as an outcome (Table 2). In such a study, researchers typically select a sample of the population of interest, gather information on vaccination status using a questionnaire, and assess HPV infection in samples collected by a clinician or by the participant themselves. The second most frequent source of HPV disease information was from registries (56, 29%), primarily cervical screening and cancer registries. Such registries exist in many high-income countries and were already established before the introduction of the HPV vaccine. As an example, a frequently studied registry in this literature review was the Victoria Cervical Cytology registry in Australia, with a total of 6 publications during years 2011–2016, reflecting that Australia implemented large-scale vaccination early after vaccine approval and thus accumulated sufficient time to observe results at an early stage compared to other nations. The remaining sources of information were electronic health records (31, 16%) or claims databases (23, 12%) to study genital warts and cervical lesions.

3.5. Exposure data sources

Generally, impact studies utilized vaccination coverage data at the national level, whereas vaccine effectiveness studies (38, 39.6%) often relied on self-reported vaccination status (Table 2). In one-third of the studies reviewed, self-reported vaccination status was verified in a proportion of the participants using claim databases (7/38, 18%) or EHR (6/38, 16%). Registries were also a frequently used source of data to ascertain HPV vaccination status (45/196, 23%). In general, these registries pre-dated the HPV vaccine and are used for all (childhood) vaccinations and all or a selection of adolescent and adult-age vaccinations. As an example, the Danish Vaccination Register covers all vaccinations, including those given outside the National Immunization Program. Registration is in real-time and data retrieval and linkage can be performed for surveillance or research [175]. In Australia, a national HPV vaccination registry was implemented at the time of the roll-out of the immunization program. Recently, this registry has been included into the Australian Immunization Registry.

3.6. Approaches to controlling confounding

As per standard epidemiological practice, a number of studies addressed potential confounding at the design stage. Several studies attempted to control for potential confounders through the inclusion of a restricted population. Most typically, studies included only sexually active young participants recruited at sexual health clinics (n = 18). While this choice may sometimes have been made for opportunistic reasons of feasibility (ease of access to, and acceptability by, the higher risk population) and statistical power (higher infection rates), it has also been done to reduce the risk of confounding by difference in/lack of sexual activity. Another approach was taken in matched case–control studies (n = 6) where controls were matched to cases for potential confounders, such as age or time since first cytology in the health system.

The large majority of studies, however, relied upon analytical rather than a priori design-based methods to address potential confounding. For most of these studies (140, 82%), simple stratification during analysis was used to assess whether the risk estimate differed by stratum, e.g. age. Other methods for assessing or minimizing the effect of potential confounders included multivariate analyses (n = 75), propensity score weighting, e.g. vaccination by age, gender and race (n = 9), or simple matching (n = 5). In ecological studies, where many potential confounding factors are unmeasured and individual-level vaccination status is unknown, a comparison to time trends of other sexually transmitted infections (e.g. chlamydia, gonorrhea) is utilized as a control. For example, if HPV infection declines in the community post-vaccination but other sexually transmitted infections remain constant, investigators would assume the HPV decline is attributable to vaccination [30,48,121].

4. Discussion

We identified 167 publications between January 2007 and March 2020, assessing vaccine effectiveness (VE) or impact of 4vHPV. The number of VE and impact studies published per year of the review steadily increased, with expanded reporting worldwide and, consistent with longer time since vaccine introduction, providing sufficient follow-up time to observe certain HPV-related diseases, as well as the progression of vaccine introduction globally. By 2020, 23 different countries reported VE/impact data, ranging from a single study in Bhutan, Colombia, India, Israel, Luxembourg, Mongolia, Rwanda, South Korea, Spain, and the UK (after the switch from bivalent to quadrivalent vaccine) to 60 studies from the US. As anogenital warts (AGWs) occur relatively quickly post-infection, many of the initial VE and impact studies focused on AGWs. Over time, there has been a significant increase in the number of publications focused on cervical lesion outcomes. With the move in many countries to HPV testing for cervical cancer screening, infection prevalence (and incidence, depending on the program) can be assessed within the population. For programs with type-specific HPV testing, the evaluation of the real-world benefit of vaccination will be enhanced, as this provides a built-in surveillance system for types targeted by the vaccines [61,176]. However, there remains limited data on other early markers of vaccine impact, for example, juvenile-onset recurrent respiratory papillomatosis (JO-RRP) and oral/anal HPV infection. Due to the lag time between infection and disease, data pertaining to HPV-related cervical cancer has been slower to obtain and may be even slower for oropharyngeal, vaginal, and vulvar cancer, where pre-stages are lacking entirely or less systematically diagnosed than for the cervix. However, after the completion of our study period, an individually linked effectiveness study utilizing the Swedish national registries was published evaluating the risk of cervical cancer by HPV vaccination status in a population of 1,672,983 women. The cumulative incidence of cervical cancer was 47 cases per 100,000 persons among women who had been vaccinated and 94 cases per 100,000 persons among those who had not been vaccinated. After adjustment for potential confounders, the incidence rate ratio was 0.12 (95% CI, 0.00 to 0.34) among women who had been vaccinated before the age of 17 years and 0.47 (95% CI, 0.27 to 0.75) among women who had been vaccinated at the age of 17–30 years [177].

Studying the VE and impact of HPV vaccines is challenging due to a number of aspects such as the multiplicity of outcomes [1,2]; the lack of routine screening in males and in non-cervical endpoints in females; the high proportion of asymptomatic HPV infections [178], and; the scarcity of vaccine registries [179–182]. Despite these challenges, HPV VE and impact studies are increasingly being conducted across the globe, using a wide variety of methods and data sources. We have summarized in Table 3 a number of considerations for conducting VE or impact studies after HPV vaccine introduction that derive from this review.

Table 3. Considerations for study design

Impact	Vaccine Effectiveness
Study design
Preferred design: Repeated cross-sectional study, large population at low cost. Best suited to assess the total effect. Disease incidence is the preferred to disease prevalence as outcome. Stratification for potential confounders (e.g. age at first sexual intercourse, condom use, positive history of STI) is possible but can be subject to recall bias.	Preferred design: Cohort study provides highest level of confidence; alternative: repeated cross-sectional study with knowledge of vaccination status. Case–control studies will be more useful for long term and more infrequent outcomes such as cancer. Disease incidence is preferred to disease prevalence as outcome. Stratification for potential confounders (e.g. age at first sexual intercourse, condom use, positive history of STI) is possible but can be subject to recall bias.
Study population
The study population is not necessary to be of age eligible for vaccination. Inclusion of males or unvaccinated female could show herd protection. With increasing GNV, preferably both women and men are included in the study population.	The vaccinated cohort should be of age eligible for vaccination. To reduce bias, controls should be selected within the same age range. In NIPs with GNV preferably both women and men are included, to assess effectiveness in both genders.
Outcome data source
AGW: as most people with AGW will be treated at some point, claims databases are good data sources. CL: cervical cancer screening registries, linked to pathology databases, have been used successfully. * INF: if HPV-based screening is performed, cervical cancer screening registries provide very useful data, although HPV type information may not be available. Anal and oral INF: no routine sample collection is available, making surveys the only option to collect data.	Outcome data need to be linked to exposure data to determine effectiveness. INF: in the absence of HPV-based screening, surveys provide the best data. Anal and oral INF: no routine sample collection is available, making surveys the only option to collect data.
Exposure data source
Vaccination status at the individual level is not necessary for impact studies, as pre- and post-vaccination periods are compared. Knowledge of the vaccination coverage at population level is important.	Vaccination registries/immunization information systems with strong organization are preferred as sources of exposure data where available, as date of immunization and number of doses are generally more reliable than self-reported vaccination status (avoiding exposure misclassification bias).
Time since start vaccination program
AGW and INF (so-called short incubation outcomes) are most likely to show early impact. CL may take longer to develop and is most easily detected in populations eligible for cervical cancer screening in an established, organized cervical cancer screening program, including appropriate registration of outcomes. Evidence of an effect on cervical cancer, as well as JO-RRP, will take relatively longer, as it concerns an outcome with a long delay or an outcome among the children of vaccinated women.
Vaccination coverage rate
Essentially, the higher the vaccination coverage rate, the more likely impact can be observed. However, for AGW and INF, a low but significant impact has been demonstrated with coverage rates as low as 30%.	Not necessary to have a high VCR. With a high VCR, unvaccinated controls may not be available. Use of other control groups (e.g. older [non-eligible] age) may impact confounders (e.g. sexual activity).
Source material for HPV detection
The drive toward HPV-based cervical cancer screening will likely result in the HPV status being recorded in the EHR. Increasingly, self-collected cervicovaginal material is considered a valid source for HPV detection, simplifying survey studies on genital infection. First-void urine may be considered as a noninvasive replacement for women.

AGW – anogenital warts; CL – cervical lesions; EHR – electronic health records; GNV – gender-neutral vaccination; INF – infection; JO-RRP – juvenile-onset recurrent respiratory papillomatosis; STI – sexually transmitted infection; VCR – vaccine coverage rate

4.1. Study design

Among observational studies, cohort studies are ranked highest in their capacity to generate high-quality evidence but are resource- and data-intensive and tend to focus on selected populations that agree to participate in long-term studies. [183,184] Cross-sectional studies, on the other hand, typically rank lowest in terms of quality of the evidence due to their inability to assess temporality, but allow large populations, with fewer selection biases, to be studied at relatively low expense. All observational studies, regardless of design, are susceptible to systematic error in terms of bias and confounding, and these can be addressed both by design and at the analytical stage. For rare outcomes, such as JO-RRP, case–control studies may be an option in settings with limited data but the ability to know the underlying population from which the JO-RRP cases arise. In some settings, a case series may suffice for JO-RPP impact. However, a cohort design, including all subjects, whether cases or controls, measuring the person-time exposed, incidence rates, etc. is more powerful and offers substantial analytical flexibility. Comprehensive cohort studies are possible in settings with national vaccine registries, such as the Nordic countries, where person-time can be divided into vaccinated and unvaccinated, with high reliability, and should thus be kept in mind also for rare outcomes.

4.2. Data sources

AGW and cervical lesions are generally recorded in registries, claims databases, and EHRs in a standardized fashion. The registration of HPV infection and/or specific genotypes is newer to the prevention field and less frequently registered in comprehensive databases. They may sporadically be captured in EHRs from managed care organizations (e.g. the US organization Kaiser Permanente) but most likely not in claims databases, often necessitating special studies to collect these data.

For exposure data, systematic vaccination registries would be the preferred source given their reliability and completeness [185]. However, not all countries have the ability to implement such, and/or they may be incomplete. Especially in countries starting a national immunization program (NIP), or incorporating HPV vaccination into the NIP, it has been recommended to establish a vaccination registry [186]. Similarly, several countries are improving healthcare infrastructure by improving completeness and accuracy of the registration of cervical cancer prevention efforts, and to monitor progress toward cervical cancer control and eventual elimination, as posited by the WHO global cervical cancer elimination campaign [187].

Regardless of which endpoint VE was assessed for, the possibility of linkage of vaccination and screening registries is a key facilitator in the study of disease rather than infection as an endpoint.

4.3. Methods for Molecular Studies

Current cervical screening programs use cervicovaginal exfoliated cells, typically for HPV detection and subsequent triage methods. In program settings that save test residuals, linkage of vaccine and screening registries can be performed to identify samples of interest and then perform HPV typing to estimate VE in vaccinated women, as well as potentially type replacement after vaccination [101]. There are several alternative sampling methods we identified in our search, apart from clinician-obtained tests. In combination with high-risk HPV assays based on polymerase chain reaction, testing of self-collected samples is as accurate as clinician samples [188]. Furthermore, first-void urine showed good agreement in HPV DNA viral load with reference cervical samples and provides a noninvasive method which may be preferable to participants [189]. Urine samples have already been used in VE studies in females [31,165], but the sensitivity of HPV detection in urine in males is lower [190–192], and male urine samples are thus currently not strongly endorsed for VE studies. To monitor the effectiveness of male HPV vaccination programs, a sentinel surveillance model offers an appropriate strategy [193]. Monitoring HPV genotypes over time will detect changes in circulating HPV types. Sampling of the penile shaft, the glans, the coronal sulcus, and the scrotum provides the highest detection rates as these sites are most likely to become colonized with HPV during intercourse [193]. Swabbing without abrasion is acceptable to participants and allows for self-collected samples [193]. Several examples of this approach have been published [38,62,123,126,149,169]. In summary, the growing utilization of molecular tests and self-sampling options will enhance our ability to conduct VE and impact studies on large populations.

In resource-limited environments, we identified several types of studies of high quality that used a pre-post study approach to measure HPV prevalence through approaches such as urine sampling and self-testing for HPV [31,59,60]. If such methodology is deemed beyond the capacity of the local context, the opportunistic re-use of existing data sources (such as medical charts or register data, where existent) may provide e.g. information to investigate overall trends in cervical cancer incidence [194]. Furthermore, although studies on HPV prevalence with cross-sectional designs could not conclude on causal effects of the HPV vaccination program, it still provides insights on changes in disease burden associated with HPV vaccination programs in such areas/settings.

We acknowledge the following limitations to our work. We only reviewed effectiveness and impact studies of the 4vHPV and 9vHPV, but not 2vHPV. However, the approach for assessment of effectiveness and impact of HPV vaccines would essentially be the same, though effectiveness/impact studies for RRP and AGW would be more appropriate in settings with vaccines that target HPV 6/11, which are responsible for 90% of genital warts cases. Similarly, in an increasing number of countries, there is mixed-use or sequential use of different HPV vaccines, due to tendering systems, making it more challenging to assign the effectiveness or impact found to a specific vaccine. How this type of delivery evolution in national immunization programs influences the real-world vaccine effectiveness and impact assessment needs to be further studied. Additionally, so far, studies have predominantly been performed in high-income settings (HIC). The good practices summarized from HIC may not be applied to LMIC, given the different settings with potentially limited resources. Finally, inevitably with any systematic review, there will be a lag time between end-of-study period (March 2020) and study publication. However, we believe this gap has limited impact on the study findings and conclusion, given the objective of this study was to review and summarize study designs, methods, and data sources.

5. Conclusions

To our knowledge, this is the first study to systematically review, appraise, and provide recommendations on the design of HPV vaccine effectiveness and impact studies, showing that both robust study methodology and healthcare data infrastructure are essential for evaluating a vaccination program. To establish robust evidence on vaccine effectiveness and impact, researchers have to overcome the methodology challenges given the complexity of HPV-related disease and the availability of well-validated data sources. Study designs and methodologies should be selected appropriately case by case, especially in resource-limited settings. Although HPV infection leads to a multitude of diseases across the life course, with attention to research methods and data sources, studies can demonstrate and track the protective effect of HPV vaccines on these disease outcomes.

6. Expert opinion

The rapid growth in access to real-world data, such as electronic health records, claims, billing data, and disease registries allows global monitoring of effects of different public health interventions, including HPV vaccination programs. But the use of data which is primarily not collected or organized to support research also underline a need to develop a robust methodology which provide insight of vaccine effects as well as on consequences of different health policy decisions. A robust study methodology and understanding both opportunities as well as limitation of different data sources are essential for generating real-world evidence. To establish robust evidence on vaccine effectiveness and impact, researchers must overcome the methodology challenges given the variety and heterogeneity of HPV-related disease, and the availability of well-validated and integrated data sources. Study designs and methodologies should be selected appropriately case by case according to the research need (e.g. to evaluate short-term or long-term effectiveness or impact of vaccination program), and the capability of the data system settings. Especially in the resource-limited settings, options are also available to measure vaccine program effect based on what we learned from existing evidence. As posited by WHO in the global cervical cancer elimination campaign, all countries must reach and maintain an incidence rate of below 4 per 100,000 women and should meet the ‘90–70-90’ targets by 2030 to get on the path to eliminate cervical cancer within the next century. To achieve the elimination goal, we could foresee a growing need for HPV vaccination program evaluation in the coming years globally. A critical and appraisal summary of a robust methodology used in the HPV vaccine effectiveness and impact evaluation will provide timely technical guidance to researchers who would like to initiate such research activities in various settings.

Declaration of interests

W Wang, S Kothari, C Velicer, J Tota and A Sinha are employees of Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA (MSD), and may own stock in Merck & Co., Inc., Kenilworth, NJ, USA; M Baay, J Skufca and T Verstraeten are employees of P95 and have received consulting fees from vaccine-producing companies, including MSD; SM Garland has received grants through her institution from Merck and has delivered lectures, received speaking fees from MSD for work performed in her personal time and is a member of the Merck HPV Global Advisory Board. ARG has received financial support from Merck for her role as a member of several advisory boards and as a speaker at conference symposia and has received research grants through her institution. M Nygard has received research grants from MSD Norway through her affiliating institute. K Ssundstrom has received research grants from MSD to her institution. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

W Wang, S Kothari, and T Verstraeten designed the study. The data were acquired and analyzed by J Skufca, M Baay, T Verstraeten, and W Wang. All authors contributed to the interpretation of the results. W Wang, S Kothari, J Skufca, M Baay, and T Verstraeten drafted the manuscript. All authors critically reviewed and revised earlier versions and approved the final version of the manuscript.

Funding

This work was supported by Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA (MSD). As MSD employees are among the co-authors, the sponsor had a role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

Supplementary material

Supplemental data for this article can be accessed here