Effectiveness of vaccination with the pentavalent rotavirus vaccine in Nicaragua as determined using the screening method

Abstract

The screening method is a surveillance tool to evaluate vaccine effectiveness (VE) using coverage data on cases and available administrative estimates of vaccine coverage in the population. The aim of this analysis was to evaluate the utility and limitations of using the screening methodology to estimate VE, particularly in a developing world country with a high coverage rate, and to compare it with the VE estimates from 2 case-control studies. Using data from 2008, the screening method employed in this study estimated that VE for 3 doses of RV5 among children < 12 mo of age to prevent wild-type severe disease, resulting in hospitalization or emergency department visits, was 92% (95% confidence interval [CI]: 78–100%). Additional sensitivity analysis demonstrated that the point estimates of VE against severe disease ranged from 72% (95% CI: 62–83%) to 92% (95% CI: 78–100%); this range of VE estimates, although wide, is relatively consistent with results reported from 2 case-control studies in Nicaragua for the same time period. When the infrastructure is in place to collect reasonably robust case data, the use of the screening method to estimate VE is possible in the developing world setting. Cases of severe wild-type rotavirus gastroenteritis were obtained through an observational, hospital-based, prospective, surveillance program to assess rotavirus acute gastroenteritis. The proportion of cases vaccinated was estimated using the child’s vaccination card or health record. The proportion of the population vaccinated was estimated using administrative population-based vaccination coverage estimates provided by the Nicaraguan Ministry of Health.

Keywords: :

• rotavirus
• vaccine effectiveness
• pentavalent rotavirus vaccine
• RV5
• screening method

Introduction

Rotavirus was responsible for an estimated 453,000 deaths globally among children < 5 y of age in 2008 and accounted for 37% of all diarrhea-related deaths among this age group.¹ The majority of these deaths occurred primarily in low-income nations.¹ From 2001 to 2005, Nicaragua—a nation classified as a high-mortality developing nation by the World Health Organization (WHO)—experienced an estimated 41,122 outpatient visits, 4,460 hospitalizations, and 60 deaths owing to rotavirus disease per year in children < 5 y of age.^2,3 Nicaragua is the second-poorest nation in Latin America and has a population of approximately 5.7 million, with 678,000 under the age of 5 y.^4,5

In 2006, a live, pentavalent, bovine–human reassortant vaccine (RV5; RotaTeq^®; Merck and Co., Inc.) was licensed in the United States for administration to infants at 2, 4, and 6 mo of age. Effectiveness studies evaluate the effect of a vaccine in a real-world setting and enable us to evaluate how the efficacy determined in the controlled environment of a clinical trial translates to real-life conditions. In addition, effectiveness studies may be important in future policy decisions in areas that have yet to adopt a vaccine, as they assess the early benefits of the vaccine and monitor them temporally. Several post-licensure effectiveness studies have demonstrated high vaccine effectiveness (VE) and safety in developed nations, with additional evidence of herd immunity causing a sustained reduction in rotavirus.^6-10 Of note, the efficacy of RV5 has been lower in developing nations. In previous studies, the vaccine efficacy of RV5 against severe rotavirus gastroenteritis (RGE) was 64.2% in Africa (Ghana, Kenya, Mali) and 51.0% in Asia (Bangladesh, Vietnam).^11,12 Thus, the assessment of VE in the developing world is an important aspect of understanding the potential impact of a vaccine in a variety of real-world settings.

The evaluation of VE in the developing world is often conducted using observational post-licensure studies; 2 common methods to evaluate VE are case-control studies and the screening method. In Nicaragua, 2 independent case-control studies with children receiving 3 doses of RV5 have been previously reported. The first (Patel et al.) was coordinated by the United States Centers for Disease Control and Prevention (CDC) in 4 hospitals from June 2007 to June 2008.¹³ The primary objective of this study was to evaluate the association between the completed 3-dose schedule of RV5 and rotavirus diarrhea requiring hospital admission or intravenous hydration in the emergency department (ED). Secondary objectives included evaluation of the association between RV5 and severe (Vesikari score ≥ 11) and very severe (Vesikari score ≥ 15) rotavirus diarrhea and VE stratified by age at onset of acute gastroenteritis (AGE). From this secondary analysis, the study reported that the VE of RV5 against severe disease (Vesikari score ≥ 11) was 58% (95% confidence interval [CI]: 30–74%) in 2008.¹³

The second study, conducted from February 2007 to October 2009, was a component of a public–private partnership formed between the Ministry of Health (MoH) in Nicaragua and Merck and Co., Inc., to demonstrate the public health benefit of universal routine immunization against rotavirus.^14,15 The primary objective of this case-control study was to determine the VE of RV5 in reducing the risk for severe (Vesikari score ≥ 11), wild-type RGE resulting in hospitalizations and ED visits among subjects administered 3 doses of vaccine.¹⁵ In addition to evaluating the effectiveness of RV5 in reducing the risk of hospitalizations or ED visits, the study also had several exploratory objectives.¹⁵ Mast et al. reported an adjusted VE of 76% (95% CI: 63–84%) against severe disease (Vesikari score ≥ 11) using community and hospital controls, combined, for the entire study period.¹⁵ Furthermore, a sub-analysis of the Nicaraguan MoH and Merck and Co., Inc., case-control study, found the unadjusted VE of 3 doses of RV5 to prevent wild-type severe disease (Vesikari score ≥ 11) resulting in hospitalization or ED visits for cases < 12 mo of age at the time of AGE onset was 92% (95% CI: 79–97%) in 2008 for hospital and community controls combined (Merck and Co., Inc., unpublished data, December 2010). While there is an 18% difference in the point estimates of VE for severe disease (comparing the 58% from the secondary analysis from Patel et al., vs. the 76% from the primary analysis from Mast et al.), it is important to note that the estimates from the 2 studies are difficult to compare owing to the relatively wide 95% CI around the VE estimates and the differing time periods of the studies during which there was increasing uptake of the newly introduced vaccine.

Another approach to estimating VE is the screening method, which has been used to estimate the VE of RV5 in Australia, a country with high coverage in the developed world.¹⁶ Using a similar approach, the present study used the screening methods to estimate the VE of RV5 in Nicaragua, a country in a developing world setting, by utilizing the available data of cases from the Nicaraguan MoH and Merck and Co., Inc., case-control study.^15,17 The screening method is a relatively simple and efficient surveillance tool that relies on finding cases, as is done in a case-control study, but without actively enrolling controls. Instead, the measure of vaccine exposure comes from the available administrative estimates of vaccine coverage. This method of evaluating VE has been shown to result in estimates that are generally similar to those generated from case-control studies and has helped to verify estimates observed in clinical trials.^16,18-21

The aim of the current study was to evaluate the utility and limitations of a screening methodology to estimate VE, particularly in a developing world country with a high coverage rate, and to compare this estimate with that obtained from the case-control studies conducted in Nicaragua during the same 1-y period.

Results

Demographic characteristics of the cases are described elsewhere.¹⁵ In 2008, 37 children had severe RGE (Vesikari score ≥ 11), were age-eligible to have received 3 doses of RV5, and had a valid vaccination record. Of those 37 children, 23 had documentation of having received 3 doses of RV5 [proportion of cases vaccinated (PCV) = 62%] and were included in the calculation of VE. Using the administrative data for children < 12 mo of age in Nicaragua, the overall proportion of the population vaccinated (PPV) with 3 doses of RV5 in 2008 was estimated to be 95.2%. Using data from 2008, the screening method employed in this study estimated that VE for 3 doses of RV5 among children < 12 mo of age to prevent wild-type severe disease, resulting in hospitalization or ED visits was 92% (95% CI: 78–100%).

As part of the sensitivity analysis to validate the vaccination coverage data obtained from the Nicaraguan MoH, the authors compared the Demographic Health Surveys (DHS) vaccine coverage estimate and the WHO administrative coverage estimate for the diphtheria, tetanus, and pertussis (DPT) and polio vaccines in Nicaragua in 2001, which was the last year that the DHS was completed in Nicaragua. Because the MoH reports vaccination coverage to the WHO, the WHO coverage estimates were considered to be a reliable proxy for the MoH statistics. In 2001, the DHS vaccine coverage for DPT and polio vaccines was estimated to be 82.7% and 84.7%, respectively. In the same year, the WHO administrative coverage for DPT and polio in Nicaragua was estimated to be 92% and 93%, respectively.²² Although these 2 different methods of estimating vaccine coverage in Nicaragua produced generally similar estimates, there was an approximate 9–10% difference between the administrative coverage estimate provided by the WHO and the DHS. To account for the potential overestimate of administrative coverage, the authors lowered the Nicaraguan MoH administrative vaccine coverage estimate from 95.2% to 85.2%. With this sensitivity adjustment in administrative coverage, the VE was 72% (95% CI: 62–83%) in 2008 for children < 12 mo of age. Therefore, the point estimates of the VE of RV5 in Nicaragua could range from 72% (95% CI: 62–83%) to 92% (95% CI: 78–100%), depending on the value used for the administrative coverage estimate.

Discussion

The data from this analysis of VE of RV5 using the screening method provide additional perspective to the 2 previously conducted case-control studies in Nicaragua.^13,15 Because the vaccine coverage (PPV) provided by the Nicaraguan MoH for the current study was based on children < 12 mo of age in 2008, it was not entirely compatible with the primary objectives of the 2 prior case-control studies which assessed VE for a broader age group and different time periods. Therefore, this study compared the VE estimate from the screening method to the appropriate sub-analyses from each of the 2 case-control studies to provide a more direct comparison. In a sub-analysis of the study conducted by the CDC, the VE of RV5 against severe disease in children aged 8 to 11 mo was 69% (95% CI: 24–87%) for the combined hospital and neighborhood controls in 2008.¹³ A sub-analysis of the Nicaraguan MoH and Merck and Co., Inc., case-control study utilizing data from 2008 determined the unadjusted VE of 3 doses of RV5 to prevent wild-type severe disease (Vesikari score ≥ 11) resulting in hospitalization or ED visits for cases < 12 mo of age at the time of AGE onset was 92% (95% CI: 79–97%) for hospital and community controls combined (Merck and Co., Inc., unpublished data, December 2010). The 92% VE estimate calculated using the screening method from this study is the same as the VE estimate from the sub-analysis of the Nicaraguan MoH and Merck and Co., Inc., case-control study. However, the VE estimate from the screening method is somewhat higher than that of the CDC case-control study and, although the 95% CI for the CDC study is wide (24–87%) and overlaps with the 95% CI around the VE estimate from this study (78–100%), some caution may be warranted in comparing the different analyses.

A strength of this study is the availability to leverage rigorously collected case data that were part of a previous case-control study. However, a caveat is that the collection of this type of case data may not be feasible in all developing world countries because of limitations in resources or infrastructure. Thus, although Nicaragua was an ideal location for a case-control study owing to the country’s strong infrastructure for disease surveillance, vaccine storage, delivery and high childhood immunization rates, the ability of conduct this type of research may not transfer to other settings.

There are several limitations of the current study. First, it has been suggested that administrative coverage estimates may, at times, be overestimated.²³ This was the rationale for conducting a sensitivity analysis to assess the impact of overestimated administrative coverage on the screening method VE estimate. Second, the administrative coverage rate was 95.2% in this study. Orenstein et al.²⁴ suggested that vaccine coverage levels in the community greater than 80% may lead to an overestimate of VE, and thus the screening method should not be used for precise VE estimates when the coverage is high. Indeed, the formula for calculating VE using the screening method demonstrates that as the PPV increases and approaches 1, the estimated VE will also increase, which could lead to an overestimate. Finally, the VE may be overestimated if there is a large temporal difference between the collection of data for PPV and PCV because the data for PCV are determined in real time as symptoms arise, and determination of PPV is less static. However, the current analysis attempted to minimize this effect by using cases and population coverage data from a single year (2008).

Despite the potential limitations of administrative vaccination coverage data, the results of the study show that the screening method can still be useful in estimating VE. The sensitivity analysis accounted for an approximate 10% difference between the DHS and WHO administrative coverage estimates, a difference that is consistent with results of a study that evaluated the validity of reported vaccination coverage in 45 countries.²³ After accounting for the potential 10% overestimate of the MoH coverage estimate, the sensitivity analysis demonstrated that the point estimates of the VE against severe disease using the screening method could be either 72% (95% CI: 62–83%) or 92% (95% CI: 78–100%), which is consistent with VE estimates from the sub-analyses (children < 12 mo of age in 2008) from both case-control studies conducted in Nicaragua. This analysis suggested that the point estimate of VE using the screening method was fairly sensitive to the accuracy of administrative vaccination coverage. If other studies were to use the screening method to assess VE in the field, it would be important to first assess the source of available administrative coverage estimates in the country. The availability of corroborating vaccine coverage data from cluster surveys or multi-stage sampling surveys (such as DHS) would provide a measure of the robustness of administrative coverage estimates and thus provide confidence in the estimates of VE derived using the screening method.

Our data indicate that when the infrastructure, such as routine surveillance, is in place to collect robust case data and other more traditional methods to estimate VE cannot be used, then using the screening method to estimate VE is possible in the developing world setting, even in a country with a high coverage rate. Overall, the screening method produced relatively reliable results, consistent with those obtained from 2 case-control studies.

Materials and Methods

Setting

As previously described, an observational, hospital-based, prospective rotavirus AGE surveillance program was established in a network of 6 hospital study sites located in 5 cities in the Western region of Nicaragua. Children identified through the surveillance program who were rotavirus-positive, were enrolled as potential cases in the case-control vaccine-effectiveness study—conducted from February 12, 2007 to October 9, 2009—that used age-matched hospital, community, and rotavirus-negative controls.¹⁵ The use of the screening methodology to estimate VE was an exploratory objective of the Nicaraguan MoH and Merck and Co., Inc., case-control study. Although the case-control study spanned 3 y (2007–2009), only data from 2008 were used in this analysis. Using only the 2008 data enabled comparison with the 2 case-control studies conducted during the same time period. Furthermore, it has been suggested that the screening method is prone to error when applied to areas with a very high proportion of vaccinated individuals;^24,25 the PCV in 2008 was 62% and in 2009 was 92%. In addition, there were no cases enrolled in the study in 2007.

Procedures

Data on rotavirus cases were collected as part of the Nicaraguan MoH and Merck and Co., Inc., case-control study. Detailed inclusion and exclusion criteria have been reported previously.¹⁵ The study was approved by the ethical review committees of the Nicaraguan MoH and the National Autonomous University of León, Nicaragua. The parents of enrolled children provided informed written consent.

AGE was defined as the first occurrence of ≥ 3 watery or looser-than-normal stools within a 24-h period and/or single forceful vomiting. Disease severity of cases was categorized as mild, moderate, or severe using the validated, 20-point Vesikari scoring system, which incorporates severity and/or duration of symptoms.²⁶ Bulk stool samples were collected from each case promptly after presentation at the ED or hospital. They were tested for rotavirus antigen in Nicaragua and later confirmed by a validated enzyme-linked immunosorbent assay.^15,27 For this analysis, cases were defined as having confirmed wild-type RGE resulting in hospitalization or ED visit, valid vaccine exposure, and severe disease (Vesikari score ≥ 11).

Vaccination history of cases

The vaccination history for cases was assessed by review of individual vaccination cards or health records if the vaccination card was unavailable. Children were considered fully vaccinated if they had been administered the third dose of RV5 at least 14 d prior to the onset of RGE.

Administrative vaccine coverage data

In Nicaragua, RV5 is administered according to the current Expanded Programme on Immunization (EPI) schedule, and administrative RV5 vaccination coverage data were provided by the Nicaraguan MoH.²⁸ Vaccine coverage was calculated using the number of administered doses (dose 3) divided by the number of age-eligible infants in the population based on the national census. This method assumes that births are uniformly distributed throughout the year.²⁸ In Nicaragua, each hospital belongs to a Local System for Integral Health, or Sistema Local de Atención Integral en Salud (SILAIS). The SILAIS reports health statistics from the regional SILAIS hospitals as well as the surrounding health centers where immunizations are conducted and submits the local data to the MoH. The MoH provided the RV5 coverage rate for each of the 5 cities in the case-control study for children < 12 mo of age. To determine an overall coverage estimate for the 5 cites, or the PPV, coverage data were weighted by the proportion of cases from each city.

Statistical analysis

The screening method used the vaccine coverage data of the cases from the case-control study. The population coverage exposure was estimated using the available administrative estimates of vaccine coverage from the Nicaraguan MoH. VE was calculated according to the following formula:¹⁷$VE=1-\frac{PCV}{1-PCV}\times \frac{1-PPV}{PPV}$where PCV is the proportion of cases vaccinated, as determined from the vaccination history of cases, and PPV is the proportion of the population vaccinated, determined using population-based vaccination coverage estimates that were provided by the Nicaraguan MoH. Approximate CIs were calculated using the process detailed by Farrington for the screening method, which provides the relationship between sampling, vaccination odds ratios, and relative risks for multiple study designs.¹⁷

A study by Murray et al. compared administrative vaccine coverage estimates in 45 countries, including Nicaragua, to that of nationally representative household surveys conducted as part of the DHS.²³ The study found that the administrative coverage estimates were up to 20% higher than the DHS estimates. Therefore, a sensitivity analysis was performed to compare the MoH administrative rates with the DHS coverage rates. The VE was recalculated by the screening method using a variant of the administrative coverage value as determined by the sensitivity analysis.

Role of funding source

The study was designed by Merck and Co., Inc., investigators with input from site investigators and staff. Merck had direct oversight or participation in every stage of the study. All authors had full access to the data, and the corresponding author had final responsibility for the decision to submit for publication.

Abbreviations:
AGE	=	acute gastroenteritis
CDC	=	Centers for Disease Control and Prevention
CI	=	confidence interval
DHS	=	Demographic and Health Surveys
DPT	=	diphtheria, tetanus, and pertussis
ED	=	emergency department
MoH	=	Ministry of Health
PCV	=	proportion of cases vaccinated
PPV	=	proportion of population vaccinated
RGE	=	rotavirus gastroenteritis
RV5	=	pentavalent rotavirus vaccine, RotaTeq®
SILAIS	=	Sistema Local de Atención Integral en Salud
VE	=	vaccine effectiveness
WHO	=	World Health Organization

Conflict of Interest

All authors are responsible for the work described in this paper and were involved in at least one of the following: conception, design, acquisition, analysis, statistical analysis, interpretation of data, and drafting the manuscript and/or revising the manuscript for important intellectual content. All authors provided final approval of the version to be published. Anna Cardellino, Shazia Khawaja, and, T. Christopher Mast are employees of Merck and Co., Inc., who may potentially own stock and/or hold stock options in the company. The authors have no other funding or conflicts of interest to disclose.

Acknowledgments

We are indebted to Xingshu Zhu for her statistical programming support on this study and to Tonya Goodman and Karen Collins, Arbor Communications, Inc., Ann Arbor, MI for manuscript preparation and editorial assistance on behalf of Merck Sharp and Dohme Corp., a subsidiary of Merck and Co., Inc. Merck Sharp and Dohme Corp. provided financial support for this study. All authors have completed the ICMJE form for disclosure of potential conflicts of interests.