How to understand the efficacy measurements for enterovirus type 71 vaccine?

Abstract

The choice of endpoint was most important for an efficacy vaccine trial. The objective of this paper is to gear toward answering questions about the rationality and scientificity of the primary endpoints choosing, case capturing and diagnosis strategy in our recently reported EV71 vaccine efficacy phase 3 trial. In order to obtain both high sensitivity and specificity in the case detecting, EV71-associated disease had been chosen as primary endpoint, a broad spectrum of clinical symptoms was surveyed, both the real-time RT-PCR and virus isolation were combined for the laboratory diagnosis, and serial specimens since disease onset were collected for assays. Though, the EV71 vaccine efficacy was well measured in the phase 3 trial, several potential factors could also have influences on the cases confirming. More evidence of EV71 vaccine efficacy will be demanded in post-marketing studies in the future.

Keywords: :

• enterovirus type 71
• vaccine
• efficacy
• sensitivity
• specificity
• hand
• foot and mouth disease

Introduction

Since the first description of enterovirus type 71 (EV71) in 1974, outbreaks of this virus have occurred periodically throughout the world, accounting for more than 6 000 000 cases and more than 2000 deaths in the past decade.^1,2 As EV71 continuously caused life-threatening outbreaks of hand, foot, and mouth disease (HFMD) especially in Asian children,^3-7 development of EV71 vaccines has been a national priority in some Asian countries and regions.^8-11 The first EV71 vaccine was tested in Bulgaria in 1976, but not further evaluated for its clinical efficacy.¹² To our knowledge, 5 EV71 vaccine candidates have been assessed in clinical trials, 3 of which manufactured in mainland China, and another 2 in Taiwan and Singapore.¹³

Recently, our phase 3 trial with an EV71 vaccine developed by Beijing Vigoo Biological Co., Ltd (1 of the 3 candidate EV71 vaccines manufactured in China) was reported. The trial was the first in which the efficacy of this novel vaccine was demonstrated.¹⁴ In the trial, both EV71-associated HFMD and EV71-associated disease (including EV71-associated HFMDs and other EV71-associated other cases) were considered as primary endpoint. To identify cases and exclude non-cases is, usually, challenging in vaccine efficacy study. When some cases can go unidentified, or some non-cases can be erroneously classified as cases, in which case the estimates of vaccine efficacy will be erroneous.¹⁵ The objective of this paper is to gear toward answering the questions about the rationality and scientificity of the primary endpoints choosing, case capturing and confirmation process, and share our experiences with this phase 3 EV71 vaccine trial.

Preventing EV71-Associated Disease or EV71 Infection

One of the most critical aspects of vaccine efficacy trials is case definition. It has to be decided whether infection or disease is the endpoint of interest.¹⁶ This choice is of importance because the EV71 infection does not necessarily imply symptomatic disease. From previous epidemic studies it was known that, subclinical infection with no clinical symptom is common, with around 30% to70% of the infected children developing visible clinical manifestations.^17,18 Subclinical EV71 infection cases do have their epidemiological significance because they can contribute to the EV71 transmission as source of infections.¹⁹ However, subclinical EV71 infection would be difficult to capture. To detect subclinical cases, biological specimens should be collected from all the subjects frequently and consistently during the whole follow-up period, which is very difficult to organize the participants to visit the investigational site for serum collection at pre-defined times, especially for such a large scale trial with around 10 000 participants in a 1-y surveillance period.¹⁴ On the other hand, case finding would be much easier when using EV71-associated disease as endpoint, because disease is usually accompanied by specific clinical symptoms. Therefore, the primary endpoint of the phase 3 trial with EV71 vaccine was diseases caused by EV71, but not EV71 infection.

Detection of Potential EV71-Associated Cases Via Clinical Manifestations

In a vaccine efficacy trial, intense efforts should be taken to ensure high sensitivity and specificity for the case finding strategy. Low specificity of case finding is known to lead to underestimation of the vaccine efficacy, while low sensitivity leads to poor accuracy and an increase in the required sample size.²⁰ A case-finding strategy depends on many factors including the nature of the disease being studied, the surveillance method, the degree to which patients with the disease will seek medical advice, the quality of records and so on.²¹

When considered how to capture as many as possible cases of the EV71-associated disease occurring during the surveillance period, we had to take into account the clinical features of disease caused by EV71 infection. EV71 can cause a wide range of disease,^2,18,22 including HFMD (presented with febrile illness with papulovesicular rash or maculopapular without vesicular lesion on palms and soles, with or without vesicles/ulcers in the mouth, buttocks, knees, or elbows), herpangina (presented with oral ulcerations on anterior tonsillar pillars, solt palate, soft palate, buccal mucosa, or uvula), neurological signs (aseptic meningitis/ encephalitis and poliomelitis like disease), and nonspecific illnesses such as febrile illness (axillary temperature greater than 37 °C without other symptoms), viral exanthema (widespread rash), respiratory infection (sore throat, coryza, or cough without herpangina or rash), acute gastroenteritis (vomiting or diarrhea with or without abdominal pain) and so on. In addition, EV71-associated disease are usually mild and self-limiting, so mild cases could be underreported significantly if patients did not seek for medical care.²³ If a case capture strategy detects moderate to severe cases easier than mild cases, or detects cases with typical symptoms easier than those with nonspecific symptoms, and if vaccinated cases are milder or more atypical than placebo cases, this will result in a too high efficacy estimate.¹⁵ Therefore, in order to avoid missed EV71-associated cases, the active surveillance system we built in the efficacy trial covered all possible cases with suspected clinical symptoms without an obvious cause, by excluding only those with a definite other cause such as burning, traffic accident and so on.¹⁴

However, several other pathogens can also cause the above-mentioned clinical manifestations. Even for HFMD which is often considered as the most typical disease caused by EV71, it can be clinically indistinguishable from HFMD cases caused by other enteroviruses such as Coxsackievirus A16 (CA16), CA4, CA5, CA6, CA7, CA9, CA10, and echoviruses 1, 4, 11, 18, and EV18.²⁴ What’s more, these viruses were frequently found to co-circulate with EV71. Hence, the surveillance covering all the possible clinical symptoms was the first step to capture all the potential EV71-associated disease, which was highly sensitive but not specific.

Laboratory Assays for EV71 for Excluding Non-Case

In a vaccine efficacy trial, specificity of the case-finding strategy is usually higher priority. The indistinguishable clinical symptoms caused by EV71 and other enteroviruses made laboratory pathogen detection especially essential for excluding non-cases, and achieving a high specificity of the case confirming. The classical “gold standard” assay for EV71 is virus isolation and culture followed by neutralization tests with serotype-specific antisera,²⁵ which is both time-consuming and labor-intensive, often requires weeks to obtain the results. Though virus isolation has been considered the most reliable method for diagnosing EV71 and claimed a perfect specificity, but with a low sensitivity due to false negative. Previous epidemiology study showed that the positive rate of EV71 isolation of the throat swab was less than 50% and of that of the rectal swabs or feces was about 30% in acute EV71-associated cases. Even combined throat plus rectal swabs enabled an increase probability of virus isolation, but still imperfect.^26,27 By contrast, molecular methods such as reverse transcription polymerase chain reaction (RT-PCR) have been proved to be more sensitive for EV71 detection than the virus isolation and cell culture method.²⁸ Among 3 common used RT-PCR tests (conventional RT-PCR, two-step semi-nested RT-PCR, and real-time RT-PCR), real-time RT-PCR was priority because it’s more rapid and with lower risk of cross-contamination, although the cost was a little higher.^29,30 In this phase 3 trial, we used a viral RNA diagnostic kit for real-time RT-PCR which was approved by China Food and Drug Administration. However, the high sensitivity of RT-PCR increases the possibility of false positive and results in an imperfect specificity. Both virus isolation and real-time RT-PCR have their own advantages and drawbacks.

In order to ensure good sensitivity without a significant compromise in specificity, EV71-specific real-time RT-PCR assay and EV71 isolation were jointly used to detect the pathogen in this phase 3 EV71 vaccine trial. Specimens were taken from each subject with suspected clinical symptoms for primary pathogen detection by real-time RT-PCR within 24 h after the illness being first reported. The result of the real-time RT-PCR could be received by the investigator within the next 24 h. If a positive in EV71-specific RNA was detected by real-time RT-PCR in any specimen of a child, then serial specimens were taken for EV71-specific RNA testing and virus isolation to exclude false positive cases. A confirmed EV71-associated case could only be identified by at least 2 consecutive positive in EV71-specific real-time RT-PCR with 2 different specimens or 1 positive in EV71 isolation.¹⁴ Usually, the results of RT-PCR and virus isolation are inconsistency, but in this phase 3 trial, 2 consecutive positive in EV71-specific real-time RT-PCR enhanced the specificity of the PCR assay and the serial collected specimens from suspected subjects for virus isolation enhanced its sensitivity. The etiology diagnosis strategy successfully reduced the inconsistency in this trial, so all the confirmed EV71-associated cases were both positive in 2 consecutive EV71-specific RNA tests and positive in EV71 virus isolation.¹⁴

Serum samples were not applied as one of the diagnostic criteria in this trial. Because 50% of the participants had been administrated with EV71 vaccines before the surveillance and the serological detection method could hardly distinguish the vaccine elicited antibody from the natural infection elicited antibody. Besides, the detection for EV71-specific IgM in the acute phase serum was not very reliable: as in the early phase of natural infection cases with a 90% sensitivity but a false positivity proportion of 11%;³¹ but in those participants who were administrated with EV71 vaccines, the detection of EV71-specific IgM showed a much lower sensitivity.¹⁴ Sometimes an increase of at least 4-fold in serum antibody titers in recovering period compared with the acute period was used clinically for diagnosis, but it could not be helpful for the rapid determination of cases as post-event evidence.

Sampling Period and the Clinical Specimen Type

Previous epidemiological studies had demonstrated that EV71 may continuously shed from upper respiratory tracts and feces of HFMD children during the latent period, the acute period, and after recovery.^32,33 The longest EV71 shedding time in throat swab was more than 3 wk and that in feces more than 5 wk after the illness onset. For severe cases the shedding duration of EV71 could be even longer.^24,27 However the sampling time is still essential for the sensitivity of diagnosis. It has been reported that most specimens collected from EV71-associated cases within 0–8 d after onset could be positive in the laboratory detection, but then the positive rate decreased significantly. Two weeks after onset, less than half of the EV71-associated cases could show EV71 positive.³²

In order to capture EV71-associated cases as early as possible, community and village doctors were instructed to update the surveillance data daily, and collect throat swabs, rectal swabs, and feces from children immediately after suspected clinical symptoms were noticed.¹⁴ Therefore, by collecting different type of specimens (throat swabs, rectal swabs, and feces) from a suspected child for assaying we could enhance the sensitivity of laboratory testing further.

After an EV71-associated case was confirmed, serial throat swabs, rectal swabs, and feces from he or she were collected with an interval of 3 d until 2 consecutive EV71 real-time RT-PCR were negative. With these serial specimens from the onset to recovery, we could get a whole picture of virus shedding pattern as the disease progressed. The prolonged EV71 shedding could act as an on-going infection source which may play an important role in outbreaks of HFMD.

Other Potential Factors Affecting the Efficacy Assessment

The prevalence of EV71 is characterized by its regional, seasonal, and periodicity. The epidemic intensity varied in different regions and time, as reported in previous studies.^18,34 The prevalence of the EV71 is largely dependent on the accumulation of the susceptible population. The newborn infants refresh the susceptible pool annually and the nonsynchronous epidemics of EV71 could occur among different regions.³⁵ In an efficacy trial, case surveillance should be performed in a highly epidemic region and the surveillance period should cover the epidemic peak. Therefore, before we chose study sites, data from the Nationwide Notifiable Infectious Diseases Reporting Information System (NIDRIS) for the past 3 y were referred. These regions with a relatively low HFMDs incidence reported in year 2010–2011 would be expected to have a high incidence in 2012 due to the accumulation of susceptible children in previous 2 y. We chose 3 counties in Jiangsu Province and 1 district in Beijing. However, the existing NIDRIS was a passive surveillance system and neither sensitive nor specific enough: a lot of mild cases may be under reported and the reported cases may lack of etiology diagnosis. So the anticipated EV71 prevalence was generally rough. Multi-center trial could increase the possibility of finding EV71 prevalence. Actually, in the surveillance period of 2012 HFMD season, EV71 prevalence was noticed mainly in 2 of the 4 centers, and CA16 dominated in the other.

Previous studies had showed that children under 3 y of age were mostly susceptible to EV71, and those less than 1 y of age were more likely to suffer from severe forms of EV71-associated disease.³⁶ In the EV71 vaccine trial, enrolled participants were aged between 6–35 mo, but nearly 75% of participants were under 24 mo of age. Because the EV71 infection risk is age-related,^36,37 the different age composition of participants may pose a higher risk of EV71 infection compare with that of the natural population aged between 6–35months. Besides, the target population (age 6–35 mo) crosses several routine vaccine times and the effect of concomitant administration of routine vaccines on EV71 vaccine efficacy was unknown.

What’s more, vaccination of a large fraction of the population may lead to herd immunity.³⁸ If a high percentage of a population was immune to an infection, the spread of the infection may be prevented. In that case, participants neither in vaccine group nor in placebo group could be exposed to EV71. The unvaccinated population could also benefit from those vaccinated, which was usually called indirect effect. However, the indirect effect was not what we planned to assess, but what we planned to reduce in the trial.¹⁴ Therefore, the overall vaccine protection against EV71-associated disease including both direct and indirect effects could be expected to be higher. Since the EV71 vaccine candidate went through the phase 3 trial was based on a genotype C4 strain which was in line with the predominant strain in mainland China and all virus strains isolated from confirmed cases in this study were genotype C4, the vaccine efficacy with cross-protection against different EV71 genotypes was not demonstrated. Though evidence showed that neutralizing antibodies induced by the EV71 vaccine could have a strong cross-reactivity to different EV71 genotypes, more solid evidence is needed to show whether the variation between genotypes could compromise the efficacy of the vaccine.

Summary and Commentaries

Two primary endpoints were used in the trial, which are EV71-associated HFMDs and EV71-associated diseases. The latter one could also be called “symptomatic EV71 infection” which consists of the EV71-associated HFMDs and EV71-associated other cases with nonspecific symptoms. The motivation for this choice is the problem that various diseases and manifestations could be caused by EV71 infection, but HFMDs is the most common and typical disease, and also gained most concerns. Given that EV71 infection can present as non-specific disease, laboratory confirmation of suspected cases will be essential for the accurate estimation of the vaccine's efficacy.

In the EV71 vaccine trial, we achieved a case finding strategy with a high specificity and a high sensitivity by a 2-stage approach. The stage 1 is to capture all potential cases presenting with clinical symptoms and have their specimens for the first assay by real-time RT-PCR; this is a very sensitive approach, but not very specificity. After that, in the stage 2, those children we test positive were resampled for a second real-time RT-PCR test, and virus isolation which is the most specific detection method was also applied. These strategies are known to exclude as many non-cases as possible and increase the case specificity. However, carrying an active surveillance on a broad spectrum of clinical symptoms and thousands of specimens collected for testing, the whole process of case capturing and confirmation were both labor-consuming and resource-demanding.

In the trial, non-specific symptoms of EV71-associated disease, e.g., fever, cough, and diarrhea made up a bigger proportion than that of previous epidemiological studies.^14,19,39 We proposed 2 explanations for this. First, active surveillance system was set up in this study to detect the potential cases while the most previous epidemiological studies were retrospective studies or based on a passive surveillance. Therefore, the mild cases and non-specific symptoms were more possible to be detected in our study. Besides, we frequently contacted the guardians of the children and instructed them to seek medical attention if any symptoms were noticed, which further decreased the underreporting rates. Therefore, in our study the sensitivity of case finding was higher than in previous epidemiological studies. On the other hand, the specificity of EV71-associated case confirming in our study was still possibly not perfect because we did not test for pathogens other than enterovirus, it was hard to rule out the possibilities that some confirmed EV71-associated cases might be caused by coinfection with other non-enterovirus pathogens, especially those with non-specific clinical symptoms. Thus, it could be a possible explanation for a numerically higher efficacy (not significant) against EV71-associated HFMDs demonstrated than against EV71-associated disease.¹⁴ EV71-associated HFMD and EV71-associated disease were applied as the co-primary endpoints, which may help to understand the efficacy of EV71 vaccine more comprehensively.

In general, potential factors were carefully considered and well controlled in order to reveal the true EV71 vaccine efficacy in this phase 3 trial. By revealing the whole background story about the efficacy measurement of this trial, we want to share our experiences and knowledge toward the EV71 vaccine efficacy, which should be understood comprehensively and scientifically.

In the future, more efficacy data will be greatly demanded inpost-marketing studies to assess any effect of concomitant routine vaccination on EV71 vaccine efficacy and the cross-immunity between EV71 genotype C4 and other genogroups.⁴⁰

Disclosure of Potential Conflicts of Interest

No potential conflicts of interest were disclosed.

Acknowledgments

This work was supported in part by China’s 12–5 National Major Infectious Disease Program (2012ZX10004703). We would like to thank Dr Jozef Nauta and Yuan-Zheng Qiu for revising and providing helpful suggestions to improve this manuscript.

10.4161/hv.27253