Beyond efficacy: A career evaluating the effectiveness of vaccines

Serendipity was important in determining the course of my career. I went to public high school in Syracuse, New York and subsequently attended Yale College. My bicycle was stolen during the second week of my freshman year as an undergraduate at Yale. Since the science buildings were far from my dorm and I could not understand a word of what the physics professor was saying, I took this as a clear sign from above that I should drop physics. I majored in English and took no science courses as an undergraduate. However, I became interested in psychology/psychiatry (probably mostly to cure myself, something at which I admit to abject failure). Despite having been accepted by a good law school, I decided to try to go to medical school to become a psychiatrist. I took the required pre-med classes and attended University of California, San Francisco School of Medicine. I soon decided that Pediatrics (not Psychiatry) was for me, and did a residency in Pediatrics at Children's Hospital of Pittsburgh, PA, in part because my girlfriend (now wife) wanted to do her Obstetrics and Gynecology residency there.

Dr. Ellen Wald, a pediatric infectious diseases specialist and a first-class clinical investigator, joined the faculty at Children's Hospital when I was a second-year resident. I decided to stay on and do a fellowship in pediatric infectious diseases to work with her. From her I learned how clinical experiences can help inform one to ask the right research questions. This was during the era when Haemophilus influenzae type b (Hib) was the most common cause of bacterial meningitis in the United States, before protein-polysaccharide conjugate vaccines were available to prevent invasive bacterial infections in young children. Working with Dr. Wald, I became interested in how to prevent these relatively common and often devastating bacterial infections. The first study that I did as a fellow helped to determine the appropriate dose of rifampin to use for chemoprophylaxis to prevent secondary cases of Hib in close contacts of children with invasive Hib infections.¹ If used perfectly, chemoprophylaxis might have prevented 1% of all Hib infections in children. Clearly, an effective vaccine was needed.

We wanted to move back to San Francisco, but there was no academic position available for me in the Bay area. A friend told me about the Robert Wood Johnson Clinical Scholars Program at Yale, at that time directed by Dr. Alvan Feinstein, one of the founders of the field of clinical epidemiology. We moved to New Haven and I did a second fellowship and learned the principles of clinical epidemiologic research from the late Dr. Feinstein, who was justifiably renowned as a tough taskmaster. Dr. John D. Clemens had recently joined the faculty and suggested that I do a case-control study of the effectiveness of pneumococcal polysaccharide vaccine in adults as my research project. He was an excellent mentor and the report of the results of the project was published in the Annals of Internal Medicine.² I submitted an R01 proposal to conduct a larger version of the same study, using incident cases of invasive pneumococcal disease throughout Connecticut in which the organism that caused the infection would be serotyped. Shortly after I joined the Pediatric faculty at Yale, the National Institute on Aging funded my proposal (for many years I would receive requests for my C.V. from centers that were looking for a geriatric immunologist!). During the subsequent 10 years, Dr. Robert Austrian serotyped for me nearly 2,500 strains of pneumococcus isolated from normally sterile sites. This was before molecular methods were readily available. He typed all strains manually using the Quellung reaction with sera from the Statenserum Institute.

By conducting that study, I learned the impact that a well-conducted case-control study of a vaccine's actual effectiveness in clinical practice could have. I realized that such studies are important; clinical trials typically are done in selected populations, so the generalizability of the results may be problematic. For example, the efficacy of pneumococcal polysaccharide vaccine was tested in young goldminers in South Africa who lived in crowded conditions and who worked in poorly ventilated mines, but the results were generalized to persons in the United States who were at risk because of their age and/or underlying medical diseases. In addition, because of logistical challenges and expense, clinical trials typically assess the efficacy of a vaccine for a short time, often in very high risk populations, so the question of the vaccine's efficacy over time often remained unanswered. I also continued to develop my interest in methodologic approaches to minimize and to assess possible bias in observational studies. For example, in the case-control study of the effectiveness of pneumococcal polysaccharide vaccine, we showed that the vaccine was effective in preventing infections by serotypes in the vaccine, but, using identical methods, was not effective in preventing infections due to types not in the vaccine, which supported the validity of the study.³

Subsequently, protein-polysaccharide vaccines for Hib were developed and eventually were approved. I was involved in some immunogenicity trials and assessments of the effectiveness of the conjugate vaccine in preventing invasive Hib infections after it was approved. Its impact was dramatic, with a rapid and large decrease in the incidence of invasive infections due to Hib. This led to accelerated interest in development of conjugate vaccines against pneumococcus for children. I also was involved with several prelicensure studies and planning for pivotal clinical trials for this vaccine, although in the end I did not conduct any clinical trials of its efficacy.

I continued to conduct studies of the effectiveness of vaccines in clinical practice, including studies of the effectiveness of Lyme vaccine and of varicella vaccine. We showed that varicella vaccine's effectiveness dropped substantially, from 97% to 84%, after the first year following immunization.^4,5 These results helped contribute to a change in recommendations for routine immunization from 1 dose to 2 doses for all children. I also continued to pursue my interest in methodologic issues to minimize and to assess the impact of confounding and bias in observational studies of a vaccine's effectiveness. This led me to conduct (and eventually to publish the results of) “sham” studies in which sham outcomes and/or sham exposures were used to assess whether confounding or bias might have affected the results of the studies. In a case-control study in which cases were children with invasive infections due to Hib, Hib vaccines were highly effective. Using exactly the same methodology for selecting controls and to ascertain prior immunization, when cases in the case-control study were children with invasive pneumococcal infections (“sham cases”), as expected Hib vaccines were not effective at all in preventing invasive pneumococcal infections in children.⁶ This was a demonstration of the validity of the case-control method of assessing the effectiveness of vaccines in clinical practice. Likewise, in the study of varicella vaccine's effectiveness, we showed that the vaccine was effective against PCR-confirmed cases of varicella but was not effective against potential cases that were PCR-negative (ie, “sham” cases who really did not have varicella). In addition, MMR vaccine (administered at about the same time as varicella vaccine—a “sham” exposure) also was not effective in preventing varicella, another demonstration that confounding and bias were not substantially affecting our results.

In recent years, teaching and mentoring have become a larger component of what I do, although I continue to pursue my interest in evaluation of vaccines. I have worked with Dr. Daniel Weinberger, who has evidence that, in countries in which there is high uptake of conjugate pneumococcal vaccine in children, the best strategy for preventing serious pneumococcal infections in adults may be to develop a separate adult-specific conjugate pneumococcal vaccine with a formulation that mostly contains different serotypes than the child-specific vaccine.^7,8 He has developed a method to determine which serotypes are most likely to emerge as causes of disease in adults in populations in which there is good uptake of the conjugate in children. I am also involved in a case-control study of the effectiveness of HPV vaccine in actual practice. Current use of HPV vaccine is a good example in which the gap between the vaccine's efficacy in clinical trials in patients who have not been infected with HPV types in the vaccine, and its effectiveness in actual practice, when patients may receive the vaccine after having been infected, may be particularly large.

I now spend a lot of my time on work related to my position as Co-Director of Education for the Yale Center for Clinical Investigation. As such I help direct Yale's KL-2 and TL-1 programs as part of our CTSA grant. In addition, I am co-PI of our T-32 training grant in Pediatric Infectious Diseases. I serve as primary mentor for several young faculty with K awards. I also direct graduate level courses in clinical epidemiology as well as 2 semester-long intensive courses on how to write grants. I think that helping young people to become successful investigators likely is the most important thing that I can do, hopefully transmitting to them some of the skills that I learned from my mentors about asking the right questions and conducting methodologically sound studies. I hope to continue to be able to do this for some time.

Additional information

Notes on contributors

Eugene Shapiro

About Dr. Shapiro. After attending public high school in Syracuse, New York, Eugene Shapiro got a B.A. degree in English literature at Yale College and an M.D. at University of California, San Francisco School of Medicine. He did a residency in Pediatrics and a fellowship in Pediatric Infectious Diseases at Children's Hospital of Pittsburgh before returning to Yale where he obtained advanced training in clinical epidemiology as a Robert Wood Johnson Clinical Scholar. He then joined the faculty at Yale where he has conducted research on the effectiveness of vaccines as well as in several different areas. He is currently Professor of Pediatrics, of Epidemiology of Microbial Diseases and of Investigative Medicine at Yale University, New Haven, CT. He is co-director of education for the Yale Center for Clinical Investigation/CTSA, Associate Chair of the Department of Pediatrics and Deputy Director of Yale's PhD Program in Investigative Medicine