Evaluation of autoimmune safety signal in observational vaccine safety studies

Abstract

Autoimmune safety evaluation is an important component of post-licensure vaccine safety evaluation. Recently, we published the findings from a large observational safety study of the quadrivalent human papillomavirus vaccine in females. From this study, based on two large managed care organizations, we have obtained some empirical data that may prove useful for the design of future vaccine safety studies within a managed care environment. For autoimmune conditions, a major challenge in vaccine safety study is to determine true incident cases in relation to the timing of vaccination. We found expert case review of medical records an indispensable component for autoimmune safety studies based on electronic health records. Case identification should also be expanded to include the use of laboratory test results or other relevant measures in addition to the disease specific ICD-9 diagnosis codes, when applicable. Furthermore, we recommend the parallel use of both safety signal evaluation that involves pattern evaluation for conditions that are more common, and statistical comparisons for conditions that are rather rare. Finally, we recommend an accompanying vaccine uptake study to understand the potential selection bias and confounding in a given study population that should be addressed with data collection and analytical techniques.

Keywords: :

• autoimmune conditions
• vaccine safety
• post-licensure safety study
• methodology
• HPV vaccine

Vaccine induced autoimmunity has been a long standing concern surrounding vaccination. Despite that fact that established vaccine induced autoimmune reactions have been very rare, given the biological plausibility and historical precedents, autoimmune evaluation should be considered an important component of post-licensure vaccine safety evaluation. In the United States, integrated managed care organizations provide one of the most efficient environments for such post-licensure safety studies. As the FDA now mandates large post-licensure safety study from vaccine manufacturers as part of the regulatory requirements, standard methodologies for detecting various types of adverse events, including autoimmune safety, within the context of a health delivery system should be discussed. This kind of methodology development is expected to be complementary to existing efforts of vaccine safety evaluation such as the Brighton Collaboration.

Recently, we published the findings from a large, post-licensure observational safety study of the quadrivalent HPV vaccine in females.¹ Autoimmune conditions are one of the three major outcomes of the safety evaluation in this study. From this study, based on two large managed care organizations, we have obtained some empirical data that may prove useful for the design of future vaccine safety studies within a managed care environment. We show the limitation of case identification approach based on disease-specific ICD-9 codes, and demonstrate the utility of expert medical chart review, laboratory test results and an expanded list of ICD-9 codes. We also provide the picture of autoimmune disease “lead time” based on cases in our study population, which can inform the design of future vaccine safety studies. In this article, we highlight some of these results and their implications for evaluating autoimmune safety signals.

Identification of New Onset Autoimmune Conditions following Vaccination

In managed care systems, the identification of most diseases relies on electronic diagnosis coding, such as ICD-9 codes applied in the context of health care visits. The sensitivity of ICD-9 diagnosis coding for new onset autoimmune conditions following vaccination, however, cannot be taken for granted. Because the initial presentation of autoimmune conditions often involves general symptoms, there is often a lag between initial symptom onset and the correct assignment of diagnosis. To address this, several strategies were employed in this study: (1) broad, highly sensitive case identification criteria were used, (2) expert panels were employed to confirm the diagnosis and date of disease onset, (3) a 180-d risk period was used to accommodate lag time for clinical work-up, and (4) only females with 12-mo health plan membership prior to vaccination were included to allow reasonable assessment by the expert panels for onset prior to vaccination. We expanded the initial case identification criteria beyond diagnosis codes to also include abnormal laboratory test results related to the autoimmune conditions of interest, such as rheumatoid factors for rheumatoid arthritis. We also used an expanded list of ICD-9 codes (such as type 2 diabetes codes to capture type 1 diabetes, see Table S1), in addition to the use of the original disease ICD-9 codes. We then submitted possible cases for detailed case review by expert physician panels. They manually reviewed medical records to confirm the incident diagnosis. With these approaches, we found that half of the potential Hashimoto’s disease cases identified by laboratory tests alone (i.e., without a diagnostic code) were confirmed to be new-onset cases. In fact, incident cases identified by lab results accounted for half of the total confirmed new on-set cases. The expanded ICD-9 codes also captured 40% of all confirmed new on-set cases of the autoimmune conditions of interest, with 24% of those identified by expanded ICD-9 codes were confirmed as new-onset cases. These findings emphasize the limited sensitivity of disease-specific ICD-9 diagnosis code based approaches for capturing new onset cases for at least some autoimmune conditions. In our study, we were able to improve the capture of Hashimoto’s thyroiditis with the lab results and with expanded ICD-9 codes for most autoimmune conditions.

The in-depth case review was also very important. We found that among the potential new-onset cases for all autoimmune conditions identified electronically, 30% were actually pre-existing cases. These subjects had symptom onset or a diagnosis in the chart note prior to the electronic coding for the disease. Of note, an “unmasking” phenomenon was discovered,² in which a diagnostic lab test was drawn on the day of vaccination, leading to the disease diagnosis at the first time in the days following vaccination. The “unmasking” refers to the fact that vaccination visit may have provided the opportunity for the evaluation of symptoms that may not have been pursued due to less regular health care visits in comparison to the pediatric population with routine well-child visit. This unmasking phenomenon was illustrated in details using the example of Grave’s disease in another manuscript.² These data again emphasize the limitation of ICD-9 diagnosis code based approaches for assessing the timing of the onset of autoimmune conditions, and highlight the need for in-depth medical record review in vaccine autoimmune safety studies based on electronic health information.

Another consideration for detecting new-onset autoimmune condition is the length of the risk window, or follow-up period allowed in the study. To adopt an appropriate length of risk period, it is necessary to consider the induction time as well as the lead time for the disease of interest. The induction time of various autoimmune conditions has not been well characterized. Experimental data are available for autoimmune disease in murine models, although it is unclear how this applies to the disease course in humans. In general, it is believed that drug induced immune-mediated events can start 10–14 d after exposure. In the case of GBS, the disease may develop in the course of days or up to 3–4 weeks.³ In addition, an extended lead time can be expected for autoimmune conditions for which the initial presentation often involves general symptoms. This leads to a delay in seeking medical attention, as well as a delay in the assignment of the correct diagnosis by the physician. It is, however, somewhat feasible to characterize the lead time for autoimmune diagnosis with epidemiologic studies. In our study, based on 41 confirmed new-onset cases, the distribution of days between first symptom onset (as noted in physician chart notes) and electronic diagnosis coding for autoimmune disease ranges from 0 to 1,210 d, with a median of 23 d and 25th and 75th percentile of 2, 59 d. Five cases (12%) had a lag between symptom onset and clinical diagnosis coding greater than 180 d, suggesting that a risk window of more than 6 mo may be needed to fully capture all true incident cases. Consideration of the length of risk windows for the completeness of case capture is therefore to be evaluated against the need for the timeliness of safety study results.

The accumulation of data such as these could be helpful to characterize the diagnostic course of the autoimmune conditions for various study populations, as this may vary by age, sex and race. For example, one study found that the use of primary health care services was greater among adolescent females compared with adolescent males.⁴ This data suggest that our “lead time” derived among adolescent females for autoimmune conditions may not be applicable to males. A longer risk window may be needed to capture the majority of incident autoimmune cases developed after vaccination, as well as to rule out prevalent cases for adolescent males. The relatively long lead time for the diagnosis of autoimmune conditions is also relevant for the validity of safety signal evaluation. Given that “lead time” (which includes the clinical undetectable phase and the delay in diagnosis) may be related to health care seeking behavior, the distribution of lead time may be different in groups defined by vaccination status.

Autoimmune Safety Signal Evaluation

There are generally two epidemiologic approaches to evaluate safety signal for autoimmune conditions. One is based on the pattern of disease onset to assess if there is clustering by age, dose or timing of onset. The other is based on a statistical comparison of disease incidence between groups defined by vaccination status. The potentially differential lead time of diagnosing autoimmune conditions therefore may affect a statistical comparison of those who received vs. did not receive the vaccine. This is probably less of a problem for evaluating patterns of disease onset among only the vaccinated subjects. However, this approach of pattern evaluation may be limited due to small sample sizes inherited to studies of rare diseases. This probably holds true for most autoimmune conditions. The statistical comparison approach is therefore essential for these rare conditions. One way to address the concern of lead time bias is to conduct case review of potential new onset cases arising from unvaccinated subjects as well. The benefit of this approach is 2-fold: (1) disease incidence in the unvaccinated population can be estimated with confidence, and (2) these data may inform how lead time may vary between those who seek the vaccination vs. not, which will inform the design of future autoimmune safety studies. While this approach may be more resource intense, focusing on only rare events that are not suitable for pattern evaluation makes it more affordable. Furthermore, sampling strategies, combined with statistical imputation techniques, may be used to contain the scope of work.

Accounting for the Impact of Potential Confounders

While there are few well established risk factors for autoimmune conditions, factors such as age, sex, race, genetic predisposition and certain environmental exposure should be considered in observational studies evaluating autoimmune safety signals. Imbalance of these factors between the vaccinated and the unvaccinated populations may introduce confounding in a crude statistical association. While information on genetic polymorphisms, family history and environmental exposures are often not readily available, other risk indicators, such as age, sex, race, history of autoimmune or other diseases related to immune dysfunction can be assessed by electronic health records within the managed care organization. Knowledge about the distribution of these factors among groups defined by exposure status, in this case, vaccination should ideally be obtained prior to the analysis. To this end, we conducted an uptake study to examine correlates for HPV4 uptake in our study population.^5,6 Interestingly, we found that females with a history of autoimmune conditions were not less likely to receive the HPV vaccine compared with females without history of autoimmune conditions. This is also the case for females with other immune modulating diseases such as asthma or allergies. These uptake studies provided insight for the potential self-selection process for vaccination for the population under study (i.e., early adopters of the vaccine for most safety studies), and are important to inform the development of analytical approaches. Should an imbalance be found, techniques such as weighting, stratified analyses or multivariable modeling should be considered.

Caveats for the data presented here include the small number of autoimmune cases these data are based on, and the fact that these data are derived from one organization which is a highly integrated health care system. Therefore, more studies are needed to help better characterize the diagnostic course and capture of incident autoimmune conditions.

In summary, we find expert case review an indispensable component for autoimmune safety studies based on electronic health records. Case review results should also be summarized and made available to inform the design of future safety studies. Case identification should also be expanded to include the use of laboratory test results or other relevant measures in addition to the disease specific ICD-9 diagnosis codes, when applicable. Furthermore, we recommend the parallel use of both safety signal evaluation that involves pattern evaluation for conditions that are more common, and statistical comparisons for conditions that are rather rare. Finally, we recommend an accompanying vaccine uptake study to understand the potential selection bias and confounding in a given study population that should be addressed with data collection and analytical techniques. The accumulation of this knowledge is critical for increasing the confidence and efficiency of autoimmune safety evaluation for newly introduced disease preventing vaccines. The goal should be to assist the development of standard methodologic recommendations for autoimmune safety evaluation in the US.

Acknowledgment of Funding

C.C. and S.J.J. received research funding from Merck and Co for studies related to the quadrivalent human papillomavirus vaccine. S.J.J served as an unpaid consultant to Merck and Co.

Supplemental Material

Supplemental material may be found here: www.landesbioscience.com/journals/vaccines/article/21268