Measurement of vaccine-derived immunity: how do we use all the data?

Keywords::

• infectious disease control
• statistics
• vaccine

The distribution of vaccine-derived immunity is central to disease control efforts, but is often poorly known. In this editorial, we outline why evaluating the degree of vaccine-derived immunity is both important and problematic, and discuss the strengths and weaknesses of different methods that have been used to approach this challenge.

Since Arthur Ransome postulated the critical role of population susceptibility in epidemic dynamics [1], it has been understood that population immunity plays an important role in disease control. Knowledge of underlying levels of population immunity allows us to appropriately target vaccination and anticipate outbreaks. A population’s immune profile is the result of its demography, its history of infection and past vaccination activities (both campaigns and routine coverage). Because vaccination results from the actions of the public health community, one might assume that measuring the effect of vaccination activities on population immunity would be straightforward. However, the information public health officials have about the distribution of a vaccine is often inadequate for making strong inferences about the immune status of the population. In nations with less developed healthcare systems, where knowing population susceptibility might be most valuable, even the size and demographics of the underlying population may not be well characterized. Hence, we rarely have accurate direct estimates of population vaccination status and resulting vaccine-derived immunity.

Measurement of vaccine-derived immunity

The numerous impediments to estimating the amount of vaccine-derived immunity in a population include uncertainty in the size and demographics of the target population; uncertainty in how many people are ever vaccinated and how many doses those that are vaccinated receive; spatial or demographic clustering in vaccine uptake; and uncertainty in the immunogenicity of the distributed vaccine. These barriers can combine to create substantial uncertainty in the amount of vaccine-derived immunity in the underlying population, especially in particular subpopulations. The combination of uncertainty in vaccine-derived immunity and the fundamental properties of long-term epidemic dynamics (e.g., the honeymoon period [2]) can lead to large outbreaks, surprising countries with apparently effective control efforts, for example, the large measles outbreaks that occurred in Malawi and Zambia recently despite years of apparently successful control and high nominal vaccination coverage [101].

Fortunately, there are several options for collecting data on population immunity and vaccination coverage. Each takes a different approach to measuring the immunization status of the population, and each has its own caveats, costs, strengths and shortcomings. Much of this information is collected regularly as part of routine surveillance and public health activities and is aggregated at the national level. However, often owing to logistics or cost, some useful data (e.g., population serostatus) are rarely collected, despite their potential utility (Table 1).

Integrative approaches

There are often differences in the vaccination levels measured in different ways, and substantial variability in data of a particular type from country to country and year to year. Heuristic approaches combining data with expert opinion have been developed to integrate this information to infer population vaccination status [3]. These methods often provide a clearer picture than that offered by any individual data stream. The UNICEF–WHO estimates are often (but not always) lower than country-reported coverage based on administrative measures [102] and appear to be more consistent with subsequently observed disease incidence. However, these approaches are not based on any systematic statistical framework, hence are hard for others to reproduce, do not include measures of uncertainty and may be difficult to use in comparative analysis or predictive models.

More systematic statistical approaches have been developed to integrate information on vaccine coverage from multiple sources of data, for example, administrative coverage and cross sectional surveys [4]. Statistically grounded approaches provide a reproducible, consistent and replicable method for estimating vaccine coverage, and therefore, can be used to infer properties of population vaccine status that have not been observed directly, such as the number of individuals who have received two doses of a vaccine. These approaches may be particularly useful in confronting one of the biggest challenges in estimating coverage after vaccination campaigns: the extent to which the populations covered by different vaccination activities are independent (i.e., the correlation between an individual’s probability of being covered in one vaccination activity versus other activities).

Despite their advantages, statistical models are only as good as the quality of the data and assumptions that go into them. Some countries’ vaccination or reporting systems may not fit into the framework assumed by the model, or there may be secular changes in a country’s immunization program not captured in the modeling approach. In this latter case, heuristic approaches may have a particular advantage, incorporating information on events and policy changes not integrated into the statistical model.

Measuring vaccination coverage may be important in its own right, but represents only the first step in estimating the distribution of a vaccine-derived immunity in the population. Vaccine-derived immunity can be measured directly through serosurveys, predicted based on the age-specific distribution of vaccine or inferred from the age distribution of cases. Direct measurement through serosurveys is relatively expensive and is unlikely to be done in a large sample of the population. Furthermore, in many cases (e.g., measles), it is not possible to separate vaccine-derived immunity from immunity resulting from previous infection, though questionnaires may help to distinguish the two. However, this approach may be important for evaluating the immune status of important subpopulations (e.g., HIV-infected vaccine recipients [5]), and empirical measures of population immunity (especially if vaccination status is known) could be used to refine predictive approaches.

Vaccine efficacy is known to vary by age, particularly for diseases like measles where maternal antibodies have a strong effect. Age-specific estimates of vaccine efficacy can be combined with data on vaccination uptake by age and population demographics to estimate vaccine-derived immunity [6]. Because this approach is based on predicting immunogenicity rather than direct measurement, it may lead to inaccurate estimates if there are unanticipated reasons for vaccine failure, for example, variation between populations and cold chain lapses.

A final method for measuring vaccine-derived immunity is the analysis of the distribution of cases during an outbreak. This approach has broad appeal, as it is based on routinely collected data and a direct indicator of immunity (protection from clinical disease). However, numerous assumptions are needed to distinguish the effect of vaccine-derived immunity from natural infection and to account for differences that naturally arise from the transmission dynamics of the disease. Perhaps the biggest drawback of this method is that the assessment of vaccine-derived immunity may come after susceptibility, which has already accumulated in the population and an outbreak has occurred. Despite these shortcomings, combining this approach with more direct assessments of vaccination coverage has proven useful in a number of contexts for assessing vaccine coverage and the reasons for outbreaks [7,8].

No matter what method is used for measuring vaccine coverage, it is important that it is reported and interpreted appropriately. In general, vaccine coverage is applicable only to some defined population, whether a particular birth cohort (for routine vaccination), a target age range (for vaccine campaigns) or a surveyed population. Even if coverage is very high in the target group, this may not generalize to the wider population. While specific measures of coverage may be well defined, their specific meaning is not always well articulated when estimates are communicated or when incorporated into policy recommendations. Specific communication of the age range that coverage is being reported at and what this implies for overall population immunity is critical, particularly when combining information from data sources with differing reference populations.

Conclusion

The global public health community and funding organizations rely on accurate measurement of vaccine coverage. Many organizations tie funding to the performance of vaccination activities and depend on accurate assessment of coverage in order to evaluate priorities. Elimination efforts for polio and measles depend on ensuring high coverage, particularly in countries where reintroductions are likely. Efforts to expand rubella vaccination to more countries are based on those countries’ ability to maintain high vaccination rates (because of the risk of increasing congenital rubella syndrome incidence). Because of the growing importance of coverage estimates, researchers and public health organizations should use data from as many sources as possible when assessing performance and making policy decisions. In doing so, they should use methods that combine these data in a logical and consistent manner, whether through statistical methods or well-validated heuristics. Because each method has its own strengths and weaknesses, vaccination coverage should not only be assessed using as many sources of data as possible but using multiple methods as well. When methods agree, it can increase our confidence in our conclusions. When methods disagree, we should proceed cautiously, and additional data collection activities should be considered. Methodological innovation and incorporation of innovations into public health practice can help ensure that policy is based on the broadest and strongest evidence base possible.

Table 1. Approaches to measuring vaccine coverage.

Survey	Cost	What is measured?
Administrative coverage (nominal doses over target population)	Low	Estimated vaccine coverage in target population
Ad hoc coverage	Moderate	Vaccine coverage
Randomized population	High	Reported, age-specific vaccine coverage
Serological	Very high	Overall or vaccine-derived immunity
Disease incidence	Low	Vaccine-derived and natural immunity estimated from models and attack rates

Financial & competing interests disclosure

The authors would like to acknowledge the support of the Bill and Melinda Gates Foundation (Vaccine Modeling Initiative, 705580-3) and the RAPIDD program of the Science & Technology Directorate, Department of Homeland Security. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.