3,544
Views
1
CrossRef citations to date
0
Altmetric
Clinical – Review

Patient-reported outcomes in vaccines research: relevance for decision-making

Pages 1-8 | Received 23 Oct 2020, Accepted 07 Jan 2021, Published online: 19 Feb 2021

ABSTRACT

The development and demand for effective vaccines have witnessed an exponential growth over the last century. In the meantime, the vaccine market involves more knowledgeable stakeholders, with a shift in emphasis by regulatory agencies on understanding the patient perception and experience. The Food and Drug Administration’s publication of the patient-reported outcomes (PRO) guidance has elevated the discipline of PROs and has resulted in a transition from clinician reports of patient outcomes to PROs. This review reports various research methods, which utilize PROs, including qualitative and quantitative research, clinical trials, and patient preference studies. With the advancement of electronic PRO data capture, additional advantages of PROs are being observed and utilized (e.g. as a trigger for clinical endpoints). We discuss uses and advantages of including PROs into the clinical trial program to improve efficiencies, clinical relevance and overall validity of the program in the vaccine field. (See Plain Language Summary)

PLAIN LANGUAGE SUMMARY

What is the context?

  • Potential vaccine recipients want to understand the benefits and risks of a vaccine directly from the patient perspective. Well-defined Patient-reported outcomes (PROs) provide this perspective.

  • The Food and Drug Administration (FDA) bases their approvals for interventions on how the patients feel, function and survive, with PROs providing important quantitative estimates of how patients feel and function.

What is new?

  • The FDA and European Medicines Agency (EMA) have developed frameworks that ensure that patients’ experiences, perspectives, needs, and priorities are captured and meaningfully incorporated into drug/vaccine development and evaluation.

  • This has led to an increased interest in PRO evaluations by other stakeholders including regulatory authorities, ministry of health, health technology assessment bodies, national immunization technical advisory groups (NITAGs), payer groups, key opinion leaders, healthcare providers and potential vaccine recipients.

  • The availability of new technologies (e.g. smartphones) has increased the role of virtual observational and clinical studies, using mobile clinical trial platforms assessing PROs with no in-person site visits required.

What is the impact?

  • PROs may be incorporated into the research and development program of a vaccine using virtual technology, resulting in a more representative sample that is easier to recruit and retain. This introduces efficiencies and improves the clinical relevance and validity of the clinical trial program.

  • The outputs of studies involving PROs are important to communicate the value of vaccination from a patient perspective.

  • PRO data may also be included as inputs in public health and cost-effectiveness models, to further inform decision-makers on the value of vaccination.

Introduction

Over the last century, there has been an exponential growth in the development of effective vaccines to prevent infections, providing substantial benefits to society.Citation1 The increasing demand for vaccines is due to factors such as identification of new pathogens, ongoing global immunization campaigns, increased use of vaccines beyond the pediatric segment, notably in older adults and pregnant women, and development of therapeutic vaccines.Citation1,Citation2 In the meantime, the vaccine market encompasses more knowledgeable customers, growing cost pressure from payers, and further need for asset differentiation in a very competitive market, i.e. going beyond traditional endpoints of disease prevention.Citation3 Potential vaccine recipients want to understand the benefits and risks of a vaccine directly from the patient perspective, driving the increased focus on patient-reported outcomes (PROs). As experts on what it is like to live with their condition, patients are uniquely positioned to inform this context for vaccine development and evaluation. In many countries, patient views are now included as part of the HTA (Health Technology Assessment) processes.Citation4

A prophylactic or preventative vaccine involves introducing antigens into an otherwise “healthy” person’s body.Citation5 The goal is to induce the individual’s immune system to produce an immune response against those foreign antigens thereby becoming immune to the associated illness, and as such, preventing infection/disease (e.g. rotavirus gastroenteritis, respiratory syncytial virus, and invasive meningococcal disease). Therefore, it is important to understand the impact of disease development and disease prevention on how a patient feels or functions from a patient/vaccine recipient perspective. It is worth noting, however, that prophylactic vaccines are not guaranteed to be 100% effective in preventing disease. Even when a vaccine may not prevent disease it may result in the reduction of disease severity in those individuals who develop disease post-vaccination (e.g. rotavirus, pertussis, influenza, herpes zoster),Citation6–8 thereby reducing the impact of disease on how the patient feels or functions.Citation9,Citation10 Therapeutic candidate vaccines are meant to help the body defend against an illness that is already present (e.g. herpes simplex virus, chronic hepatitis B), and against health threats including viruses such as oncoviruses.Citation2 In such a setting, it will be important to demonstrate that how a patient feels and functions is maintained or improved through vaccination compared to the outcome where individuals are not vaccinated.

PRO instruments are used to measure the experience of disease-related symptoms (e.g. zoster specific pain, using the zoster brief pain inventory [ZBPI]),Citation11 disease impact (e.g. sleep or mood disturbance from zoster pain using the ZBPI),Citation11 and larger health-related quality of life (HRQL) concepts which may be disease-specific (e.g. cancer specific such as the European Organization for Research and Treatment of Cancer quality of life questionnaire [EORTC QLQ-C30]),Citation12 or more general (e.g. Short-Form-36 [SF-36]).Citation13

The Food and Drug Administration (FDA) has developed a patient-focused drug development (PFDD) initiative.Citation14 This framework is a systematic approach to help ensure that patients’ experiences, perspectives, needs, and priorities are captured and meaningfully incorporated into drug/vaccine development and evaluation. Similarly, the European Medicines Agency (EMA) has developed a framework for interaction between the EMA, patients, consumers and organizations.Citation15 FDA approvals are based on demonstrated improvements in how patients feel, function or survive. The FDA defines PRO endpoints as direct assessments of the impact of an intervention on how a patient feels or functions.Citation16 Traditional objectives such as pulmonary function, tumor size, or biological markers do not directly assess improvements in how patients feel or function and are consequently defined as indirect assessments. Thus, the FDA requires evidence that the treatment effect for an indirect assessment reflects a treatment effect on how patients survive, feels or functions (direct assessment), to demonstrate evidence of clinical meaningfulness.Citation16,Citation17

PROs have served as a primary endpoint in pivotal phase III clinical trials when developing a vaccine, as seen in the Merck Shingles Prevention Study, but have also emerged as important vaccine secondary and/or exploratory endpoints for regulators, HTA bodies, payers, policy experts, healthcare providers and patients.Citation18,Citation19

Patient-reported outcomes (PROs)

PROs are a subset of a larger family of assessments known as clinical outcome assessments (COA) that may be used to understand the broad impact of a new health technology. The four types of COAs are PRO measures, clinician-reported outcome (ClinRO) measures, observer-reported outcome (ObsRO) measures and performance outcome (PerfO) measures.Citation16,Citation17 A PRO is a measurement based on a report that comes from the patient or “healthy” individual about the status of their health condition without amendment or interpretation of the report by a clinician or anyone else.Citation17 PROs include patient symptom diaries which may assess attributes of duration or severity of symptoms, questions on aspects of function such as physical function or activities of daily living or emotional impacts (e.g. fear of contagion), in addition to multi-item, multi-domain instruments measuring aspects of HRQL. HRQL is a multidimensional measure of the health and treatment experience of the patient, generally involving physical, social, role, and emotional domains. In the US, 75% of PRO label claims were granted for signs and symptoms, 13% for activity limitations and 13% for HRQL. While in the EMA, a higher proportion (31%) of PRO label claims were granted based on HRQL endpoints.Citation20

Robustness of PROs as an endpoint within a clinical trial

Although PROs are sometimes considered to be subjective in nature, it is important to note that prior to inclusion of a PRO instrument into a clinical trial to support endpoints for regulatory claims, the instrument needs to have established criteria for content and construct validity, reliability, responsiveness, interpretation and acceptability.Citation17 The instrument also needs to measure a defined concept of patient experience of disease or treatment that is relevant for the target population (content validity). Development of a PRO instrument is a long and complex process involving multiple steps: establishing the concepts to be measured based on qualitative research with patients (content validity); identifying the items which represent the concepts and defining an appropriate response scale and recall period; determining how items should be combined to form domains; appropriate scoring, and assessing psychometric properties including scale validity and measurement precision; and developing guidelines for interpretation. Several guidelines for assessing the properties of PRO instruments have been developed by international societies.Citation21,Citation22 In addition, the FDA has published recommendations for establishing content validity and assessing the psychometric properties of PROs.Citation17 When instruments that are fit for purpose for the target population are employed together with appropriate statistical methods, consistent and robust results across clinical studies are observed.Citation9,Citation10,Citation23

PRO instrument selection

The choice of PRO instrument will be strongly influenced by the value proposition of the product, which includes statements on the target population, the clinical benefit (e.g. prevention of painful conditions), the health economic benefit (e.g. utility or quality adjusted life year [QALY] gain) and the societal perspective (e.g. promotes healthy aging\HRQL and/or quality of life of a caregiver).

Broadly speaking, disease\symptom-specific instruments may be more influential from a regulatory perspective, utility questionnaires from a recommendation and reimbursement perspective and generic questionnaires from a patient perspective. However, all instruments may influence decisions across different areas of the decision-making process.

Research methods using PROs

To build credible PRO arguments, one needs data showing that the new intervention has a significant impact on endpoints that are meaningful to healthcare decision-makers, patients and/or society. These endpoints can be disease severity, disease impact, or patient preferences for one health state over the other.

Qualitative research

The FDA has expanded the role of patient-focused drug development and encourages the collection of patient experience data through qualitative research to augment traditional clinical trials as part of the regulatory submission. Qualitative research provides an opportunity to elicit patient’s voices to better understand the patient experiences including disease symptoms and their related impacts and treatment experience of disease, and support a development of a well-defined PRO strategy. Qualitative research with patients may be performed through focus groups, one-on-one interviews (as seen in concept elicitation studies), or novel methods such as real-world data analysis of social media postings or/and moderated on-line discussions.

To establish content validity to support a PRO, it is fundamental to include the patient, or their caregiver, when the patient is not able to self-report (e.g. young children; adults with dementia), in developing new instruments and when investigating if existing PRO instruments are fit-for-purpose to assess and measure the concepts of interest (e.g. cognitive debriefing) in the target population. In-depth interviews, as opposed to focus groups, allow for more granular individual insights and experiences to be uncovered. The analysis of qualitative research data involves coding and analysis of patient data (e.g. transcripts of interviews) using structured methods to identify and group concepts.

Quantitative research on disease impact

Quantitative research provides an opportunity to quantify the burden of disease and quality of life impact on a sample of individuals. These data are important in increasing the level of knowledge and awareness about the disease and its complications. PRO instruments measure concepts ranging from discrete symptoms or signs such as cough severity or frequency of cough, to the overall state of a condition, where both specific symptoms and the impact of the condition on function, activities, or feelings can be measured alongside feelings about the condition or treatment/vaccine (e.g. fear of getting infected; infecting others, disease impact; having to avoid certain situations).

The question of what instruments to use should be driven by both qualitative research and literature reviews to ensure that the concepts that are relevant to the patients are appropriately captured using instruments that are fit for purpose in the target population. For example, for herpes zoster, it is important to assess the pain/allodynia/pruritis and the associated impact on activities of daily living, and health-related quality of life. As such, the ZBPI disease-specific instrument and generic PRO tools such as the SF-36 and EQ-5D may be appropriate. However, if existing PRO instruments are not adequate, it may be necessary to either modify an existing instrument or develop a new instrument.

Clinical trials

When developing the product profile and value proposition, it is important to consider how the vaccine could potentially impact the patient from a patient-centric perspective. Even at this early stage, it is important to consider what wording could be included in the regulatory label to support the product profile, the value proposition and promotional claims (i.e. starting with the end in mind). Concepts should be identified which best illustrate how the disease is altered by the vaccine under development in the study population. It is imperative to understand the impact of disease on PROs (i.e. disease process, signs and symptoms) from previous qualitative and quantitative research and the rationale for any potential vaccine effect on the disease. This may include assessing the overall reduction in disease impact and/or reduction in disease impact in breakthrough subjects.Citation9 Based on literature reviews and external opinion, a conceptual framework should be hypothesized to support the measurement of the concept of interest and identify the domains and items to be measured, which must be subsequently validated by patient involvement.Citation17 Various PRO instruments may already exist which measure this concept in the disease area of interest and in the relevant population matching the development program.Citation17

It is important to define the role of a PRO endpoint in the clinical trial as either a primary, key secondary, or exploratory endpoint. Note that historically for FDA label inclusion, generally only primary or secondary endpoints are considered,Citation17 whereas the EMA has considered exploratory endpoints if those show clinically meaningful effects.Citation15 The objectives and endpoints developed within the clinical trial should be developed bearing in mind the potential label wording to support promotional claims.Citation17 Consequently, the instrument to be used, the frequency of assessment, recall period, definition of the endpoint and statistical methods are all critical elements in assuring that the endpoint can be assessed appropriately.Citation15,Citation17 Other important considerations are the patient population, languages, and mode of administration.

Patient preference studies

A Discrete Choice Experiment (DCE) is a method for eliciting preferences regarding alternative scenarios or options.Citation24 Participants are presented with alternative hypothetical scenarios and are asked to indicate their preferred option, with each option involving several attributes (outcomes) with different levels.Citation24 DCEs have been commonly used to (1) estimate the difference in utilities between health states; and (2) help to prospectively identify which outcome of a trial is more relevant to the patient.Citation24

In the case of vaccines, it may be informative to explore what attributes individuals are willing to trade off to prevent disease and consequently have the peace of mind from protection for themselves and/or potential family members, including through maternal vaccine protection or through herd immunity.

Preference methods to measure utilities also include time-trade-off (TTO) and standard gamble, while the estimation of utilities can also be done using PROs such as the health utilities index (HUI), EQ-5D or SF-36.Citation25

Assessment of PROs

It is important to note that a PRO strategy should be developed and integrated early into the research and development process. For example, it is important to identify the concepts that are relevant for patients involving qualitative interviews. The potential development of an instrument, ensuring that the instrument is fit for purpose in the specific population and in the disease area of interest, takes a considerable amount of time. It is important to perform quantitative research to identify the appropriate schedule of assessments prior to the phase III clinical trial. Additional quantitative research, clinical trials and preference studies may then be performed in parallel.

Note that in prophylactic vaccine studies PRO assessments are frequently completed at baseline, in otherwise “healthy” individuals prior to infection, and subsequently during the episode. PRO assessments can also be done after the initial episode has resolved to explore long-term effects of an episode. Consequently, PRO-studies in vaccine clinical trials are more cumbersome than in treatment trials because the sample size is much larger in vaccine clinical trials. It may also be important to assess disease-specific questionnaires at baseline, as some individuals may already have underlying conditions associated with symptoms that are consistent with infection (e.g. shortness of breath associated with asthma or chronic obstructive pulmonary disease (COPD) which is an important concept for individuals with a respiratory infection).

Statistical analysis of vaccine studies

Several standards have been developed to harmonize the statistical analysis of PRO data including the more recent Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints (SISAQOL) guidelines.Citation26,Citation27 The guidelines are also relevant for PRO data from vaccine clinical trials, yet they primarily focus on individuals who are ill or have disease. Consequently, endpoints such as the change in relief of symptoms from baseline, time to reduction in disease symptoms and responder analysis are utilized to show the proportion of patients experiencing meaningful improvement.Citation26,Citation27 The statistical analysis of PRO data from a prophylactic vaccine trial introduces additional specificities in that “healthy” individuals are vaccinated, who may later develop disease. Prophylactic vaccines can reduce the impact of disease on an individual’s HRQL by preventing disease but can also attenuate disease severity in subjects who develop disease despite receiving vaccination.Citation9

In vaccine studies, it is important to compare the PRO data in subjects who developed disease in the unvaccinated and vaccinated groups. For example, in herpes zoster (HZ) studies, it was demonstrated that the vaccine attenuated disease severity in individuals who developed disease manifesting in significantly lower pain scores among patients with confirmed HZ in the vaccinated group.Citation9,Citation23 It should however be noted that comparing PRO data between those individuals who have developed disease in the unvaccinated and vaccinated groups is subject to selection bias.Citation28 Consequently, a burden-of-illness (BOI) statistic has been developed as a composite endpoint to incorporate disease incidence with disease severity and duration. In the herpes zoster (HZ) studies mentioned above, the severity of illness due to HZ was calculated as the area under the curve of the patient self-assessment of pain related to HZ from day 0 to day 182 of the HZ episode for subjects with HZ and as 0 for subjects without an HZ episode.Citation9,Citation23 The BOI was then calculated as the sum of the severity of illness scores in a group divided by the sum of the subject years of follow-up in that group.Citation28 Vaccine efficacy was then estimated as the relative reduction in the BOI score in the vaccinated group as compared to the score in the placebo group and calculated as one minus the relative risk (i.e. the BOI score in the vaccinated group divided by the BOI score in the placebo group). The BOI statistic has been used in vaccine decision-making,Citation29 although some HTA bodies have questioned the use of composite endpoints in a clinical trial setting.Citation30 Nevertheless, the BOI statistic is unbiased and has been demonstrated to be a robust measurement of vaccine benefit.Citation9,Citation10,Citation18,Citation23,Citation28 As such, the BOI statistic should be considered as an important efficacy endpoint from a PRO perspective in vaccine clinical trials.

PRO stakeholders

A PRO strategy must be developed considering the decision-makers of vaccination programs at both a regional and country level. A clear PRO strategy is of prime strategic importance to provide direct evidence of patient benefit which will ultimately contribute to the success of a product as it serves as an important communication of the product’s value proposition, expressed in clear PRO value messages, and as a potential differentiator over a competitor or compared to the existing situation. A PRO strategy should be planned and implemented in the earliest stages of product development. In most countries the decision-makers interested in PROs include:

  • Regulatory Authorities,

  • The Ministry of Health,

  • The Advisory Bodies making recommendations on immunization programs (Health Protection Agencies, HTA, National Immunization Technical Advisory Groups [NITAGs] etc.),

  • Payers such as: insurance companies, sickness funds, funding bodies, etc.,

  • Key opinion leaders (physicians, health economists, etc.),

  • Healthcare providers if they have a choice between different options,

  • Vaccine recipients, parents, caregivers and patient advocacy groups.

Additional uses of PROs

PROs may be used in various ways to enhance research within a development program. Recently, FDA has noted that PRO instruments could be applied to help determine patient´s eligibility inclusion criteria, measure safety and/or effectiveness endpoints, either as a standalone or as a component of a composite endpoint.Citation31 provides an overview of examples where PROs have been utilized, including as inclusion or exclusion criterion or as a case definition within a clinical trial.Citation19,Citation32 When a PRO is assessed electronically (i.e. ePRO) the identification of a case can trigger other events such as a telephone call, site visit or the completion of other assessments to be performed according to the study protocol. Electronic capture of PROs also ensures that the source data are ALCOA compliant (i.e. attributable, legible, contemporaneous, original, and accurate) and ALCOA+ compliant (i.e. Complete, Consistent, Enduring and Available).Citation34 The availability of new technologies has increased the role of virtual clinical trials and the use of mobile clinical trial platforms assessing PROs in conjunction with wearable devices to track participants with no in-person site visits required.Citation34 Virtual studies have the potential to make clinical studies more inclusive, faster and more cost-effective.Citation34 This can be achieved using technology, such as smartphones, that study participants may already have available. Virtual studies bring studies directly to the patients at home, creating a more human approach to clinical research. In doing so, studies may be more representative and easier to recruit as well as help retain subjects in an ongoing trial.Citation34

Table 1. Additional uses of PROs

An additional example of the use of PROs can be seen among frail individuals who are at increased risk of poor clinical outcomes.Citation39 Frailty may be influenced by an age-related decline in innate and adaptive humoral and cell-mediated immunity which impairs their ability to resist infection and respond to interventions such as vaccination.Citation40 Frailty Indices have been constructed using PROs allowing vaccine efficacy and safety to be evaluated by frailty status.Citation36

In addition to the acute period of infection, vaccine-preventable disease may be associated with long-term sequelae in children and young adults,Citation25 while infected older adults may not completely recover and return to their pre-disease functional ability, thus loosing independence.Citation41 In such cases, PRO data are necessary to describe the long-term impact of an infection and consequently the value of vaccination, e.g. by preventing disease, the functional ability is maintained, thus enabling healthy aging.Citation36,Citation41 To estimate the public health benefit and cost-effectiveness of vaccination, it is imperative that robust estimates of both the short- and long-term impacts, including the probability of developing sequelae and the QALY loss, are available.Citation37,Citation38

Vaccines may also have negative implications for certain aspects of life deemed important to the individual; vaccine reactogenicity is often a necessary but unwanted implication of a vaccine. As such, it may also be important to address the potential negative effects of a vaccine, including the impact of vaccine reactogenicity on an individual’s functioning, to better inform potential vaccine recipients and consequently to support their decision-making on whether they are willing to receive it or not. Healthcare providers, payers (e.g. due to associated costs) and regulators should also be informed of any negative consequences of vaccination. Both vaccine-specific and generic questionnaires have been used to assess the impact of reactogenicity.Citation42,Citation43 The rationale for using generic questionnaires is that they allow a comparison of the impact of vaccine reactogenicity and disease impact on concepts such as pain, physical function, vitality, quality of life, and utility loss.Citation43

Discussion

Traditionally, clinical studies were based on clinician-reported outcomes of patients’ health. However, it is now well recognized that PRO measures should be used when assessing concepts best known by the patient or best measured from the patient’s perspective, without interpretation by clinicians or others.Citation17 This transition from clinician reports of patient outcomes to PROs, in conjunction with the FDA’s publication of the PRO Guidance, has transformed the discipline of PRO measurement.

Constructing a comprehensive PRO strategy, following well-documented guidelines for the development and validation of PRO instruments, defining endpoints that are relevant for the patient, while matching the development program, and using appropriate statistical methodology will lead to a more robust outcome while increasing the probability of a successful label claim. Once labeling is achieved, the information in the label can be informative to prescribing clinicians and facilitate raising awareness amongst patients and decision-makers at all levels regarding the value of the vaccine.

PROs may be incorporated into the research and development program of a vaccine in many additional ways. As mentioned above, virtual clinical and epidemiological studies using mobile trial platforms may result in a more representative sample as well as being easier to recruit and retain subjects. In addition, events are triggered (e.g. cases or outcomes) resulting in less cases and events being missed and a more streamlined follow-up of such events and cases. These elements can introduce efficiencies and improve both the clinical relevance and validity of the clinical trial program.Citation16 The outputs of studies involving PROs may also be included in public health and cost-effectiveness models, informing decision-makers at all levels on the value of vaccination.

It should be noted that the many uses of PRO data may not materialize if study subjects do not receive adequate training on the value of PROs and how to adequately complete PRO assessments and the requirements of the study protocol. The importance of providing training on the rationale and implementation of PROs to patients and clinical staff to obtain high-quality data should not be underestimated.

As with other study procedures, an assessment needs to be performed to understand the benefits of including PROs into a study. For example, a study of a prophylactic vaccine assessing immunological outcomes where few subjects may develop the disease under investigation may be considered inappropriate or unethical for study participants to complete an assessment, which has limited benefit. It is therefore important that sample size and/or associated power calculations are performed in studies that include PROs to ensure that the sample size is sufficiently large to have a high probability of detecting clinically meaningful differences, in order to obtain reliable results and to be able to convince others of their validity.

Disclosure of potential conlficts of interest

All authors are employed by and hold shares in the GSK group of companies. All authors declare no other financial and non-financial relationships and activities.

Acknowledgments

Business & Decision Life Sciences platform provided editorial assistance and publication coordination, on behalf of GSK. Carole Desiron coordinated publication development and editorial support.

Additional information

Funding

GlaxoSmithKline Biologicals SA funded this study and all costs related to the development of the publications.

Notes on contributors

Desmond Curran

All authors reviewed the literature, provided substantial input, contributed to writing, and reviewed this publication. Desmond Curran led this publication development. All authors gave their final approval and are accountable for all aspects of the work.

References

  • Rappuoli R, De Gregorio E. Editorial overview: vaccines: novel technologies for vaccine development. Curr Opin Immunol. 2016;41:v–vii. doi:10.1016/j.coi.2016.07.001.
  • Melief CJM, van Hall T, Arens R, Ossendorp F, van der Burg SH. Therapeutic cancer vaccines. J Clin Invest. 2015;125(9):3401–12. doi:10.1172/JCI80009.
  • Zagadailov E, Fine M, Shields A. Patient-reported outcomes are changing the landscape in oncology care: challenges and opportunities for payers. Am Health Drug Benefits. 2013;6(5):264–74.
  • Hämeen-Anttila K, Komulainen J, Enlund H, Mäkelä M, Mäkinen E, Rannanheimo P, Sipilä R. Incorporating patient perspectives in health technology assessments and clinical practice guidelines. Res Social Adm Pharm. 2016;12(6):903–13. doi:10.1016/j.sapharm.2015.12.005.
  • Brotherton JML. HPV prophylactic vaccines: lessons learned from 10 years experience. Future Virol. 2015;10(8):999–1009. doi:10.2217/fvl.15.60.
  • Ehrlich HJ, Singer J, Berezuk G, Fritsch S, Aichinger G, Hart MK, El-Amin W, Portsmouth D, Kistner O, Barrett PN. A cell culture-derived influenza vaccine provides consistent protection against infection and reduces the duration and severity of disease in infected individuals. Clin Infect Dis. 2012;54(7):946–54. doi:10.1093/cid/cir959.
  • Kendrick P, Eldering G. A study in active immunization against pertussis. Am J Epidemiol. 1939;29-SectionB(3):133–53. doi:10.1093/oxfordjournals.aje.a118485.
  • Vesikari T, Giaquinto C, Huppertz HI. Clinical trials of rotavirus vaccines in Europe. Pediatr Infect Dis J. 2006;25(1 Suppl):S42–7. doi:10.1097/01.inf.0000197565.45345.4e.
  • Curran D, Matthews S, Rowley SD, Young J-AH, Bastidas A, Anagnostopoulos A, Barista I, Chandrasekar PH, Dickinson M, El Idrissi M, et al. Recombinant zoster vaccine significantly reduces the impact on quality of life caused by herpes zoster in adult autologous hematopoietic stem cell transplant recipients: a randomized placebo-controlled trial (ZOE-HSCT). Biol Blood Marrow Transplant. 2019;25(12):2474–81. doi:10.1016/j.bbmt.2019.07.036.
  • Schmader KE, Johnson GR, Saddier P, Ciarleglio M, Wang WWB, Zhang JH, Chan ISF, Yeh SS, Levin MJ, Harbecke RM, et al. Effect of a zoster vaccine on herpes zoster-related interference with functional status and health-related quality-of-life measures in older adults. J Am Geriatr Soc. 2010;58(9):1634–41. doi:10.1111/j.1532-5415.2010.03021.x.
  • Coplan PM, Schmader K, Nikas A, Chan ISF, Choo P, Levin MJ, Johnson G, Bauer M, Williams HM, Kaplan KM, et al. Development of a measure of the burden of pain due to herpes zoster and postherpetic neuralgia for prevention trials: adaptation of the brief pain inventory. J Pain. 2004;5(6):344–56. doi:10.1016/j.jpain.2004.06.001.
  • Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, Gundy C, Koller M, Petersen MA, Mirjam AG, et al. EORTC QLQ–C30 Reference Values; 2008. [accessed 2021 Jan 20]. https://www.eortc.org/app/uploads/sites/2/2018/02/reference_values_manual2008.pdf
  • Ware JE, Kosinski M SF-36 physical & mental health summary scales: a manual for users of version 1. Quality Metric Incorporated, 2001.
  • United States Department of Health and Human Services Food and Drug Administration. Patient-focused drug development: collecting comprehensive and representative input. Guidance for industry, Dood and Drug Administration staff, and other stakeholders. MD: Food and Drug Administration, 2020. https://www.fda.gov/media/139088/download .
  • Isaac M, Vamvakas S, Pavlovic M. EMA perspective on PRO instrument qualification and harmonization. First annual patient‐reported outcomes (pro) consortium workshop march 23, 2010 – Bethesda, MD: pro consortium. [accessed 2020 Aug 11]. https://c-path.org//wp-content/uploads/2013/09/2010PRO-Workshop-Isaac.pdf
  • Powers JH 3rd, Howard K, Saretsky T, Clifford S, Hoffmann S, Llorens L, Talbot G. Patient-reported outcome assessments as endpoints in studies in infectious diseases. Clin Infect Dis. 2016;63(Suppl2):S52–6. doi:10.1093/cid/ciw317.
  • United States Department of Health and Human Services Food and Drug Administration. Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims. MD: Food and Drug Administration, 2009. [accessed 2020 Aug 11]. https://www.fda.gov/media/77832/download
  • Oxman MN, Levin MJ, Johnson GR, Schmader KE, Straus SE, Gelb LD, Arbeit RD, Simberkoff MS, Gershon AA, Davis LE, et al. A vaccine to prevent herpes zoster and postherpetic neuralgia in older adults. N Engl J Med. 2005;352(22):2271–84. doi:10.1056/NEJMoa051016.
  • Yu J, Powers JH 3rd, Vallo D, Falloon J. Evaluation of efficacy endpoints for a phase IIb study of a respiratory syncytial virus vaccine in older adults using patient-reported outcomes with laboratory confirmation. Value Health. 2020;23(2):227–35. doi:10.1016/j.jval.2019.09.2747.
  • Doward LC, Gnanasakthy A, Baker MG. Patient reported outcomes: looking beyond the label claim. Health Qual Life Outcomes. 2010; 8: 89. doi:10.1186/1477-7525-8-89.
  • Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, Schwartz C, Revicki DA, Moinpour CM, McLeod LD, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. 2013;22(8):1889–905. doi:10.1007/s11136-012-0344-y.
  • Revicki DA, Gnanasakthy A, Weinfurt K. Documenting the rationale and psychometric characteristics of patient reported outcomes for labeling and promotional claims: the PRO evidence dossier. Qual Life Res. 2007;16(4):717–23. doi:10.1007/s11136-006-9153-5.
  • Curran D, Oostvogels L, Heineman T, Matthews S, McElhaney J, McNeil S, Diez-Domingo J, Lal H, Andrews C, Athan E, et al. Quality of life impact of an adjuvanted recombinant zoster vaccine in adults aged 50 years and older. J Gerontol A Biol Sci Med Sci. 2019;74(8):1231–8. doi:10.1093/gerona/gly150.
  • Hauber AB, González JM, Groothuis-Oudshoorn CGM, Prior T, Marshall DA, Cunningham C, Ijzerman MJ, Bridges JFP. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016;19(4):300–15. doi:10.1016/j.jval.2016.04.004.
  • Wells CD, Murrill WB, Arguedas MR. Comparison of health-related quality of life preferences between physicians and cirrhotic patients: implications for cost-utility analyses in chronic liver disease. Dig Dis Sci. 2004;49(3):453–8. doi:10.1023/b:ddas.0000020502.46886.c1.
  • Calvert M, Kyte D, Mercieca-Bebber R, Slade A, Chan AW, King MT, Hunn A, Bottomley A, Regnault A, Chan AW, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the SPIRIT-PRO extension. Jama. 2018;319(5):483–94. doi:10.1001/jama.2017.21903.
  • Coens C, Pe M, Dueck AC, Sloan J, Basch E, Calvert M, Campbell A, Cleeland C, Cocks K, Collette L, et al. International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: recommendations of the SISAQOL consortium. Lancet Oncol. 2020;21(2):e83–96. doi:10.1016/s1470-2045(19)30790-9.
  • Callegaro A, Curran D, Matthews S. Burden-of-illness vaccine efficacy. Pharm Stat. 2020;19(5):636–45. doi:10.1002/pst.2020
  • Public Health England. Vaccination against shingles. Information for healthcare professionals. London, UK: Public Health England, 2018 [accessed 2020 Sep 24]. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/696498/Shingles_information_for_healthcare_professionals.pdf
  • EUnetHTA Joint Action 2. 2015. Guideline. Endpoints used for relative effectiveness assessment. Composite endpoints. [accessed 2020 Sep 24]. https://eunethta.eu/wp-content/uploads/2018/03/composite_endpoints.pdf .
  • United States Food and Drug Administration. Principles for Selecting, Developing, Modifying, and Adapting Patient Reported Outcome Instruments for Use in Medical Device Evaluation. Draft Guidance for Industry and Food and Drug Administration Staff, And Other Stakeholders. MD: U.S. Department of Health and Human Services; 2020 [accessed 2020 Sep 24]. https://www.fda.gov/media/141565/download
  • Betsuyaku T, Kato M, Fujimoto K, Hagan G, Kobayashi A, Hitosugi H, James M, Jones PW. A study to assess COPD symptom-based management and to optimise treatment strategy in Japan (COSMOS-J) based on GOLD 2011. Int J Chron Obstruct Pulmon Dis. 2013;8:453–9. doi:10.2147/COPD.S48298.
  • Global Initiative for Chronic Obstructive Lung Disease (GOLD). Global strategy for the diagnosis, management, and prevention of COPD. 2019 Global initiative for Chronic Obstructive Lung Disease, 2019. [accessed 2020 Aug 11]. https://goldcopd.org/wp-content/uploads/2018/11/GOLD-2019-v1.7-FINAL-14Nov2018-WMS.pdf .
  • Grove BE, Ivarsen P, de Thurah A, Schougaard LM, Kyte D, Hjøllund NH. Remote follow-up using patient-reported outcome measures in patients with chronic kidney disease: the PROKID study - study protocol for a non-inferiority pragmatic randomised controlled trial. BMC Health Serv Res. 2019;19(1):631. doi:10.1186/s12913-019-4461-y.
  • Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, Rogak L, Bennett AV, Dueck AC, Atkinson TM, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34(6):557–65. doi:10.1200/JCO.2015.63.0830.
  • Curran D, Kim JH, Matthews S, Dessart C, Levin MJ, Oostvogels L, Riley ME, Schmader KE, Cunningham AL, McNeil SA, Schuind AE, et al. Recombinant zoster vaccine is efficacious and safe in frail individuals. J Am Geriatr Soc. 2020 Nov 16. doi:10.1111/jgs.16917.
  • Curran D, Schmidt-Ott R, Schutter U, Simon J, Anastassopoulou A, Matthews S. Impact of herpes zoster and postherpetic neuralgia on the quality of life of Germans aged 50 or above. BMC Infect Dis. 2018;18(1):496. doi:10.1186/s12879-018-3395-z.
  • Van Oorschot D, Anastassopoulou A, Poulsen Nautrup B, Varghese L, von Krempelhuber A, Neine M, Lorenc S, Curran D. Cost-effectiveness of the recombinant zoster vaccine in the German population aged ≥60 years old. Hum Vaccin Immunother. 2019;15(1):34–44. doi:10.1080/21645515.2018.1509645.
  • Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K. Frailty in elderly people. Lancet. 2013;381(9868):752–62. doi:10.1016/s0140-6736(12)62167-9.
  • Curran D, Andrew MK, Levin MJ, Turriani E, Matthews S, Fogarty C, Klein NP, Grupping K, Oostvogels L, Schmader KE. Evaluation of two frailty indices, with practical application in a vaccine clinical trial. Hum Vaccin Immunother. 2019;15(12):2960–8. doi:10.1080/21645515.2019.1622974.
  • Branche A, Granieri E, Walsh E, Finelli L, Greendyke W, Falsey AR, Barrett A, Vargas C, Saiman L. 733. Incidence and evaluation of the change in functional status associated with respiratory syncytial virus infection in hospitalized older adults. Open Forum Infect Dis. 2018;5(Suppl1):S263. doi:10.1093/ofid/ofy210.740.
  • Hernández-Guerra M, González-Méndez Y, de Molina P, Gimeno-García AZ, Carrillo M, Casanova C, Pumarola T, Jimenez A, Hernández-Porto M, Torres A, et al. Immunogenicity and acceptance of influenza A (H1N1) vaccine in a cohort of chronic hepatitis C patients receiving pegylated-interferon treatment. PloS One. 2012;7(11):e48610. doi:10.1371/journal.pone.0048610.
  • Schmader KE, Levin MJ, Grupping K, Matthews S, Butuk D, Chen M, Idrissi ME, Fissette LA, Fogarty C, Hartley P, et al. The impact of reactogenicity after the first dose of recombinant zoster vaccine on the physical functioning and quality of life of older adults: an open-label, phase III trial. J Gerontol A Biol Sci Med Sci. 2019;74(8):1217–24. doi:10.1093/gerona/gly218.