Adaptation of a previously validated vaccination report card for use in adult vaccine clinical trials to align with the 2007 FDA Toxicity Grading Scale Guidance

Abstract

The Adult/Adolescent Vaccination Report Card (VRC) was developed and validated by Merck in 1998 for use in vaccine clinical trials to collect information from trial subjects on complaints for both local and systemic events after vaccination. This short report describes the revision to the original validated VRC in order to align with the guidelines outlined in the 2007 FDA Toxicity Grading Scale for Healthy Adult and Adolescent Volunteers Enrolled in Preventive Vaccine Clinical Trials. Since the VRC elicits trial subjects' self-reports of any adverse experiences (AE) occurring post vaccination, it was important that subsequent modifications of the VRC retained the original user-friendly characteristics while gathering the appropriate information to align with the FDA Guidance. A convenience sample of 15 participants (71% females, 87% white and mean (SD) age 45 (13) years was recruited to obtain feedback in order to revise the Adult/Adolescent VRC. Based on the feedback received, the following were slightly revised: ruler for the measurements of local systemic reactions, severity ratings, and general instructions. The revised VRC is currently being used in Merck vaccine clinical trials.

Keywords: :

• vaccine report card
• patient reported outcomes
• vaccine safety
• vaccination complaints
• adverse event collection
• injection site reaction

Introduction

The Vaccination Report Card (VRC) was developed and validated by Merck Sharp and Dohme Corp. (Merck) in 1998¹ with the intent of creating a standardized tool for collecting trial subject-reported complaints occurring post vaccination. The VRC is a tool used to supplement standard trial adverse event (AE) collection in order to have a comprehensive and systematic collection of AEs. Two versions of the original VRC were created: a pediatric version for children 12 y of age and younger and an adult/adolescent version for individuals 13 y of age and older. Both VRC versions have been used in Merck clinical trials since 1998^2-10 and have proven to be a useful method for identifying and characterizing injection site and other adverse experiences occurring after vaccination (e.g., temperature, medication use and administration of non-study vaccinations).

In an effort to allow for standardized comparisons across similar vaccine products by manufacturers, the Food and Drug Association (FDA) Toxicity Grading Scale for Healthy Adult and Adolescent Volunteers Enrolled in Preventive Vaccine Clinical Trials (hereinafter, “FDA Guidance”) was released in September of 2007.¹¹ The FDA Guidance includes recommendations for grading the severity of clinical and laboratory abnormalities. Due to the variable nature of different clinical trials, the FDA Guidance also suggests that the parameters monitored should be appropriate for the specific study vaccine. To keep up to date with the FDA's current assessment of toxicity grading in vaccine clinical trials, the original, validated Adult/Adolescent VRC was revised. The pediatric VRC was not modified as the FDA Guidance is not applicable to this age group.

The original VRC consists of core sections (injection site complaints, temperature, medications, and any other complaints or illnesses) as well as vaccine- and study-specific modules (e.g., rash, diarrhea/vomiting) that can be selected based on protocol needs.

A multidisciplinary working group within Merck was created to review the FDA Guidance and determine how to incorporate the recommendation into the original VRC. The objectives of this effort were to adhere to the new guidelines while maintaining the structure of the original VRC and to maximize comparability with previously collected safety data from Merck vaccine trials. In order to meet these objectives, feedback on the revised VRC was collected from a convenience sample of adults.

The purpose of this short report is to describe the rationale and methods used to implement the FDA Guidance recommendations into the original VRC and to present the main results obtained from this effort. This short report describes the major changes made to the core sections of the adult/adolescent VRC. All final changes are reflected in the local and systemic sections of the revised VRC.

Results

A total of 15 participants were interviewed during the two focus groups (Focus Group A, n = 8; Focus Group B, n = 7), none of the participants underwent vaccination for these interviews. Table 2 summarizes the distribution and general characteristics of participants overall. The mean (SD) age was 45 (13) years old with a range of 29–68 y, the majority were female (71%) and white (87%). In general, the participants' characteristics were similar between both focus groups.

Table 1. General Characteristics of Focus Group Participants

General Characteristics	Total (n = 15)
Age (yrs) mean (± SD) range	45 ( ± 13) 29–68
Gender Male Female	29% 71%
Race White Asian	87% 13%
Education HS/equivalent Some college Bachelor Degree Graduate	13% 13% 40% 33%
Marital Status Married Not Married	87% 13%
Children Yes No	87% 13%
Previously seen VRC Yes No	13% 87%
Received vaccination in last 2 y (self/child) Yes No	73% 27%
Previously used VRC Yes No	0% 100%

All key revisions to the original validated VRC are summarized in Table 1. Specific revisions are summarized below.

Table 2. Comparison of Sections Related to Injection Site Complaints Grading Between the FDA Guidance and Merck. Original and Revised VRC

	Merck Original VRC Standard						FDA Guidance					Merck Revised VRC
Comparison of Redness and Swelling Sections	Grading Definition Category	Ruler Size 1	Ruler Size 2	Ruler Size 3	Over Ruler Size 3		Mild (Grade 1)	Moderate (Grade 2)	Severe (Grade 3)	Potentially Life-Threatening (Grade 4)		Ruler Size A	Ruler Size B	Ruler Size C	Ruler Size D	Ruler Size E or >
	Swelling^†	0 to 2.5 cm	2.6 to 5 cm	5.1 to 7.5 cm	> 7.5 cm		2.5 to 5 cm and does not interfere with activity	5.1 to 10 cm or interferes with activity	> 10 cm or prevents daily activity	Necrosis		0 to < 2.5 cm	≥ 2.5 to < 5.1 cm	≥ 5.1 to < 7.4 cm	≥ 7.4 to ≤ 10 cm	> 10 cm
	Redness^†						2.5 to 5 cm and does not interfere with activity	5.1 to 10 cm	> 10 cm	Necrosis or exfoliative dermatitis		1 to < 2.5 cm	≥ 2.5 to < 5.1 cm	≥ 5.1 to < 7.4 cm	≥ 7.4 to ≤ 10 cm	> 10 cm
Comparison of Pain and Tenderness Sections	Grading Definition Category	Mild (Grade 1)	Moderate (Grade 2)	Severe (Grade 3)			Mild (Grade 1)	Moderate (Grade 2)	Severe (Grade 3)	Potentially Life-Threatening (Grade 4)		Mild (Grade 1)	Moderate (Grade 2)	Severe (Grade 3)
	Pain^‡	Awareness of symptom but easily tolerated	Discomfort enough to cause interference with usual activities	Incapacitating with inability to work or do usual activity			Does not interfere with activity	Repeated use of non-narcotic med > 24 h or interferes with activity	Any use of narcotic pain reliever or prevents daily activity	ER Visit or Hospitalization		Awareness of symptom but easily tolerated	Discomfort enough to cause interference with usual activities	Inability to do work or usual activities
	Tenderness^‡						Mild discomfort	Discomfort with movement	Significant discomfort at rest	ER Visit or Hospitalization

^† Grade 4 as recommended in FDA Guidance captured in separate section of VRC (not shown in this table) asking whether redness and swelling required medical attention. ^‡Grade 4 as recommended in FDA Guidance captured in separate section of VRC (not shown in this table) asking whether pain and tenderness.

Measurement of Redness and Swelling

The focus group participants understood the redesigned ruler labeled “A through E→.” However, the participants expressed a need for further instruction on how to distinguish and measure redness and swelling; therefore instructions were included on how to differentiate redness from swelling (Fig. 1).

Figure 1. Comparison of original VRC and revised VRC measurement examples for redness and swelling.

Severity Ratings

Focus group participants had no difficulty rating the severity of redness, swelling, pain/tenderness. In addition, although the FDA Guidance suggests rating pain and tenderness severity separately, focus group participants' feedback indicated little differentiation between these two concepts, and some participants perceived no difference at all. Therefore, as in the original development work,¹ pain and tenderness were kept together under one severity item and the severity rating response options were slightly modified to align with grades 1 to 3 in the FDA Guidance (Table 1).

General Formatting

Finally, the focus group participants also provided suggestions that led to reformatting the instructions to make them more user-friendly. Some of these changes include placing the instructions on a separate page before each recording instead of having all directions together in one page.

Based on the results listed above, changes to the ruler for the measurements of local reactions, the severity ratings, and the general instructions were all included in the revised VRC. All other sections of the original VRC remained unchanged because focus group participants did not identify any needed modifications.

Methods

The Merck working group drafted a revised VRC based on the FDA Guidance by aligning each section with the corresponding recommendations (Table 1). In the FDA Guidance, the toxicity grading recommendations are for investigators to assess severity of clinical abnormalities. The original validated VRC was completed by clinical trial subjects and provides clinical staff with the information needed to properly grade toxicity scores. The challenge was to adapt this guidance while maintaining as much of the original version as possible.

The FDA Guidance provides suggestions for severity assessment based on a 4-part grading scale, including Mild (Grade 1), Moderate (Grade 2), Severe (Grade 3), and Potentially Life Threatening (Grade 4). This scale includes interference with activity level, use of narcotic pain reliever, size measurements of local reaction at the injection site, and/or whether urgent medical attention was required. The revised VRC incorporated components of these suggested severity assessments while retaining as much of the original validated VRC rating scale as possible. For example, the severity option in the FDA Guidance, “Potentially Life Threatening (Grade 4) – Emergency room (ER) visits or hospitalizations,” was excluded from the revised VRC because ER visits may not accurately reflect a life-threatening event, given the widely varied healthcare systems, especially in international trials. Therefore, events requiring use of urgent medical care is captured as a separate item on the revised VRC (not shown). In addition, the use of narcotic pain reliever was retained and is recorded on a separate page within the revised VRC to maintain consistency with the original, validated VRC.

The revised VRC was evaluated by two focus groups of participants who were unfamiliar with using the original VRC. Participants consisted of a convenience sample of Merck employees separated into two demographically similar groups. In addition, within each group, every attempt was made to recruit diverse subjects in terms of demographic characteristics such as age, gender, education, and ethnic background. The purpose of this effort was to gain feedback on the functionality, general understanding and user-friendliness of sections of the revised VRC (i.e., temperature, measurement, redness/swelling, pain/tenderness, severity, injection site reactions, instructions, etc.). At the beginning of each focus group, participants received a copy of the revised Adult/Adolescent VRC and were asked questions from the moderator, with a focus on three areas as specified below, based on a prepared script.

Measurements of Redness and Swelling

The original VRC used a ruler divided into 8 one-inch (~2.5cm) sections (labeled 1 to 8) to assist trial subjects in measuring the size of any redness or swelling observed after vaccination at the injection site¹ (Fig. 1). Trial subjects were instructed to measure the size of the redness or swelling and mark the appropriate corresponding box. In order to adapt to the VRC to the FDA Guidance (Table 1), the ruler was redesigned into increments of five 2.5 cm sections, each labeled “A” through “E→.”

Severity of Redness and Swelling

To minimize revision to the original validated VRC, redness and swelling assessments were retained in their original format and did not include severity rating options for swelling as recommended in the FDA Guidance. Participants were instructed to review the content of the VRC and provide feedback on their understanding of the questions, response scale, instructions and if any areas needed to be revised.

Severity of Pain and Tenderness

Because the original development of the VRC revealed that subjects could not meaningfully distinguish pain and tenderness, the combined assessment, pain and tenderness was kept as a single item as in the original VRC, and not separated as the FDA guidance recommends. However, during focus groups, participants were specifically asked if they could distinguish between the two concepts.

Conclusions

The Merck working group implemented the recommendations from the FDA Guidance Toxicity Grading Scale in order to revise the original, validated Adolescent/Adult VRC for use in vaccine clinical trials. The authors recognize that the focus group participants used in this study are not fully representative of vaccine clinical trial subjects and included a higher percentage of highly educated subjects compared with the original VRC development sample. The scope of this current study was not to replicate the original development work but to confirm that the FDA guidance recommendations included in the revised VRC were easy to understand. Although small, the current sample did include some subjects with lower education levels. The authors believe that this work (although with limitations) reinforces that the recommendations included in the FDA guidance and incorporated in the revised VRC can be understood by subjects and supports the work performed to generate the original VRC.

The revised VRC remains a user-friendly tool for subjects to report any complaints after vaccination and provides clinical staff with the information needed to properly grade toxicity scores according to the FDA Guidance. The revised VRC is currently being used in new Merck clinical trial programs to provide supplemental AE information in addition to the AE experiences routinely collected by investigators as specified in all Merck vaccine trials, and is available, free of charge, for use outside of Merck trials (contact [email protected] for licensing information).

Abbreviations:
AE	=	adverse events
ER	=	emergency room
FDA	=	Food and Drug Administration
HS	=	high school
SD	=	standard deviation
VRC	=	Vaccination Report Card

Acknowledgments

Merck Sharp and Dohme Corp., a subsidiary of Merck and Co., Inc. Whitehouse Station, NJ provided financial support for this study. All authors have completed the ICMJE form for disclosure of potential conflicts of interests. We would like to thank Allison Martin Nguyen for her contribution in moderating the focus group sessions and Michelle Brown for her scientific input.

Disclosure of Potential Conflicts of Interest

No potential conflicts of interest were disclosed.