Comparing reactogenicity of COVID-19 vaccines: a systematic review and meta-analysis

ABSTRACT

Objectives

A number of vaccines have now been developed against COVID-19. Differences in reactogenicity and safety profiles according to the vaccine technologies employed are becoming apparent from clinical trials.

Methods

Five databases (Medline, EMBASE, Science Citation Index, Cochrane Central Register of Controlled Trials, London School of Hygiene and Tropical Medicine COVID-19 vaccine tracker) were searched for relevant randomized controlled trials between 1 January 2020 and 12 January 2022 according to predetermined criteria with no language limitations.

Results

Forty-two datasets were identified, with 20 vaccines using four different technologies (viral vector, inactivated, mRNA and protein sub-unit). Adults and adolescents over 12 years were included. Control groups used saline placebos, adjuvants, and comparator vaccines. The most consistently reported solicited adverse events were fever, fatigue, headache, pain at injection site, redness, and swelling. Both doses of mRNA vaccines, the second dose of protein subunit and the first dose of adenovirus vectored vaccines were the most reactogenic, while the inactivated vaccines were the least reactogenic.

Conclusions

The different COVID-19 vaccines currently available appear to have distinct reactogenicity profiles, dependent on the vaccine technology employed. Awareness of these differences may allow targeted recommendations for specific populations. Greater standardization of methods for adverse event reporting will aid future research in this field.

KEYWORDS:

• Systematic review
• Meta-analysis
• COVID-19 vaccines
• reactogenicity
• adverse events

1. Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection (COVID-19) has been associated with more than 513 million cases and 6.2 million deaths worldwide as of 6 May 2022 [1]. To control this pandemic, safe and effective vaccines were developed rapidly and several vaccine candidates have emerged. Currently, 344 COVID-19 vaccine candidates are in various stages of development and 126 candidates have reached clinical trials [2]. Thirty-eight vaccines have now been approved for emergency use in 197 countries [3].

Different strategies and technologies have been utilized for the development of vaccines against SARS-CoV-2. While these include conventional approaches such as inactivated and protein subunit vaccines, they also include novel technologies such as messenger ribonucleic acid (mRNA) and viral-vectored vaccines. Already, clear differences in the efficacy of these vaccines are emerging from clinical trials [4,5] and it is also apparent that the reactogenicity and safety profiles differ according to the platform employed. A detailed understanding of the side effect profiles of different vaccines is required for decisions to be made about their deployment and in informing the general public about the risk-benefit ratios of vaccination. Additionally, such data may have implications for the populations in which they are to be used, for example, vaccines associated with low rates of adverse events (AEs) may be prioritized for use in pregnant women, young children, the immunocompromised and the elderly.

An early systematic review and meta-analysis of COVID-19 vaccines by Yuan et al. [6] (to October 2020) showed that there were significant differences between vaccine and placebo recipients in terms of local and systemic AEs. Pormohammad et al. [5] concluded that mRNA-based vaccines had the highest level of side effects (except for diarrhea and arthralgia) and aluminum-adjuvanted vaccines had the lowest side effect profile (except for injection site redness). Another review by Ling et al. [4] concluded that the incidence of adverse reactions was highest for the adenovirus vector vaccines. Chen et al. [7] drew a similar conclusion and also observed that the overall incidence of adverse events was higher for vaccinees aged 16–55 years than older adults (aged over 55 years), an observation also reported in the systematic review by Wang et al. [8]. The most recently published meta-analyses of the safety of COVID-19 vaccines (to 17 June 2021) concluded that all vaccines increased the risk of non-serious AEs. Due to the inconsistencies reported in past reviews and the speed at which new data in this area is published, we undertook a systematic review and meta-analysis of the reactogenicity of COVID-19 vaccines assessed in randomized controlled trials (RCTs), with a focus on commonly reported systemic and local AEs.

2. Methods

2.1. Database and search terms

A systematic review and meta-analysis was carried out to compare the reactogenicity of COVID-19 vaccines developed using different technologies: viral vector, mRNA, inactivated and protein subunit vaccines. The following databases were searched: Medline, EMBASE, Science Citation Index (Web of Science), Cochrane Central Register of Controlled Trials (CENTRAL) and the London School of Hygiene and Tropical Medicine (LSHTM) COVID-19 vaccine tracker [2]. Medical subject headings (MESH) terms and free text synonyms were used to search the databases for the following search themes: ‘vaccines,’ ‘reactogenicity,’ and ‘COVID-19.’ The Cochrane highly sensitive search strategy was used to narrow the search results to RCTs [9]. The search results were limited to human studies published between January 2020 – 12 January 2022. The full search strategy for each database can be found in supplementary Table S1.

This review was registered in the PROSPERO International prospective register of systematic reviews (13 April 2021, PROSPERO 2021 CRD42021248766).

2.2. Systematic review: inclusion and exclusion criteria

The search results were imported into the web application Rayyan, a recommended screening tool for systematic reviews [10]. Duplications were removed and the remaining papers were independently assessed in duplicate by NS, ASFR and EB against the inclusion and exclusion criteria, disagreements were resolved by consensus. Papers were deemed suitable for inclusion if they described a blinded randomized control trial of a COVID-19 vaccine, in participants aged 12 years and over, with either a placebo or control arm. Only studies of vaccines that were in active phase III clinical trials (recruitment or follow-up) before the 6th of January 2022, according to the LSHTM vaccine tracker [11] were included. A full list of eligible vaccines can be found in supplementary Table S2. Studies were excluded if they described only i) heterologous or booster regimes, ii) immunogenicity or efficacy, iii) the study protocol, vi) vaccines that were not administered intramuscularly, and v) COVID-19 vaccines which were co-administered with other vaccines. Pre-print papers, not yet peer-reviewed and listed on the LSHTM tracker were not included.

2.3. Risk of bias assessment

Each paper was independently assessed in duplicate for risk of bias by NS, ASFR, EB, SI, DS, and YH using the Revised Cochrane risk-of-bias tool for randomized trials (RoB 2) [12]. Any discrepancies were resolved by discussion between the two authors. If they were unable to resolve their differences, discrepancies were resolved by discussion with the rest of the review team. Studies which were assessed to have a high risk of bias were not included in the meta-analysis, and only data regarding study characteristics were extracted for these papers.

2.4. Data extraction

The following data regarding the study characteristics were extracted: the countries where the trials were conducted, participant characteristics (age, sex, and ethnicity), vaccine characteristics (vaccine platform, dose, and schedule) and the placebo or control used. For the studies which were assessed to have a ‘low’ or ‘some concerns’ risk of bias, data on reactogenicity were extracted. For each dose of vaccine or control, the number of participants who experienced fever, fatigue, headache, pain at injection site, redness, swelling, any local AE, any systemic AE or any AEs were extracted. Although differences in the grading of AEs used in the different trials were small, in order to minimize any potential bias, data was analyzed for ‘all’ AEs in each category and not further categorized by AE grade. If the trials reported data on different doses of vaccines, only data related to the dose that was taken forward into Phase 3 trials were collected.

2.5. Missing data requested from authors

For papers which did not report the reactogenicity separately for each vaccine dose administered, or papers where the data were only presented in graphs, the authors were contacted to request the data. Raw data received by authors were summarized in R studio statistical software (version 1.4.1717). The data received from authors had two potential denominators – number of participants who received a vaccine dose and should have completed the reactogenicity diary and number of participants who completed any part of the diary (per-protocol). We used the former in our analysis to remain consistent with the approach taken by other papers included in this review. When data was not provided by authors but was available in graph format in the manuscript, a web-based plot digitizer tool (WebPlotDigitiser V 4.5) [13] was used for data extraction.

2.6. Statistical analysis

The descriptive analyses were performed and summarized using percentage, frequency, and median with minimum-maximum ranges. Meta-analysis was carried out in RevMan Version 5.4. The studies were divided into two groups: studies which had data for each vaccine dose and studies in which the data were combined for the whole vaccine course. Single dose vaccines were included in the analysis of studies where data were available by dose. The Mantel-Haenszel random-effects model was performed to estimate risk ratios (RRs) and 95% confidence intervals (CIs) for each symptom by vaccine platform for dose 1 and dose 2, or all doses combined. This model was selected because of the high potential heterogeneity across trials. Low, moderate, and high heterogeneity were defined as I² values of 25%, 50%, and 75%, respectively [14].

2.7. Sensitivity analysis

A sensitivity analysis was carried out comparing intention to treat (ITT) and per-protocol populations by plotting separate forest plots with the same denominators for the first and second doses. In the per-protocol population, the denominators for participants who received dose one and dose two were different as some participants did not receive the second dose. In the ITT population, the denominators for doses one and two were the same, based on the number of participants who received dose one of the vaccine. If there was no significant difference between the two analyses, the per-protocol data was reported in order to be consistent with the published trial reports.

2.8. Investigating heterogeneity and publication bias

Factors that could contribute to high heterogeneity were identified as age, phase of vaccine trial, type of placebo or active control used by different trials, and multiple vaccines being included in each vaccine platform group. Four additional groups of forest plots were created which included only adult participants aged 16–65 years, only trials which used a placebo control (0.9% saline or water for injection), only phase II and III studies, and only vaccines which had three or more papers published. The I² statistic was compared to assess which of the factors was contributing to high heterogeneity. Due to lack of granularity in the data, we were unable to perform meta-regression to further investigate heterogeneity in this review.

To assess publication bias of included clinical trials, funnel plots of the RRs against the standard error for each individual study were performed.

2.9. Comparison of control types and analysis of individual vaccines

Further analysis was carried out to assess the impact of the different controls that were used across trials. For the vaccines with trials using different control groups, forest plots were constructed with each control group as an independent subgroup. Additional forest plots were constructed to compare individual vaccines which had at least three papers published.

3. Results

The database search was carried out on 12 January 2022 and yielded 1335 results. After duplications were deleted, 865 papers were screened. See Figure 1 for the study selection flowchart. Forty-eight papers describing 20 vaccines met the inclusion criteria for the review and underwent a risk of bias assessment. Most papers were assessed as having a low risk of bias, six had some concerns [15–20] and four had a high risk of bias [21–24]. A breakdown of the risk of bias assessment for each paper can be found in the supplementary Table S3. Seven papers were excluded from the meta-analysis: four [21–24] due to a high risk of bias and three [25–27] due to missing data which was not available from the authors. One of the papers [28] included data on two vaccines, resulting in 42 datasets describing 17 vaccines. Thirty-two of the datasets[15–18,20,29–55] presented data divided by dose, whereas ten [19,28,56–61] combined data for all doses.

Figure 1. Study selection flow chart.

Of the 20 vaccines included in the systematic review, the vaccine platforms were: viral vector (4), inactivated (6), mRNA (3), and protein sub-unit (7). For the control groups, 29 of the trials used a placebo (0.9% saline, water for injection, or vaccine excipients), 18 used an adjuvant (aluminum hydroxide, Algel, or Algel-IMDG) and two used a Meningococcal ACWY (MenACWY) conjugate vaccine. Most trials recruited adults over the age of 18 years. Four trials included adolescent participants (aged over 12 years) [38,41,44,62]. Overall, there was an equal mix of male and female participants in the trials, although three trials had less than 30% female trial participants [28,31,49]. The trials took place across all continents (see Figure 2), which resulted in a mix of ethnicities among participants. See Table 1 for a summary of the study characteristics.

Figure 2. World map of countries hosting vaccine trial sites by vaccine type (note: some trials took place in more than one country).

Table 1. Study characteristics.

Vaccine characteristics							Participant characteristics
Vaccine platform	Vaccine name (developer)	Study	Phase	Dose (b)	Dosing schedule	Control	Age	% Female	Ethnicity (%) (c)
Viral Vector	Ad26.COV2.S (Janssen Pharmaceutical Companies)	Sadoff et al [15].	I/IIa	5 × 10^10 vp	Single dose	0.9% Saline	18–55, ≥65	51.3	White (94.5), Black (2.9)
		Sadoff et al (d) [27].	III	5 × 10^10 vp	Single dose	0.9% Saline	18–59, ≥60	45	White (58.7), Black (19.4)
	Ad5-nCov (CanSino Biological Inc, Beijing Institute of Biotechnology)	Halperin et al [31].	III	5 × 10^10 vp	Single dose	Vaccine excipients	≥18	29.2	Hispanic/Latino (37), Other (63)
		Zhu et al [30].	II	5 × 10^10 vp	Single dose	Vaccine excipients	≥18	50	Not reported
		Zhu et al [29].	IIb	5 × 10^10 vp	D0-D56	Vaccine excipients	18–55, >56	41	Not reported
	ChAdOx1 (University of Oxford, AstraZeneca)	Asano et al [36].	I/II	5 × 10^10 vp	D0-D28	0.9% Saline	18–55, 56–69, ≥70	33.9	Japanese (100)
		Falsey et al [16].	III	5 × 10^10 vp	D0-D28	0.9% Saline	18–64, ≥65	44.4	White (79), Black (8.3)
		Folegatti et al [32].	I/II	5 × 10^10 vp	D0-D28	MenACWY Vaccine	18–55	49.8	White (90.9), South Asian (3.2)
		Madhi et al [34].	Ib/II	5 × 10^10 vp	D0-D28	0.9% Saline	18–64	43.5	Black African (70.6), Mixed (15.0)
		Madhi et al [33].	Ib/IIa	5 × 10^10 vp	D0-D28	0.9% Saline	18–65	61	Black (99), White (1)
		Ramasamy et al [35].	II	5 × 10^10 vp	D0-D28	MenACWY Vaccine	18–55, 56–69, ≥70	49.8	White (94.9), Asian (3.4)
	Gam-COVID-Vac (Gamaleya Research Institute)	Logunov et al (e) [21].	III	10^11 vp	D0-D21	Vaccine excipients	≥18	38.8	White (98.5), Asian (1.4)
Inactivated	BBV152 (Bharat Biotech)	Ella et al [49].	I	6 µg	D0-D14	Algel-IMDG only	18–55	20.8	Not reported
		Ella et al [50].	III	6 µg	D0-D28	Algel only	18–60, ≥60	32.7	Indian (100)
	BIBP (Sinopharm’s Beijing Institute of Biological Products)	Al Kaabi et al [28].	III	4 µg	D0-D21	Aluminum Hydroxyde	≥18	15.3	Not reported
		Xia et al [56].	I + II	4 µg	D0-D21	Aluminum Hydroxyde	18–59	55	Not reported
		Xia et al [44].	I/II	4 µg	D0-D28-56	Aluminum Hydroxyde	13–17	47.9	Han Chinese (97.5), Hui (2.3)
	CoronaVac (Sinovac)	Bueno et al [48].	III	3 µg	D0-D14	Aluminum Hydroxyde	18–59, ≥60	61.8	White (93.8), Other (6.2)
		Fadlyana et al [47].	III	3 µg	D0-D14	Water for Injection	18–59	35.4	Not reported
		Han et al [62].	I/II	3 µg	D0-D28	Aluminum Hydroxyde	12–17	46.0	Han Chinese (100)
		Tanriover et al [57].	III	3 µg	D0-D14	Aluminum Hydroxyde	18–59	42.2	Not reported
		Wu et al [45].	I/II	3 µg	D0-D28	Aluminum Hydroxyde	≥60	51.1	Han Chinese (99.8)
		Zhang et al [46].	I/II	3 µg	D0-D14, D0-D28	Aluminum Hydroxyde	18–59	53.4	Han Chinese (100)
	Inactivated Vaccine (IMBCAMS(a))	Che et al (e) [22].	II	150 U	D0-D14	Aluminum Hydroxyde	18–59	62.3	Han Chinese (70.1), Yi (19.2)
		Pu et al [20].	I	150 U	D0-D14	Aluminum Hydroxyde	18–59	55.2	Han Chinese (96.9), Tibetan (1.6)
	QazCovid-in (Scientific Research Institute for Biological Safety Problems of the Republic of Kazakhstan)	Zakarya et al (e) [23].	I	5 µg	D0-21	0.9% Saline	18–50	34.1	Not reported
	WIBP (Sinopharm’s Wuhan Institute of Biological Products)	Al Kaabi et al [28].	III	4 µg	D0-D21	Aluminum Hydroxyde	≥18	15.6	Not reported
		Guo et al [58].	II	5 µg	D0-21, D0-28	Aluminum Hydroxyde	18–59	50.4	Not reported
mRNA	BNT162b2 (Pfizer, BioNTech)	Frenck et al [38].	III	30 µg	D0-D21	0.9% Saline	12–15	50.1	White (84.7), Black (6.1)
		Haranaka et al [40].	I/II	30 µg	D0-D21	0.9% Saline	20–64, 65–85	49.3	Asian (100)
		Polack et al (d) [26].	II/III	30 µg	D0-D21	0.9% Saline	16–55, ≥55	49.4	White (82.9), Black (9.3)
		Thomas et al (d) [25].	II/III	30 µg	D0-D21	0.9% Saline	≥16	49.1	White (82), Black (9.6)
		Walsh et al [39].	I	30 µg	D0-D21	0.9% Saline	18–55, 65–85	55.2	White (88.6); Asian (8.6)
	CVnCoV (CureVac N.V.)	Kremsner et al [37].	IIb	12 µg	D0-D28	0.9% Saline	18–60, >61	45	White (25.3), Latin American (74.7)
	mRNA-1273 (Moderna)	Ali et al [41].	II/III	100 µg	D0-D28	0.9% Saline	12–17	49	White (84), Asian (6)
		Chu et al [42].	II	100 µg	D0-D28	0.9% Saline	18-<55, ≥55	65.0	White (94.8), Black (2.7)
		El Sahly et al [43].	III	100 µg	D0-D28	0.9% Saline	18-<65, ≥65	47.4	White (79.2), Black (10.2)
Protein Subunit	EpiVacCorona (Vector Institute)	Ryzhikov et al (e) [24].	I/II	225 μg	D0-D21	0.9% Saline	18–60	44.0	Not reported
	MVC-COV1901 (Medigen Vaccine Biologics Corporation, Dynavax, NIAID)	Hsieh et al [55].	II	15 µg	D0-D28	0.9% Saline	20–64, ≥65	43.5	Asian (99.9)
	NVX-CoV2373 (Novavax)	Dunkle et al [54].	III	5 µg	D0-D21	0.9% Saline	18–64, ≥65	48.2	White (75.9), Black (11)
		Formica et al [53].	II	5 µg	D0-D21	0.9% Saline	18–59, 60–84	50.9	White (86.8), Asian (7.6)
		Heath et al [51].	III	5 µg	D0-D21	0.9% Saline	18–84	48.4	White (94.5), Asian (2.9)
		Keech et al [17].	I/II	5 µg	D0-D21	0.9% Saline	18–59	49.6	White (78.6), Hispanic/Latino (14.5)
		Shinde et al [52].	II	5 µg	D0-D21	0.9% Saline	18–84	42.6	Black (95.3), White (3.5)
	S-Trimer/SCB-2019 (Clover Biopharmaceuticals, GSK, Dinavax)	Richmond et al [18].	I	30 µg	D0-D21	0.9% Saline	18–54, 55–75	65	Asian (13), White (87)
	Sf9 (West China Hospital, Sichuan University)	Meng et al [19].	I/II	40 µg	D0-D14-D28	Aluminum Hydroxyde	18–55, ≥56	60.2	Not reported
	V-01 (Guandong Provincial CDC, Gaozhou CDC, Livson Pharmaceutical Group)	Zhang et al [46].	I	10 µg	D0-D21	Aluminum Hydroxyde	18–59, >60	40	Han Chinese (100)
		Ya-Jun et al [61].	II	10 µg	D0-D21	Aluminum Hydroxyde	18–59, >60	47	Han Chinese (100)
	ZF2001 (Anhui Zhifei Longcom Biopharmaceutical, Chinese Academy of Sciences)	Yang et al [59].	II	25 μg	D0-D30-D60	Aluminum Hydroxyde	18–59	58.0	Han Chinese (99.3)

a- IMBCAMS – Institute of Medical Biology, Chinese Academy of Medical Sciences; b– Dose which is being taken forward to Phase 3 trials; c– The top two ethnicities are listed; d– Not included in meta-analysis due to missing data; e- Not included in meta-analysis due to high risk of bias; vp = viral particles

Analysis of solicited AEs focused on the six symptoms which were consistently reported in most papers: fever, fatigue, headache, pain at injection site, redness, and swelling. There was variability in the number of local and systemic AEs that were solicited in the individual trials (median four [range 3–7] and eight [range 4–16] respectively). For this reason, data on total local AEs, total systemic AEs and total ‘any AEs’ were not analyzed.

3.1. Systemic adverse events

3.1.1. Fever

Forest Plot 1 shows the RR of developing fever, as compared with the control, by each vaccine type, for those papers that divided data by dose. The overall RR of developing fever after any vaccine type was 4.21 (95% Confidence Interval [CI] 2.56–6.94). The mRNA vaccines had the highest RR for fever, especially after the second vaccination: 6.64 (95% CI 2.21–19.96) after dose one and 31.17 (95% CI 15.91–61.05) after dose two, compared to 5.97 (95% CI 2.95–12.09) for dose one of the adenovirus-vectored vaccines and 5.61 (95% CI 1.94–16.23) for dose two of the protein subunit vaccines. Inactivated vaccines had the lowest RR of fever against control for both doses (1.38 [95% CI 1.05–1.81] for dose 1 and 1.10 [95% CI 0.82–1.47] for dose two), while the risk after dose one for the adenovirus vectored (1.52 [95% CI 0.89–2.59]) and protein subunit vaccines (1.14 [95% CI 0.85–1.54]), was no greater than that of the control group.

3.1.2. Fatigue

Data on fatigue according to dose is shown in Forest Plot 2. Overall, pooled RR for fatigue after any dose was 1.69 (95% CI 1.50–1.90). A second dose of vaccine was associated with a higher RR for both mRNA (1.54 [95% CI 1.31–1.82] for dose 1 and 2.65 [95% CI 2.44–2.87] for dose 2) and protein subunit vaccines (1.14 [95% CI 1.10–1.19] and 2.00 [95% CI 1.63–2.45]). In contrast, adenovirus-vectored vaccines showed similar RRs for dose one (1.68 [95% CI 1.50–1.59]) and dose two (1.40 [95% CI 1.25–1.57]). RR for fatigue in the inactivated vaccine studies were not different to control for either dose one or two (1.34 [95% CI 0.97–1.86] and 1.27 [95% CI 0.68–2.34] respectively).

3.1.3. Headache

The RR of headache for all subgroups was 1.59 (95% CI 1.41–1.80). Data divided by dose (Forest plot 3) show that the second dose of mRNA vaccines had the highest RR (2.63 [95% CI 2.42–2.85]) compared to 1.44 [95% CI 1.16–1.77] for the first dose. An increase between doses was also seen for protein subunit vaccines (1.09 [95% CI 1.05–1.14] for dose one and 1.73 [95% CI 1.33–2.23] for dose two) while it was similar between doses for the adenovirus-vectored vaccines (1.64 [95% CI 1.47–1.83] for dose one and 1.38 [95% CI 1.24–1.54] for dose two). Overall, inactivated vaccines had smaller RRs (1.21 [95% CI 1.00–1.45] and 1.18 [95% CI 0.91–1.55] for the first and second doses respectively).

3.2. Local adverse events

3.2.1. Pain at the injection site

Data for localized pain divided by dose is presented in Forest plot 4. Overall risk of developing pain at the injection site versus control after any dose of vaccine was 3.30 [95% CI 2.92–3.72]. Second doses of mRNA (5.22 [95% CI 4.04–6.74]) and protein subunit vaccines (4.67 [95% CI 3.76–5.80]) had the highest RR, followed by mRNA first dose (4.75 [95% CI 3.62–6.24]), adenovirus-vectored vaccines (3.59 [95% CI 2.65–4.88] for first dose and 2.62 [95% CI 1.53–4.49] for second dose) and the second dose of protein subunit vaccines 3.23 [95% CI 2.78–3.76]. Inactivated vaccines had the lowest RRs (1.38 [95% CI 1.11–1.72] for dose one and 1.19 [95% CI 0.96–1.46] for dose 2).

3.2.2. Redness

RR of redness at the injection site after any vaccine dose versus control was 3.71 [95% CI 2.52–5.48]. Data divided by dose are shown in forest plot 5. Adenovirus vaccines had comparable rates following each dose (1.77 [95% CI 0.96–3.27] and 2.03 [95% CI 1.46–2.81] respectively). Conversely, there was a marked increase in the risk of redness compared to control for mRNA (first dose 6.32 [95% CI 3.69–10.82] versus second dose 12.77 [95% CI 7.64–21.34]) and protein subunit vaccines (first dose 3.39 [95% CI 2.37–4.83] versus second dose 18.78 [95% CI 10.82–32.57]). Inactivated vaccines did not show statistically significant RRs for redness compared to controls.

3.2.3. Swelling

Data analysis divided by dose is shown in forest plot 6. Inactivated vaccines had non-significant RR versus placebo for both doses and RR were similar for adenovirus-vectored vaccine dose 1 (2.50 [95% CI 1.29–4.84]) and dose 2 (1.95 [95% CI 1.52–2.50]). mRNA vaccines were at least 10 times more likely to cause local swelling compared to placebo (RR 10.78 [95% CI 7.40–15.70] for dose 1 and 17.06 [95% CI 9.97–29.18] for dose 2). There was a substantial increase in the risk of swelling after the second dose of protein subunit vaccines (18.43 [95% CI 12.47–27.23]) compared to the first (RR 3.42 [95% CI 2.41–4.84]). The RR of swelling for pooled subgroups was 4.23 (95% CI 2.74–6.52).

3.3. Analysis of data not divided by dose

Forest plots for all six symptoms for studies which did not report data divided by dose can be found in the supplement (Forest plots S1-S6). These only included inactivated and protein subunit vaccines. There were non-statistically significant RRs for fever, fatigue, headache, pain at injection site and redness for both vaccine types. For swelling, protein subunit vaccines had an overall RR of 5.87 (95% CI 1.93–17.86) whereas inactivated vaccines showed no statistically significant increased risk compared to control.

3.4. Sensitivity analysis

The sensitivity analyses were performed to compare ITT and per-protocol population for the six selected solicited AEs. Results for sensitivity analyses are presented in the supplement (Forest plots S7-S12). For all six events, there were no significant differences in the RRs for the second doses in the ITT versus the per-protocol populations except for dose two injection site pain for inactivated vaccines (original RR 1.19 [95% CI 0.96–1.46] versus 1.50 [95% CI 1.01–2.24]). This reflects the fact that the vast majority of participants who received a first dose of vaccine went on to receive their scheduled second dose.

3.5. Heterogeneity and publication bias

Forest plots showing I² values for the following analyses: younger adult population only (16–65 years), 0.9% saline control studies only and phase II/III studies can be found in the supplement (Forest plots S13 – S30). When all studies were plotted, the heterogeneity was moderate to high for adenovirus-vectored vaccines dose one (59–92%), mRNA vaccines dose one (54–97%) and dose two (65–96%), generally lowest for inactivated vaccines (both doses), the first dose of protein subunit vaccines and variable for the remaining subgroups. Heterogeneity could not be calculated for inactivated vaccines (both doses) for the 0.9% saline control analysis due to only one study being in the group. A reduction in the heterogeneity values was seen within each vaccine platform subgroup; the effect was greater when the extremes of age (teenagers and older adults) were removed from the analysis, and the heterogeneity was smaller when Phase I trials or non-0.9% saline-controlled studies were not included in the analysis. See Table S4 in the supplement for a summary of I² values. The total heterogeneity remained high (90–98%) in all the analyses in keeping with an expected difference in reactogenicity between vaccine types.

Analysis of publication bias of included clinical trials is shown in Funnel plots S1-6 in the supplement. The publication bias was assessed for each individual AE. Funnel plot asymmetry varied for each symptom. One potential reason for this may be due to the high heterogeneity observed between different trials and vaccine platforms.

3.6. Choice of control groups

Most trials for the same vaccine type used the same control for all included trials, for example, all the mRNA and protein subunit vaccine trials used 0.9% saline as a control. Four trials of the ChAdOx-1 (Oxford-AstraZeneca) vaccine used 0.9% saline as the control group [16,33,34,36] and two used a MenACWY vaccine [32,35]. Forest plots for selected symptoms (fatigue and injection site pain) are shown in the supplement (Forest plots S31 and S32). Trials with 0.9% saline control had higher RR of local pain after each vaccine dose compared to those with MenACWY control (dose one: 4.01 [2.71–5.95] vs 1.80 [1.59–2.04]; dose two 3.61 [3.10–4.21] vs 0.90 [0.62–1.330]), with a similar trend for other symptoms (data not shown). One trial of the CoronaVac vaccine [47] used water for injection (WFI) as the control while three [45,46,48] used an adjuvant (aluminum hydroxide). Forest plots for fatigue and pain are shown in the supplement (Forest plots S33 and S34). RRs were smaller for trials which used an adjuvant as the control compared to those that used WFI, although the differences were not statistically significant.

3.7. Individual vaccines

Six individual vaccines had at least three papers with data available for meta-analysis: Ad5-nCov (Cansino, adenovirus-vector), BNT162b2 (Pfizer-BionTech, mRNA), ChAdOx-1 (Oxford-AstraZeneca, adenovirus-vector), CoronaVac (inactivated), mRNA-1273 (Moderna, mRNA) and NVX-CoV2373 (Novavax, protein subunit). The pooled RR were estimated as individual vaccine subgroups rather than vaccine technology (forest plots S35 – S40 in the supplement).

All vaccines except CoronaVac had statistically significant RRs for fatigue against control, with mRNA-1273 (dose one RR 1.36 [95%CI 1.31–1.40] and dose two RR 2.64 [95%CI 2.28–3.06]) and NVX-CoV2373 (dose one RR 1.14 [95%CI 1.09–1.19] and dose two RR 2.19 [95%CI 1.83–2.62]) having higher risk after the second dose compared to the first.

All vaccines were associated with increased risk of fever against control except for CoronaVac (both doses), ChAdOx-1 (second dose), and NVX-CoV2373 (first dose). The second doses of both mRNA vaccines: BNT162b2 (RR 27.26 [95%CI 15.38–48.33] versus 8.88 [95%CI 5.48–14.39]) and mRNA-1273 (RR 25.9 [95%CI 7.78–86.46] versus 2.56 [95%CI 1.90–3.47]) and the protein subunit vaccine NVX-CoV2373 (RR 6.76 [95%CI 2.21–20.60] versus 1.13 [95%CI 0.83–1.52]) had significantly higher risk of fever compared to the first dose.

For headache, all vaccines except CoronaVac had significant RRs against control, which were higher for the second doses of BNT162b2 (2.67 [95%CI 2.43–2.92]), mRNA-1273 (2.46 [95%CI 2.28–2.65]) and NVX-CoV2373 (1.90 [95%CI 1.49–2.43]) compared to the first dose (1.54 [95%CI 1.42–1.66], 1.22 [95%CI 1.18–1.26] and 1.09 [95%CI 1.05–1.14] respectively).

All doses, except for the second dose of CoronaVac, were associated with increased risk of pain at the injection site, with no statistically significant differences between both doses. Analysis for redness showed that all vaccines except CoronaVac had increased risk versus control, statistically higher for dose two in NVX-CoV2373 (17.82 [95%CI 7.57–41.97] versus 3.25 [95%CI 2.26–4.66]) for dose one. Risk of swelling was significantly higher than control for all vaccines except CoronaVac, with increased risk for dose two compared to dose one in mRNA-1273 (28.01 [95%CI 22.21–35.32] versus 14.62 [95%CI 11.64–18.36]) and NVX-CoV2373 (16.81 [95%CI 9.71–29.08] versus 3.01 [2.07–4.38]).

4. Discussion

This systematic review and meta-analysis focuses on data reported in blinded RCTs of COVID-19 vaccines utilizing either placebo or control arms. We believe it is the most comprehensive systematic review and meta-analysis of COVID-19 vaccine reactogenicity available, with data on approximately 200,000 administered vaccine doses. Only studies of vaccines that had reached phase III clinical trials by the start of 2022 were included and thus the results of this review provide a perspective on the reactogenicity of vaccines that are currently relevant to the global community. In contrast to earlier reviews, studies of all relevant vaccine technologies were available for analysis.

Several conclusions can be drawn from these analyses. It is clear that the vaccine type (technology) does influence the likelihood of AEs occurring. This will reflect inherent biological differences between such vaccines (e.g. mRNA vs protein) as well as the inclusion of different adjuvants or vaccine dosage. In general, the mRNA vaccines are associated with the highest risk of AE and the inactivated vaccines with the lowest. This does, however, vary by dose, with higher rates of events after the second dose for both mRNA and protein subunit vaccines (compared to the first dose), higher rates after the first dose for adenovirus vectored vaccines (compared to the second dose) and equivalent rates of AEs after first and second doses for the inactivated vaccines. In fact, for the inactivated vaccines such rates were often no different from those seen in the respective control groups.

We hypothesized that the nature of the control group employed in the different trials would have a significant impact on the relative risk of AEs. The controls used in the trials varied from 0.9% saline placebo, to aluminum or another adjuvant, through to a (non-COVID-19) active vaccine. As each of these ‘controls’ will have a different intrinsic AE profile, describing the relative risk of AEs of a specific COVID-19 vaccine will vary according to the control group chosen, as demonstrated in our analysis of the ChAdOx-1 and CoronaVac vaccines by control type. The choice of a control group may be influenced by different considerations; for example, a control group that is likely to be associated with few AEs (such as 0.9% saline) might allow a ‘blinded’ trial participant to work out whether they have in fact received the COVID-19 vaccine, and thus change their behavior, with a potential impact on trial integrity. However, use of such an inert placebo will allow a full description of the AEs associated with the COVID-19 vaccine being tested. This analysis may be too simplistic however, as it ignores the ‘nocebo’ effect. Nocebo responses are thought to be caused by misattribution of routine background symptoms, anxiety, and expectations of AEs. In their recent systematic review, Haas et al. [63] focused only on the frequencies of AEs reported in the placebo groups of COVID-19 vaccine trials by excluding studies using a non-inert placebo. They estimated that 76% of systemic AEs and 24% of local AEs after the first vaccination were attributed to nocebo responses and 52% of systemic AEs and 16% of local AEs after the second dose.

Another cause for heterogeneity was extremes of age. Other reviews have drawn similar conclusions, observing that the overall adverse event incidence is higher for vaccinees aged 16–55 years than among older adults aged over 55 years [7,8].

The information provided in this review is important for health-care workers, policy makers and the general public when making decisions around receipt of COVID-19 vaccines. It may also allow better matching of specific vaccines with specific populations; for example, the preferred use of the least reactogenic vaccine platform in pregnant women. All of these considerations, however, are likely to be significantly offset by considerations around the efficacy of the respective vaccines, as an individual may prefer a more effective vaccine despite its greater reactogenicity. Clear differences in efficacy and effectiveness are now evident among the different vaccine types [4,5]. It does raise the possibility that heterologous vaccine schedules may allow matching of different vaccines with different reactogenicity and efficacy profiles in order to provide an overall schedule with lower reactogenicity and preserved efficacy. Such a hypothesis requires further study.

There are a few limitations to our study. We focused on the side effect profile of homologous primary dosing vaccine schedules and excluded studies evaluating booster doses or heterologous vaccine regimens, which are the focus of multiple on-going COVID-19 vaccine trials.

We were unable to fully evaluate several factors which may affect reactogenicity, for example age, ethnicity and prior COVID-19 infection, due to lack of granularity of the data. Some of the trials reported in this review are still on-going, therefore full safety results are not yet published or available for researchers to include in meta-analyses. Once larger datasets are available and additional trials on the pediatric population are published, meta regression would be useful to infer the effect of these variables.

Despite not limiting language in our search method, we only used English language databases and therefore will have missed publications in other languages, resulting in publication bias. In addition to this, there is an underrepresentation of trials from developing countries.

Lack of a standardized study design for the COVID-19 vaccine studies made comparing studies challenging. The three most common differences noted were: a variation in the number and type of symptoms participants were asked to report, the choice of control used, and whether data was reported by single dose or by combined dose.

We have demonstrated considerable variability in the number of local and systemic AEs that were solicited in the individual trials. Inviting participants to report a greater number of symptoms may result in the overall vaccine reactogenicity appearing to be more severe than studies that stipulate reporting of fewer symptoms. Reporting of a standard list of symptoms (as well as using standard definitions of events and of severity) would allow a more accurate and complete comparison of the reactogenicity profile of different vaccines. Such calls have been made previously by the Brighton Collaboration for vaccine studies in general [64].

The majority of trials reported reactogenicity data for each of the vaccine doses given. Some combined the data and reported it for the whole vaccine course. As the data was most commonly presented as occurrence of each symptom per participant for the whole vaccine course, this may underestimate the reactogenicity, as the same participant could have experienced the symptom twice, once with each vaccine dose. This is reflected in our results, as there was no statistically significant difference in the occurrence of reactogenicity symptoms between the control and vaccine group for the studies with results presented in this way.

5. Conclusions

Among COVID-19 vaccines currently available and/or in Phase III trials, the four vaccine types (platforms) appear to have a distinct reactogenicity profiles, which also varies between the first and second dose of each individual vaccine. Both doses of mRNA vaccines, the second dose of protein subunit and first dose of adenovirus vectored vaccines were the most reactogenic, while the inactivated vaccines were the least reactogenic. Awareness of the reactogenicity profiles of different vaccine types can allow different vaccines to be recommended for specific populations. The lack of standardization of COVID-19 vaccine trials and the way data is reported made comparisons challenging. Greater standardization of this will aid research in the future.

6. Expert opinion

Current COVID-19 vaccine trials have shifted their original focus on safety and efficacy of doses in unvaccinated participants to booster studies (third or fourth doses), which commonly include heterologous ‘mix-match’ schedules combining more than one vaccine platform, rather than the homologous primary dosing schedules which are the subject of this review. As a result, there may be significant differences in the reactogenicity of booster dosing schedules. In addition to this, previous COVID-19 immunity may impact the range and intensity of side effects experienced after vaccination. Furthermore, novel studies are being performed on population groups previously excluded from the earlier trials: children, pregnant women and immunocompromised patients, who may have a different reactogenicity profile compared to the general public.

Side effect data reporting on vaccine trials are still very heterogeneous, despite international efforts to unify definitions such as through the Brighton collaboration. More efforts should be made to standardize COVID-19 vaccine trial design: from the choice of control, follow-up duration, definitions of solicited and unsolicited adverse events to safety data reporting and choice of denominator to calculate adverse event rates (participants receiving a vaccine versus participants reporting adverse events).

Ultimately, the choice of COVID-19 vaccine will need to consider multiple factors, including reactogenicity, frequency of rare and serious adverse events, efficacy against circulating COVID-19 variant strains, availability of doses and costs.

For obvious reasons, the development, testing, and implementation of COVID-19 vaccines have occurred at an unprecedented pace. This now means there are likely to be fewer opportunities to assess new COVID-19 vaccine candidates in the context of a placebo-controlled trial. Comparisons will need to be made against licensed COVID-19 vaccines, rather than against non-COVID vaccine control groups. This may complicate any ongoing analysis of the reactogenicity of COVID-19 vaccines and will require creative trial designs to ensure that the reactogenicity of new candidate vaccines can be accurately reported. In addition to this, safety monitoring will shift from RCTs to large community cohort longitudinal follow-up data or Phase IV pharmacovigilance studies to detect infrequent adverse events.

It is likely that studies of the reactogenicity of COVID-19 vaccines as booster doses, as part of heterologous schedules and in other important groups such as children will become more common over the next 5 years.

Forest plot 1. Risk ratio of fever compared to control, by vaccine type and dose.

Forest plot 2. Risk ratio of fatigue compared to control, by vaccine type and dose.

Forest plot 3. Risk ratio of headache compared to control, by vaccine type and dose.

Forest plot 4. Risk ratio of pain at injection site compared to control, by vaccine type and dose.

Forest plot 5. Risk ratio of redness compared to control, by vaccine type and dose.

Forest plot 6. Risk ratio of swelling compared to control, by vaccine type and dose.

Article highlights

• The scientific community’s response to the COVID-19 pandemic has resulted in the development of numerous COVID-19 vaccines in a short period of time.

• Four main vaccine types (mRNA, adenovirus-vector, protein subunit and inactivated) are in advanced clinical trials or approved for general use in different populations.

• Each vaccine type has a unique reactogenicity profile with mRNA vaccines being most reactogenic and inactivated vaccines being least reactogenic.

• High heterogeneity is evident within each vaccine type. This reduced when age was restricted to adults aged 16–55 years and individual vaccines were analyzed separately.

• Lack of standardization of COVID-19 vaccine trial design makes comparison of different vaccines challenging. The main differences noted were in variation in the number and type of post-vaccine symptoms participants were asked to report, the choice of control used and whether data was reported by single dose or combined doses.

• Use of a control (adjuvants or MenACWY vaccine) instead of a placebo (0.9% saline) reduced the risk ratios of AEs, thereby underestimating reactogenicity.

• Standardization of vaccine trial design and reporting will aid comparison of vaccines in the future.

• Awareness of the reactogenicity profile of different vaccine types will aid health-care workers and policy makers to make decisions around the use of different vaccine types in different settings and populations. For example, the use of less reactogenic vaccines for pregnant women and children.

Declaration of interest

PT Heath coordinates research on behalf of St Georges, University of London which is funded by vaccine manufacturers, including those that manufacture COVID-19 vaccines (Pfizer, AZ, Novavax, Moderna, Valneva, Janssen). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

PH, NS, SI, DS, EG, and YH conceptualized and designed the work. NS, ASFR, and EB carried out the database search and identified eligible studies. NS, ASFR, EB, DS, SI, and YH carried out risk of bias assessments and data extraction. DS summarized raw data from authors using R. Funnel plots were created by ASFR. Data analysis and interpretation was guided by YH. PH, NS, ASFR, EB, and EG drafted the manuscript. All authors contributed to, reviewed, and approved the final manuscript.

Acknowledgments

We thank Jonathan Lewis-Bridgeman and Atul Kumar-Beurg for their help with Excel and data management, and Caroline Albrecht for creating Figure 2.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/14760584.2022.2098719

Additional information

Funding

This paper was not funded.