The efficacy and effectiveness of the COVID-19 vaccines in reducing infection, severity, hospitalization, and mortality: a systematic review

ABSTRACT

With the relatively rapid development of the COVID-19 pandemic, vaccine development has become crucial for limiting disease transmission. The accelerated growth in the approved COVID-19 vaccines has sparked concerns about their efficacies which have been assessed by many studies. This systematic review compares the efficacy and effectiveness of seven COVID-19 vaccines. A comprehensive systematic literature search was performed using several databases to identify studies reporting the effectiveness or the efficacy of the vaccines. Only 42 studies met our inclusion criteria, which revealed that the COVID-19 vaccines have successfully reduced the rates of infections, severity, hospitalization, and mortality among the different populations. The full-dose regimen of the Pfizer/BioNTech vaccine is the most effective against infections with the B.1.1.7 and B.1.351 variants. Despite of the high effectiveness of some of the COVID-19 vaccines, more efforts are required to test their effectiveness against the other newly emerging variants.

KEYWORDS:

• COVID-19
• vaccine
• effectiveness
• efficacy

1. Introduction

As of June 2, 2021, the COVID-19 pandemic has caused almost 172 million infections and 3.5 million deaths worldwide.¹ COVID-19, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was first identified in December 2019 in Wuhan, China.

SARS-CoV-2 is analogous to the coronaviruses that cause severe acute respiratory syndrome (SARS-CoV) and Middle East respiratory syndrome (MERS-CoV).² The relatively rapid sequencing of its genome allowed for diagnostic testing, epidemiologic tracking, and the development of both preventative and treatment methods.³ Indeed, the development of vaccines is central for limiting SARS-CoV-2 transmission.⁴

1.1. Types of COVID-19 vaccines

According to the McGill COVID-19 vaccine tracker system, there are currently ongoing trials on more than 120 vaccine candidates, of which 17 have been granted approval in multiple countries.⁵

1.1.1. Nucleic acid vaccines (mRNA vaccines)

In general, the development of nucleic acid vaccines is faster and less expensive than protein subunit vaccines, which may be a driving factor to the production of several mRNA-based SARS CoV-2 vaccines;^6,7 however, the tendency of mRNA to degrade remains an obstacle in developing mRNA vaccine candidates.^6,8 To stabilize the vaccine structure, strategies such as encapsulating the mRNA in lipid nanoparticles are implemented.^6,9 Furthermore, these lipid nanoparticle structures can act as adjuvants causing increased T- and B-cell responses⁶. This was the case with the first two vaccinations, Pfizer/BioNTech (BNT162b2) and Moderna (mRNA-1273), which were delivered internationally.⁴

1.1.1.1. BNT162b2 (Pfizer/BioNTech)

In the collaboration between BioNTech and Pfizer, four RNA-based vaccine candidates were developed, of which two advanced to further testing.⁶ Pfizer and BioNTech selected BNT162b2 to progress to a Phase 2/3 study based on preclinical and clinical data, including immune response and tolerability.¹⁰ The vaccine, also encapsulated in lipid nanoparticle, encodes a perfusion-stabilized, membrane-anchored SARS-CoV-2 full-length spike protein (BNT162b2).⁶ The Pfizer/BioNTech vaccine has been approved in 85 countries.⁵

1.1.1.2. mRNA-1273 (Moderna)

The vaccine candidate mRNA-1273, developed in a collaboration between Moderna and the National Institute of Allergy and Infectious Diseases (NIAID), encodes the spike-2 protein antigen, which is made of SARS-CoV-2 glycoprotein with the transmembrane anchor and an intact S1-S2 cleavage site.^6,11 In initial trials on nonhuman primates, the vaccine generated a strong anti-SARS-CoV-2 neutralizing antibody response.^6,12

1.1.2. Adenoviral-based vaccines

Adenoviral-based vaccines are the most extensively used virally vectored option for non-replicating SARS-CoV-2 vaccinations.⁶

1.1.2.1. AZD-1222/ChAdOx1-nCoV (Oxford/AstraZeneca)

In partnership with AstraZeneca, the University of Oxford has developed a chimpanzee adenovirus-vectored vaccination that expresses the full S protein (AZD-1222, known prior as ChAdOx1-nCoV).⁶ To guarantee the development of long-term immunity, the first injection is followed by a booster dose 28 days later.^6,13,14 The Oxford/AstraZeneca vaccine has been approved in 99 countries.⁵

1.1.2.2. Ad26.COV2.S (Janssen by Johnson and Johnson) (J&J)

Another example of an adenovirus-based vaccine is the Janssen Pharmaceuticals vaccine by Johnson and Johnson, which is a recombinant adenovirus serotype 26 (Ad26) vector that encodes a full-length, stabilized SARS-CoV-2 spike (S) protein.^15,16 The vaccine uses a recombinant human-based adenovirus vector to deliver the genetic material.⁴ In comparison to the other alternatives, the J&J vaccine has the advantage of being delivered in only one dosage, which decreases production costs.⁴ The J&J vaccine has been approved in 42 countries.⁵

1.1.2.3. rAd26-S + rAd5-S (Sputnik V)

A further example of an adenovirus-based vaccine is the Sputnik V or Gam-COVID-Vac-Adeno-based (rAd26-S + rAd5-S) vaccine developed by the Gamaleya Research Institute of Epidemiology and Microbiology.⁴ The vaccine is designed with two recombinant adenovirus vectors, recombinant adenovirus type 26 (rAd26) and recombinant adenovirus type 5 (rAd5), both of which carry the gene for the SARS-CoV-2 full-length glycoprotein S (rAd26-S and rAd5-S).¹⁷ Preliminary clinical trial results for nonreplicating viral vector vaccine candidates for SARS-CoV-2 have shown safety and immunogenicity, defined as the detection of antibodies induced against the spike protein using enzyme-linked immunosorbent assays (ELISA) or, as in some studies, cellular immunity development defined by measurements of interferon-gamma (IFNγ), interleukin-2 (IL-2), and tumor necrosis factor (TNFα).⁶

1.1.3. Protein subunit vaccines

The synthetic-protein subunit approach is an alternate way of vaccine production. These vaccine candidates are made of a recombinant spike protein expressed in diverse cell lines.⁶ Similar to mRNA-based vaccines, peptides are generally unstable and hence are often bundled into nanoparticles to stabilize delivery.⁶

1.1.3.1. NVX-CoV2373 (Novavax)

The most promising SARS-CoV-2 protein vaccine candidate is Novavax’s NVX-CoV2373, which is now in phase 3 clinical trials.⁶ This vaccine is made up of nanoparticles that contain the full-length wild-type spike glycoprotein, which has been designed to be resistant to proteolytic cleavage and capable of binding to ACE2 receptors with high affinity.¹⁸ Protein synthesis was improved using the well-established baculovirus Spodoptera frugiperda (Sf9) insect cell expression system, and Novavax’s Matrix-M1 adjuvant was employed to boost vaccination immunogenicity.^6,18 Novavax’s vaccine is currently being tested in 6 trials in 6 different countries.⁵

1.1.4. Inactivated virus vaccines

The inactivated virus vaccines are based on the idea of eliminating the infectivity of the virus to make it safe, while maintaining its immunogenic potential. Even though this kind of immunization has historically been the most successful, its long production time has put it at a disadvantage in the current COVID-19 epidemic.⁶

1.1.4.1. CoronaVac (Sinovac)

A promising vaccine utilizing this methodology is Sinovac, an inactivated vaccine candidate developed by Sinovac, currently being assessed in Phase IV.⁶ Variants of the SARS-CoV-2 virus are produced using Vero (African Green Monkey) cell lines in this vaccination. Beta-propiolactone is utilized to inactivate the virus once it is extracted, and the viral particle is subsequently adsorbed onto an adjuvant (aluminum hydroxide) .^6,19,20 The inactivated viral vaccine seems to have less side effects when compared to other vaccination forms; the majority of systemic side effects were mild, with no severe ones reported. The Sinovac vaccine has been approved in 26 countries.⁵

1.2. The development and approval of the COVID-19 vaccines

The development of a new vaccine may take 10–15 years. The phases of vaccine development involve the pre-clinical studies including testing the safety and immunogenicity of the vaccine using animal, models. This will be followed by 3 phases of human clinical trials starting by testing the safety and immunogenicity in small groups in phase one then larger groups in phases 2 and 3. Before the vaccine is used for public, it has to be approved by the Food and Drug Administration (FDA) of the USA, or European Medicines Agency in EU. Due to the urgent need to develop a protective vaccine against SARS-CoV-2, the phases of vaccines development have been combined to accelerate the process.²¹

Developing COVID-19 vaccines within a short timeframe has raised several concerns about the safety and efficacy of the vaccines. For example, in one of the UK hospitals, Robbins et al., 2021 reported that 27 of the 174 (16%) COVID-19 inpatients during weekend in February had previously received a COVID-19 vaccine.²² However, the study did not specify the type of vaccine or the number of doses received. Furthermore, in a study conducted in New York City and included all the employees and students of Rockefeller University campus, two women tested COVID-19 positive and were symptomatic after more than 2 weeks following receiving the second dose of the Pfizer/BioNTech or the Moderna vaccines; viral analysis showed different SARS-CoV-2 variants of clinical importance that indicated a potential risk infection following vaccination.²³ This raised a concern that vaccinated individuals may remain susceptible to COVID-19.

While vaccine efficacy is determined in the clinical trials under controlled conditions, vaccine effectiveness assesses the vaccine among the real populations.²⁴ In the setting of this new pathogen, it is challenging to evaluate potential vaccine candidates’ clinical effectiveness. Because there has been a scarcity of large-scale analysis comparing the efficacy of the multiple available candidate vaccines against confirmed COVID-19 infection, the goal of this systematic review was to compare the efficacy/effectiveness of candidate vaccines in reducing the number of infections, deaths, and severity of infection. Figure 1 diagrammatically illustrates the different sections of the review to facilitate the comprehensive understanding of its structure.

Figure 1. A diagrammatic illustration summarizing the structure of the review.

2. Methods

We conducted a comprehensive search that prioritized sensitivity for comprehensiveness. The following databases were searched in the end of April 2021 (see appendix I): PubMed, Medline (Ovid, 1946 – April 2021), Embase (Ovid, 1974 − 2021), Scopus, Web of Science, Science Direct, MedRxiv, and Lens.org. All searches were limited by year to 2020 through 2021 (or current date). No language restrictions were used, and all searches, where allowed, employed a combination of controlled vocabulary and keywords. A total of 48,292 articles were retrieved through initial searching. Results were imported into EndNote (version 19) and initial de-duplication was conducted using the Bramer methodology,which reduced the total number of articles to 19,736 An additional 3,428 duplicates were removed when references were imported into Covidence, leaving 16,308 unique articles for initial screening.

During the screening phase, the studies reporting the efficacy, or the effectiveness of the COVID-19 vaccines were selected. We mainly focused on the studies that reported the reduction of COVID-19 infection, severity, hospitalization, mortality or viral load. No restrictions were made about country, age or gender. Any duplicated articles were removed and reviews or any articles that did not include primary data were excluded from the study. Studies that were not in English or those that did not specify the type of COVID-19 vaccine were excluded. Articles devoid of original patient data were excluded from the study. Title and abstract as well as full-text screening were conducted by two different reviewers for each study using Covidence and disagreements were resolved by consensus. Demographic and clinical data of patients reported in each study (whenever data were available) were extracted independently by two different reviewers and disagreements were resolved by consensus. Extracted data included age, sex, type of vaccines and the efficacy/effectiveness of each vaccine in reducing COVID-19 infections, severity, hospitalization and mortality rate. Categorical variables were expressed as percentages while continuous variables were expressed as mean standard deviation or range of results. Data were extracted from each study by two different reviewers.

3. Results

3.1. Studies selection and their specifications

Figure 2 shows the results of database search and screening. The flow diagram summarizes the details of our protocol. After removing the duplicates, a total of 16,308 studies were retrieved, and among those, 277 studies were selected for full-text screening. Only 42 studies that met the inclusion criteria were included.^25–66 A total of 235 studies were excluded as 214 studies were irrelevant to the data of interest, 18 were without primary data, 1 was an ongoing study, 1 was not in English, and 1 used animal models. Tables 1 and 2 illustrate the highest efficacy and effectiveness values reported for each vaccine after the first and second dose respectively. In most of the included studies, the results were reported as 95% confidence intervals (CI). For simplicity, we specified the confidence level only whenever a value other than 95% was reported. Table 3 summarize the results from 5 studies where vaccine effectiveness was not assessed by determining the number of reduced COVID-19 infections, infection severity, hospitalization or mortality rate.^{35,44,47,51,53} The different methods of assessment of vaccine effectiveness is shown for each study.

Table 1. The highest effectiveness/efficacy values obtained for each category (infection, severity, mortality, and hospitalization) for each vaccine’s 1st dose (partial vaccination). The highest value recorded against a variant is also included. If the highest value was obtained in analysis toward a particular variant, only that value is included

Vaccine	Infection		Severity		Hospitalization		Mortality		Efficacy total no. of studies	Effectiveness total no. of studies
	Efficacy	Effectiveness	Efficacy	Effectiveness	Efficacy	Effectiveness	Efficacy	Effectiveness
Pfizer	NR	91 (90% CI 83–98) at ≥21 days after vaccination^[Citation43] B.1.1.7* 70 (CI 55–85) at ≥21 days after vaccination^[Citation41] B.1.351 16.9 (CI 10.4–23)^[Citation25]	NR	85 (CI 71–92) at 15–28 days after vaccination^[Citation26] B.1.1.7* 62 (CI 39–80)^[Citation34] B.1.351¶ 0 (CI 0–19)^[Citation25]	NR	85 (CI 71–92) at 15–28 days after vaccination^[Citation26] B.1.1.7* 74 (CI 56–86)^[Citation34]	NR	B.1.1.7* 72 (CI 19–100) at 14–20 days after vaccination^[Citation34] B.1.351¶ 0 (CI 0–19)^[Citation25]	0	15^[Citation25,Citation26,Citation30,Citation31,Citation33,Citation34,Citation38–41,Citation43,Citation50,Citation52,Citation54,Citation55,Citation60]
Moderna	95.2 (CI 91.2–97.4) at ≥14 days after vaccination^[Citation28]	51.7 (CI 37.3–63.0) at ≥7 days after vaccination^[Citation55]	NR	NR	NR	NR	NR	NR	1^[Citation27]	1 ^[Citation55]
AstraZeneca	63.9 (46.0–75.9) at 22–90 days after vaccination^[Citation65]	74% (HR 0.26 (CI 0.19–0.35) reduction in risk of testing positive. at ≥28 days after vaccination^[Citation38]	NR	NR	100 (97.5% CI one sided with lower limit:72 · 2) at ≥22 days after vaccination^[Citation65]	37% protection against hospitalization^[Citation30]**	NR	NR	2^[Citation49,Citation65]	2 ^[Citation30,Citation38]
J&J	-	-	-	-	-	-	-	-	-	-
Sputnik	91.6 (CI 85.6–95.2) at ≥21 days after vaccination^[Citation48]	NR	100 (CI 94.4–100.0) at ≥21 days after vaccination^[Citation48]	NR	NR	NR	NR	NR	1^[Citation48]	0
Novavax	-	-	-	-	-	-	-	-	-	-
Sinovac	NR	P.1 49.4 (CI 13.2–71.9)* at ≥14 days after vaccination^[Citation42]	NR	NR	NR	NR	NR	NR	0	1^[Citation42]
Pfizer/BioNTech (BNT162b2) Moderna (mRNA-1273)	NR	80 (CI 59–90) at ≥14 days after vaccination^[Citation63] 80 (CI 59–90)^[Citation36]	NR	NR	NR	NR	NR	NR	0	7^[Citation27,Citation32,Citation36,Citation52,Citation62–64]

*This was the dominant variant in the location of the study’s participants during the study period, suggesting that the indicated vaccine in the table is effective against this variant.

**This study concluded that the variant B.1.1.7 has a minor effect on the observed vaccine effectiveness.

*** Data was obtained from the USA study cohort, where the dominant variant was (Wuhan-Hu-1), the South Africa study cohort, where the dominant variant was (B.1.351) and from the Brazil study cohort, where the dominant variants were (P.2) and (Wuhan-Hu-1).

¶Includes fatal disease as well as severe or critical illness.

Table 2. The highest effectiveness/efficacy values obtained for each category (infection, severity, mortality, and hospitalization) for each vaccine’s 2nd dose (full vaccination). The highest value recorded against a variant is also included. If the highest value was obtained in analysis toward a particular variant, only that value is included

Vaccine	Infection		Severity		Hospitalization		Mortality		Efficacy total no. of studies	Effectiveness total no. of studies
	Efficacy	Effectiveness	Efficacy	Effectiveness	Efficacy	Effectiveness	Efficacy	Effectiveness
Pfizer	95.0 (CI 90.3–97.6) at ≥ 7 days after vaccination^Citation56	99.5 (CI 97.0–99.9) for symptomatic infections at ≥35 days after vaccination^Citation66 B.1.1.7** 94 (CI 87–98) for symptomatic infections. At >7 days after vaccination^Citation34 B.1.351: 75.0 (CI 70.5–78.9) at ≥14 days after vaccination^Citation25	NR	B.1.1.7: 100 (CI 81.7–100)* at 14 days after vaccination^Citation25 B.1.351: 100 (CI 73.7–100)* at 14 days after vaccination^Citation25	NR	B.1.1.7** 100 (CI 51.4–100)^Citation55	NR	B.1.1.7: 100 (CI 81.7–100)* at 14 days after vaccination^Citation25 B.1.351: 100 (CI 73.7–100)* at 14 days after vaccination^Citation25	1^[Citation56]	13 ^[Citation25,Citation27,Citation29,Citation31,Citation34,Citation39–41,Citation50,Citation52,Citation54,Citation55,Citation60,Citation66]
Moderna	94.1 (CI 89.3–96.8) at ≥14 days after vaccination^Citation28	93.3 (CI 85.7–97.4) at ≥7 days after vaccination^Citation55	100 (CI could not be estimated to 1.0)^Citation28	86.0 (CI 71.6–93.9) at ≥7 days after vaccination (severe infection requiring hospitalization)^Citation55	NR	100 (43.3–100) at ≥7 days after vaccination (ICU admission)^Citation55	NR	NR	1^Citation28	2 ^[Citation27,Citation55]
AstraZeneca	B.1.1.7 80.7% (CI 69.2–87.9) for all other strains. at ≥14 days after vaccination^Citation46	NR	NR	NR	NR	NR	NR	NR	4^Citation45,Citation46,Citation49,Citation65	0
Sputnik	-	-	-	-	-	-	-	-	-	-
Novavax	60.1 (CI 19.9–80.1) at ≥7 days after vaccination B.1.351*** 43.0 (CI – 9.8–70.4)^Citation61	NR	NR	NR	NR	NR	NR	NR	1^Citation61	0
J&J	NR	66.9 (CI 59.1–73.4) ¶ at ≥14 days after vaccination^Citation59	NR	83.5 (54.2 to 96.9) ¶ at ≥28 days after vaccination^Citation59	NR	100 (CI 74.3–100.0) ¶ at ≥28 days after vaccination^Citation59	NR		1^Citation59	0
Sinovac	NR	73.8 (CI 57.0–84.8) at ≥5 weeks after vaccination^Citation37	NR	NR	NR	NR	NR	NR	0	2^[Citation37,Citation42]
Pfizer/BioNTech (BNT162b2) Moderna (mRNA-1273)	NR	91.3 (CI 79.7–96.3) for symptomatic infection at ≥15 days after vaccination^Citation27	NR	96 (CI 95–99)^Citation64	NR	96 (CI 95–99)^Citation64	NR	98.7 (CI 91.0–99.8)^Citation64	0	6^[Citation27,Citation36,Citation52,Citation62–64]
J&J (Ad26.COV2.S) Pfizer/BioNTech (BNT162b2) Moderna (mRNA-1273)	NR	Number of new cases for those 65+ years of age was 82% lower, and 70% lower for those <65 years of age^Citation58	NR	NR	NR	Number of hospitalizations for those 65+ years of age was 80% lower, and 60% lower for those <65 years of age^Citation58	NR	Number of deaths for those 65+ years of age was 92% lower, and 87% lower for those <65 years of age^Citation58	0	1^[Citation58]

*Includes fatal disease as well as severe or critical illness.

**This was the dominant variant in the location of the study’s participants during the study period, suggesting that BNT162b2 is effective against this variant.

*** Variant was identified in 93% of sequenced samples.

¶ Data was obtained from the USA study cohort, where the dominant variant was (Wuhan-Hu-1), the South Africa study cohort, where the dominant variant was (B.1.351) and from the Brazil study cohort, where the dominant variants were (P.2) and (Wuhan-Hu-1).

Table 3. Data from 5 studies where vaccine effectiveness was assessed by different methods. . The different methods of assessment of vaccine effectiveness is shown for each study

Study	Type of study and country	Vaccine	Unvaccinated (N)	Vaccinated (N)	Method of assessment	Recorded difference post-vaccination	Notes
McEllistrem et al.^Citation51	Single-Center retrospective study USA	Pfizer-BioNtech	5	5	Nasopharyngeal viral load (NVL)	A single dose of vaccination was associated with 2.4 mean log10 lower NVL than detected in the absence of vaccination (p = .004)	Mean Log10 NVL in unvaccinated: 9.5 (9.3–9.8); vaccinated: 7.1 (5.4–8.8) The difference is statistically significant
Lillie et al.,^Citation47	Retrospective cohort study UK	Pfizer-BioNtech	NR	NR	RT-PCR tests	Significant negative correlation between cumulative vaccination and PCR positive cases (Pearsons R = −0.9061, p = .0019)	This study assessed the general effectiveness of the vaccine by looking at a large cohort of healthcare workers without dividing them into vaccinated and unvaccinated.
Jones et al.,^Citation44	Retrospective study UK	Pfizer-BioNtech	NR	NR	RT-PCR tests	26/3252 (0.8%, Wilson’s interval 0.6–1.2%) positive tests in unvaccinated health care workers (HCWs) (Ct < 36). 13/3535 (0.4%, Wilson’s interval 0.2–0.6%) positive tests in HCWs <12 days post-vaccination. 4/1989 (0.2%, Wilson’s interval 0.1–0.5%) positive tests in HCWs ≥12 days post-vaccination (p = .023 and p = .004)	Population numbers were not reported (only test numbers)
Daniel et al.,^Citation35	Observational study USA	Pfizer-BioNtech Moderna	8969 (6282 F, 2558 M, 129 unknown)	8121 fully vaccinated (5181 F, 2924 M, 16 unknown) 6144 partially vaccinated (3891 F, 2232 M, 21 unknown)	RT-PCR tests	Infections in: 234 of 8969 unvaccinated employees (2.61%; CI 2.29 to 2.96). 112 of 6144 partially vaccinated employees (1.82%; 95% CI 1.50 to 2.19). 4 of 8121 fully vaccinated employees (0.05%; CI 0.01 to 0.13) (P < .01 for all pairwise comparisons)	A greater than 90% decrease in the number of employees who were either in isolation or quarantine was observed. Real-world experience with SARS-CoV-2 vaccination at University of Texas Southwestern Medical Center (UTSW) has shown a marked reduction in employee’s incidence of infections
Mor et al.,^Citation53	Matched pairs analysis US	Pfizer-BioNtech Moderna	13221 (8091 F/5130 M	12,157 (7443 F/4714 M)	RT-PCR and antigen tests	1 week post vaccination: 2.5 fewer infections per 100 at-risk residents per week (CI 1.2–4.0). At 5  weeks, the predicted reduction in new infections was 5.2 cases per 100 at-risk residents (CI 3.2–7.3). By 5 to 8  weeks post-vaccine clinic, early vaccinated facilities had a predicted 1.1 to 3.8 fewer hospitalizations and/or deaths per 100 infected residents per day	Late vaccinated group was considered as the unvaccinatedgroup. CI

Figure 2. Screening and study selection protocol.

3.2. Efficacy and effectiveness of the vaccines determined by reduction of infection, severity, hospitalization, and mortality

Supplementary Tables (ST) 1–4 summarize the types of studies and number of vaccinated and control participants in each study for each vaccine. ST1 and 2 summarize the reported efficacy values for each vaccine after the first and second doses, respectively, while ST3 and 4 summarize the reported effectiveness values for each vaccine after the first and second doses respectively. Furthermore, Tables 1 and 2 compare the highest efficacy and effectiveness values for each vaccine after the first and second doses, respectively. Results for the J&J vaccine have been reported only in the tables of the second dose (ST2 and Table 2) as only one dose is required for full vaccination.

3.2.1. Pfizer/BioNTech

Our results yielded 20 studies on the Pfizer/BioNTech vaccine: 1 randomized, controlled trial and 19 population-based studies. The trial included 18,198 vaccinated participants who were compared to 18,325 controls in efficacy and effectiveness analyses.⁵⁶ The number of vaccinated and unvaccinated individuals in the 19 included population studies are reported in ST1-ST4.

3.2.2. Moderna

Our results yielded three studies on the Moderna vaccine: 1 randomized, observer-blinded, placebo-controlled, phase 3 trial and 2 population-based studies. The trial included 15,170 unvaccinated participants who were compared to 15,181 vaccinated persons in efficacy and effectiveness analyses²⁸ The population-based study by Pawlowski et al. included 31,069 unvaccinated individuals and 31,069 vaccinated subjects.⁵⁵ Furthermore, the study by Andrejko et al. included 767 unvaccinated individuals and 256 vaccinated subjects who received the Moderna vaccine.²⁷

3.2.3. Oxford/AstraZeneca

Six studies were retrieved by our search on the Oxford/AstraZeneca vaccine: 1 was a pooled analysis of 4 randomized controlled trials, 3 were randomized, controlled trials, and 2 were population-based studies. The pooled analysis included 8,581 unvaccinated subjects who were compared to 8,597 vaccinated subjects.⁶⁵ The trial by Knoll et al. involved 5,829 unvaccinated subjects who were compared to 5,807 vaccinated subjects.⁴⁵ The trial by Emary et al. involved 4,270 unvaccinated subjects who were compared to 4,236 vaccinated subjects.⁴⁶ The trial by Madhi et al. included 717 unvaccinated subjects who were compared to 750 vaccinated subjects.⁴⁹ The population-based study by Glampson et al. involved 1,794,352 unvaccinated subjects who were compared to 389,587 vaccinated subjects.³⁸ The population-based study by Bernal et al. involved 18,061 unvaccinated subjects who were compared to 138,869 vaccinated subjects.³⁰

3.2.4. Janssen (Johnson & Johnson)

The results of our search yielded 1 randomized controlled trial on the J&J vaccine, which included 21,895 vaccinated participants who were compared to 21,888 unvaccinated persons in an efficacy analysis.⁵⁹

3.2.5. Sputnik V

Our results yielded 1 randomized controlled trial on the (Sputnik V) vaccine, which included 14,964 vaccinated participants with one dose who were compared to 4,902 unvaccinated persons in an efficacy analysis.⁴⁸

3.2.6. Novavax

The results of our search yielded 1 randomized controlled trial on the Novavax vaccine, which included 2,188 unvaccinated participants who were compared to 2,199 vaccinated persons in an efficacy analysis.⁶¹

3.2.7. Sinovac

The results of our search yielded 2 population-based studies on the Sinovac vaccine. In the population-based study by Faria et al., there were 22, 402 vaccinated participants compared to 22,402 unvaccinated persons in an effectiveness analysis.³⁷ The second study by Hitchings et al. reported the result based on the number of RT-PCR tests done for the cohorts of vaccinated and unvaccinated persons, not the number of individuals in each group.⁴²

3.2.8. Studies reporting effectiveness of multiple vaccines

The results of our search yielded 8 population-based studies reporting vaccine effectiveness in populations receiving ≥1 vaccine type: 6 with populations receiving the Moderna or the Pfizer/BioNTech vaccines,^{27,32,36,52,63,64} 1 with a population receiving Moderna, Pfizer/BioNTech, or the J&J vaccine,⁵⁸ and 1 where the population received the Moderna or the Pfizer/BioNTech vaccine with <0.1% of the included participants with unknown data on the given vaccine.⁶² The number of vaccinated and unvaccinated individuals in the 6 included population studies are reported in ST1-ST4. In the study involving the J&J vaccine, data from 4,338,099 vaccinated persons was used to determine vaccine effectiveness.⁵⁸ Finally, in the study where <0.1% of the included participants had unknown data on the given vaccine, 45,327 unvaccinated participants were compared to 3,006 vaccinated subjects in an effectiveness analysis.⁶²

3.3 Other methods of assessing the effectiveness of the vaccines

Our research yielded five studies where vaccine effectiveness was reported based on nasopharyngeal viral load or the correlation between vaccination rates and positive PCR tests^{35,44,47,51,53} (Table 3).

4. Discussion

The accelerated growth in the approved COVID-19 vaccines has sparked concerns about their efficacies which have been assessed by many studies. This systematic review compares the efficacy and effectiveness of seven COVID-19 vaccines through 42 included studies. Furthermore, multiple SARS-CoV-2 virus subtypes were discovered worldwide during the second wave of the pandemic. eThe nomenclature system established by WHO on May 31 for naming and tracking SARS-CoV-2 variants of concern are as follows: 1) Alpha (B.1.1.7) first documented in the UK in September 2020; 2) Beta (B.1.351) first documented in South Africa in May 2020; 3) Gamma (P.1) first documented in Brazil in November 2020; 4) Delta (B.1.617.2) firstly documented in India in October 2020.⁶⁷ As a result of the emergence of the new variants, several research groups conducted studies to assess the efficacy of the approved vaccines against each variant. The next sections summarize the efficacy versus the effectiveness of the newly developed COVID-19 vaccines against SARS-CoV-2 in general and against the new variants whenever reported.

4.1. Efficacy vs effectiveness of the vaccines

Vaccine efficacy has been defined by multiple outcomes in several studies. Laboratory-confirmed COVID-19 or SARS-CoV-2 infection, according to the US Food and Drug Administration (FDA), are valid primary endpoints for vaccination effectiveness trials (FDA). To quantify the efficacy attributable to the vaccination, outcome data from randomized controlled trials (RCTs) are frequently reported as a proportionate decrease in illness between vaccinated and control participants.^68,69 Provided that an immunological correlate of protection is established, vaccine efficacy can also be tested by assessing the percent of recipients who elicit a certain immune response.^68,70

Vaccine efficacy is usually expressed as a relative risk reduction (RRR). It employs the relative risk (RR) which is the ratio of attack rates with and without vaccination.⁷¹ While RRR only evaluates those who could benefit from the vaccination, the absolute risk reduction (ARR), the difference between attack rates with and without the vaccine, takes into account the entire population.⁷¹ ARRs are often overlooked since they provide a considerably smaller effect magnitude than RRRs.⁷¹ ARR is used to calculate vaccine effectiveness, which is defined as the number of people who need to be vaccinated to prevent one additional case of COVID-19 (1/ARR).⁷¹ To understand the difference between effectiveness and efficacy results, it is important to note the variables affecting both measures. For instance, the ARR and the number needed to vaccinate are sensitive to background risk with a greater risk translating to a higher effectiveness.⁷¹ When only RRRs are used and ARRs are not considered, reporting bias is seen, affecting vaccination efficacy interpretation.⁷¹ Differing study protocols in primary endpoints, placebo types, study populations, and varying background infection risks make comparing vaccines with the currently available data more difficult.⁷¹ Furthermore, the question of whether a vaccination of a certain efficacy in one group would have the same efficacy in another population with different characteristics remains unresolved.⁷¹

Vaccine efficacy is determined in a controlled clinical trial by comparing how many people who received the vaccine acquired the infection to how many people who received the placebo developed the same infection.²⁴ Vaccine effectiveness, on the other hand, is an assessment of how well vaccines perform in a less controlled setting.²⁴ While the controlled setting in a clinical trial involves a diverse population including people of all ages, genders, races, and medical conditions, it cannot perfectly represent the whole population, hence the need for studies assessing vaccine effectiveness.²⁴

4.2. Efficacy vs effectiveness of the COVID-19 vaccines in partially vaccinated individuals

4.2.1. Pfizer/BioNTech

4.2.1.1. Effectiveness against infection, severe infection, hospitalization, and mortality

The reported effectiveness for the first dose against infection ranged from 16.9 (CI 10.4–23), a value seen against the B.1.351 variant,²⁵ up to 91 (90% CI 83–98) at ≥21 days after vaccination.⁴³ The lowest effectiveness was seen by Abu-Raddad et al. against the B.1.351 variant.²⁵ In this study, data on vaccinations and SARS-CoV-2 Polymerase Chain Reaction (PCR) testing from February 1 to March 31, 2021, were extracted from national, federated COVID-19 databases in Qatar. To assess vaccine effectiveness against the B.1.1.7 and B.1.351 variants, a test-negative, case-control study was done where PCR positive cases (stratified into B.1.1.7 cases, B.1.351 cases, or severe or critical or fatal disease cases) were matched to controls by age, gender, nationality, and reason for PCR testing.²⁵ This study highlights the lower effectiveness of the first dose of the Pfizer/BioNTech vaccine against infection with this particular variant. The highest effectiveness was observed by Hunter and Brainard where effectiveness was assessed in a reanalysis of a retrospective study conducted by Chodick et al. in an Israeli cohort. Vaccine effectiveness was calculated for each day from day 13 to day 24.^43,72 The reanalysis revealed that the effectiveness of the Pfizer/BioNTech vaccine increased gradually, beginning 14 days after administration of the first dose, and eventually reached a peak of 91% effectiveness on day 21. As highlighted by Abu-Raddad et al., the effectiveness is significantly low against the B.1.351 variant.²⁵

Effectiveness of the first dose against severe infection ranged from 0 (CI 0–19), a value seen against the B.1.351 variant,²⁵ up to 85 (CI 71–92) at 15–28 days after vaccination.²⁶ The lowest effectiveness was reported by Abu-Raddad et al. against the B.1.351 variant. The highest effectiveness was observed by Amit et al., 2021, in a retrospective analysis on 9,109 health care workers (HCWs) eligible for vaccination. Amit et al. assessed vaccine effectiveness by comparing infection rates among vaccinated and unvaccinated individuals.²⁶ In a total of 170 SARS-CoV-2 infections documented in this population, 89 (52%) were unvaccinated, 78 (46%) were positive after the first dose, and 3 (2%) were positive after the second dose. Infection rate in the unvaccinated cohort was 7.4 per 10,000 person-days compared to 5.5 per 10,000 person-days 1–14 days after the first dose and 3.0 per 10,000 person-days 15–28 days after the first dose. Adjusted rate reductions of infection were 30% (CI 2–50) for days 1–14, increasing to 75% (72–84) for days 15–28 after the first dose. The study results confirmed a significant reduction in SARS-CoV-2 infection after the first vaccine dose. In addition, authors suggested that the reduction supports postponing the second vaccine dose in countries which would benefit from increased population coverage with a single vaccine dose.²⁶ The vast difference in effectiveness of the first dose against severe infection can be explained by the fact that the lowest value was seen specifically against the B.1.351 variant. As highlighted by Abu-Raddad et al., the effectiveness was significantly low against the B.1.351 variant.²⁵

Effectiveness of the first dose against infection requiring hospitalization ranged from 43 (CI 33–52)³⁰ up to 85 (CI 71–92)²⁶ at 15–28 days after vaccination. The lowest effectiveness was reported by Bernal et al. in a test negative case-control study where they examined the effectiveness of the Pfizer/BioNTech vaccine in individuals aged ≥70 old in England.³⁰ Cases vaccinated with a single dose of the vaccine had a 43% (CI 33–52%) lower risk of hospitalization and a 51% (CI 37–62%) lower risk of death. This data translates to an effectiveness of approximately 80% for a single dose of the vaccine in preventing hospitalization, with one dose of Pfizer/BioNTech vaccine being 85% effective at preventing COVID-19 – related death.³⁰ The authors concluded that a single-dose vaccination with Pfizer/BioNTech vaccine significantly reduced symptomatic SARS-CoV-2 infections and provided protection against severe illness. This protection was maintained for the follow-up duration of >6 weeks. Additionally, with the variant of concern (VOC 202012/01) being the predominant variant in the study period in the UK, the authors suggest this likely reflects effectiveness of the Pfizer/BioNTech vaccine against this variant.³⁰ The highest effectiveness was observed by Amit et al.²⁶ The effectiveness of the first vaccine dose against infections requiring hospitalization is very similar in both population-based studies.

Effectiveness of the first dose against fatal disease ranged from 0 (CI 0–19), a value seen against the B.1.351 variant,²⁵ up to 72 (CI 19–100)³⁴ at 14–20 days after vaccination The lowest effectiveness was reported by Abu-Raddad et al., 2021 against the B.1.351 variant, while the highest value was reported by Dagan et al . In the latter study, vaccinated individuals in the period of December 20, 2020, to February 1, 2021 were matched to unvaccinated controls by demographic and clinical characteristics.³⁴ With 596,618 individuals per study group, vaccine effectiveness was assessed by SARS-CoV-2 symptomatic infection, COVID-19-related hospitalization, severe disease, and death. During the study period, there was an increase in B.1.1.7 variant SARS-CoV-2 infections in Israel.⁷³ Therefore, this study suggests an average effectiveness of the vaccine over multiple strains. The observed plateau in infection incidence in later periods for vaccinated persons potentially suggests that the Pfizer/BioNTech vaccine is also effective for the B.1.1.7 variant.³⁴ This study suggests high effectiveness of the Pfizer/BioNTech vaccine for preventing symptomatic COVID-19 and for hospitalization, severe disease, and death. The vast difference in effectiveness of the first dose against fatal disease can be explained by the fact that the lowest value was seen specifically against the B.1.351 variant. In addition, Abu-Raddad et al. reported a combinedeffectiveness value against both fatal disease and severe or critical illness, hence possibly lowering the observed value.²⁵ This difference highlights the concept that different study protocols and endpoint variables make comparing the effects of vaccines more difficult.⁷¹ No studies in our search measured the vaccine efficacy of the first dose.

4.2.2. Moderna

4.2.2.1. Efficacy against infection, severe infection, hospitalization, and mortality

The efficacy of the first vaccine dose against infection was 95.2 (CI 91.2–97.4) at ≥14 days after vaccination.²⁸ Baden et al. conducted a phase 3 randomized, placebo-controlled, observer-blinded study, where individuals, enrolled from 99 US sites, were assigned in a 1:1 ratio to receive the Moderna vaccine or placebo (saline).²⁸ With the aim of determining vaccine efficacy in protection against symptomatic COVID-19 infection at least 14 days after the second dose in patients with no prior SARS-CoV-2 history, efficacy analysis was conducted on a population of 14,073 individuals in the placebo group, compared to fully vaccinated 14,134 individuals. In the total 196 infection cases included in the primary efficacy analysis, 11 occurred in the vaccinated cohort (3.3 per 1000 person-years; CI, 1.7–6.0), compared to 185 in the placebo cohort (56.5 per 1000 person-years; CI, 48.7–65.3). In addition, a total of 30 individuals had severe COVID-19 infection, all of whom belonged to the placebo cohort; this data suggests that the vaccine has significant efficacy in preventing severe COVID-19 disease.²⁸

4.2.2.2. Effectiveness against infection, severe infection, hospitalization, and mortality

The effectiveness of the first dose against infection was 51.7 (CI 37.3–63.0) at ≥7 days after vaccination against infection.⁵⁵ As discussed earlier, Pawlowski et al. noted that the lower effectiveness observed in their studies, as compared to the efficacy values seen in clinical trials, could be attributed to the fact that individuals receiving the vaccination are mostly at high risk of infection. Threfore, there may be an overrepresentation in the cohort leading to undervalued effectiveness. In addition, vaccinated individuals may engage in higher risk behaviors, such as social gatherings.⁵⁵ Importantly, the observed difference highlights the notion mentioned earlier that in efficacy studies that tend to utilize mostly RRRs, and overlook ARRs, a reporting bias might be seen, hence affecting vaccination efficacy interpretation.⁷¹

4.2.3. Oxford/AstraZeneca

4.2.3.1. Efficacy against infection, severe infection, hospitalization, and mortality

The efficacy of the first dose against infection ranged from 33.5 (CI −13.4–61.7) at ≥14 days after vaccination⁴⁹ up to 63.9 (46.0–75.9) at 22–90 days after vaccination.⁶⁵ Madhi et al. conducted a multicenter, double blinded, phase 1b-2 randomized control trial in South Africa and aimed to assess the efficacy and safety of the Oxford/AstraZeneca vaccine among people not infected with human immunodeficiency virus (HIV).⁴⁹ A total of 2,026 HIV-negative participants between the ages of 18 and 65 were randomized 1:1 to receive two doses of either the vaccine or the control or the placebo, 0.9% NaCl solution, 21–35 days apart. A total of 1467 seronegative participants (750 to vaccine and 717 to placebo) were included in the primary efficacy analysis. A total of 42 cases of COVID-19 were reported. Of these, there were 15 mild cases amongst the vaccinated group and 17 among the placebo group. There were also 4 moderate cases among the vaccinated group and 6 among the placebo group. Vaccine efficacy was determined to be 21.9% (CI −49.9 to 59.8) among seronegative participants and 10.6% (CI −66.4 to 52.2) among seropositive patients. Sequencing was available for 41 of the 42 participants and 39 (95.1%) of those were due to the B.1.351 variant and 2 (4.9%) were due to the B.1.1.1 and B.1.144 lineages. Vaccine efficacy against the B.1.351 variant was found to be 10.4% (CI −76.8 to 54.8). Importantly, these findings suggest that two doses of the Oxford/AstraZeneca vaccine have no efficacy in preventing mild-to-moderate COVID-19 disease due to the B.1.351 variant.⁴⁹ Voysey et al. conducted a multinational study comprised three single-blinded randomized controlled trials and one double-blinded randomized controlled trial.⁶⁵ The three single-blinded randomized controlled trials include a phase 1/2 UK trial (COV001), which recruited adults between 18 and 55 years old, a phase 2/3 UK trial (COV002), which recruited adults >18 years old with a focus on high-risk personnel and healthcare workers, and a phase 3 Brazil trial (COV003), which had the same recruiting criteria as COV002. The double-blinded randomized controlled trial (COV005) took place in South Africa and recruited adults between 18 and 65 years old. All 4 aforementioned trials were conducted between April 23, 2020 and December 6, 2020, and a total of 24,422 participants were recruited. With an assigning ratio of 1:1, each participant either received 2 doses of Oxford/AstraZeneca vaccine or 2 doses of saline placebo. The overall vaccine efficacy after 14 days of second-dose recipience was 66.7% (CI 57.4–74.0), compared to 76.0% (CI 59.3–85.9) after 22 days of first-dose recipience.⁶⁵ Voysey et al. noted that the discrepancy in the first- and second-dose vaccine efficacy could be attributed to several factors, such as the intensity of the COVID-19 pandemic in different countries as well as the length of the prime-boost interval between first- and second-dose vaccine recipience.⁶⁵ As shown in the study, longer prime-boost interval (≥ 12 weeks) indicated higher vaccine efficacy, 81.3% (CI 60.3–91.2%), whereas shorter prime-boost interval (< 6 weeks) indicated lower vaccine efficacy, 55.1% (CI 33.0–69.9%).⁶⁵ Efficacy of the first dose against infection requiring hospitalization was only reported in one study and was 100 (97.5% CI one sided with lower limit: 72.2) at ≥22 days after vaccination.⁶⁵

4.2.3.2. Effectiveness against infection, severe infection, hospitalization, and mortality

The effectiveness of the first dose against infection ranged from 73 (CI 27–90) at ≥35 days after vaccination³⁰ up to an observed 74% (HR 0.26 (CI 0.19–0.35)) reduction in the risk of infection at ≥28 days after vaccination.³⁸ In a retrospective cohort study by Glampson et al., unvaccinated and vaccinated individuals who had received at least one vaccine dose (57% Pfizer/BioNTech and 42% Oxford/AstraZeneca) were included; the data were obtained directly from active hubs or electronic health records and fed into a multivariable Cox regression model for analysis.³⁸ The infection rates were found to be 0.09% and 0.13% among those vaccinated in weeks 1 and 2 post-vaccination, before falling to 0.10% in week 3 and further declining with time. This translated to a decrease in infection rates 2 weeks after vaccination to rates equal to or lower than in the general population (0.19%). The authors also suggested that there could be a depreciation of the effects of single-dose vaccination due to a rise in infection rates in frail care home residents 7 weeks post vaccination.³⁸ The effectiveness of the first dose against infection requiring hospitalization was reported in one study as a 37% protection against infection requiring hospitalization.³⁰

4.2.4. Sputnik V

4.2.4.1. Efficacy against infection, severe infection, hospitalization, and mortality

The efficacy against infection was 91.6 (CI 85.6–95.2) at ≥21 days after vaccination and 100 (CI 94.4–100.0) against severe infection at ≥21 days after vaccination.⁴⁸ In a study by Logunov et al., the investigators conducted a double-blinded, phase 3 randomized control trial in Russia on participants aged ≥18 years old with the aim of determining the efficacy and safety of the Gam-COVID-Vac vaccine.⁴⁸ 21,977 participants were stratified into groups by age (18–30 years; 31–40 years; 41–50 years; 51–60 years; and >60 years) and were randomized 3:1 to receive either the vaccine or a placebo. With 14,964 participants assigned to the vaccine group and 4,902 to the placebo group, results indicated 78 cases of COVID −19 in participants who had received the 2nd dose, 16 belonging to the vaccine group and 62 to the placebo group. Vaccine efficacy was determined to be 91.6% (Cl 85.6–95.2) from day 21 after the 1st dose that is the day of receiving the 2nd dose. No cases of moderate or severe COVID−19 21 days after the 1st dose was noted in the vaccine group, and 20 cases were reported in the placebo group, translating to an efficacy against moderate or severe COVID −19 of 100% (94.4–100).⁴⁸ The study highlights a high efficacy of the Sputnik V vaccine against COVID-19 infection, with an even higher efficacy value against severe infection. No studies in our search measured the vaccine effectiveness.

4.2.5. Sinovac

4.2.5.1. Effectiveness against infection, severe infection, hospitalization, and mortality

The reported effectiveness against infection was 49.4 (CI 13.2–71.9) at ≥14 days after vaccination, a value suggested to be valid against the P.1 variant.⁴² In a retrospective, test-negative, matched case-control study by Hitchings et al., investigators sought to assess the effectiveness of the Sinovac vaccine in HCWs in Manaus, Brazil, which was widely affected by the epidemic Gamma (P.1) variant.⁴² Analysis of effectiveness was conducted after administration of one dose and two doses. The main outcome assessed was symptomatic SARS-CoV-2 infection. Of the one dose case-control pairs that were matched by age, neighborhood, and calendar time, analysis revealed that vaccination with at least one dose was associated with a 0.50-fold reduction, a vaccine effectiveness of 49.6 (CI 11.3–71.4) in the odds of developing symptomatic SARS-CoV-2 infection ≥14 days after first-dose administration. On the contrary, analysis of the two-dose case-control pairs that were also matched revealed a low effectiveness of 36.8 (CI 54.9–74.2) against symptomatic SARS-CoV-2 infection ≥14 days after second-dose administration. It was also found that vaccinated HCWs had a much greater likelihood of being infected than their unvaccinated counterparts, 0–13 days after first dose (OR 2.11, CI 1.36–3.27). The authors concluded that administration of at least one dose of Sinovac is effective against Gamma (P.1) variant transmission.⁴² However, the observed low effectiveness of the two-dose regimen questions the vaccine’s efficacy and suggests that more studies on the vaccine’s efficacy are needed. No studies in our search measured the efficacy of the first dose.

4.2.6. Studies reporting data on multiple vaccine types

4.2.6.2. Effectiveness against infection, severe infection, hospitalization, and mortality

The observed effectiveness of the first vaccine dose in studies with populations receiving the Moderna or the Pfizer/BioNTech vaccine ranged from 67.1 (CI 30.9–84.5) against infection at ≥15 days after vaccination²⁷ up to 80 (CI 59–90) at ≥14 days after vaccination.⁶³ Effectiveness values of the first dose against severe infection or infection requiring hospitalization, and fatal illness were reported in only one study and were 77 (CI 71–82) and 64.2 (CI 13.0–85.2), respectively.⁶⁴ In the study where <0.1% of the included participants had unknown data on the given vaccine, a 72% reduction in the risk of infection was observed after the first dose.⁶¹

4.3. Efficacy vs effectiveness of the COVID-19 vaccines in fully vaccinated individuals

4.3.1. Pfizer/BioNTech

4.3.1.1. Efficacy against infection, severe infection, hospitalization, and mortality

Polack et al. used an ongoing multinational, placebo-controlled, observer-blinded trial for individuals ≥16 years old.⁵⁶ The primary end points included confirmed infection after 7 days of second dose and efficacy in participants with and without prior infection. Vaccine efficacy was 95% (CI: 90.3–97.6%) in participants with no prior infections and 94.6%% (CI: 89.9–97.3%) including those with prior infection history. Further analyses showed that the vaccine efficacy was largely consistent over subgroups such as age, race, coexisting conditions when compared to the overall population. Incidence of SARS-CoV-2 cases when comparing both the vaccine and control groups showed divergence after 12 days after the first dose of the vaccine – where the vaccine group had a lower incidence – and improved efficacy for vaccine recipients 7 days after the second dose.⁵⁶ The two studies highlight the significantly high efficacy of the full-dose regimen of the Pfizer/BioNTech vaccine in preventing infection.

4.3.1.2. Effectiveness against infection, severe infection, hospitalization, and mortality

The observed effectiveness of the second dose against infection ranged from 46 (CI 28–59) within 0–7 days after second dose⁵⁴ up to 99.5 (CI 97.0–99.9) at ≥35 days after vaccination.⁶⁶ In their study, Moustsen-Helms et al. aimed to assess the effectiveness of two doses of Pfizer/BioNTech vaccine in long-term care facility residents (LTCF) and healthcare workers in Denmark.⁵⁴ The study is a retrospective population-based cohort study where all long-term facility residents and all HCW living in Denmark either at the start of vaccination or immigrated before the end of the study were included. In LTCF residents, findings showed the vaccine effectiveness was 52% 0–7 days after the second dose and this was increased to 64% >7 days after the second dose. In HCW population, vaccine effectiveness was 46% 0–7 days after second dose which significantly increased to 90% >7 days after the second dose. The authors concluded that the analysis confirms that two vaccine doses offer significant protection against COVID-19 in two critical groups in Denmark, the elderly population and the healthcare workers, who are both at a higher risk of infection.⁵⁴ Yelin et al. conducted a prospective, non-randomized, uncontrolled study in Maccabi Healthcare Services (MHS), Israel between December 19, 2020 and February 25, 2021.⁶⁶ All data used in this study were extracted from MHS electronic health record (EHR) and a total of 1,723,509 participants were included. Since the data were extracted from the EHR, all participants had their RT-qPCR test results, city of residence, age, sex, and comorbidities tagged. The outcome of this study was defined as a participant having a positive RT-qPCR test on a specific calendar day and was termed observation. Using all the observations in the study, vaccine effectiveness was found to increase starting at day 12 after first-dose recipience and ultimately plateau at 95% effectiveness after 35 days of first-dose recipience.⁶⁶Regarding age, the odds ratio of vaccine effectiveness, 0.74 (CI, 0.52–1.06), in the elder participants (81–90 years old), was mildly lower compared to the younger participants (17–80 years old). As for comorbidities, vaccine effectiveness decreased significantly for type 2 diabetics, immunosuppressed participants, and Chronic Obstructive Pulmonary Disease (COPD) patients, with odds ratios of vaccine effectiveness being 0.73 (CI 0.59–0.91), 0.67 (CI 0.53–0.83), and 0.55 (CI 0.38–0.80), respectively. Apart from vaccine effectiveness in infection prevention, Pfizer/BioNTech vaccine also showed great symptomatic infection prevention, with 99.5% vaccine effectiveness (CI, 097.0–99.9%) in symptomatic infection prevention after 35 days of first-dose recipience.⁶⁶ Overall, Pfizer/BioNTech vaccine appeared effective in both infection prevention and symptomatic infection prevention for the B.1.1.7 variant, which was the main variant circulating in Israel during the study period.⁶⁶ The observed results from population-based studies verify the high efficacy seen in the controlled studies and support the conclusion two doses of the Pfizer/BioNTech vaccine are significantly effective in preventing infection.

Effectiveness against severe infection ranged from 88.8 (CI 75.5–95.7)⁵⁵ up to 100.0 (CI 81.7–100.0) at ≥14 days after vaccination, a value reported for the B.1.1.7 variant, and also up to 100.0 (CI 73.7–100.0) at ≥14 days after vaccination, a value reported for the B.1.351 variant.²⁵ The lowest effectiveness was reported by Pawlowski et al. in a retrospective study that included 31, 069 vaccinated subjects who were propensity matched to an unvaccinated group of 31, 069 patients.⁵⁵ The study assessed vaccine effectiveness via Kaplan Meier analysis for incidence of SARS-CoV-2. Pawlowski et al. suggested that the lower effectiveness, as compared to the efficacy values seen in clinical trials, could be attributed to the fact that individuals receiving the vaccination are mostly at a high risk of infection so there may be an overrepresentation in the cohort leading to undervalued effectiveness and also to the fact that vaccinated individuals may engage in higher risk behaviors, such as social gatherings.⁵⁵ The highest values, against the B.1.1.7 and the B.1.351 variants were reported by Abu-Raddad et al. and strongly support the conclusion that the two-dose regimen is highly efficacious against severe infection due to both variants of concern.²⁵

Effectiveness of the second dose against infections requiring hospitalization ranged from 75.6 (52.8–87.6) at 35–41 days after the first dose⁵⁰ up to 100 (CI 51.4–100).⁵⁵ The lowest effectiveness was reported by Mason et al. in a study conducted on the older population of England. The study was a matched case-control study where vaccinated individuals aged 80–81 were matched to controls (those younger who became eligible for vaccination at a later time) aged 76–77 and vaccinated individuals aged 82–83 to controls aged 78–79.⁵⁰ Also, vaccinated individuals aged 80–81 were matched to controls aged 72–73, and vaccinated individuals aged 82–83 to controls aged 74–75. Three outcomes were examined: SARS CoV-2 infection, COVID-19 related emergency hospital attendance, and COVID-19 related hospitalizations. The findings demonstrated BNT1262b vaccine effectiveness against the B.1.1.7 variant across all three outcomes over the follow-up period between vaccinated and control groups (unvaccinated). The effectiveness was found to be 50.1% for hospital admissions from days 21–27. By day 35–41, the estimated effectiveness increased to 75.6%. The authors concluded that the Pfizer/BioNTech vaccine is effective in reducing COVID-19 related hospitalizations and that the vaccination of older adults in England on a national aspect reduced the burden of the virus.⁵⁰ The highest value was reported by Pawlowski et al.⁵⁵ Overall, the observed results from the population-based studies confirm the high efficacy in preventing hospitalizations seen in studies with controlled settings. Furthermore, the results from the study by Mason et al. suggest that the vaccine is effective against the B.1.1.7 variant.

Effectiveness of the second dose against fatal infections ranged from 74 (90% CI 58–81)⁶⁰ up to 100.0 (CI 81.7–100.0) at ≥14 days after vaccination, a value reported for the B.1.1.7 variant, and also up to 100.0 (CI 73.7–100.0) at ≥14 days after vaccination, a value reported for the B.1.351 variant.²⁵ Effectiveness of the second dose against fatal infections ranged from 74 (90% CI 58–81)⁶⁰ up to 100.0 (CI 81.7–100.0) at ≥14 days after vaccination, a value reported for the B.1.1.7 variant, and up to 100.0 (CI 73.7–100.0) at ≥14 days after vaccination, a value reported for the B.1.351 variant.²⁵ The lowest value was reported by Salazar et al. in a study where investigators quantified using regression models, the effect of administering the Pfizer/BioNTech vaccine on to residents of long-term care facilities in terms of COVID-19 related deaths and infections.⁶⁰ The study estimated that vaccination of 70% of residents prevented 74% of deaths. In addition, results suggested that high vaccination coverage was able to prevent 3 out of 4 deaths in subsequent weeks. The highest values against the B.1.1.7 variant and the B.1.351 variant were reported by Abu-Raddad et al. and strongly support the conclusion that the two-dose regimen is highly efficacious against fatal infection due to both variants of concern.²⁵

4.3.2. Moderna

4.3.2.1. Efficacy against infection, severe infection, hospitalization, and mortality

The efficacy of the second dose against infection was 94.1 (CI 89.3–96.8) at ≥14 days after vaccination, and 100 (CI could not be estimated to 1.0) against severe infection.²⁸

4.3.2.2. Effectiveness against infection, severe infection, hospitalization, and mortality

The effectiveness of the second dose against infection ranged from 85.6 (CI 69.1–93.9) at ≥2 weeks after vaccination²⁷ up to 93.3 (CI 85.7–97.4) at ≥7 days after vaccination.⁵⁵ In a case-control study, Andrejko et al. assessed effectiveness of the Pfizer/BioNTech and Moderna vaccines in California.²⁷ Among a total of 1,023 participants enrolled from the California Department of Public Health, there were 525 cases and 498 controls matched by age, gender, and geographic region. Results showed that among the identified cases, 71 (13.5%) received Pfizer/BioNTech or Moderna vaccines, and 20 (3.8%) were fully vaccinated following one vaccine protocol. Among the controls, there were 185 (37.1%) subjects who received either Pfizer/BioNTech or Moderna, and 86 (16.3%) who were fully vaccinated following one vaccine protocol. Pfizer/BioNTech vaccine effectiveness was 86.8% (CI 68.6–94.7), and Moderna effectiveness was 85.6% (CI 69.1–93.9), both 2 weeks after the second dose. In participants fully vaccinated with either vaccine, vaccine effectiveness against symptomatic infection was 91.3% (CI 79.7–96.3) and was 68.3% (CI 28.5–86.0%) against asymptomatic infection. The authors concluded that currently available mRNA-based vaccines induce significant protection against both symptomatic and asymptomatic SARS-CoV-2 infection and against severe infection, as evident by having no hospitalizations observed in the fully vaccinated cases cohort.²⁷

Effectiveness of the second dose against severe infection and infection requiring hospitalization were reported in one study and were 86.0 (CI 71.6–93.9) at ≥7 days after vaccination and 100 (43.3–100) at ≥7 days after vaccination, respectively.⁵⁵ The observed effectiveness values, along with the efficacy values, strongly support the conclusion that the two-dose regimen of the Moderna vaccine is highly efficacious in preventing COVID-19 infections, including those severe and requiring hospitalization.

4.3.3. AstraZeneca

4.3.3.1. Efficacy against infection, severe infection, hospitalization, and mortality

For the second dose of the Oxford/AstraZeneca vaccine, the efficacy against infection ranged from 10.4 (CI −76.8–54.8) at ≥14 days after vaccination, a value seen against the B.1.351 variant⁴⁹ up to 80.7 (CI 69.2–87.9) at ≥14 days after vaccination.⁴⁶ The results by Madhi et al. suggest that even the two-dose regimen of the Oxford/AstraZeneca vaccine does not confer significant efficacy against infection with the B.1.351 variant, as seen with the first dose as well.⁴⁹ The study by Emary et al. is an ongoing, multicenter, single-blinded trial in phase 2/3, conducted in the UK where participants aged 18 or above are randomized 1:1 to receive either the Oxford/AstraZeneca vaccine or a control meningococcal vaccine (Men ACWY).⁴⁶ The aim of the study was to determine the efficacy of the Oxford/AstraZeneca vaccine against the B.1.1.7 variant as compared to the other variants. The results indicate that 499 participants had developed COVID-19 infection. From the participants, 1524 NAAT positive swabs were collected, and 323 swabs collected from 256 participants were also sequenced. Thirty-four (28.3%) of the primary symptomatic cases that had sequencing available belonged to the B.1.1.7 variant and 86 (71.7%) belonged to non-B.1.1.7 variants. Among the asymptomatic or those with unknown symptoms, 14 (32%) were due to B.1.1.7 variant and 30 (68%) were due to non-B.1.1.7 variants. Vaccine efficacy against primary symptomatic B.1.1.7 COVID −19 infection was 74.6%, (CI 41.6–88.9) and against symptomatic non-B.1.1.7 COVID −19 infection was 84.1%, (CI 70.7–91.4). For asymptomatic or unknown symptoms disease due to the B.1.1.7 infections, vaccine efficacy was 26.5% (CI −112.0–74.5) and 75.4%, (CI 39.9–89.9) for non-B.1.1.7 infections. Overall vaccine efficacy for all B.1.1.7 cases was 66.5% (CI 37.1–82.1) and 80.7% (CI 69.2–87.9) for all other variants.⁴⁶ These findings support the conclusion that (AZD1222) vaccine is potentially efficacious against the B.1.1.7 variant. No studies in our search measured the effectiveness of the second dose.

4.3.4. Janssen (Johnson & Johnson) vaccine

4.3.4.1. Efficacy against infection, severe infection, hospitalization, and mortality

The efficacy of one dose was 66.9 (CI 59.1–73.4) at ≥14 days after vaccination against infection, 83.5 (54.2–96.9) at ≥28 days after vaccination against severe infection, 100 (CI 74.3–100.0) at ≥28 days after vaccination against infection requiring hospitalization.⁵⁹ In a study by Sadoff et al., the investigators assessed the efficacy of the J&J vaccine by performing a randomized, double blind, placebo controlled, phase 3 trial.⁵⁹ This was done by injecting participants with either the vaccine or a placebo and comparing the number of moderate-to-severe cases in each group 14 days and 28 days after injection, as well as the safety of the injections. The results showed that the overall vaccine efficacy was 66.9 (CI 59.1–73.4) at ≥14 days after vaccination, specifically 63.7% in the 18–59 years old aged group and 76.3% in the group over 60 years of age. There were 5 COVID-19 related deaths in the placebo group and none in the vaccine group. The authors concluded that a single dose of the J&J vaccine protected against symptomatic and asymptomatic infections and was efficacious in preventing severe or critical disease requiring hospitalizations or deaths.⁵⁹ Furthermore, the study’s cohorts in the USA, South Africa, and Brazil had the following dominant variants Wuhan-Hu-1, B.1.351, and P.2 and Wuhan-Hu-1, respectively. The results of the study support the conclusion that the J&J vaccine is significantly efficacious against these variants after 1 dose. No studies in our search measured the vaccine effectiveness.

4.3.5. Novavax

4.3.5.1. Efficacy against infection, severe infection, hospitalization, and mortality

In a study by Shinde et al., the safety and efficacy of the Novavax vaccine against the B.1.351 variant was assessed by conducting a randomized, placebo-controlled, observer-blinded trial consisting of participants randomly allocated to receive either two doses (21 days apart) of the NVX-CoV2373 vaccine or saline placebo.⁶¹ HIV-negative patients and those with controlled HIV were included. Efficacy analysis results from 2,684 participants who were seronegative at baseline (with 94% of them being HIV-negative and 6% being HIV-stable) showed 15 and 29 symptomatic COVID-19 positive cases (post day 28) among NVX-CoV2373 vaccine-given participants and placebo-given participants respectively, translating to a vaccine efficacy of 49.4%. Among baseline seronegative participants without HIV, there were 11 and 27 symptomatic COVID-19 positive cases in participants given NVX-CoV2373 and in placebo-given participants respectively, translating to a vaccine efficacy of 60.1%. Of the 44 COVID-19 positive cases, 41 had genome sequencing data available and, from these, 38 were identified as the B.1.351 variant. Vaccine efficacy was then calculated to be 51.0% against this variant in solely HIV-negative participants and 43.0% in both HIV-negative and HIV-stable participants.⁶¹ Based on these data, the NVX-CoV2373 vaccine is significantly efficacious against COVID-19 infection, with document efficacy against the B.1.351 strain of SARS-COV-2. No studies in our search measured the vaccine effectiveness.

4.3.6. Sinovac

4.3.6.1. Effectiveness against infection, severe infection, hospitalization, and mortality

The effectiveness of the second dose against infection ranged from 37.9 (CI 46.4–73.6) at ≥14 days after vaccination⁴² up to 73.8 (CI 57.0–84.8) at ≥5 weeks after vaccination.³⁷ In a study by Faria et al., between February 23, 2020 and March 28, 2021, data were recorded on the weekly numbers of symptomatic COVID-19 cases confirmed by RT-PCR in HCWs in Hospital das Clinicas (HC).³⁷ These data were compared to the weekly number of COVID-19 cases in São Paulo.⁷⁴ After HCW vaccination, the number of COVID-19 cases in São Paulo increased, while the number of cases in the HCWs did not. Estimated vaccine effectiveness was 50.7 (CI 33.3–62.5); 51.8 (CI 30.0–66.0); 68.4 (CI 51.0–80.8); and 73.8 (CI 57.0–84.8) for the weeks 2, 3, 4, and 5 after the second dose of Sinovac, respectively. In 2021, there were 9 HCW hospitalizations in HC due to COVID-19 (6 unvaccinated, 1 had one vaccine dose, and 2 had 2 doses), one of which died and was not vaccinated. Among randomly examined 142 HCWs’ respiratory samples, 67 (47%) variants of concern (VOC) were detected: 57 (P1), 5 (B.1.1.7), and 5 were other VOC not identified by the researchers’ methods. The study outlines a reduction of confirmed symptomatic cases of COVID-19 compared to the expected numbers considering the epidemiological situation in the community.³⁷ The high prevalence of P1 is likely due to the epidemiological circumstances of the region. P1 became the predominant strain in Brazil,³⁷ and São Paulo reported 64% of P1 and 7% of B.1.1.7 in samples collected from February 16 to March 6, 2021.^75(p15) The observed discrepancy in both studies on the effectiveness of the second vaccine dose can be explained by the time at which effectiveness was assessed, since de Faria et al. obtained the high effectiveness of 73.8 at 5 weeks after the second dose, with only 50.7 at the 2-week time period.

4.3.7. Studies reporting data on multiple vaccine types

4.3.7.1. Effectiveness against infection, severe infection, hospitalization, and mortality

The observed effectiveness of the second vaccine dose in studies with populations receiving the Moderna or the Pfizer/BioNTech vaccine ranged from 64 (CI 14–84) against infection at ≥7 days after vaccination⁵² up to 91.3 (CI 79.7–96.3) at ≥15 days after vaccination.²⁷ Effectiveness values of the second dose against severe infection, and fatal illness were reported in only one study and were 96 (CI 95–99), and 98.7 (CI 91.0–99.8), respectively.⁶⁴ Effectiveness against infection requiring hospitalization ranged from 96 (CI 95–99)⁶⁴ up to 100 (undefined CI).²⁷ In the study where <0.1% of the included participants had unknown data on the given vaccine, an 80% reduction was seen after the second dose.⁶²

In the study on the population receiving the Moderna, Pfizer/BioNTech, or the J&J vaccine, the second vaccine dose resulted in an 82% reduction in the number of new infections for patients older than 65 years and a 70% reduction in patients younger than 65 years.⁵⁸ Also, an 80% reduction in the number of hospitalizations for those older than 65 years and a 60% reduction lower for those younger than 65 years were seen.⁵⁸ In addition, a reduction of 92% was seen in the number of fatalities in patients older than 65 years and a reduction of 87% in those younger than 65 years.⁵⁸

4.4. Partial vs full vaccination

Based on the highest reported effectiveness values in the included studies, the second dose of Pfizer was more effective in reducing the rates of infection, severity, hospitalization and mortality compared with the first dose. The reduction in the rate of infection in a Pfizer fully vaccinated population reached 99.5% while 100% protection against severity, hospitalization and mortality was reported after the second dose. Similarly, the second dose of Moderna, increased the infection reduction rate compared with the first dose. Moderna full vaccination achieved 80–100% protection against severity and hospitalization. While none of our included studies reported any effectiveness data following the second dose of AstraZeneca, the first dose achieved up to 74% and 37% reduction in the rate of infection and hospitalization, respectively. No data were available to compare the effectiveness of the first and second doses of Sputnik and Sinovac vaccines. Full vaccination with J&J vaccine requires a single dose, which provides relatively lower protection rates against infection and severe infections compared with the results obtained from the Pfizer fully vaccinated populations. However, a single dose of J&J vaccine achieved a 100% reduction in the rate of hospitalization.

In order to compare the effectiveness of full vaccination in reducing infection, severity, hospitalization and mortality, Pfizer vaccine is taken as an example due to data availability. While 100% protection was reported against severity, hospitalization, and mortality following Pfizer full vaccination, including the B1.1.7 and B.1.351 variants, lower level of protection was reported against infection in general especially against the B.1.351 (75%).

4.5. Other methods of assessing the effectiveness of the vaccines

Several studies reported vaccine effectiveness by documenting the number of positive PCR tests or the NVL. In a study by McEllistrem et al., the viral loads of partially vaccinated nursing home residents with asymptomatic infection within 21 days of their first dose were retrospectively compared to those of their unvaccinated peers within the same time period.⁵¹ The mean log10 viral loads, representing the amount of measurable virus in nasopharyngeal samples, were found to be significantly lower (p = .004, non-overlapping ranges) in vaccinated residents (7.1 (CI 5.4–8.8)) compared to unvaccinated residents (9.5 (9.3–9.8)). In this study, the significantly lower NVL measured in vaccinated residents as compared to unvaccinated residents (9.5 (9.3–9.8)) supports the authors’ conclusion that nationwide single-dose strategies are viable public health approaches.⁵¹

At Hull University Teaching Hospitals, Lillie et al. analyzed SARS-CoV-2 Nucleic Acid Amplification Tests (NAAT) results conducted on all symptomatic staff, along with routine asymptomatic clinical staff testing with Lateral Flow Device (LFD), beginning January 4^th, 2021.⁴⁷ By this date, 827 (8.3%) subjects received their first vaccine dose, increasing to 8243 (82.5%) by the end of the week of February 22, 2021. Results showed that the number of positive cases decreased from 120 in the week of January 4 to 10 in the week of February 22. In addition, significant negative correlations between PCR positive cases and cumulative vaccination (Pearson's R = −0.9061, p = .0019), and between vaccine coverage and symptomatic PCR testing rates (Pearson's R = −0.8972, p = .0025) were identified. Positive tested staff members who self-isolated decreased by 72% from the week of January 11 to that of February 23. In this study, the significant negative correlations between PCR positive cases and cumulative vaccination, and between vaccine coverage and symptomatic PCR testing rates outline a significant efficacy of the BNT162b2 vaccine against infection, with a single dose associated with a significant decrease in positive PCR tests in both symptomatic and asymptomatic HCWs.⁴⁷

In a study by Jones et al., an equal number of vaccinated and unvaccinated asymptomatic HCWs were PCR-tested at the Cambridge University Hospitals NHS Foundation Trust.⁴⁴ PCR tests were done on an HCW population of ~9000. Vaccinated HCWs were further stratified into <12 days or ≥12 days post-vaccination, since this time point, in the phase III clinical trial, was when protection against symptomatic infection appeared.⁵⁶ In comparison to 13/3535 positive tests in HCWs <12 days post-vaccination and 4/1989 positive tests in HCWs ≥12 days post-vaccination, 26/3252 tests from unvaccinated HCWs were positive, implying a fourfold lower incidence of asymptomatic SARS-CoV-2 infection in HCWs ≥12 days after vaccination compared to unvaccinated HCWs, with a relatively intermediate impact in HCWs <12 days after vaccination. When analyses were performed on both symptomatic and asymptomatic HCWs testing positive, a significant reduction in infections was also observed: 56/3370 positive tests in the unvaccinated cohort compared to 8/2018 tests ≥12 days post-vaccination, implying a 4.2-fold reduction. It is important to note that, in all groups, the incidence of previous SARS CoV-2 infection was comparable, indicating that study results were not impacted by prior seropositivity. In this study, the significantly lower incidence of SARS-CoV-2 infection in HCWs ≥12 days after vaccination compared to unvaccinated HCWs suggests that a single dose of BNT162b2 provides short-term immunity against asymptomatic SARS-CoV-2 infection.⁴⁴

In an observational study by Daniel et al., data was reported from a vaccination campaign launched by the University of Texas Southwestern Medical Center (UTSW) that aimed to vaccinate 23,234 UTSW employees eligible to receive the Pfizer/BioNTech and Moderna vaccines.³⁵ Within the first 31 days, 59% of the eligible employees received the first dose of one vaccine, and 30% received both doses. Until January 28, 2021, 1.5% of eligible UTSW employees (350/23,234) reported a new COVID-19 case. Upon stratifying new infections by vaccination status, differences were found in the percentage of infected subjects. Out of 8969 unvaccinated employees, there were 234 cases (2.61%, CI 2.29–2.96%), compared to 112/6144 (1.82%, 95% CI 1.50–2.19%) in those partially vaccinated, and 4/8121 (0.05%, 95% CI 0.01–0.13%) in fully vaccinated individuals (p > .01 for all results). In this study, the decreased incidence of infections seen in partially and fully vaccinated individuals as compared to the unvaccinated subjects supports the conclusion that mRNA vaccines provide a significant protection against infection.³⁵

In a matched pairs analysis of electronic health records of two groups of nursing homes, one where residents were partially vaccinated with an mRNA vaccine 12–16 days earlier than residents in the second group, Mor et al. revealed that the earlier vaccinated group (12,157 residents over 136 facilities) had a predicted 2.5 fewer infections per 100 at-risk residents per week (CI 1.2–4.0) than the later vaccinated group (13,221 residents over 144 facilities).⁵³ Similarly, the reduction in the 5 week-cumulative infection rates was an estimated 5.2 cases per 100 at-risk residents (CI 3.2–7.3) as a result of vaccination. A 1.1–3.8 fewer hospitalizations and/or deaths per 100 infected residents per day was also observed in the earlier vaccinated group. In this study, lower infections, hospitalizations, and deaths were seen in the earlier vaccinated group as compared to the later vaccinated group, suggesting that the mRNA vaccines protect against new COVID-19 infections and reduce morbidity and mortality.⁵³

4.6. Summary of the efficacy/effectiveness of the COVID-19 vaccines against the new variants of SARS-CoV-2

4.6.1. B.1.1.7

The highest reported effectiveness for the first dose of any vaccine against infection with B.1.1.7 variant in the included studies was 70 (CI 55–85) at ≥21 days after vaccination reported by Hall et al. for the Pfizer/BioNTech vaccine.⁴¹ Hall et al. conducted a prospective cohort study on staff aged ≥18 years old working in hospitals in the UK. Participants were split into either the positive cohort that includes positive antibodies or a history of infection, or the negative cohort that includes negative antibodies and no history of infection. Vaccine effectiveness against COVID-19 infection 21 days after the first dose was 70 (CI 55–85) and increased to 85 (CI 74–96) after the second dose of the vaccine. These findings suggest that the vaccine is able to prevent both symptomatic and asymptomatic infections and is effective against the B1.1.7 variant that was the most dominant at the time of the study.⁴¹ The highest effectiveness for the first dose of any vaccine against severe infection with B.1.1.7 variant was 62 (CI 39–80), reported by Dagan et al. for the Pfizer/BioNTech vaccine.³⁴ The highest effectiveness for the first dose of any vaccine against infection with B.1.1.7 variant requiring hospitalization was 74 (CI 56–86), reported for the Pfizer/BioNTech vaccine.³⁴ The highest effectiveness for the first dose of any vaccine against fatal infection with B.1.1.7 variant was 72 (CI 19–100) at 14–20 days after vaccination, reported for the Pfizer/BioNTech vaccine.³⁴ Based on this data, we conclude that the first dose of the Pfizer/BioNTech vaccine is the most effective first dose against the B.1.1.7 variant. In all the studies that we have analyzed, the highest effectiveness for the second dose of any vaccine against infection with B.1.1.7 variant was 94 (CI 87–98) for symptomatic infections at >7 days after vaccination reported for the Pfizer/BioNTech vaccine.³⁴ The highest effectiveness for the second dose of any vaccine against severe or fatal infection with B.1.1.7 variant was 100 (CI 81.7–100) at 14 days after vaccination, reported for the Pfizer/BioNTech vaccine.²⁵ The highest effectiveness for the second dose of any vaccine against infection with B.1.1.7 variant requiring hospitalization was 100 (CI 51.4–100) reported for the Pfizer/BioNTech vaccine.⁵⁵ Based on this data, we conclude that the full-dose regimen of the Pfizer/BioNTech vaccine is the most effective against infections with the B.1.1.7 variant.

4.6.2. B.1.351

In all the studies that we have analyzed, the highest effectiveness for the first dose of any vaccine against infection with the B.1.351 variant was 16.9 (CI 10.4–23), reported for the Pfizer/BioNTech vaccine.²⁵ Furthermore, the first dose of the Pfizer/BioNTech vaccine was not effective against severe or fatal infections with the B.1.351 variant (0 (CI 0–19)).²⁵ Based on these data, we conclude that with the current vaccines, none are sufficient, in the first dosage, to protect against infections with the B.1.351 variant. In all the studies that we have analyzed, the highest effectiveness for the second dose of any vaccine against infection with the B.1.351 variant was 75.0 (CI 70.5–78.9) at ≥14 days after vaccination, reported for the Pfizer/BioNTech vaccine.²⁵ The highest effectiveness for the second dose of any vaccine against severe or fatal infection with the B.1.351 variant was 100 (CI 73.7–100) at 14 days after vaccination, reported for the Pfizer/BioNTech vaccine.²⁵ Based on these data, we conclude that the full-dose regimen of the Pfizer/BioNTech vaccine is the most effective against infections with the B.1.351 variant.

4.6.3. P.1

In all the included studies, the only effectiveness for the first and second doses of any vaccine against infection with the P.1 variant was 49.4 (CI 13.2–71.9) at ≥14 days after vaccination and 37.9 (CI 46.4–73.6) at ≥14 days after vaccination, respectively.⁴² These results were reported for the Sinovac vaccine and suggest that Sinovac may be potentially effective against the P.1 variant.

4.6.4. B.1.617.2

Our original systematic search results did not retrieve any studies assessing the effectiveness of the vaccines against the B.1.617.2 as the variant has newly emerged. However, we discuss here some of the recent publications addressing this issue. A study in the USA revealed that the prevalence of the B.1.617.2 (Delta) variant increased rapidly over the month of July 2021 with decreasing prevalence of other variants in the meantime. For instance, prevalence of variant B.1.1.7 (Alpha) in Minnesota decreased dramatically from 75% to 15% in July while B.1.617.2 increased from 5% to 72%.⁷⁶ Monitoring outcomes over this period thus provide an insight over effectiveness of vaccines against this variant. In this study, the effectiveness of Moderna decreased from 86% (95% CI: 81–90.6%) during the January – July period to 76% (95% CI: 58–87%) in July, while the effectiveness of Pfizer decreased from 76% (95% CI: 69–81%) in the period of January – July to 42% (95% CI: 13–62%).⁷⁶ Considering the significant change in prevalence of variant B.1.617.2 and the large decrease in effectiveness, this points to a decreased effectiveness of both vaccines on the B.1.617.2 variant, with a more pronounced decrease seen with the Pfizer vaccine. A similar comparison by the same study of other states found similar changes in effectiveness coinciding with the month of July’s increase in prevalence of variant B.1.617.2. Another study by Pouwels et al. showed decreased effectiveness during the increased prevalence of variant B.1.617.2 period after first-dose vaccination of AstraZeneca (VE 46% with 95% CI 35–55%) but no reduction in effectiveness after the second dose (67%, 62–71% vs 79%, 56–90% in the Alpha-dominant period, heterogeneity p = .23).⁷⁷ The Pfizer vaccine showed no decreased effectiveness after either the first or second dose. The study by Bernal et al. showed no decrease in vaccine effectiveness against the B.1.617.2 variant compared to the B.1.1.7 variant when two doses of Pfizer or AstraZeneca were administered, compared to a 18% reduction when only the first dose was administered, indicating the necessity of two doses especially for effectiveness against the B.1.617.2 variant.⁷⁸

5. Limitations of the study

Some of the analyzed studies highlighted flaws in their methods, such as the lack of sufficient ethnic varieties in the population sampled to determine a vaccine’s efficacy. Furthermore, studies assessed the vaccines’ efficacy values at different follow-up times, hence affecting the efficacy value seen, and making it more difficult to compare data to studies with much shorter or longer follow-up times. Additionally, in several studies, the vaccinated population was not stratified into those who received the first dose and the second dose. In multiple studies, vaccine effects were assessed after a relatively long follow-up time (for instance, at 35–41 days after the first dose in the study by Mason et al.,⁵⁰ hence first dose effects may have been assessed in patients who already received the second dose. Such methodology may have exaggerated the effects of vaccines in those particular populations.

6. Conclusion

Despite the speed with which multiple vaccines candidates were developed and the consistently emerging approvals in multiple countries worldwide, our results revealed that COVID-19 vaccines successfully reduced the rates of infections, severity, hospitalization and mortality among the different populations since the vaccines rollout started. The Pfizer/BioNTech vaccine was the most extensively studied among the COVID-19 vaccines with >90% effectiveness against infection, severe infection, infection requiring hospitalization and mortality after the second dose. The effectiveness of the Moderna vaccine after the second dose was >80% against infection, severe infection and infection requiring hospitalization. While none of the included studies reported the effectiveness of the AstraZeneca vaccine after the second dose, it was 80.7% efficacious against infection after the second dose and 74% effective against infection after the first dose. A single dose of the J&J vaccine was >60% effective against infection, severe infection and infection requiring hospitalization. While no effectiveness values were reported for the Sputnik, Novavax, Sinovac vaccines after the second dose, the efficacies of the later 2 were 60.1% and 73.8%, respectively, against infection after the second dose. It is important to note that the relatively low effectiveness values of some vaccines that were obtained in some studies could be attributed to the dominance of certain viral variants within certain populations. The full-dose regimen of the Pfizer/BioNTech vaccine is the most effective against infections with the B.1.1.7 and B.1.351 variants. Despite of the high effectiveness of the newly developed COVID-19 vaccines in reducing the rates of infections, hospitalization/severity and mortality, more efforts are required to test the efficacy/effectiveness of these vaccines against the other newly emerging variants.

Data sharing

The data that supports the findings of this study are available in the supplementary material of this article.

Acknowledgments

We would like to thank Mr. Sa’ad Laws for his help during the early phases of this project including developing the search strategy and importing papers. We would like also to thank Weill Cornell Medicine-Qatar for the continuous support and to specifically thank Weill Cornell Medicine - Qatar Distributed eLibrary for funding the publication of this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2022.2027160.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.