Assessment of COVID-19 vaccine effectiveness in a nation with a low vaccination coverage: insights from real-world data and propensity score matched analyses

Abstract

We conducted a comprehensive evaluation of COVID-19 vaccine effectiveness in reducing mortality rates in a country with a low vaccination coverage. A retrospective cohort study analysed data from 1,048,574 adult (≥ 18) patients spanning from March 2020 to April 2022, using data from the national digital medical record repository. Univariate analysis and logistic regression calculated odds ratios and their significance. Propensity score matching was utilised to strengthen the statistical results. Among the 1,048,574 patients diagnosed with COVID-19, 73% (n = 780,718) were unvaccinated, and 27% (n = 267,856) were reported as fully vaccinated. Unadjusted statistical analyses revealed a significant reduction in mortality rates among the vaccinated cohort (1,608 deaths; 0.6%) compared to the unvaccinated group (40,985 deaths; 5.2%) [OR 0.1090 (95%CI 0.1037 to 0.1146), p < 0.0001]. This outcome was consistent across all subgroups, including patient sex, age group, in-hospital setting, vaccine type, SARS-CoV-2 variant, and high-risk patient groups (i.e. with a solid malignancy, cardiovascular disease, chronic pulmonary disease or diabetes mellitus). Logistic regression revealed that the highest fatal risk was in non-vaccinated males aged >63. Propensity score matching substantiated the observed reduction in mortality rates across the entire vaccinated cohort and within all patient subgroups. Individuals infected with SARS-CoV-2 face an elevated risk of premature mortality. Vaccination, especially when utilising mRNA-based platforms, significantly mitigates this risk, particularly among high-risk populations.

Keywords:

• COVID-19
• Vaccination
• High-risk population
• Mortality
• Real-world data

Introduction

Bulgaria stands as one of Europe’s lowest-vaccinated nations and is also among the front-runners in terms of chronic disease prevalence and mortality, including cardiovascular (CVD), malignancies, chronic pulmonary diseases (CPD) and poorly controlled diabetes mellitus (DM), even prior to the onset of the COVID-19 pandemic [1–3]. The country’s average life expectancy has been estimated as 71 years, in contrast to the average in the European Union (EU) reaching about 80 years of age [2,4]. This discrepancy can be attributed to inadequate chronic disease management, socioeconomic factors, healthcare inequalities, ineffective healthcare policies, limited patient health culture, and significant vaccine hesitancy, including reluctance towards COVID-19 vaccination, in Bulgaria, along with instances of falsified vaccination records [2,5,6]. The COVID-19 pandemic had a unique trajectory in Eastern Europe, with major surges in cases and deaths occurring towards the end of 2020, particularly in Bulgaria, which saw high excess mortality due to insufficient testing, delayed lockdowns, high cardiovascular disease prevalence, and disparities in medical resources [7].

Numerous prospective studies and retrospective analyses have consistently demonstrated the effectiveness of the COVID-19 vaccines in reducing premature mortality rates [8–10]. Nevertheless, it is crucial to approach the extrapolation of these findings to the Bulgarian population with caution, given the overall challenges associated with the nation’s health culture and disparities in healthcare quality and access [11].

In this study, our aim was to analyse real-world data (RWD) to explore the relationship between vaccination status and COVID-19-related outcomes within the Bulgarian population on a national level. We also delved into several clinically relevant subgroups, encompassing both ambulatory and in-hospital settings, stratified by vaccine type, SARS-CoV-2 variant and the presence of specific socially significant chronic comorbidities, such as solid malignancy, cardiovascular disease, chronic pulmonary disease or diabetes mellitus.

Methods

Ethics approval

Approval for this retrospective analysis was granted by the Bulgarian Ministry of Health (94-4750/09.11.2022). The study adhered to the ethical standards outlined in the Helsinki Declaration, the International Ethical Guidelines for Health-related Research Involving Humans (2016), and the Personal Data Protection Law No. 25,326, as well as the resolution of the Ministry of Health of the Nation No. 1480/11.

Patient cohort criteria

This retrospective, multicentric real-world study encompassed a cohort of 1,048,574 individuals diagnosed with COVID-19 between March 2020 and April 2022. To obtain the necessary data for analysis, we submitted a formal access request to the United Information Portal of the Bulgarian Ministry of Health, which served as the comprehensive data source for all 28 provinces in Bulgaria [12]. The primary objective of this study was to evaluate the effectiveness of vaccines in reducing COVID-19 associated mortality rates. We conducted subgroup analyses considering sex, age, vaccine type, SARS-CoV-2 variant, and the presence of specific socially significant chronic comorbidities: solid malignancy, cardiovascular disease, chronic pulmonary disease, or diabetes mellitus (Figure 1). Having any of these chronic conditions significantly increases hospitalisation and mortality risks in COVID-19 patients [13].

Figure 1. A flow chart of the the patient cohort selection process.

During the period of interest, patients were considered fully vaccinated if they had received either two doses of an mRNA-based vaccine (e.g. BNT162b2 or CX-024414), two doses of ChAdOx1-SARS-COV-2, or one dose of Ad26.COV2-S. Given that most patients included in this analysis received a single vaccine type, our study did not specifically investigate differences in vaccination regimens within the reported as vaccinated cohort and excluded cases involving booster doses. The dataset of 1,048,574 patients diagnosed with COVID-19 between March 2020 and April 2022 was also utilised in other published studies [14,15].

Statistical analysis

Statistical evaluation employed SAS version 9.4 (SAS, Cary, North Carolina, USA). Odds ratios (OR) were calculated to examine the association between COVID-19 vaccination and mortality using a logistic regression model. The probability of death was modeled as a dependent variable, incorporating gender, virus type, grouped age and vaccination status. The significance of these relations was estimated using the Wald Chi-square test. Additionally, a similar model, excluding vaccination status, was applied to calculate propensity scores.

Patient and public involvement

The research question formulation and outcome measure selection did not involve patient input. Moreover, patients were not engaged in the study’s design or implementation. There are no plans to directly communicate the research findings with the study participants or the relevant patient communities.

Results

Among the 1,048,574 confirmed COVID-19 patients, 73% (n = 780,718) remained unvaccinated, while 27% (n = 267,856) had received full vaccination. Unadjusted statistical assessments, as displayed in Table 1, unveiled a substantial reduction in mortality rates across the reported-as-vaccinated cohort (1,608 fatalities; 0.6%) compared to the unvaccinated group (40,985 fatalities; 5.2%) [Odds Ratio (OR) 0.1090 (95% Confidence Interval (CI) 0.1037 to 0.1146), p < 0.0001] (Figure 2).

Figure 2. Forest plot of the unadjusted univariate statistical analyses.

Table 1. Unmatched cohorts.

Criteria	TOTAL DEATHS (N = 42593)	UNVACCINATED CASES (N = 780718)	VACCINATED CASES (N = 267856)	OR (95% CI)	p-value
Overall deaths	n = 42593	40985/780718(5.2%)	1608/267856(0.6%)	0.1090[0.1037 to 0.1146]	<0.0001
Age ≤ 63	n = 8925	8598/603123(1.4%)	327/206073(0.2%)	0.1099[0.0984 to 0.1227]	<0.0001
Age > 63	n = 33668	32387/177595(18.2%)	1281/61783(2.1%)	0.0949[0.0897 to 0.1005]	<0.0001
Male	n = 23504	22437/358619(6.3%)	1067/127563(0.8%)	0.1264[0.1188 to 0.1344]	<0.0001
Female	n = 19089	18548/422099(4.4%)	541/140293(0.4%)	0.0842[0.0773 to 0.0918]	<0.0001
In-hospital (non-ICU)	n = 24624	23737/759147(3.1%)	887/265786(0.3%)	0.1037[0.0970 to 0.1109]	<0.0001
In-hospital (ICU)	n = 17969	17248/21571(80.0%)	721/2070(34.8%)	0.1340[0.1217 to 0.1475]	<0.0001
mRNA based vaccine	n = 41984	40724/778383(5.2%)	1260/240424(0.5%)	0.0954[0.0902 to 0.1010]	<0.0001
Viral vector based vaccine	n = 41365	40676/770661(5.3%)	689/87814(0.8%)	0.1419[0.1316 to 0.1531]	<0.0001
Alpha (B.1.1.7)	n = 9600	9580/127232(7.5%)	20/85564(0.0%)	0.0029[0.0019 to 0.0045]	<0.0001
Beta (B.1.351)	n = 11299	11271/156965(7.2%)	28/52780(0.1%)	0.0069[0.0047 to 0.0099]	<0.0001
Delta (B.1.617.1)	n = 16935	15838/260676(6.1%)	1097/50615(2.2%)	0.3425[0.3219 to 0.3644]	<0.0001
Omicron (B1.1.529)	n = 4759	4296/235845(1.8%)	463/78897(0.6%)	0.3182[0.2890 to 0.3503]	<0.0001
Solid malignancies	n = 330	315/1474(21.4%)	15/323(4.6%)	0.1792[0.1052 to 0.3053]	<0.0001
Cardiovascular diseases	n = 9865	9495/43888(21.6%)	370/11507(3.2%)	0.1203[0.1082 to 0.1338]	<0.0001
Chronic pulmonary diseases	n = 486	460/2163(21.3%)	26/536(4.9%)	0.1887[0.1256 to 0.2836]	<0.0001
Diabetes mellitus	n = 709	686/3008(22.8%)	23/784(2.9%)	0.1023[0.0670 to 0.1562]	<0.0001

The median age of the patients was 63 years. Both age groups, i.e. reported-as-COVID-19-vaccinated patients aged ≤63 (n = 20,673) and >63 (n = 61,783) years, displayed significant reductions in mortality rates [OR 0.1099 (95% CI 0.0984 to 0.1227), p < 0.0001] and [OR 0.0949 (95% CI 0.0897 to 0.1005), p < 0.0001], respectively. Vaccination proved equally beneficial for both genders, with males [OR 0.1264 (95% CI 0.1188 to 0.1344), p < 0.0001] and females [OR 0.0842 (95% CI 0.0773 to 0.0918), p < 0.0001] experiencing reduced mortality rates. Significant reductions in mortality rates were observed in both non-intensive care unit [OR 0.1037 (95% CI 0.0970 to 0.1109), p < 0.0001] and intensive care unit-related in-hospital deaths [OR 0.1340 (95% CI 0.1217 to 0.1475), p < 0.0001].

When comparing primary vaccine platforms concerning mortality, both vector-based vaccines [OR 0.1419 (95% CI 0.1316 to 0.1531), p < 0.0001] and mRNA-based vaccines [OR 0.0954 (95% CI 0.0902 to 0.1010), p < 0.0001] demonstrated statistically significant reductions in fatal outcomes, with the mRNA-based vaccine cohort showing a more pronounced benefit. Assessing the impact of vaccination on mortality across various SARS-CoV-2 variants, statistically significant reductions in premature mortality rates were consistent throughout all clinically relevant pandemic waves: B.1.1.7 [OR 0.0029 (95% CI 0.0019 to 0.0045), p < 0.0001], B.1.351 [OR 0.0069 (95% CI 0.0047 to 0.0099), p < 0.0001], B.1.617.1 [OR 0.3425 (95% CI 0.3219 to 0.3644), p < 0.0001] and B1.1.529 [OR 0.3182 (95% CI 0.2890 to 0.3503), p < 0.0001].

In evaluating vaccine effectiveness among high-risk patient groups, the unadjusted analysis showed statistically significant reductions in mortality rates in the reported-as-vaccinated cohorts with solid malignancy [OR 0.1792 (95% CI 0.1052 to 0.3053), p < 0.0001], cardiovascular disease [OR 0.1203 (95% CI 0.1082 to 0.1338), p < 0.0001], chronic pulmonary disease [OR 0.1887 (95% CI 0.1256 to 0.2836), p < 0.0001], or diabetes mellitus [OR 0.1023 (95% CI 0.0670 to 0.1562), p < 0.0001]. Logistic regression analysis indicated that the lowest mortality rates occurred in reported-as-vaccinated females aged 63 or younger, while the highest mortality rates were observed in unvaccinated males aged >63 as shown in Table 2. The findings from propensity score matching support the unadjusted statistical analyses, as depicted in Table 3 and Figure 3.

Figure 3. Forest plot of the propensity score matched univariate statistical analyses.

Table 2. Logistic regression.

Analysis of Maximum Likelihood Estimates
Parameter		DF	Estimate	Standard Error	Wald Chi-Square	Pr > ChiSq
Intercept		1	−4.1024	0.0135	91804.2884	<.0001
Vaccination	Yes	1	−1.1862	0.0129	8413.8214	<.0001
Gender	F	1	−0.2755	0.00532	2683.9966	<.0001
Virus type	Alpha	1	0.2837	0.00984	831.6416	<.0001
	Beta	1	0.1367	0.00931	215.7782	<.0001
	Delta	1	0.2513	0.00829	918.3604	<.0001
Age groupr	< = 63 yrs.	1	−1.3417	0.00620	46878.0576	<.0001
Odds Ratio Estimates and Wald Confidence Intervals
Effect				Estimate	95% Confidence Limits
Vaccination Yes vs No				0.093	0.089	0.098
Gender F vs M				0.576	0.565	0.589
Virus type Alpha vs Omicron				2.600	2.505	2.697
Beta vs Omicron				2.244	2.165	2.326
Delta vs Omicron				2.517	2.433	2.603
Age group < = 63 yrs. vs > 63 yrs.				0.068	0.067	0.070
Analysis of Effects
Effect	DF			Wald Chi-Square	Pr > ChiSq
Vaccination	1			8413.8214	<.0001
Gender	1			2683.9966	<.0001
Virus type	3			3218.3015	<.0001
Age group	1			46878.0576	<.0001

Table 3. Propensity score matched cohorts.

Criteria	TOTAL DEATHS (N = 13087)	UNVACCINATED CASES (N = 267856)	VACCINATED CASES (N = 267856)	OR (95% CI)	p-Value
Overall deaths	n = 13087	11479/267856(4.3%)	1608/267856(0.6%)	0.1349[0.1280 to 0.1422]	<0.0001
Age ≤ 63	n = 2951	2624/206073(1.3%)	327/206073(0.2%)	0.1232[0.1098 to 0.1382]	<0.0001
Age > 63	n = 10136	8855/61783(14.3%)	1281/61783(2.1%)	0.1266[0.1192 to 0.1343]	<0.0001
Male	n = 7171	6104/127563(4.8%)	1067/127563(0.8%)	0.1679[0.1572 to 0.1792]	<0.0001
Female	n = 5917	5376/140293(3.8%)	541/140293(0.4%)	0.0972[0.0889 to 0.1062]	<0.0001
In-hospital (non-ICU)	n = 8853	7966/263294(3.0%)	887/265786(0.3%)	0.1073[0.1001 to 0.1151]	<0.0001
In-hospital (ICU)	n = 4234	3513/4562(77.0%)	721/2070(34.8%)	0.1595[0.1424 to 0.1788]	<0.0001
mRNA based vaccine	n = 12739	11479/267856(4.3%)	1260/240424(0.5%)	0.1177[0.1110 to 0.1247]	<0.0001
Viral vector based vaccine	n = 12168	11479/267856(4.3%)	689/87814(0.8%)	0.1766[0.1635 to 0.1908]	<0.0001
Alpha (B.1.1.7)	n = 5800	5780/85564(6.8%)	20/85564(0.0%)	0.0032[0.0021 to 0.0050]	<0.0001
Beta (B.1.351)	n = 1944	1916/52780(3.6%)	28/52780(0.1%)	0.0141[0.0097 to 0.0205]	<0.0001
Delta (B.1.617.1)	n = 3128	2031/50615(4.0%)	1097/50615(2.2%)	0.5301[0.4920 to 0.5711]	<0.0001
Omicron (B1.1.529)	n = 2216	1753/78897(2.2%)	463/78897(0.6%)	0.2598[0.2344 to 0.2879]	<0.0001
Solid malignancies	n = 15	0/217(0.0%)	15/323(4.6%)	NA
Cardiovascular diseases	n = 6573	6203/10065(61.6%)	370/11507(3.2%)	0.0207[0.0185 to 0.0231]	<0.0001
Chronic pulmonary diseases	n = 26	0/458(0.0%)	26/536(4.9%)	NA
Diabetes mellitus	n = 23	0/1257(0.0%)	23/784(2.9%)	NA

NA = Not Applicable.

Discussion

In this extensive analysis, we harnessed a comprehensive real-world dataset, involving over a million confirmed COVID-19 patients. We examined a wide array of subgroups, facilitating a holistic evaluation of vaccination outcomes within a population characterised by low vaccination coverage. The outcomes unveiled in this study make a substantial contribution to the ever-expanding body of evidence, strongly endorsing the pivotal role of vaccination in the management of COVID-19 and the prevention of severe outcomes. This significance extends even to high-risk populations grappling with a heavy burden of chronic diseases, as demonstrated by the findings of our study.

However, while the risk of SARS-CoV-2 infection, severe disease and premature mortality has diminished in vaccinated populations [16], it is essential to acknowledge that the risk for breakthrough infections persist. Our data uncovered a noticeable difference in vaccine effectiveness when comparing the alpha (B.1.1.7) and beta (B.1.351) variants to the delta (B.1.617.1) and omicron (B1.1.529) variants, illustrated by an increase in fatality cases in the reported-as-vaccinated cohort with the latter two variants. However, it is crucial to emphasise that despite this numerical disparity, a statistically significant benefit was still apparent when compared to unvaccinated controls during the same time period.

Convincing evidence suggests that individuals aged 65 and older, as well as those with underlying health conditions, remain at the greatest risk from severe COVID-19. This might be due to the potential waning of post-vaccination immunity in these demographic groups [17, 18]. Moreover, it is noteworthy that the burden of COVID-19 appears considerably higher in developing countries compared to high-income countries. As Bulgaria is ranked among the EU’s economically disadvantaged states, this association warrants attention and future exploration [19].

In 2018, a report published by the European Commission shed light on a concerning trend: among all EU countries, Bulgaria stood out as the nation ‘least likely to agree that vaccines are safe’ [20]. This perception predates the emergence of SARS-CoV-2, which, in turn, has fueled a proliferation of baseless conspiracy theories, further exacerbating these statistics [21]. As a result, Bulgaria continues to be one of the European countries with the lowest COVID-19 vaccination rates [22]. Despite the development of an etiological antiviral treatment that has demonstrated its effectiveness in reducing hospitalisation rates and mortality [23], Bulgaria faces an additional healthcare challenge. The primary ambulatory anti-viral agent against SARS-CoV-2 is currently not reimbursed by the Bulgarian National Health Insurance Fund [24], which significantly restricts public access.

Even though vaccination remains the most cost-effective strategy for preventing severe COVID-19-related disease and premature deaths [25], addressing healthcare access disparity is of paramount importance, especially in light of persistently high vaccination hesitancy rates. Such measures are indispensable for safeguarding the health and well-being of a population grappling with inadequately managed chronic diseases and life expectancies that lag behind the international average.

Recent recommendations advocate for booster doses or an "up-to-date" vaccination strategy due to incomplete immunity against SARS-CoV-2 after "full-dose" vaccination [26,27]. Studies demonstrate that frequent boosters (every 6–12 months) significantly reduce severe COVID-19 risk, particularly in older adults and immunocompromised individuals. Annual boosters for those over 75 could decrease severe cases by 199 per 100,000 compared to one-time boosters. Immunocompromised individuals benefit more [28]. Variants with immune evasion heighten the need for targeted boosters [29]. Therefore, tailored booster strategies based on patient characteristics and risk factors are crucial.

Limitations

While our study has provided valuable insights, it is vital to acknowledge and address certain limitations that could affect how we interpret and apply our findings. This study utilised a retrospective design, which inherently presents constraints in data collection and potential biases. Relying on existing electronic medical records and databases may have introduced selection biases, as not all relevant variables or outcomes might have been included in the available data. To enhance the robustness of our evidence, the inclusion of prospective studies with meticulously designed protocols is crucial, as they can offer more reliable and rigorous insights. However, the rapid viral evolution, emergence of new variants, and continually shifting epidemiological landscape present challenges to conducting real-time prospective studies, as noted by other published work following a similar methodology [14,30].

Conclusions

This study highlights the pivotal role of COVID-19 vaccination in reducing premature mortality rates, especially in a population characterised by low vaccination coverage and a high burden of chronic diseases. The findings reaffirm the significance of vaccination as a critical tool in managing the pandemic.

Ethics approval

Approval for this retrospective analysis was granted by the Bulgarian ministry of health - document number 94-4750 from 09.11.2022.

Authors contributions

GD and TV designed, implemented the study, analysed the data and wrote the manuscript. KK conducted the statistical analyses.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data supporting the findings of this study are available upon formal request from the Ministry of Health of the Republic of Bulgaria. Access to these data is subject to restrictions and was obtained with permission for this specific study. For data access, please visit https://coronavirus.bg/ in compliance with the Ministry of Health’s policies and guidelines.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.