A Nomogram-Based Prediction for Severe Pneumonia in Patients with Coronavirus Disease 2019 (COVID-19)

Abstract

Background

The outbreak of a novel coronavirus disease 2019 (COVID-19) is currently ongoing worldwide. A proportion of COVID-19 patients progress rapidly to acute respiratory failure.

Objective

We aimed to build a model to predict the risk of developing severe pneumonia in patients with COVID-19 in the early stage.

Methods

Data from patients who were confirmed to have COVID-19 and were admitted within 7 days from the onset of respiratory symptoms were retrospectively collected. The patients were classified into severe and non-severe groups according to the presence or absence of severe pneumonia during 1–2 weeks of follow-up. The clinical characteristics and laboratory indicators were screened by cross-validation based on LASSO regression to build a prediction model presented by a nomogram. The discrimination and stability, as well as the prediction performance of the model, were analysed.

Results

The neutrophil–lymphocyte ratio, monocyte counts, eosinophil percentage, serum lactate dehydrogenase level and history of diabetes mellitus were collected for the model. Bootstrap resampling showed the apparent C-statistics, and the brier scores were 0.929 and 0.098. The optimism of the C-statistics and brier score was 0.0172 and −0.019, respectively. The adjusted C-statistics and brier score were 0.9108 and 0.1169, respectively. The optimal cut-off value of the total nomogram score was determined to be 119 according to the maximal Youden index. The sensitivity, specificity, positive predictive value, and negative predictive value for differentiating the presence and absence of severe pneumonia were 83%, 89%, 74%, and 94%, respectively.

Conclusion

In our study, the neutrophil–lymphocyte ratio, monocyte counts, eosinophil percentage, serum lactate dehydrogenase level and history of diabetes mellitus showed great discrimination and stability for the prediction of the presence of severe pneumonia and were selected for the model.

Keywords:

• COVID-19
• prediction
• nomogram

Introduction

An epidemic caused by coronavirus disease 2019 (COVID-19) has occurred unexpectedly around the world.^1–4 The clinical spectrum of COVID-19 varies from asymptomatic or pauci-symptomatic forms to clinical conditions characterized by acute respiratory failure that necessitates mechanical ventilation and support in an intensive care unit (ICU) to multi-organ and systemic manifestations in terms of sepsis, septic shock, and multiple organ dysfunction syndromes.^5,6 Previous studies showed that nearly 20% of patients with COVID-19 developed a severe or critical clinical course with higher morbidity and mortality rates, more than 25% of patients required ICU admission, nearly 20% of patients developed acute respiratory distress syndrome, and 10% of patients required invasive mechanical ventilation or extracorporeal membrane oxygenation for refractory hypoxemia.^6–8 Different treatments and managements between non-severe and severe COVID-19 patients are thus required. The proper management and distribution of medical resources critical factors that rely on the accurate estimation at an early stage for COVID-19 patients at high risk of developing severe pneumonia. Early recognition can not only facilitate patients’ classified management but benefit those at high risk of receiving close supervision and timely special intervention. Previous studies^9,10 have attempted to screen haematological biomarkers as predictors for the severity of COVID-19, but further clinical validation is lacking. Nomograms, currently available prediction tools, show good discrimination characteristics in predicting outcomes and are simple and convenient for clinical use.^11,12 Therefore, the purpose of this study was to build a model to predict the risk of developing severe pneumonia at an early stage in patients with COVID-19 and to present the model by nomogram.

Methods

Participants

The data of patients who were admitted between January 28, 2020, and March 20, 2020 at the First Affiliated Hospital of Nanchang University and were confirmed to have COVID-19 were retrospectively collected. The study was approved by the Institutional Ethics Committee of the First Affiliated Hospital of Nanchang University. Participant consent for approval was waived due to the retrospective study design. We confirmed that all patient data accessed complied with relevant data protection and privacy regulations. The inclusion criteria were as follows: (1) patients were admitted within 7 days from the onset of respiratory symptoms or in the absence of any symptoms following contact with confirmed COVID-19 patients; (2) more than two positive polymerase chain reaction (PCR) tests of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2); and (3) more than 18 years old. None of the patients had developed severe pneumonia at the time of admission.

Routine Examination and Severe Pneumonia Diagnostic Criteria

The clinical information of the patients was collected. Routine tests such as complete blood count, blood chemistries and lactate dehydrogenase were performed when upon admission and were included as predictive factors in the study. The test time from the onset of symptoms was collected. All variables including basic information and laboratory indictors in this study are reported in Table 1. The outcome of the prediction model was the presence of severe pneumonia after admission. The patients were classified into severe and non-severe groups according to the presence or absence of severe pneumonia during 1–2 weeks of follow-up. Severe pneumonia in this study included severe and critical pneumonia. The diagnostic criteria of the severe type or critical pneumonia were (1) respiratory distress (respiratory rate, ≥30 cycles per minute), (2) oxygen saturation 93% or below or arterial partial pressure of oxygen (PaO2)/oxygen concentration FiO2 less than or equal to 300 mm Hg in the resting state, (3) respiratory failure requiring mechanical ventilation, (4) shock, and (5) other forms of organ failure requiring monitoring and treatment at the intensive care unit.¹³ Severe pneumonia was determined if one of the diagnostic criteria was met.

Table 1 Description and Comparisons of Characteristics Between COVID-19 Patients in the Severe and Non-Severe Groups

	Non-Severe Pneumonia	Severe Pneumonia	P value
	(n = 82)	(n = 30)
Patient demographics
Age, years, mean ± SD	42 ± 14	55 ± 16	<0.001
Male, No. (%)	43 (52.4)	23 (76.7)	0.021
Time interval between tests and onset, days, Median (Quartiles)	3.5 (2, 5.25)	6 (4, 7)	<0.001
Comorbid conditions, No. (%)
Diabetes Mellitus	7 (8.5)	11 (36.7)	0.001
High Blood Pressure	5 (6.1)	6 (20)	0.067
Hepatitis B	9 (11)	6 (20)	0.353
Others	4 (4.9)	3 (10)	0.582
Initial signs and symptoms, No. (%)
Fever	73 (89)	29 (96.7)	0.378
Cough	32 (39)	20 (66.7)	0.009
Sputum production	11 (13.4)	4 (13.3)	1
Fatigue weakness	13 (15.9)	7 (23.3)	0.36
Myalgia	8 (9.8)	1 (3.3)	0.475
Sore throat	13 (15.9)	6 (20)	0.605
Headache	11 (13.4)	2 (6.7)	0.513
Chills	14 (17.1)	5 (16.7)	0.96
Diarrhea	1 (1.2)	1 (3.3)	0.489
Nausea	2 (2.4)	1 (3.3)	0.8
Vomit	1 (1.2)	1 (3.3)	0.481
Non-symptoms	4 (4.9)	0	0.11
Laboratory variables, Median (Quartiles)
WBCs, ×10^9/L	4.92 (3.67, 6.21)	7.47 (4.09, 10.1)	0.001
Neutrophil counts, ×10^9/L	3.16 (2.09, 4.45)	6.21 (2.8, 9.63)	<0.001
Neutrophil %	68.45 (56.5, 74.5)	82.1 (69.69, 90.3)	<0.001
Lymphocyte counts, ×10^9/L	1.18 (0.88, 1.56)	0.74 (0.46, 0.89)	<0.001
Lymphocyte %	23.7 (18.58, 34.43)	12.1 (4.85, 21.38)	<0.001
Neutrophil-lymphocyte ratio	2.9 (1.64, 4.08)	7.43 (3.35, 19.07)	<0.001
Monocyte counts, ×10^9/L	0.33 (0.25, 0.44)	0.44 (0.29, 0.57)	0.029
Monocyte %	7.15 (5.79, 8.33)	6.29 (3.96, 8.25)	0.133
Eosinophil counts, ×10^9/L	0.015 (0.01, 0.04)	0 (0, 0.013)	<0.001
Eosinophil %	0.35, (0.165, 0.813)	0.1 (0, 0.208)	<0.001
Basophil counts, ×10^9/L	0 (0, 0.01)	0 (0, 0.0025)	0.224
Basophil %	0.075 (0, 0.1)	0 (0, 0.1)	0.032
Serum lactate dehydrogenase, U/L	207 (179.75, 233.5)	305.75 (223.25, 492.5)	<0.001
Serum creatine, μmol/L	66 (51.96, 78.64)	73.72 (58.72, 85.68)	0.023

Notes: Quantitative data were presented as the mean ± standard deviation or median (quartiles). The counting data were presented as the percentage of the total. P value < 0.05 is marked bold.

Abbreviation: SD, standard deviation.

Feature Selection and Assessment

Cross-validation based on the Least Absolute Shrinkage and Selection Operator (LASSO) method was conducted to select significant features from all variables mentioned in Table 1. The process was mainly performed in the glmnet package of R, version 3.6.0 (http://www.r-project.org/).

Development and Assessment of a Multi-Predictor Nomogram

A nomogram was constructed using selected features. A nomogram is based on proportionally converting each regression coefficient in a multivariate logistic regression point scale. The points are added across independent variables to derive total points, which are converted to predicted probabilities.¹⁴ In this study, the predictive power was measured by the concordance index (C-index), namely, C-statistics. To prove the stability of the model, bootstrapping validation with 100 resamples was conducted to overcome the overfitting problem.¹⁵ The calibration curve provided a comparison between the expected and observed conversion probabilities. The entire process was performed in the rms package of R, version 3.6.0 (http://www.r-project.org/).

Estimation for the C-Statistics and Brier of the Model by Internal Validation

Internal validation was used to evaluate the stability of the prediction model by the principle of random changes in sample composition as implemented by the bootstrap resampling technique, in which the regression models were fitted in 100 bootstrap replicates drawn with replacement from the total sample. Then, the model was refitted in each bootstrap replicate and tested on the original sample to estimate optimism in model performance. The process was performed in the stats and pROC package of R, version 3.6.0 (http://www.r-project.org/).

Applying Nomogram Assessment

For clinical use of the model, the total scores of each patient were calculated based on the nomogram. Receiver operating characteristic curve analysis was used to calculate the optimal cut-off values that were determined by maximizing the Youden index (sensitivity + specificity − 1). The accuracy of the optimal cut-off value was then assessed by the sensitivity, specificity, predictive values, and likelihood ratios. The process was performed in the pROC and epiR package of R, version 3.6.0 (http://www.r-project.org/).

Statistical Analysis

Continuous variables with normal distributions are expressed as the mean (SD) and compared using an unpaired, 2-tailed t-test. Continuous variables without normal distributions are expressed as medians (quartiles) and compared using the Mann–Whitney test. Categorical variables were compared using the χ2 test or Fisher’s exact test.

In all analyses, P < 0.05 was considered to indicate statistical significance. All analyses were performed using SPSS, version 21 and R, version 3.6.0.

Results

Demographic and Clinical Characteristics

One hundred and twelve patients who were identified as a confirmed COVID-19 were included in this study from January 28 to March 20, 2020. The 112 patients were divided into two groups according to the presence or absence of severe pneumonia after admission. The patients with (n = 82) or without (n = 30) severe pneumonia were described in Table 1. All laboratory results were obtained according to the tests, which were performed on the same day as admission. Age, male percentage, time of admission from onset, percentage of diabetes mellitus history, cough percentage, haemocyte tests including white blood cell counts, neutrophil counts, percentage of neutrophils, neutrophil-lymphocyte ratio, monocyte counts and basophil percentage, as well as serum lactate dehydrogenase and serum creatine levels, were significantly higher in the severe pneumonia group compared to the non-severe group (p < 0.05) (Table 1). The lymphocyte counts, lymphocyte percentage, eosinophil counts and eosinophil percentage in the severe group were significantly lower than the non-severe group (p < 0.05) (Table 1).

Extraction of Feature

Feature selection was conducted by cross-validation-based LASSO regression in all patients. When the lambda value was collected as 1 standard error, eight variables (age, sex, lymphocyte counts, neutrophil-lymphocyte ratio, monocyte counts, eosinophil percentage, serum lactate dehydrogenase level and history of diabetes mellitus) were selected (Figure 1). After estimating the regression coefficient in the refitting model, five variables, including the neutrophil-lymphocyte ratio, monocyte counts, eosinophil percentage, serum lactate dehydrogenase level and history of diabetes mellitus, were finally collected.

Figure 1 A graph showing the cross validation in LASSO regression to screen variables.

Establishment and Calibration of a Nomogram for the Prediction Model

A nomogram that combined the five significant predictors was constructed. Figure 2A shows the predictive nomogram, which obtained a C-statistic of 0.929 (95% CI, 0.875–0.984). The calibration plots of the nomogram are shown in Figure 2B using bootstrapping with 100 resamples. The closer the calibration curve is to the diagonal, the better the predictive power of the nomogram. Figure 2C shows a large area under the receiver operating characteristic (AUROC) curve, which was equal to the C-statistics.

Figure 2 (A) Nomogram for estimating the risk of severe pneumonia after admission in COVID-19 patients. (B) Validity of the predictive performance of the nomogram in estimating the risk of the presence of severe pneumonia by 100 bootstrap tests. (C) Receiver operating characteristic (ROC) curves of the nomogram model. The ROC curve is based on a series of different dichotomous methods (cut-off value or determination threshold), with a true positive rate (sensitivity) as the ordinate and false positive rate (1-specificity) as the abscissa.

Estimation for the C-Statistics and Brier of the Model by Internal Validation

As shown in Table 2, bootstrap resampling showed negligible model optimism. The apparent C-statistics and apparent brier score was 0.929 and 0.098, respectively. The optimism of the C-statistics and brier score was 0.0172 and −0.019, respectively. The adjusted C-statistics and brier score was 0.9108 and 0.1169, respectively.

Table 2 The Discrimination and Calibration of the Predictive Model

	C-Statistics	Brier Score
Apparent value	0.929	0.098
Optimism value	0.0172	−0.019
Adjusted value	0.9108	0.1169

Notes: optimism = mean (resample - original) in both C-statistics and Brier by 100 resamples.

The Predictive Performance of the Nomogram

The performance of the model was good in predicting the presence of severe pneumonia. The predictive accuracy was an AUC of 0.929. The optimal cut-off value of the total nomogram score was determined to be 119. The sensitivity, specificity, positive predictive value, and negative predictive value when used in differentiating the presence from absence of severe pneumonia was 83%, 89%, 74%, and 94%, respectively (Table 3).

Table 3 Accuracy of the Prediction Score of the Nomogram for Estimating the Risk of Severe Pneumonia

	Value (95% CI)
Variables
Area under ROC curve	0.929 (0.875, 0.984)
Cut-off score	119
Sensitivity, %	83 (0.65, 0.94)
Specificity, %	89 (0.8, 0.95)
Positive predictive value, %	74 (0.56, 0.87)
Negative predictive value, %	94 (0.86, 0.98)
Positive likelihood ratio	7.59 (4.02, 14.35)
Negative likelihood ratio	0.19 (0.08, 0.42)

Abbreviations: CI, confidence interval; ROC, receiver operating characteristic.

Discussion

In our study, we found that many characteristics and laboratory indicators at the early stage showed a significant difference between COVID-19 patients in severe versus non-severe groups (Table 1). After statistical layers of screening, five significant features, including the neutrophil-lymphocyte ratio, monocyte counts, eosinophil percentage, serum lactate dehydrogenase level and history of diabetes mellitus, were finally selected for the model and showed great discrimination and stability for the prediction of the presence of severe pneumonia (Figure 2 and Table 2). For clinical use of the model, we summarized the prediction performance and found that the sensitivity, specificity, positive predictive value and negative predictive value was 83%, 89%, 74%, and 94%, respectively, in estimating the risk of severe pneumonia by using the total score and 119 as the cut-off value (Table 3). The included indicators are all routine examinations performed upon patient admission. Thus, the model has a high value of practical application in the clinic. Based on the prediction, the nomogram might serve as a tool to predict patients at high risk of severe pneumonia. The model will not only help physicians perform reasonable classified management for COVID-19 patients but also facilitate randomized clinical trials to evaluate the efficacy of early inhaled oxygen or intensive care unit management on improving the prognosis of patients at high risk.

Recent research showed that COVID-19 patients deteriorated over the course of 7–14 days after the onset of initial symptoms.¹⁶ This finding is consistent with our observation, which indicates that the prediction should be carried out within 7 days after the onset of symptoms. Of the five features, the neutrophil-lymphocyte ratio and serum lactate dehydrogenase levels have been reported to be significantly correlated with severe COVID-19 pneumonia.^16,17 Additionally, comorbid conditions are also important in the development of COVID-19 pneumonia. Most COVID-19 patients with severe pneumonia or those that died have underlying diseases, including diabetes, hypertension, and cardiovascular disease, and the overall case-fatality rate of these patients is higher than the non-severe patients.⁶ Diabetes, including undiagnosed diabetes, is more prevalent compared to other comorbidities, and makes patients more vulnerable to any infection. Previous studies reported that diabetes mellitus was one of the major factors associated with severe pneumonia following influenza infection.¹⁸ These studies support our clinical findings that diabetes is an important risk factor for predicting severe pneumonia at an early stage.

At present, few studies have been conducted to predict the development of severe pneumonia in COVID-19 patients. Of the currently available systems for assessing pneumonia severity, the pneumonia severity index (PSI) and CURB-65 score are widely validated and used; however, these scales do not play an early predictive role in the development of severe pneumonia and have low specificity for the severity of COVID-19, as well as poor performance in predicting the presence of severe COVID-19 pneumonia.¹⁶ A study combining the neutrophil-lymphocyte ratio and age to predict severe pneumonia in COVID-19 patients also showed good predictive performance.¹⁶ The current study demonstrated superior predictive performance by adding other significant indicators to the neutrophil-lymphocyte ratio.

This study has several limitations. The data were obtained from a single institution, and the model only underwent internal validation. A prospective research study is required to validate the feasibility and effectiveness of the nomogram.

Conclusions

In this study, the neutrophil-lymphocyte ratio, monocyte counts, eosinophil percentage, serum lactate dehydrogenase level and history of diabetes mellitus were selected for inclusion in the model that showed great discrimination and stability for the prediction of the presence of severe pneumonia. The model may help physicians identify COVID-19 patients at risk of developing severe pneumonia in the early stage and provide timely treatment and optimal management for medical resources.

Acknowledgments

We thank all the staff who participated in the data collection.

Disclosure

The authors report no conflicts of interest in this work.