Development and deployment of interpretable machine-learning model for predicting in-hospital mortality in elderly patients with acute kidney disease

Figures & data

Figure 1. A flowchart for the selection of elderly patients with AKD. AKD: acute kidney disease; AKI: acute kidney injury; eICU-CRD: the eICU Collaborative Research Database; ESRD: end-stage renal disease; ICU: intensive care unit; MIMIC-IV: medical information mart for intensive care IV.

Table 1. Baseline characteristics of elderly patients with AKD.

Variables	Training cohort			External validation cohort
	Survival (n = 1876)	Death (n = 790)	P-value	Survival (n = 363)	Death (n = 172)	P-value
Demographic
Age, years	72.7 [66.6, 80.1]	76.1 [68.7, 82.8]	<0.001	70.0 [64.0, 75.0]	72.0 [66.0, 79.0]	0.002
Gender, n (%)			0.131			0.569
Male	1087 (57.9)	432 (54.7)		200 (55.1)	100 (58.1)
Female	789 (42.1)	358 (45.3)		163 (44.9)	72 (41.9)
Ethnicity, n (%)			0.015			0.006
White	1275 (68.0)	517 (65.4)		264 (72.7)	147 (85.5)
Black	196 (10.4)	69 (8.7)		60 (16.5)	11 (6.4)
Asian	48 (2.6)	14 (1.8)		3 (0.8)	1 (0.6)
Other	357 (19.0)	190 (24.1)		36 (9.9)	13 (7.6)
BMI (kg/m^2)	29.0 [24.8, 34.2]	27.7 [23.8, 33.0]	<0.001	29.9 [25.8, 36.0]	28.3 [24.6, 33.5]	0.018
Disease severity
Aki stage, n (%)			0.583			0.141
I	1452 (77.4)	598 (75.7)		301 (82.9)	150 (87.2)
II	389 (20.7)	178 (22.5)		58 (16.0)	18 (10.5)
III	35 (1.9)	14 (1.8)		4 (1.1)	4 (2.3)
GCS	10.0 [6.0, 13.0]	7.0 [3.0, 11.0]	<0.001	8.0 [3.0, 13.0]	8.0 [3.0, 13.0]	0.75
Delta GCS	5.0 [2.0, 9.0]	7.0 [4.0, 12.0]	<0.001	4.0 [0.0, 8.0]	1.0 [–2.0, 5.0]	<0.001
SOFA	8.0 [5.0, 11.0]	9.0 [7.0, 12.0]	<0.001	8.0 [5.0, 10.0]	7.0 [5.0, 10.0]	0.385
Delta SOFA	–2.0 [–4.0, 0.0]	0.0 [–3.0, 1.8]	<0.001	–1.0 [–4.0, 1.0]	1.0 [–1.2, 4.0]	<0.001
Comorbidities, n (%)
Sepsis	1706 (90.9)	756 (95.7)	<0.001	107 (29.5)	71 (41.3)	0.009
Hypertension	964 (51.4)	372 (47.1)	0.047	60 (16.5)	33 (19.2)	0.525
Diabetes mellitus	757 (40.4)	291 (36.8)	0.098	53 (14.6)	35 (20.3)	0.121
Cerebrovascular disease	116 (6.2)	37 (4.7)	0.153	35 (9.6)	9 (5.2)	0.118
CHF	435 (23.2)	201 (25.4)	0.231	45 (12.4)	30 (17.4)	0.151
CKD	554 (29.5)	273 (34.6)	0.012	45 (12.4)	26 (15.1)	0.466
Interventions, n (%)
MV	1849 (98.6)	780 (98.7)	0.866	65 (17.9)	44 (25.6)	0.052
RRT	311 (16.6)	228 (28.9)	<0.001	9 (2.5)	2 (1.2)	0.499
Renal toxic drugs	1811 (96.5)	767 (97.1)	0.541	54 (14.9)	46 (26.7)	0.002
Vasopressor use	1388 (74.0)	683 (86.5)	<0.001	124 (34.2)	56 (32.6)	0.789
Vital signs
Heart rate (bpm)	85.1 [77.0, 94.3]	88.0 [78.4, 97.5]	<0.001	91.3 [80.4, 104.8]	88.9 [77.6, 98.6]	0.01
Delta heart rate (bpm)	–15.7 [–23.0, −8.6]	–17.9 [–30.1, −8.4]	<0.001	–21.8 [–34.3, −9.1]	–17.2 [–26.8, −6.4]	0.002
SBP (mmHg)	118.1 [109.5, 129.4]	112.7 [105.6, 123.2]	<0.001	116.0 (17.9)	115.0 (15.1)	0.499
Delta SBP (mmHg)	–27.7 [–38.0, −19.3]	–33.1 [–46.6, −22.5]	<0.001	–20.1 [–34.3, −3.2]	–28.6 [–43.0, −13.8]	<0.001
DBP (mmHg)	59.2 [54.2, 65.1]	57.0 [52.1, 62.4]	<0.001	62.7 [56.3, 69.8]	60.3 [55.4, 66.3]	0.003
Delta DBP (mmHg)	–16.2 [–21.8, −11.3]	–18.5 [–24.9, −12.7]	<0.001	–15.5 [–25.2, −8.8]	–17.3 [–27.3, −10.5]	0.139
MAP (mmHg)	76.4 [71.6, 82.5]	73.3 [69.4, 79.0]	<0.001	76.9 [68.7, 84.7]	74.5 [67.7, 81.4]	0.035
Delta MAP (mmHg)	–20.0 [–26.5, −13.9]	–22.4 [–32.0, −16.2]	<0.001	–16.2 [–25.4, −6.6]	–18.3 [–29.1, −9.4]	0.023
Temperature (°C)	37.0 (0.5)	36.9 (0.5)	<0.001	37.0 [36.7, 37.3]	36.7 [36.4, 37.2]	0.001
Delta Temperature (°C)	–0.6 [–0.9, –0.3]	–0.6 [–1.0, –0.3]	0.516	–0.6 [–1.1, –0.2]	–0.4 [–1.0, 0.0]	0.063
Respiratory rate (bpm)	20.3 (3.1)	21.0 (3.4)	<0.001	19.8 [17.2, 23.3]	20.0 [17.4, 24.0]	0.359
Delta Respiratory rate (bpm)	–7.1 [–9.7, –4.9]	–7.8 [–11.2, –5.3]	<0.001	–6.7 [–10.9, –3.0]	–5.1 [–10.2, –1.6]	0.038
SpO2 (%)	96.9 [95.9, 97.9]	97.0 [95.8, 97.9]	0.958	96.9 [95.2, 98.2]	96.9 [95.2, 98.7]	0.345
Delta SpO2 (%)	–5.1 [–7.3, –3.2]	–6.3 [–12.7, –3.8]	<0.001	–4.9 [–8.5, –2.0]	–7.0 [–14.9, –3.4]	<0.001

AKI: acute kidney injury; CHF: congestive heart failure; CKD: chronic kidney disease; DBP: diastolic blood pressure; GCS: Glasgow coma scale; MAP: mean artery pressure; MV: mechanical ventilation; RRT: renal replacement therapy; SBP: systolic blood pressure; SOFA: sequential organ failure assessment; SpO2: peripheral oxygen saturation.

Figure 2. (a) The ROC curves of the SVM, Simplified SVM, LRM, MLP, RF, and XGB in the training cohort. (b) The ROC curves of the SVM, Simplified SVM, LRM, MLP, RF, and XGB in the external validation cohort. LRM: logistic regression model; MLP: multilayer perceptron; RF: random forest; SVM: support vector machine; XGBoost: extreme gradient boosting.

Figure 3. (a) Calibration curves. (b) Decision curve analysis. (c) Precision-Recall curves of the SVM and Simplified SVM in the external validation cohort. SVM: support vector machine.

Table 2. The predictive performance of models in the training and external validation cohorts.

Models	Training cohort
	AUC	Cutoff	Sensitivity	Specificity	PPV	NPV
LRM	0.812 (0.793–0.832)	0.335	0.673 (0.639–0.706)	0.809 (0.790–0.826)	0.597 (0.564–0.629)	0.855 (0.837–0.871)
RF	0.855 (0.838–0.873)	0.316	0.697 (0.663–0.728)	0.845 (0.828–0.861)	0.653 (0.621–0.686)	0.868 (0.852–0.884)
XGBoost	0.899 (0.884–0.914)	0.315	0.799 (0.769–0.826)	0.830 (0.812–0.847)	0.664 (0.633–0.694)	0.907 (0.893–0.921)
MLP	0.824 (0.805–0.843)	0.356	0.675 (0.641–0.707)	0.821 (0.803–0.838)	0.613 (0.580–0.646)	0.857 (0.840–0.873)
SVM	0.823 (0.804–0.842)	0.295	0.722 (0.689–0.753)	0.787 (0.768–0.806)	0.588 (0.557–0.619)	0.870 (0.853–0.886)
Simplified SVM	0.810 (0.790–0.830)	0.271	0.713 (0.680–0.744)	0.768 (0.748–0.787)	0.564 (0.532–0.595)	0.864 (0.846–0.880)
Models	External validation cohort
	AUC	Cutoff	Sensitivity	Specificity	PPV	NPV
LRM	0.739 (0.691–0.786)	0.293	0.634 (0.557–0.706)	0.733 (0.684–0.778)	0.529 (0.459–0.599)	0.809 (0.762–0.850)
RF	0.689 (0.639–0.739)	0.273	0.663 (0.587–0.733)	0.651(0.599–0.699)	0.473 (0.409–0.538)	0.803 (0.753–0.847)
XGBoost	0.716 (0.668–0.765)	0.246	0.622 (0.545–0.695)	0.738 (0.690–0.783)	0.530 (0.458–0.600)	0.805 (0.758–0.846)
MLP	0.731 (0.683–0.779)	0.323	0.610 (0.533–0.684)	0.755(0.707–0.798)	0.541 (0.468–0.613)	0.804 (0.757–0.844)
SVM	0.759 (0.713–0.805)	0.337	0.634 (0.557–0.706)	0.796 (0.751–0.836)	0.595 (0.521–0.667)	0.821 (0.777–0.860)
Simplified SVM	0.776 (0.731–0.821)	0.238	0.738 (0.666–0.802)	0.686 (0.635–0.733)	0.527 (0.462–0.591)	0.846 (0.801–0.886)

AUC: the area under the receiver operating characteristic curve; LRM: logistic regression model; MLP: multilayer perceptron; NPV: negative predictive value; PPV: positive predictive value; RF: random forest; SVM: support vector machine; XGBoost: extreme gradient boosting.

Additional file 1. The optimal hyperparameters for five models based on the ten-fold cross-validation.

Models	Hyperparameters	Settings
LRM	C	0.248
	max_iter	100000
	solver	liblinear
RF	max_depth	9
	max_features	10
	min_samples_leaf	8
	min_samples_split	4
XGBoost	gamma	0.2
	max_depth	4
	min_child_weight	6
	max_delta_step	0
MLP	kernel_initializer	uniform
	activation	hard_sigmoid
SVM	C	1000
	gamma	0.0001
	kernel	rbf
Simplified SVM	C	100
	gamma	0.001
	kernel	rbf

LRM: logistic regression model; MLP: multilayer perceptron; RF: random forest; SVM: support vector machine; XGBoost: extreme gradient boosting.

Figure 4. (a) The distribution of the impacts of features on the SVM model. (b) The ranking of features importance. DBP: diastolic blood pressure; GCS: Glasgow coma scale; MAP: mean artery pressure; MV: mechanical ventilation; SBP: systolic blood pressure; SpO2: peripheral oxygen saturation; SOFA: sequential organ failure assessment.

Figure 5. The partial dependence plots of (a) Delta SOFA, (b) GCS, (c) Delta GCS, (d) Delta SpO2, (e) SOFA, (f) Delta SBP, (g) MAP, (h) Delta MAP, (i) Age, (j) SBP, (k) Respiratory rate, and (l) Heart rate. GCS: Glasgow coma scale; MAP: mean artery pressure; SBP: systolic blood pressure; SOFA: sequential organ failure assessment; SpO2: peripheral oxygen saturation.

Data availability statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.