Significance of platelets in the early warning of new-onset AKI in the ICU by using supervise learning: a retrospective analysis

Abstract

Objective

To explore a machine learning model for the early prediction of acute kidney injury (AKI) and to screen the related factors affecting new-onset AKI in the ICU.

Methods

A retrospective analysis was performed used the MIMIC-III data source. New onset of AKI defined based on the serum creatinine changed. We included 19 variables for AKI assessment using four machine learning models: support vector machines, logistic regression, and random forest. and XGBoost, using accuracy, specificity, precision, recall, F1 score, and AUROC (area under the ROC curve) to evaluate model performance. The four models predicted new-onset AKI 3–6–9–12 h ahead. The SHapley Additive exPlanation (SHAP) value is used to evaluate the feature importance of the model.

Results

We finally respectively extracted 1130 AKI patients and non-AKI patients from the MIMIC-III database. With the extension of the early warning time, the prediction performance of each model showed a downward trend, but the relative performance was consistent. The prediction performance comparison of the four models showed that the XGBoost model performed the best in all evaluation indicators in all the time point at new-onset AKI 3–6–9–12 h ahead (accuracy 0.809 vs 0.78 vs 0.744 vs 0.741, specificity 0.856 vs 0.826 vs 0.797 vs 0.787, precision 0.842 vs 0.81 vs 0.775 vs 0.766, recall 0.759 vs 0.734 vs 0.692 vs 0.694, Fl score 0.799 vs 0.769 vs 0.731 vs 0.729, AUROC 0.892 vs 0.857 vs 0.827 vs 0.818). In the prediction of AKI 6, 9 and 12 h ahead, the importance of creatinine, platelets, and height was the most important based on SHapley.

Conclusions

The machine learning model described in this study can predict AKI 3–6–9–12 h before the new-onset of AKI in ICU. In particular, platelet plays an important role.

Key message

• The new-onset of AKI in ICU is a common and important problem, which early be identified the risk of AKI can improve patients’ outcomes.

• We explored MIMIC-III and determined the exact time point of occurrence of AKI as the basis for the new-onset of AKI in ICU.

• XGBoost model performed the best prediction in all the time point at new-onset AKI 3–6–9–12 h ahead.

• For patients with the new-onset of AKI in ICU, platelets become an important factor associated with AKI.

Keywords:

• Acute kidney injury
• critical care
• machine learning
• predictive model
• influencing factors
• platelet

Introduction

Acute kidney injury (AKI) is a complex syndrome with multiple causes and multiple clinical manifestations. The incidence of AKI is high; approximately, AKI occurs in 22–57% of ICU patients [1,2]. In addition, AKI can cause serious complications and even death. It also leads to a longer hospital stay, which in turn increases the cost of medical care to the family and society and creates a heavy burden [3]. If AKI is not prevented or controlled in time, it can also lead to serious kidney problems, such as chronic kidney disease and the development of dependence on dialysis [4]. Unfortunately, despite extensive research, there is currently no effective treatment to promote or accelerate kidney recovery. Therefore, the current methods are dedicated to identifying high-risk populations of AKI, identifying AKI at the early stage, and intervening early before it progresses to more severe stages. Previous diagnosis of AKI and assessment of renal function were mainly based on serum creatinine, cystatin C, urine output, and urinary deposits [5]. The diagnosis is neither sensitive nor specific. When creatinine is elevated, the kidneys often have substantial damage [6]. The diagnosis of AKI based on the increase in Scr and the decrease in urine volume alone can delay the diagnosis of renal injury for several hours to several days, resulting in irreversible renal injury or increasing the mortality of patients. There is no good correlation between the severity of clinical AKI using creatinine-based criteria and the severity of histological renal tubular injury [7]. Studies have shown that, if timely prediction is performed, up to 30% of hospital-acquired AKI is preventable [8]. If diagnosed and managed in a timely manner, AKI may also be reversible. Therefore, early diagnosis, prediction, and intervention can delay the progression of AKI to the severe stage and even improve the adverse outcomes of patients.

Studies have shown that because AKI is a heterogeneous syndrome, its etiology, pathophysiology and clinical manifestations are different and highly complex [9]. Pathological, physiological, and immune responses alone cannot effectively and timely diagnose AKI. In the past decade or so, many AKI early warning studies have been carried out, and various biological markers have emerged, but few can be effectively applied in clinical practice [10]. Combining clinical characteristics and biochemical markers, we found that data mining and machine learning (ML) methods may be applied to the diagnosis and early warning of AKI. Therefore, methods based on artificial intelligence and machine learning can be used as diagnostic tools or to predict prognosis. At present, with the improvement and progress of electronic medical record information systems, many researchers try to use the clinical data of patients to develop predictive models to solve this bottleneck problem [11]. Le et al. developed a convolutional neural network prediction system that extracts patient data from the electronic medical record system to predict AKI 48 h before the onset of disease [12]. Malhotra et al. developed and validated risk scores through logistic regression to predict the occurrence of AKI in the ICU environment [13]. In this study, we used the Medical Information Mart for Intensive Care III (MIMIC-III) to explore the relevant models for the early prediction of AKI and to screen the relevant factors that affect the occurrence of new-onset AKI in the ICU.

Method

Study design

This study was a retrospective study used MIMIC III database. This study aimed at ICU patients and predicted the occurrence of AKI 3/6/9/12 h in advance. For patients with multiple occurrences of AKI, only the first occurrence of AKI was predicted. The definition of AKI used in this study was as follows: two serum creatinine (Scr) measurements were performed within 48 h, and the measured value increased by ≥0.3 mg/dL (≥26.5 μmol/l) or increased by ≥1.5 times [14].

Considering that the new onset of AKI in the ICU mainly occurred within 96 h after entering the ICU, only the data of 0–96 h after entering the ICU were extracted. ICU patients without AKI were used as the control cohort. The 96-h block is divided into 32 blocks according to the 3-h block, and the average value of the relevant indicators in the time period is calculated for each block.

Patients with new-onset AKI in the ICU: The time of AKI occurrence (i.e., the time of the second serum creatinine measurement) was set as time 0, and the dynamic feature data were extracted from time 0 forward.

Control cohort: The time of entering the ICU was taken as time 0, the dynamic feature data were extracted from time 0 onward, and the data every 3 h were averaged until 96 h.

As shown in Figure 1, after 12 h of entry into the ICU, the number of patients with AKI was summed up every 3 h, and the number of patients with AKI (94/1130) was the highest 15 to 18 h after entering the ICU. Based on this, for the control cohort that did not have AKI, we predicted that AKI would not occur 15 to 18 h after entering the ICU 3/6/9/12 h in advance.

Figure 1. Time distribution of AKI occurrence. From the time of entering the ICU for 12 h, the number of patients with AKI was summed every 3 h to obtain the time distribution of AKI (unit: person).

The specific prediction design is as follows:

1. For the prediction of 3 h ahead (the time interval of feature value is 3 h, the model response time is 0 h):

   Patients with AKI: Data of −3 to 0 h were used to predict the occurrence of AKI at time 0.

   Non-AKI patients: Data from 12 to 15 h were used to predict the occurrence of AKI in 15 to 18 h patients.

2. For the 6-h advance prediction (the feature value time interval is 3 h, and the model response time is 3 h ahead):

   Patients with AKI: The data of −6 to −3 h were used to predict the occurrence of AKI at time 0.

   Non-AKI patients: Data from 9 to 12 h were used to predict the occurrence of AKI in 15 to 18 h patients.

3. For the prediction 9 h ahead (the time interval of feature value is 3 h, the model response time is 6 h ahead):

   Patients with AKI: The data of −9 to −6 h were used to predict the occurrence of AKI at time 0.

   Non-AKI patients: Data from 6 to 9 h were used to predict the occurrence of AKI in 15 to 18 h patients.

4. For the 12-h advance prediction (the feature value time interval is 3 h, and the model response time is 9 h ahead):

   Patients with AKI: The data of −12 to −9 h were used to predict the occurrence of AKI at time 0.

   Non-AKI patients: Data from 3 to 6 h were used to predict the occurrence of AKI in 15 to 18 h patients.

The above design can fully ensure that the patient is newly diagnosed with AKI in the ICU, and the department will provide early warning in advance.

Data source

The data of this study came from the Medical Information Mart for Intensive Care III (MIMIC-III). It is a free large-scale database that contains data related to more than 40,000 patients in the intensive care unit of the Beth Israel Deaconess Medical Center between 2001 and 2012. The database includes demographic information, bedside vital sign data (approximately one data point per hour), laboratory test results, medical measures, medications, nurses’ records, imaging reports, and mortality (inside and outside the hospital) information [15]. In this study, data were screened from the MIMIC III database.

Study cohort and variable selection

Patients who were hospitalized for the first time and entered the ICU for the first time in the MIMIC III dataset were selected, and those who were younger than 18, stayed in the ICU for less than 24 h, had creatinine measured less than two times in the first 96 h of the ICU and were in the first 48 h in the ICU were excluded. Patients with a creatinine level ≥ 1.2 mg/dL at the first measurement, patients with a history of kidney disease/renal obstruction/ESRD in the preliminary diagnosis, patients with renal replacement therapy (CRRT) within 24 h after entering the ICU, and patients with the first diagnosis of AKI who have already left the ICU and entered the ICU for less than 12 h have developed AKI. Finally, 6885 patients were enrolled, including 1130 AKI patients and 5755 non-AKI patients (Figure 1). To balance the positive and negative samples, 1130 non-AKI patients were randomly selected based on the demographics. The final model included 2260 patients, including 1130 patients with AKI and 1130 patients without AKI.

We performed tests for the interaction between variables and multicollinearity in multivariate models. Parameters were excluded for collinearity. Finally, the variables included in the model were sex, height, weight, BMI, age, body temperature, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, blood oxygen saturation, white blood cell count, hemoglobin, hematocrit, platelet count, blood creatinine, blood urea nitrogen, bicarbonate and potassium. Among them, sex, height, weight, BMI, and age were static characteristics, and the others were dynamic characteristics.

Data preprocessing

The raw data were mainly processed according to the following steps:

(1) Gender is coded as male 0, female 1. (2) For the static feature data (gender, height, weight, age), we used the average of all patients to impute the missing values and then calculated the BMI. (3) For the dynamic feature data, if the data were measured at least once, we used linear interpolation so that the data were counted every 3 h; if the data were not measured once, we used the average of all patients to impute the missing values. (4) The data were subjected to min-max standardization.

Model establishment and validation

In this study, we used support vector machine, logistic regression, and random forest [16], XGBoost [17]. The four machine learning models were trained and tested on the data, and the effects of the models were compared. We used accuracy, specificity, precision, recall, F1 score, and AUROC (area under the ROC curve). These six evaluation indicators are used to evaluate the performance of the model. Accuracy refers to the proportion of samples with correct predictions to the total samples; precision refers to the proportion of samples with positive predictions that are actually positive; recall refers to the proportion of samples with positive predictions that are actually positive; specificity F1 score = (2 * precision rate * recall rate)/(accuracy rate + recall rate); the ROC (receiver operating characteristic) curve is usually used to identify one sample. The degree of model prediction, AUROC (Area Under ROC Curve), is defined as the area under the ROC curve. In general, the larger the AUROC is, the better the model effect. We randomly divided the data into the training set and the test set at a ratio of 4:1 while maintaining the ratio of positive to negative samples at 1:1. Fivefold cross-validation was used.

Results

Cohort screening

A total of 61,532 hospitalizations were conducted in the MIMIC III dataset. According to the exclusion and inclusion criteria, 6885 patients were eventually included, including 1130 AKI patients and 5755 non-AKI patients (Figure 2). To balance the positive and negative samples, 1130 non-AKI patients were randomly selected. The final model included 2260 patients, including 1130 patients with AKI and 1130 patients without AKI.

Figure 2. Flowchart of cohort screening.

Static feature statistics

As shown in Table 1, the four characteristic variables used in the study were simply statistically analyzed: sex, height, weight, body mass index (BMI), and age. Among the four variables, there was no significant difference between the new-onset AKI patients and the control cohort.

Table 1. Statistics of static characteristic data.

Static features	AKI	Non-AKI
N = 6885	1130	5755
Gender, n (%)
Male	525 (46.5)	2813 (48.9)
Female	605 (53.5)	2942 (51.1)
Height (m), mean (SD)	1.66 (0.11)	1.68 (0.11)
Weight (kg), mean (SD)	74.54 (20.64)	74.57 (22.90)
Age (year), mean (SD)	69.46 (14.58)	69.43 (15.45)

New-onset AKI prediction results

The results of the four models for predicting new-onset AKI at 3/6/9/12 h in advance are shown in Table 2. It can be seen that the prediction performance of all models decreases as the lead time increases.

Table 2. Prediction results of four machine learning models.

	Accuracy	Specificity	Precision	Recall	F1 score	AUROC	95% CI
a. 3 h in advance:
Support vector machines	0.750	0.802	0.782	0.699	0.736	0.811	0.711–0.884
Logistic regression	0.744	0.783	0.765	0.705	0.733	0.793	0.624–0.955
Random forest	0.778	0.853	0.828	0.702	0.760	0.857	0.791–0.921
XGBoost	0.809	0.856	0.842	0.759	0.799	0.892	0.74–0.997
b. 6 h in advance:
Support vector machines	0.710	0.779	0.745	0.641	0.687	0.766	0.644–0.909
Logistic regression	0.694	0.727	0.709	0.662	0.683	0.741	0.604–0.808
Random forest	0.748	0.822	0.791	0.675	0.728	0.827	0.725–0.915
XGBoost	0.780	0.826	0.810	0.734	0.769	0.857	0.798–0.938
c. 9 h in advance:
Support vector machines	0.675	0.747	0.706	0.606	0.651	0.736	0.656–0.833
Logistic regression	0.661	0.689	0.672	0.631	0.651	0.704	0.495–0.763
Random forest	0.722	0.790	0.756	0.653	0.701	0.801	0.7–0.913
XGBoost	0.744	0.797	0.775	0.692	0.731	0.827	0.716–0.931
d. 12 h in advance:
Support vector machines	0.661	0.739	0.693	0.585	0.632	0.717	0.586–0.858
Logistic regression	0.640	0.660	0.646	0.621	0.633	0.676	0.595–0.797
Random forest	0.709	0.768	0.737	0.650	0.689	0.783	0.695–0.875
XGBoost	0.741	0.787	0.766	0.694	0.729	0.818	0.705–0.853

In the 3-h early-stage model performance, XGBoost performed the best among all evaluation indicators (accuracy 0.809, specificity 0.856, precision 0.842, recall 0.759, Fl score 0.799, and AUROC 0.892). This was followed by random forest (accuracy 0.778, specificity 0.853, precision 0.828, recall 0.702, Fl score 0.760, AUROC 0.857). SVM is superior to logistic regression in terms of accuracy, specificity, precision, and AUROC, while the performance of the two is similar in terms of the Fl score. In the 6/9/12 h advance stage, the relative trends of the prediction performance of each model were consistent. The XGBoost model performed optimally in all evaluation indicators.

Model interpretability and important features analysis

We used the SHapley Additive exPlanation (SHAP) value to evaluate the feature importance of the XGBoost model (Figure 3). The results showed that the importance of creatinine, platelets, and height in the 6/9/12 h advance prediction were all ranked in the top three.

Figure 3. Feature importance analysis of the XGBoost model. (a) Prediction 3 h ahead. (b) Prediction 6 h ahead. (c) Prediction 9 h ahead. (d) Prediction 12 h ahead.

Discussion

At present, many studies have attempted to develop early warning models for the occurrence of AKI, but the truly widely used clinical prediction method is mainly clinical scoring [18]. In addition, most patients underwent renal replacement, but not all ICU patients, which would lead to a small sample size in the study. In addition, most of these scoring systems are validated in the same center, which lacks high-level discrimination or calibration and often cannot be widely applied [19].

In clinical practice, we speculate that some factors may be related to AKI, and many studies have attempted to include different variables for the establishment of early warning models. A research team found that an early warning model was established for community-acquired acute kidney injury. They included 55 indicators for modeling. Finally, they found serum creatinine, urea, age, glomerular filtration rate, serum calcium, serum phosphorus, and use. The 10 variables, i.e., diuretics, diabetes, severe liver insufficiency, and chronic renal insufficiency, are important features for the establishment of an early warning system [20]. Yang et al. found that four factors, including mean arterial pressure, serum albumin, uric acid, and lymphocyte count, were independently associated with the development of AKI in patients with minimal disease status [21]. For high-risk AKI patients with volume responsiveness, urinary creatinine, blood urea nitrogen (BUN), age, and albumin are important predictors for identifying these patients [22]. Bhatraju et al. included biomarkers that reflect endothelial dysfunction, such as angiopoietin-1, angiopoietin-2, and soluble vascular cell adhesion molecules, and biomarkers that reflect inflammation, such as soluble TNF receptor-1. sTNFR-1], IL-6, IL-8, and soluble Fas were combined with clinical data to establish the model. The results showed that age, liver cirrhosis, and soluble TNF receptor-1 levels had good predictive performance [23].

Our study used the indicators in the MIMIC III database and used advanced machine learning techniques to identify and confirm the important clinical factors associated with AKI and explore the relevant models for the early prediction of AKI. Early warning. In our study, sex, height, weight, BMI, age, body temperature, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, blood oxygen saturation, white blood cell count, hemoglobin, hematocrit, platelet count, blood creatinine, and blood urea nitrogen, bicarbonate, and serum potassium were used to establish a clinical warning model. We use the SHapley Additive exPlanation (SHAP) value to evaluate the importance of each feature of the XGBoost model. Finally, we found that the top three indicators that contribute the most to the prediction of AKI are serum creatinine (Scr), platelets, and height. The KDIGO guidelines for the improvement of the prognosis of kidney disease proposed AKI is defined as follows: Scr increased by ≥ 0.3 mg/dl (≥26.5 µmol/l) or increased to ≥ 1.5 times within 48 h and/or urine volume <0.5 mL/(kg·h) for more than 6 h. Kang et al. [24] suggested that slight changes in creatinine (Scr) were associated with poor prognosis. The results of this study showed that the Scr level was the most influential among all physiological measurement indicators, and the Scr level was one of the most important indicators in the prediction of AKI. AKI is characterized by a rapid decrease in the glomerular filtration rate, which suggests that renal hemodynamics are impaired for various reasons. Both ischemia–reperfusion injury and systemic inflammation can lead to changes in renal macrocirculation and microcirculation, leading to hypoxic injury and upregulation of inflammatory responses. Platelets are a type of nonnucleated cell and are present in large quantities in the circulation. It is well known that platelets play an important role in hemostasis and coagulation. A number of studies have shown that platelets can release a large amount of bioactive mediators. In addition, their role in the immune system has also attracted attention. Under physiological conditions, the endothelium remains physically and biochemically intact, and platelets remain stable in the circulation. When the endothelium is damaged, platelets release cytokines, recruit leukocytes, interact with bacteria and endothelial cells, and promote microthrombus formation [25]. Microthrombosis may produce persistent microvascular occlusion, and microvascular occlusion and leukocytes and activated platelets may be the key to the pathogenesis of AKI [26]. Platelets carry microbubbles and three different types of granules: α granules, dense granules, and lysosomes. Singbartl et al. found that p-selectin stored in the α granules of platelets and endothelial cells was involved in the recruitment of septic renal leukocytes, and blocking p-selectin protected mice by reducing the entry of neutrophils into the kidney [27]. After being activated by proinflammatory cytokines, platelets may also trigger the complement system to mediate endothelial cell inflammation and interact with leukocytes to promote inflammatory responses, thereby promoting the progression of AKI [28]. In addition, this study used the ICU population to construct the algorithm, and the role of platelets in this population may be more critical. These may be the reason and mechanism of the use of machine learning in this study to find that platelets can become an important factor for the occurrence of AKI. In Figure 3, height importance ranks third in the prediction of AKI. However, in fact, due to the large lack of height and the difference in the proportion of missing AKI and non-AKI, it is very likely that the height can be caused by using the average to check the height. The importance of features is overestimated.

In our study, by comparing the prediction performance of the four models, the results show that XGBoost performs best in AKI prediction. XGBoost is an ensemble machine learning model that contains multiple decision trees, which provides better predictions by converting a group of weak learners into strong learners. It can estimate the importance of features by considering the contribution of specific features of each tree in the learning process. This feature of XGBoost has great clinical significance because it indicates that the model in this study can be used to prospectively support the clinical decision-making system in real time to rapidly screen and identify patients at risk of AKI in the ICU. At present, relevant studies on prediction models have confirmed that XGBoost is significantly better than traditional regression prediction models and has better prediction performance [20,22].

Of course, this study also has shortcomings. First, the MIMIC-III database is a database based on the US population. Therefore, the applicability of its predictive effect in other population cohorts still needs to be verified. Second, the time segmentation of AKI prediction in this study was designed based on the idea of trying to diagnose AKI as early as possible, and it needs to be confirmed by sufficient clinical validation. Third, the basal creatinine value was considered at the time of ICU admission, not considering the values obtained previously to the ICU admission or the Modification of Diet in Renal Disease (MDRD) formula. Moreover, patients with medical history or associated diseases, and related laboratory tests such as urine test and proteinuria assessment, eGFR, CRP, etc. were not included in the study. This is subject to the limitations of the database itself. Incorporating relevant indicators in future studies is of great significance for the recognition of AKI. In addition, the role of platelets in AKI in the ICU is currently not very clear. This study only suggests its important significance and role in the data, and more mechanistic explanations are needed.

Conclusions

This study used machine learning methods to establish a model for the early warning of AKI in ICU patients. The XGBoost modeling technique has good predictive performance and can identify creatinine changed. Platelets can also be used as a very important predictive factor in the development and progression of acute kidney injury. However, the mechanism of action of platelets in AKI is still not very clear. More basic research is needed to verify our results. Furthermore, the contribution of platelets to the pathology of AKI was further elucidated to provide prevention and treatment targets for avoiding the occurrence of AKI in the future. It may also improve the clinical outcome of AKI patients.

Author contributions

Substantial contributions to the conception or design of the work: L Su; the acquisition, analysis, or interpretation of data for the work: Y Liu, F Xie, Z Duan, L Li, H Gu; Drafting the wrote: P Pan, L Su; Revising it critically for important intellectual content: X Lu; Final approval of the version to be published: L Su and X Lu.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets in this study are available from the corresponding author on reasonable request.

Additional information

Funding

This work was supported by the Military Logistics Research Independent Research Project 2023; Chinese PLA General Hospital Youth Independent Innovation Research Project (22QNFC146); the key project of the Eighth Medical Center of the People's Liberation Army General Hospital (2021ZD001); the young talent promotion project of Beijing Science and Technology Association (BYESS2022035); Beijing Nova Program from the Beijing Municipal Science and Technology Commission (Z201100006820126).