Dynamic prediction of hypoxemia risk at different time points based on preoperative and intraoperative features: machine learning applications in outpatients undergoing esophagogastroduodenoscopy

Figures & data

Figure 1. Flow chart of patient inclusion and exclusion.

Figure 2. The PRC curve (a) and ROC curve (b) of 4 final models in the internal testing set. AUPRC, the area under the precision-recall curve; AUROC, the area under the receiver operating characteristic curve; LR, logistic regression; SVM, support vector machine; XGBoost, extreme gradient boosting; Type A, the early prediction model of intraoperative hypoxemia; Type B, the late prediction model of intraoperative hypoxemia; Type C, the early prediction model of induction period hypoxemia; Type D, the late prediction model of induction period hypoxemia. The single file for each plot can be found in the figure details of the supplemental materials.

Table 1. The performance of the 4 final models under the optimal threshold in the internal testing set.

Model	Threshold	Dataset	Accuracy	Sensitivity	Specificity	F1 Score	MCC	Brier Score
Type A (XGBoost)	0.140	Training	87.6%	96.7%	86.6%	0.602	0.60	0.051
		Testing	86.6%	90.9%	86.2%	0.563	0.55	0.058
Type B (SVM)	0.129	Training	87.8%	93.3%	87.2%	0.598	0.59	0.051
		Testing	88.4%	95.5%	87.6%	0.609	0.61	0.052
Type C (XGBoost)	0.083	Training	84.6%	100.0%	83.1%	0.534	0.55	0.051
		Testing	82.8%	95.0%	81.6%	0.487	0.50	0.056
Type D (LR)	0.088	Training	83.9%	98.8%	82.5%	0.521	0.54	0.052
		Testing	83.2%	95.0%	82.1%	0.494	0.50	0.050

LR: logical regression; SVM: support vector machine; XGBoost: extreme gradient boosting; MCC: Matthews correlation coefficient; Type A: early prediction model of intraoperative hypoxemia; Type B: late prediction model of intraoperative hypoxemia; Type C: early prediction model of induction period hypoxemia; Type D: late prediction model of induction period hypoxemia.

Table 2. Comparison of the performance between the prediction model and STOP-BANG score for predicting intraoperative hypoxemia.

Model	Internal Testing Cohort
	AUPRC	Bootstrap CI	AUROC	P-value
STOP-BANG	0.374 (0.201-0.586)	–	0.850 (0.770-0.931)	–
Type A (XGBoost)	0.583 (0.323-0.718)	0.209 (0.136-0.313) *	0.938 (0.895-0.970)	0.0139*
Type B (SVM)	0.650 (0.436-0.817)	0.276 (0.198-0.366) *	0.952 (0.924-0.979)	0.0051*

XGBoost: extreme gradient boosting; SVM: support vector machine; AUPRC: the area under the precision recall curve; Bootstrap CI: the values outside the parentheses represent the difference between AUPRCs (bootstrap confidence interval), and if the confidence interval does not include 0, the difference between AUPRCs is considered to be significant (p-value < 0.05); AUROC: the area under the receiver operating characteristic curve; P-value: the p-values of AUROC difference between prediction models and STOP-BANG score; *: P-Value < 0.05; Type A: the early prediction model of intraoperative hypoxemia; Type B: the late prediction model of intraoperative hypoxemia.

Table 3. Comparison of the performance between the prediction model and STOP-BANG score for predicting induction period hypoxemia.

Model	Internal Testing Cohort
	AUPRC	Bootstrap CI	AUROC	P-value
STOP-BANG	0.319 (0.155–0.545)	–	0.833 (0.747–0.920)	–
Type C (XGBoost)	0.525 (0.315–0.727)	0.207 (0.125–0.323)*	0.933 (0.898–0.969)	0.0085*
Type D (LR)	0.617 (0.395–0.799)	0.298 (0.220–0.401)*	0.947 (0.914–0.981)	0.0013*

XGBoost: extreme gradient boosting; LR: logical regression; AUPRC: the area under the precision-recall curve; Bootstrap CI: the values outside the parentheses represent the difference between AUPRCs (bootstrap confidence interval), and if the confidence interval does not include 0, the difference between AUPRCs is considered to be significant (P-value < 0.05); AUROC: the area under the receiver operating characteristic curve; P-value: the P-values of AUROC difference between prediction models and STOP-BANG score; *: P-value < 0.05; Type C: early prediction model of induction period hypoxemia; Type D: late prediction model of induction period hypoxemia.

Figure 3. SHAP summary plot of the 4 types of the final model. (1) SHAP bar plot for the ranking of variables’ importance. The sorted result was based on the mean absolute SHAP value of each variable in every Type of final model. (2) SHAP beeswarm plot for the entire distribution of SHAP values for each variable in every Type of final model. Each line represents a variable and each dot represents a case. The redder the color of the dot, the higher the value of the variable for the case. Conversely, the bluer the color of the dot, the lower the value of the variable for the case. The abscissa represents the SHAP value, where a positive value helps the model to predict the case of developing hypoxemia, and vice versa. Type A, the early prediction model of intraoperative hypoxemia; Type B, the late prediction model of intraoperative hypoxemia; Type C, the early prediction model of induction period hypoxemia; Type D, the late prediction model of induction period hypoxemia; ASA, American Society of Anesthesiologists; BMI, Body mass index; GIT, Gastroscopy intubating times; RHSMD, Height-to-sternomental distance; RHTMD, Height-to-thyromental distance; SMD, Sternomental distance; TMD, Thyromental distance; TMH, Thyromental height. The single file for each plot can be found in the figure details of the supplemental materials.

Figure 4. SHAP force plot for individual case explanation on three random patients from the testing cohort of Type A model (SVM). Red represents that the variable increases the chance that the model predicts the patient developing hypoxemia, while blue represents that the variable reduces the chance that the model predicts the patient developing hypoxemia. The figure after the equal sign of the variable name represents the value of the variable (not the value of the SHAP value), and the length of the color bar represents the amount of contribution. (a) true positive case. (b) true negative case. (c) False negative case. Type A, the early prediction model of intraoperative hypoxemia.

Data availability statement

The data that support the findings of this study are available from the corresponding author, [Yanna Si], upon reasonable request. The related code to the study is available at https://github.com/zdznj/hypoxemia.