A new model based on artificial intelligence to screening preterm birth

Abstract

Objective

The objective of this study is to create a new screening for spontaneous preterm birth (sPTB) based on artificial intelligence (AI).

Methods

This study included 524 singleton pregnancies from 18th to 24th-week gestation after transvaginal ultrasound cervical length (CL) analyzes for screening sPTB < 35 weeks. AI model was created based on the stacking-based ensemble learning method (SBELM) by the neural network, gathering CL < 25 mm, multivariate unadjusted logistic regression (LR), and the best AI algorithm. Receiver Operating Characteristics (ROC) curve to predict sPTB < 35 weeks and area under the curve (AUC), sensitivity, specificity, accuracy, predictive positive and negative values were performed to evaluate CL < 25 mm, LR, the best algorithms of AI and SBELM.

Results

The most relevant variables presented by LR were cervical funneling, index straight CL/internal angle inside the cervix (≤ 0.200), previous PTB < 37 weeks, previous curettage, no antibiotic treatment during pregnancy, and weight (≤ 58 kg), no smoking, and CL < 30.9 mm. Fixing 10% of false positive rate, CL < 25 mm and SBELM present, respectively: AUC of 0.318 and 0.808; sensitivity of 33.3% and 47,3%; specificity of 91.8 and 92.8%; positive predictive value of 23.1 and 32.7%; negative predictive value of 94.9 and 96.0%. This machine learning presented high statistical significance when compared to CL < 25 mm after T-test (p < .00001).

Conclusion

AI applied to clinical and ultrasonographic variables could be a viable option for screening of sPTB < 35 weeks, improving the performance of short cervix, with a low false-positive rate.

Keywords:

• Preterm birth
• cervical length
• transvaginal ultrasound
• artificial intelligence

Introduction

The prediction of preterm birth (PTB) plays a crucial role in addressing the significant mortality rates among children [1]. Currently, the most widely accepted predictor for PTB is a cervical measurement (CL) < 25 mm, which serves as the recognized cutoff value for positive screening in singleton pregnancies [2]. Obstetric history, particularly the number of prior preterm births occurring before 37 weeks or 34 weeks, can also serve as a screening method. However, despite the potential for improved screening performance through a combination of obstetric history, especially the number of previous preterm birth < 37 weeks or < 34 weeks, and cervical length, there is currently no readily available tool to facilitate such integration [3,4].

In most centers, patients in the mid-trimester of gestation were considered a positive screening if they had a short cervix (<25 mm) [2], or in case of previous PTB, with 10 percent false positive for each case separately [3]. According to Iams et al. [2], the sensitivity of the short cervix <25 mm was 37.3%. When compared to screenings for other pregnancy complications, such as preeclampsia [5], the current performance of screening for preterm birth, based solely on cervical length (CL), is significantly inferior.

Considering the global number of live births in 2014 (139.9 million), preterm birth (PTB) occurred in approximately 10.6% (14.84 million) of cases [6]. According to DATASUS data from 2019, Brazil had a PTB rate of 11.4% before 37 weeks of gestation (http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def). PTB is the leading cause of mortality among children under the age of 5 [7]. It is estimated that approximately 1.4 million deaths occur annually worldwide due to prematurity [8]. Consequently, interventions to reduce PTB are of utmost importance, and the utilization of novel technologies such as Artificial Intelligence (AI) may offer a viable alternative [9].

Over the last 10 years, the increase of data in modern society created big data. Artificial Intelligence has been extensively used to analyze it [10]. This process of machine learning may use different models to increase the performance of traditional approaches such as CL < 25 mm, as Logistic Regression, XGradient Boosting, Bagging Classifier, LDA, Artificial Neural Network, Adda Boost Classifier, staking ensemble learning model (SBLEM) and others [11,12]. Machine learning techniques have been increasingly applied in medicine and are regularly accepted in image recognition and other screening methods [13,14].

The main objective is to create a new screening for spontaneous preterm birth (sPTB) < 35 weeks, based on demographic, clinical, and ultrasonographic variables analyzed by AI.

Methods

This is a retrospective cohort analysis of 755 singleton pregnancies in the second trimester evaluated from October 2010 to August 2018 in the Sector for Prediction of Preterm Birth at the Federal University of São Paulo (UNIFESP). During this period, all patients between 18 to 24 weeks were offered, in addition to the anomaly scan, a transvaginal scan for the assessment of CL. This study was approved by the Ethics Committee (IRB) of The Federal University of Sao Paulo (Plataforma Brasil - http://plataformabrasil.saude.gov.br) number CAAE 30873613.8.0000.5505, on 3rd Sep 2014, with an amendment on 8th Aug 2020. All participants signed the consent form. Mechanical treatment for PTB was appreciated with the use of pessary and cervical cerclage. Cervical cerclage, McDonald technique, was systematically offered for typical cervical insufficiency [15].

Only cases of preterm rupture of ovular membranes and spontaneous onset of labor were included in the analysis. All non-spontaneous deliveries (n = 99) were excluded from this study, encompassing cases of labor induction (n = 62) and cesarean sections performed due to maternal or fetal complications (n = 37), such as gestational diabetes, preeclampsia, and fetal growth restriction (medically indication of delivery). Furthermore, mechanical treatments for PTB were also excluded (n = 22). Mechanical treatments referred to the utilization of pessary (n = 11) and cervical cerclage (n = 11). A total of 110 patients were lost to follow-up, resulting in 524 patients remaining for analysis (Figure 1). Pregnancy outcome data were obtained from medical records or the government’s database of live birth registrations (SINASC). For cases of preterm delivery < 35 weeks, the classification considered whether it occurred spontaneously or was medically indicated.

Figure 1. Flowchart of the included patients.

All patients included in this study were participants in international randomized trials focusing on the systematic prediction and prevention of preterm birth (PTB) [16–18]. Among them, those identified as high risk for PTB (defined as CL < 25 mm) were randomly assigned to receive either isolated progesterone treatment at a dosage of 200 mg/day or a combination of progesterone at 200 mg/day and a cervical pessary. However, patients who were randomized for the pessary intervention were subsequently excluded from the analysis, leaving only those who were randomized to receive progesterone. The administration of antibiotics followed a predefined protocol [19].

Discriminatory aspects of measurements of cervix

Cervical length was measured with a transvaginal (TV) probe, with an empty maternal bladder. An adequate view of the cervical gland area was obtained, low pressure over de cervix was applied and appropriate magnification was used. The cervical study consisted of three consecutive measures without abdominal compression of the uterine fundus and the other three measures after abdominal compression of the uterine fundus by at least 20 s, to simulate a uterine contraction. From the TV images, CL (A-B), cervical outer diameter (X-Z), a two-time measure (A-Y-B length) [17]; and the inside cervical angle (A-Y-B angle) were measured (Figure 1), this variable will be better explained on discussion section. Care was taken to exclude from the CL the isthmus (B-C), where there was no cervical mucosa [20]. Another measure applied as a variable in this study was an index straight CL/internal angle inside the cervix (index = A-B/A-Y-B angle). The smallest straight measure was considered as its respective A-Y-B measure and angle. Other measures which were performed were A-C, A-C two-times, A-C trace line, as presented in Figure 2. This study aims to expand the understanding of cervical characteristics by investigating new aspects and measurements not previously described in the literature, specifically focusing on the variables of index straight cervical length (CL) and internal angle within the cervix (index = A-B/A-Y-B angle). By exploring these variables, we seek to enhance the available information on the cervix and contribute to its comprehensive assessment.

Figure 2. Image representative of the cervix to demonstrate the principal measurements performed during TVS in our unit.

Selection of variables

In the whole study, 33 variables were evaluated and could be compared. After the determination of variables, the missing data were evaluated. More than 30% of missing data were criteria of exclusion of the variable and all possible correction on the description of data was performed.

The ultrasound was preceded by a detailed appointment to collect all relevant data for screening sPTB < 35 weeks as a history of preterm or term birth, and the gestational age when PTB occurs, cervical surgery or known uterine malformation, history for cervical incompetence, habits, addictions (smoking or drugs), medications (including antibiotics and progesterone), history of vaginal bleeding, bacterial infection during the current pregnancy, and most common demographic characteristics [as age, weight, height, body mass index (BMI), ethnics, schooling, number of gestations, parity, miscarriage, and curettage]. A variable “Any Risk Factor to PTB” was created including, in the same variable, patients with previous PTB, cervical surgery, uterine malformation, or cervical incompetence.

First of all, the dummies variables creation was performed, i.e. all categorical variables were modified into a numerical variable that takes on one of two values: zero or one. Subsequently, the vectorization and the dummies were created by odds ratio (OR) pointing out sPTB < 35 weeks, and finally, the dummies were reassembled by the similarity of the OR. For non-categorical variables, the variable was separated in decile scale and regroup by the similarity of the OR in each decile; and for categorical (zero or one) variables the OR was considered directly [21].

Cervical length as control group

All screening techniques applied during this study considered 524 patients. The CL < 25 mm is considered the gold standard for PTB screening and in terms of comparison can be considered a control group, thus the results achieved with AI techniques were compared with the ones established using the CL < 25 mm.

Criteria of selection of the variables

The inclusion of the variables and algorithms in the model obeys two predefined rules: 1) Forward selection based on logistic regression (LR) to select variables (it needs to have a significance of at least 0.05 to be elected by the model) [22]; 2) There was verified Pearson Correlation among variables and no elimination of variables was necessary because all variables were in low correlation (0.30 to 0.50; − 0.30 to − 0.50) or very low correlation (0.00 to 0.30; 0.00 to −0.30) [23].

Cross-validation method

The analysis was conducted using the cross-validation method, whereby 70% of the total number of cases were utilized for adjusting and training the techniques, while the remaining 30% constituted the Test Group for testing and validating all the models subsequent to the training phase [24].

Stepwise forward multivariate unadjusted logistic regression

A stepwise process is a mathematical approach used to select the most relevant variables for predicting spontaneous preterm birth (sPTB) occurring before 35 weeks of gestation [25]. In this study, the stepwise analysis involved the inclusion of all 33 variables in a multivariate analysis [26] and subsequently identified the statistically significant variables to compose the new predictive model. Once the most important variables were determined through logistic regression, their individual importance was further evaluated using SHAP analysis.

Artificial intelligence – evaluated algorithms (models)

The utilization of Artificial Intelligence (AI) involves inherent complexity, and this novel mathematical calculation introduces additional learning strategies, some of which may be regarded as “black box” approaches that offer limited insight into the underlying data structure [27].

Different algorithms of machine learning were applied in the same database to evaluate the performance in Receiver Operating Characteristics (ROC) curve (area under ROC curve - AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) by different techniques of Machine Learning (ML) to discriminate the target (sPTB < 35 weeks). Among different possibilities in AI models existing, the following algorithms were tested: X Gradient Boosting (XGBoost); Linear Discriminate Analysis (LDA); AdaBoost Classifier; Randon Forrest Classifier; Decision Tree [11,27,28]. The statistical analysis to choose the best algorithm was the highest number in the Kolmogorov-Smirnov test [29]. The statistical analysis to choose the best algorithm was the highest number in the Kolmogorov-Smirnov test in the 30% of the Test Group.

Stacking-based ensemble learning method

A combination of multiple models was employed to generate a unified score for the creation of the Stacking-Based Ensemble Learning Model (SBELM), utilizing a Neural Network approach. This method synergistically combines different models, leveraging the strengths of each, to achieve optimal performance [30]. Based on the calculated metrics obtained from logistic regression, a comparison between the best Machine Learning (ML) algorithm and CL < 25 mm was conducted, revealing distinct performance outcomes. The Kolmogorov-Smirnov test was employed to identify the ML algorithm that demonstrated the highest performance. To uphold the current screening method (CL < 25 mm), a composite performance measure combining CL, logistic regression, and the best-performing AI algorithm was chosen (Figure 3).

Figure 3. Schematic representation of SBELM, stacking different algorithms to achieve better performance to explain the event (sPTB < 35 weeks) stacked by Neural Network. The stacking was composed of CL < 25 mm, LR, and the best algorithm was obtained by machine learning.

Statistical analysis

The significance was established by p-value < .05, two-sided, and the confidence interval was established in 95% for logistic regression, IA models, and the comparison between ROC curves. The ROC curve and AUC were created for CL < 25 mm, LR, the principal algorithm of AI and SBELM. The significance of each model was tested comparing with CL < 25 mm and p-value was determined by the Shapiro-Wilk test and Student T-test, and it was considered statically significant when p < .05. Sensitivity, specificity, accuracy, predictive positive and negative values were performed with a false-positive rate fixed at 10% and 20%.

The process of collection of data, missing data validation, and Dummy vectorization was developed with Microsoft Excel for Mac (Microsoft 365) and SQL Management Studio (SQL - Structured Query Language). Statistical analysis of CL < 25 mm, LR, algorithms of AI, and SBELM required IBM SPSS Statistics, version 23.0 (SPSS Inc., Chicago, IL, USA) and Python (Anaconda) [31].

Results

Demographic characteristics

There was a total of 524 cases, 488 (93.1%) deliveries occurred ≥ 35 weeks and 36 (6.9%) deliveries < 35 weeks. Demographic characteristics were expressed in Table 1.

Table 1. Demographic characteristics of the population in the study and odds ratio.

Demographic Characteristics	Delivery ≥ 35 weeks – n (%)	Delivery < 35 weeks - n (%)	Odds Ratio
	(Total n = 488)	(Total n = 36)
Age (years)
Missing Data	7 (1.43)	0 (0)	–
<20	41 (8.40)	2 (5.56)	0.66
20-35	332 (68.03)	25 (69.44)	1.02
>35	108 (22.13)	9 (25.00)	1.13
Race
Caucasian	249 (51.02)	18 (50.00)	0.98
African	108 (22.13)	3 (8.33)	0.38
Latin	123 (25.20)	14 (38.89)	1.54
Asian	8 (1.64)	1 (2.7)	1.64
Height (cm)
Missing Data	2 (0.41)	0 (0)	–
145.00−152.00	54 (9.84)	6 (17)	1.72
152.01−181.00	468 (89.75)	30 (83)	0.92
Weight (Kg)
Missing data	1 (0.21)	0 (0.0)	–
39.00−58.00	109 (22.33)	9 (25.0)	1.12
58.10−64.00	94 (19.27)	5 (13.89)	0.72
64.10−124.00	284 (58.19)	22 (61.11)	1.05
BMI
Missing data	2 (0.41)	0 (0)	–
15.99−26.09	244 (50.00)	15 (41.67)	0.83
26.10−44.96	242 (49.59)	21 (58.33)	1.18
Gestation
1	109 (22.34)	7 (19.44)	0.87
2-3	229 (46.93)	12 (33.33)	0.71
>4	150 (30.74)	17 (47.22)	1.53
Parity
0	186 (38.11)	15 (41.67)	1.09
1	176 (36.07)	7 (19.44)	0.54
>2	126 (25.82)	14 (38.89)	1.51
Miscarriage
0	302 (61.89)	19 (52.78)	0.85
1	78 (15.98)	6 (16.67)	1.04
>2	108 (22.13)	11 (30.56)	1.38
Curettage
Missing data	209 (42.83)	19 (52.78)	1.23
0	224 (45.90)	14 (38.89)	0.85
1	51 (10.45)	2 (5.56)	0.53
>2	4 (0.82)	1 (2.78)	3.39
Smoking during pregnancy
No	459 (94.06)	32 (88.89)	0.94
Yes	29 (5.94)	4 (11.11)	1.87
Active vaginal bleeding until 2^nd trimester
No	403 (82.58)	27 (75.00)	0.91
Yes	85 (17.42)	9 (25.00)	1.43
Antibiotics use before inclusion
No	357 (73.16)	30 (83.33)	1.14
Yes	131 (26.84)	6 (16.67)	0.62
Progesterone use before Inclusion
No	424 (86.89)	25 (69.44)	0.80
Yes	64 (13.11)	11 (30.56)	2.33
Anterior Term Delivery
Missing data	5 (1.02)	0 (0.0)	–
No	251 (51.43)	22 (61.11)	1.19
Yes	232 (47.54)	14 (38.89)	0.82
Any risk factor to PTB
No	400 (81.97)	21 (58.33)	0.71
Yes	88 (18.03)	15 (41.67)	2.31
Cervical incompetence
No	487 (99.80)	31 (86.11)	0.86
Yes	1 (0.20)	5 (13.89)	69.45
Previous Preterm Birth < 37 weeks
No	416 (85.24)	24 (66.67)	0.78
Yes	72(14.76)	12 (33.33)	2.25
Previous Preterm Birth < 34 weeks
No	446 (91.39)	28 (77.78)	0.85
Yes	42 (8.61)	8 (22.22)	2.58
Previous Preterm Birth < 32 weeks
No	458 (93.85)	28 (77.78)	0.83
Yes	30 (6.15)	8 (22.22)	3.61
Previous Preterm Birth < 28 weeks
No	464 (95.08)	29 (80.55)	0.85
Yes	24 (4.92)	7 (19.45)	3.95
Uterine Malformation
No	486 (99.59)	35 (97.22)	0.98
Yes	2 (0.41)	1 (2.78)	6.78
Previous cervical surgery
No	484 (99.18)	36 (100.00)	1.00
Yes	4 (0.82)	0 (0)	–
Bacterial Infection during pregnancy
No	376 (77.05)	31 (86.11)	1.12
Yes	112 (22.95)	5 (13.89)	0.60
Cervical Length (CL)
<10 mm	8 (1.53)	6 (16.67)	10.89
10.1 to 20 mm	16 (3.05)	3 (8.33)	2.73
20.1 to 25 mm	21 (4.01)	3 (8.33)	2.08
25.1 to 30 mm	98 (18.70)	8 (22.22)	1.19
30.1 to 35 mm	186 (35.50)	7 (19.44)	0.55
>35 mm	195 (37.21)	9 (25.00)	0.67
Angle inside cervix (A-Y-B angle)
Missing data	244 (50.00)	24 (66.67)	1.33
78.09−147.42	139 (28.48)	3 (8.33)	0.29
147.43−180.00	105 (21.52)	9 (25.00)	1.16
Index CL / Angle inside cervix
Missing data	138 (28.28)	17 (47.22)	1.67
0.003−0.2005	78 (15.98)	11 (30.56)	1.91
0.2006−0.4330	272 (55.74)	8 (22.22)	0.40
Cervix Measure A_C
0.00−25.50	38 (7.79)	11 (30.56)	3.92
25.51−36.10	192 (39.34)	16 (44.44)	1.13
36.11−94.00	258 (52.87)	9 (25.00)	0.47
Cervix Measure B_C
0.00 (absent)	257 (52.66)	25 (69.44)	1.32
0.01−5.00	84 (17.21)	1 (2.78)	0.16
5.01−52.00	147 (30.12)	10 (27.78)	0.92
Trace line measure
0.00−27.00	42 (8.61)	12 (33.33)	3.87
27.10−37.60	237 (48.57)	19 (52.78)	1.09
37.70−73.00	209 (42.83)	5 (13.89)	0.32
Two-times cervix measure
Missing data	258 (52.87)	25 (69.44)	1.31
17.40−29.90	18 (3.69)	1 (2.78)	0.75
29.91−45.80	159 (32.58)	8 (22.22)	0.68
45.81−73.20	53 (10.86)	2 (5.56)	0.51
Presence of Funneling
No	475 (97.34)	27 (75.00)	0.77
Yes	13 (2.66)	9 (25.00)	9.40
Presence of AFS
No	293 (60.04)	13 (36.11)	0.60
Yes	195 (39.96)	23 (63.89)	1.60
Absence of CGA
No	476 (97.54)	30 (83.33)	0.85
Yes	12 (2.46)	6 (16.67)	6.78

Cervical length distribution and sPTB detection rate by the cervix

The ROC curve for CL < 25 mm has an AUC of 0.318 and it was considered the principal parameter of comparison for LR, Algorithms of AI, and SBELM. The sensitivity, specificity, accuracy, PPV and NPV obtained exclusively by CL < 25 mm for sPTB < 35 weeks (gold standard method) were, respectively, 33.3%, 91.8%, 87.8%, 23.1%, and 94.9%, keeping fixed a false positive rate of 10%.

Stepwise forward multivariate unadjusted logistic regression

A total of 8 variables were considered relevant to explain sPTB < 35 weeks based on 129 dummies, which were developed through a similar OR that discriminates higher risk for PTB. After the LR the AUC obtained was 0.749.

The results of LR were expressed in Table 2, considering Coef β, SD, Wald, P-value, OR, and confidence interval (CI) of 95%. The obtained variables were the presence of funneling, index A-B/A-Y-B angle (index ≤ 0.200), weight (≤ 58 kg), previous preterm delivery < 37 weeks, no antibiotic treatment during pregnancy, previous curettage, no-smoking, and CL (< 30.9 mm) (Table 2).

Table 2. Selected variables using logistic regression (LR).

Selected variables	Coef. Beta	S.D.	Wald	p-value	Odds Ratio	C.I. 95%
						Inferior	Superior
Presence of Funneling	1.787	0.804	4.934	.026	5.971	1.234	28.891
Index A-B/A-Y-B angle (≤ 0.200)	0.427	0.592	0.521	.470	1.533	0.481	4.887
No antibiotic treatment during preg.	0.530	0.508	1.088	.297	1.698	0.628	4.594
Previous Preterm Delivery ≤ 37w	1.579	0.547	8.336	.004	4.851	1.661	14.172
One Previous Curettage	−1.245	1.092	1.301	.254	0.288	0.034	2.447
Weight (≤ 58 kg)	0.699	0.519	1.810	.178	2.012	0.727	5.568
No Smoking	−0.949	0.656	2.095	.148	0.387	0.107	1.399
Cervical Length < 30,9 mm	0.622	0.560	1.237	266	1.863	0.622	5.580
Constant	−3.116	0.770	16.393	.000	0.044

The most important variables were selected according to LR, the P-value was expressed, and the value of Coef. ß is the number applied on the Logit function to discriminate the cases with a higher risk for PTB in this population.

After logistic regression determines the current variables, a test of correlation (described in Table 3) was performed to confirm the low correlation. Any variable selected has less than 50% of correlation, and in fact, the absolute majority have less than 10% of correlation.

Table 3. Correlation matrix.

Correlation Matrix	Cervical Funneling	Index A-B/ A-Y-B angle (≤ 0.200)	Previous PTB < 37 w	One Previous Curettage	Weight (≤ 58 kg)	No Smoking	No Antibiotic treatment during pregnancy	Cervical Length < 30,9 mm
Cervical Funneling	1	0.057	0.058	−0.039	−0.090	0.015	−0.091	−0.235
Index A-B/ A-Y-B angle (≤0.200)	0.057	1	0.091	0.034	0.048	−0.092	−0.098	−0.466
Previous PTB <37 w	0.058	0.091	1	0.017	−0.010	−0.083	−0.004	−0.079
One Previous Curettage	−0.039	0.034	0.017	1	0.031	0.009	0.139	−0.004
Weight (≤58 kg)	−0.090	0.048	−0.010	0.031	1	−0.030	−0.026	−0.054
No Smoking	0.015	−0.092	−0.083	0.009	−0.030	1	−0.013	0.016
No Antibiotic treatment during pregnancy.	−0.091	−0.098	−0.004	0.139	−0.026	−0.013	1	0.071
Cervical Length <30,9 mm	−0.235	−0.466	−0.079	−0.004	−0.054	0.016	0.071	1

The SHAP dependence plot was generated to elucidate the interplay among variables and their impact on the probability prediction of preterm birth (Figure 4).

Figure 4. The SHAP dependence plot to evaluate the interaction between variables selected by logistic regression.

Regarding logistic regression (LR), the sensitivity for preterm deliveries occurring before 35 weeks was determined to be 38.9%, with a predictive positive value of 26.9%. This corresponds to a false-positive rate of 10% (Table 5).

Artificial Intelligence

All variables were included in several algorithms of AI simulation. The models successively interacting with the variables after testing (70.8% of the cases – n = 371) and training (29.2% of the cases - n = 153), at the end of the process regularly improving the result of LR. This interaction could be measured by the Kolmogorov-Smirnov test (KS2) and the ROC curve could be obtained after the use of IA, and the KS2 best performance was obtained by X Gradient Boosting and all results were expressed in Table 4.

Table 4. – Different algorithms and respective Kolmogorov-Smirnov tests. The XGBoost achieved the best performance, with the higher Kolmogorov-Smirnov test index (KS test = 0.489899).

Kolmogorov Smirnov Training	Kolmogorov Smirnov Test	Algorithm
0.4575	0.4315	Logistic
0.4764	0.4899	XGBoost
0.7092	0.4646	LDA
0.7825	0.3384	AdaBoost
0.8928	0.3384	Bagging
0.9307	0.4394	Rforest
0.9342	0.3510	Dtree
0.5922	0.4293	Nbayes
0.8721	0.4621	Neural Network
0.9342	0.4040	KNN
0.9342	0.2424	ExtraTreeClassifier
0.9032	0.3712	BaggingClassifier
0.7029	0.1970	LGBMClassifier
0.7897	0.3914	KneighborsClassifier

The SHAP analysis was conducted using the variables selected by XGBoost, and the results were visually presented in Figure 5. The significance of each variable is depicted on the scale provided in the graphic.

Figure 5. The SHAP analysis for XGBoost demonstrates the importance of each variable.

Stacking-based ensemble learning model

The stacking-based ensemble learning model (SBELM), implemented with a neural network, exhibited the highest performance in terms of the area under the curve (AUC) compared to other techniques. SHAP analysis revealed the varying importance of each individual model in contributing to its overall performance. Figure 6 illustrates the contributions of CL< 25 mm, logistic regression (LR), and XGBoost, with the importance of these models presented as normalized values to facilitate comprehension.

Figure 6. SHAP analysis of the different models. Normalized importance of each algorithm which composed SBELM.

The neural network was created with one hidden layer with 3 (three) units (artificial neurons). One dependent variable in the output layer with 2 (two) units in the output layer, with SoftMax which is an activation function. The activation function of the hidden layer was Hyperbolic Tangent, and the error function was Cross-Entropy (Figure 7).

Figure 7. The Schematic representation of neural network with different models compound SBELM (left column) with hidden layers (Middle column) and output layers (right column). the thickness of the lines represents the importance of each algorithm in the respective hidden layer and the output layer.

The ROC curve analysis and AUC to predict the target was created for CL < 25 mm, LR, XGBoost, and SBELM (Figure 8), and statistical significance was calculated after comparison with CL < 25 mm by Student T-test. Sensitivity, specificity, accuracy, PPV, and NPV and AUC (ROC) considering 10% of false-positive rate, were expressed in Table 5.

Figure 8. Performance of ROC curve of the CL < 25 mm (green line), logistic regression (red line), X Gradient Boosting (blue line), and SBELM (purple line). considering the ROC curve area, the best-isolated model performed (SBELM) performed better than CL < 25 mm and achieved AUC = 0.808 for the detection of sPTB < 35 weeks.

Table 5. Considering different models to screening sPTB < 35 weeks, it Presents AUC (ROC) with p-value, sensitivity, specificity, accuracy, predictive positive value, and a predictive negative value when was fixed the false-positive rate (FPR) in 10%.

Fixed 10% FPR	AUC (ROC)	Sensitivity (%)	Specificity (%)	Accuracy (%)	Predictive Positive Value (%)	Predictive Negative Value (%)	p-value (AUC comparison)
CL < 25mm	0.318	33.3	91.8	87.8	23.1	94.9	–
LR	0.749	38.9	92.2	88.5	26.9	95.3	<.0001
XGBoosting	0.788	44.4	92.6	89.3	30.8	95.8	<.0001
SBELM	0.808	47.2	92.8	89.7	32.7	96.0	<.0001

Table 5 provides the most significant findings, showcasing a 10% false-positive rate. However, it is worth considering another perspective by hypothetically setting a fixed 20% false-positive rate for the stacking-based ensemble learning model (SBELM). In this scenario, the SBELM demonstrates superior values for sensitivity (75.0%), specificity (84.2%), accuracy (83.6%), positive predictive value (PPV) (26.0%), and negative predictive value (NPV) (97.0%) compared to sensitivity (41.5%), specificity (82.0%), accuracy (78.8%), PPV (16.3%), and NPV (94.3%) when considering values for short cervix < 28.5 mm, which represents similar 20% of FPR in terms of the cervical length. Notably, the increase in sensitivity and PPV is 60% greater with the utilization of the machine learning technique (SBELM), in addition to improvements observed in all other aspects.

A Student T-test for the LR, XGBoost, and SBELM comparing with CL < 25 mm were performed and the values of p were < .0001, for all algorithms. Predict pseudo-probability obtained by SBELM can be observed in Figure 9, it is important to notice that there was a reduced intersection between 0 and 1.

Figure 9. Predict pseudo-probability of preterm birth < 35 weeks, with small Points of intersection between 0 and 1.

Discussion

A significant change in data generation and technology consumption based on AI has been experienced. During 2019 and 2020, the amount of processed information was larger than all the production developed by mankind so far [32]. Health technologies have continuously been further developed since the beginning of medicine. Increasing knowledge in diagnostic, prevention, treatment, and rehabilitation has led to significant changes in healthcare systems [33]. This is a unique moment in using AI techniques, where the current computational power linked to data production allows us to use algorithms which in the past were not possible and provides better adjustment of statistical models and patients’ research-based ratings.

Observing the current AI moment, it is possible to identify this technology in several everyday applications, such as search engines, engines for a credit decision, the process of image identification, among others, which inside an application could provide an automated and efficient process offering a differentiated experience for mankind.

According to Gartner [10], the Data & Analytics area is increasingly gaining space in the market and is expected to grow even more in the forthcoming years. Until 2022, 90% of corporative strategies mention data as an essential asset for the results of public or private companies.

Main findings

Therefore, the objective of this application is to enhance the process of PTB risk assessment, which can have significant implications for the perinatal outcome. The current methods exhibit low accuracy, leading to delays in the development of therapeutic interventions.

The implementation of AI, specifically the SBELM, led to a notable improvement in the screening sensitivity for sPTB occurring before 35 weeks, increasing from 33.3% with the use of CL < 25 mm to 47.2% in our specific population. It is worth highlighting that these patients were referred to a tertiary center specifically for CL screening. This advancement in PTB screening holds significant implications and has the potential to pave the way for the development of novel therapeutic interventions. Moreover, this model can be extrapolated to a larger population of women as it relies solely on ultrasound, a widely accessible method, and a computer program. It is important to emphasize that the primary objective of this study was not to serve as a diagnostic tool.

New variables were created by this research to describe better the cervix and one of them was selected with relevance by the model, they could be evaluated in other studies.

Implication for research

Beta et al. [4] has created a similar screening method based on maternal characteristics, intending to associate serum markers, but without sonographic variables. The principal variables selected were previous PTB 34-36 weeks (OR 2.980 CI95% 1.887 − 4.704; p < .0001), smoking (OR 1.577 CI95% 1.143 − 2.174; p = .005 and assisted human reproduction (OR 1.722 CI95% 1.114 − 2.661; p = .014), these numbers were very similar to OR of this study.

Another study by Celik et al. [34] evaluated different scenarios to screen PTB using CL, associated with maternal demographic characteristics and obstetric history. Fixed 10% of the false-positive rate the detection rate for CL < 25 mm was 24.2% and for the combination CL < 25 mm, obstetric history and maternal characteristics was 29.3% for 34–36 weeks’ gestation. The comparison from this study and Celik et al. [34], regarding the similar target (gestational age 34-36 weeks), the similar false-positive rate (10%), and the similar analyzed variables (CL < 25 mm + maternal characteristics + obstetric history) could improve detection for sPTB < 35 weeks, achieving 47.2% after AI assessment and other variable assessment.

Future studies can be conducted based on the findings of this article, and it would be of great importance to develop multicenter studies utilizing the same strategy. This approach can lead to the development of models tailored to different populations, thereby increasing the sensitivity for PTB and ultimately resulting in improved therapeutic options.

This article holds relevance in the context of PTB screening. The combination of high sensitivity with a low false-positive rate can effectively discern the risk of PTB in a specific population. These findings can serve as the basis for the formulation of new guidelines, not only for different scenarios such as longer cervix and infections but also for exploring new applications of transvaginal ultrasound examinations at different stages of pregnancy.

Implication for clinicians

The selection of the optimal criteria for the utilization of this new method is feasible. In order to enhance sensitivity while maintaining the same false-positive rate for CL< 25 mm, a fixed false-positive rate of 10% was implemented and applied. However, it is worth considering the high values of sensitivity (75.0%), positive predictive value (PPV), negative predictive value (NPV), specificity, and accuracy achieved with a false-positive rate of 20%. This threshold of 20% false-positive rate, accompanied by increased sensitivity, could be considered a second evaluation between 26 to 28 weeks gestation.

The applicability of this calculator to the entire population is feasible. The improved performance of PTB screening, surpassing the sole use of cervical length, has the potential to provide therapeutic options that may not have been considered for certain individuals. Nonetheless, validation of these results in a low-risk population through randomized clinical trials or other retrospective studies utilizing a similar technique is imperative.

New modalities of AI were tested using exclusively these variables selected for this study and the performance of the Kolmogorov-Smirnov test was superior when using SBELM [35].

Reflection of other results

The group created for research in PTB developed a single form to measure de cervix in clinical appointments. An unusual measurement trying to explain PTB was created. Including the inside cervical angle (A-Y-B angle), index A-B/A-Y-B angle, B-C measure (isthmus), the A-C trace line, the A-C two straight measures, transverse measure, and anteroposterior measure (X-Z). New studies must be conceived to validate, or not, these findings with these new measures, as inside cervical angle (A-Y-B angle), index A-B/A-Y-B angle, A-C straight measure, and B-C measure which participated of logistic regression, and index A-B/A-Y-B angle was included from final variables selected by stepwise LR, which can demonstrate the importance of this new non-validated variable.

Over the years the CL < 25 mm was chosen for screening of PTB even despite the regular sensitivity to discriminate PTB around 37% [2]. Otherwise, 63% or almost 2/3 of PTB are not screened by short cervix. Consequently, new more accurate predictors would be welcome. So, the facility to screening patients with a single measure with regular accuracy is the most important factor to accept this mediocre sensitivity [3]. With AI, it is possible to consider screening PTB with better performance, without increasing the false-positive rate and achieving an acceptable performance for a screening.

Strengths and limitation

One of the strengths of this research lies in the innovative methodology based on SBELM, which resulted in the development of a novel AI algorithm specifically tailored for this study, in conjunction with logistic regression and superior AI algorithms for preterm birth prediction. While it may appear as a mere maneuver, it was a deliberate strategy aimed at optimizing the assessment of key factors associated with PTB risk.

Consequently, it becomes evident that cervical length is the most influential factor in PTB risk across the entire cohort, particularly in cases where CL is less than 20 mm. However, the question arises: Is CL equally significant in assessing PTB risk for pregnant women with CL measurements between 20 and 35 mm? Moreover, does CL hold any relevance in predicting PTB risk for patients with CL measurements exceeding 35 mm [34]? These aspects are explored in the context of this new SBELM approach, which enables the screening of patients at risk of PTB regardless of their CL measurements, with high sensitivity and specificity.

The utilization of SBELM offers a valuable opportunity to identify patients at risk of PTB beyond the reliance on CL alone, thereby enhancing the screening process with improved sensitivity and specificity [13].

These research results demonstrated that different characteristics could be involved in different CLs and this strategy to achieve better sensitivity without changes in false-positive rate or the number of positive and negative predictive values. This retrospective study is just the start of the entire process to increase the detection of PTB. After this initial phase, a prospective study with this new methodology will be performed to check the real efficiency of the algorithm SBELM.

Preterm birth (PTB) imposes a significant economic and social burden, accounting for approximately 80% of child mortality before the age of 5 [6]. In the United States alone, there are approximately 4.1 million preterm deliveries each year [36]. When considering the cost associated with each PTB occurring between 28 and 36 weeks, which is estimated at around $12,500 per case [37], the financial implications reach staggering figures, exceeding $51.2 billion annually.

Given the enormity of these numbers, it is crucial to urgently explore new and effective technologies for the detection of PTB. The current reliance on the measurement of cervical length alone, with a detection rate of approximately 37% [2], necessitates the pursuit of alternative screening methods. In this regard, this study demonstrates the potential of AI technology in achieving a detection rate of nearly 50%. It is important to note that while this improvement in sensitivity is promising, it does have limitations, particularly in terms of diagnostic capability. However, for the purpose of screening, this level of sensitivity is considered acceptable.

Efforts to enhance PTB detection and screening are vital to mitigate its profound impact. The utilization of AI technology holds promise in this endeavor, offering a substantial improvement in sensitivity compared to the traditional approach based solely on CL< 25 mm.

Another strength of this study lies in the utilization of logistic regression as the foundation of the mathematical process. This approach enabled the demonstration of each step in the mathematical process, including the determination of algorithmic parameters such as Coef. ß, Wald, OR, and P-value for each cervical length measurement. Subsequently, the application of machine learning techniques was introduced, enhancing the sensitivity of PTB detection. Although the AI models employed in this study may be considered as “black boxes” in terms of their internal workings, it is important to highlight that they were built upon a well-established and transparent stepwise univariate forward LR framework. This approach ensures the reproducibility and transparency of the research process [38].

Furthermore, this study demonstrates another strength by promptly developing an online calculator to facilitate future research calculations, prospective phases, and technology validations. This calculator can also be utilized to benefit other populations by providing improved sensitivity and specificity. Given the complexity of PTB screening, involving various parameters such as different cervical measurements, clinical history, and obstetrical antecedents, having a dedicated calculator for this purpose is invaluable. The study by Nicolaides et al. [39], which developed a similar calculator based on ultrasonographic and biochemical findings in the first trimester of pregnancy for screening aneuploidies, serves as an inspiration for this study. It is widely recognized as the leading approach for screening fetal aneuploidies on a global scale.

One of the limitations of this article is its methodology. A retrospective study, in essence, does not have the best statistical power in the medical area to discriminate an event, and this cohort was not conceived for this methodology. A randomized double-blind prospective multicentric clinical trial developed exclusively with this objective has the best statistical power. However, a retrospective cohort to test this new technology was used, it could be considered adequate for this innovation and the database could be useful during the implementation of the second phase, a prospective randomized clinical trial [40].

Another limitation of this article is the unique research center in a tertiary screening referred center for PTB. This aspect could produce less reproducibility of our results in low-risk populations, because of the demographic characteristics of our population. However, this singular population will be prospectively evaluated in the future and these findings could be consistent, probably this methodology could be applied in a wider form provided that calculation has been performed in a low-risk population in a multicentric study which better represents the population in general, in comparison with CL < 25 mm.

The exclusion of cases with CL < 25 mm treated with a cervical pessary or cerclage is also a limitation of this study. These exclusions could produce bias, although this bias could not be improper, because the presence of a short cervix in a higher frequency could uncover the other less relevant aspects evaluated in a longer cervix, which regularly could have a fewer performance in the determinism of PTB, and maybe this risk of bias was important to give more relevance for aspects regularly less valorized.

The small number of cases (n = 524) in this study could be another limitation, after exclusion of 33 cases by missing data, and other cases with cervical pessary and cervical cerclage, we necessarily made the statistical analysis with this low number of patients. However, increasing the number of cases will require minimal time and effort to train the model.

Research about screening methods must be conceived without any treatment [41], but in our days is unethical not to offer vaginal progesterone for short cervix, and also not offer antibiotics for amniotic fluid sludge, regarding meta-analyses evidence [42,43]. Thus, excluding mechanical treatment was an alternative to exclude overwhelming treatments for PTB whose does not achieve worldwide agreement [44], therefore reducing the risk of bias for the new screening method, even considering the bias which could be created by vaginal progesterone and antibiotics. If this screening method were effective in the use of progesterone/antibiotics it could be useful in clinical practice, once the patients would also be using it, if they had medical indication.

This study introduces a novel cervical measurement, the angle inside the cervix (A-Y-B angle), and its association with cervical length (CL) forms the CL/angle index (index = A-B/A-Y-B angle). Notably, this variable demonstrates significant discriminatory effects within the group of cervix measurements ranging from 20 to 35 mm. Additionally, the isolated cervical angle holds importance in longer cervix measurements (> 35 mm), although these specific results are not presented in this article. The inclusion of this innovative measurement should be emphasized, as it contributes to the pool of variables analyzed in this study and may have relevance in machine learning models. Logistic regression analysis further supports the significance of this novel measurement, identifying it as the most relevant variable with an odds ratio of 1.533 (95% CI 0.481–4.887).

So, if this calculator will be applied in the real context, these authors suggest that all measures be executed exactly as the methodology was described, and in the future new research could be developed to explain better the importance of each variable.

Hence, the incorporation of machine learning techniques in screening for PTB before 35 weeks gestation holds great promise and can be implemented in clinical practice shortly after the availability of the calculator. The World Health Organization emphasizes the importance of screening tests being valid, reliable, cost-effective, acceptable, and accompanied by appropriate follow-up services. This new method shows the potential in meeting these essential criteria for application in clinical settings [41]. By enhancing the screening process, new possibilities emerge in the realm of PTB, as potential treatments can be offered to patients who would not have been identified through traditional short cervix screening alone. This advancement offers the possibility of reducing PTB rates. To validate these findings, further investigation is warranted, including larger retrospective studies involving low-risk populations and prospective randomized clinical trials.

The utilization of artificial intelligence and machine learning improves the ability to identify individuals at a higher risk of PTB before 35 weeks, irrespective of cervical length or previous PTB history. This approach represents a new perspective in medical screening, significantly enhancing the sensitivity and specificity compared to traditional methods such as cervical length less than 25 mm while maintaining a similar false-positive rate of 10%.

Author’s contribution

The authors’ contribution, the interpretation of the data, the article’s writing, the critical review of the intellectual content, and the final approval of the version to be published were similar. All authors accept responsibility for the paper as published.

Acknowledgments

These authors would like to thank the Health and Medical Equipment Division of Samsung Brazil for offering the WS80A ultrasound system and HERA W9 ultrasound system to perform the exams during the study. We are also grateful to Mr. Rudolf Wiedemann for his support with the present article’s English version.

Disclosure statement

The authors report no conflict of interest: Including relevant financial, personal, political, intellectual, or religious interests.

Data availability statement

Data available on request from the authors.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.