Predicting cervical cancer biopsy results using demographic and epidemiological parameters: a custom stacked ensemble machine learning approach

Figures & data

Figure 1. Internal structure of the cervix.

Table 1. Cervical cancer various stages

Stage	Symptoms	Five-year survival rate
Stage 0	Carcinoma present in situ. The cervix’s innermost membrane contains malignant cells	97–100%
Stage 1	Cervical cancer that is localized to the cervix itself.	81–96%
Stage 2	The malignancy has progressed beyond the uterus, although not to the pelvic sidewalls or the lower part of the vaginal cavity.	65–87%
Stage 3	Cancer cells spread to lower third of the vaginal canal and side walls. The kidneys start to malfunction as well.	35–50%
Stage 4	The cancer had migrated beyond the pelvis. The cancerous cell spread to the rectum and bladder too.	15–20%

Table 2. Various researches that diagnose cervical cancer using machine learning approaches

Reference	Dataset Source	Dataset Size	Number of Features Chosen	Models used	Accuracy	Sensitivity	Specificity	AUC	Critical Analysis/ Findings
(Nithya & Ilango, 2019)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes	C5.0, rpart, KNN, random forest, SVM	97%	-	-	91%	The use of Boruta algorithm to assist random forest algorithm.
(Fernandes, Chicco, et al., 2018)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes	Various ML methods.	-	-	-	68%	Joint dimensionality reduction with accurate classification
(Ijaz et al., 2020)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes	Various ML methods	96%	97%	94%	-	Outlier detection using DBSCAN.
(Suman & Hooda, 2019)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes	Various ML methods	96%	-	-	95%	The use of decision stump algorithms. MCC was used as a performance metric.
(Lee et al., 2021)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes	Various ML methods	-	-	-	-	In-depth mathematical intuition for cervical cancer diagnosis.
(Soğukkuyu & Ata, 2022)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes	Various ML methods	97%	-	-	-	Novel ensemble methods
(Aloss et al., 2022)	Hospital Universitario de Caracas in Caracas, Venezuela	858 Patients	36 attributes -	Various ML methods	97%	-	-	-	The use of crow search algorithm for feature selection

Table 3. Description of the dataset

Feature	Range (Min: Max)	Datatype	Missing Values	Description	Feature	Range (Min: Max)	Datatype	Description	Missing Values
Biopsy result: Target variable	0:1	Boolean	0	Result obtained after the biopsy test	STD- Vulvo Perineal Condylomatosis	0:1	Boolean	It indicates the presence of this particular STD.	105
Cytology test	0:1	Boolean	0	Result obtained after the cytology test	STD- Vaginal Condylomatosis	0:1	Boolean	It indicates the presence of this particular STD.	105
Schiller’s test	0:1	Boolean	0	Result obtained after the Schiller’s test	STD- Cervical Condylomatosis	0:1	Boolean	It indicates the presence of this particular STD.	105
Hinselmann’s test	0:1	Boolean	0	Result obtained after the Hinselmann’s test	STD- HPV	0:1	Boolean	It indicates the presence of this particular STD.	105
Age	13:84	Integer	0	Age of the patient	STD: Hepatitis B	0:1	Boolean	It indicates the presence of this particular STD	105
Age during the first sexual intercourse	10–32	Integer	7	Age during the first sexual intercourse	STD: HIV	0:1	Boolean	It indicates the presence of this particular STD.	105
Number of sexual partners	0–28	Integer	26	The number of sexual partners	STD: AIDS	0:1	Boolean	It indicates the presence of this particular STD.	105
Pregnancies	0:11	Integer	56	The number of pregnancies	STD: Mollusc-um Contagio-sum	0:1	Boolean	It indicates the presence of this particular STD.	105
Smokes (years/pack)	0:37	Integer	13	It represents the sum of cigarette packets smoked by the patient in a year	STD: Genital herpes	0:1	Boolean	It indicates the presence of this particular STD	105
Smokes (Years)	0:37	Integer	13	Time in years since the patient started smoking	STD: Pelvic Inflammatory	0:1	Boolean	It indicates the presence of this particular STD	105
Smokes	0:1	Boolean	13	It indicates whether the patient smokes or not	STD: Syphilis	0:1	Boolean	It indicates the presence of this particular disease	105
Hormonal Contraceptives (years)	0:30	Integer	108	The number of years since the patient is using hormonal contraceptives	STD: Number of Diagnosis	0:3	Integer	Total number of times tested.	707
Hormonal Contraceptive	0:1	Boolean	108	It indicates whether the patient uses contraceptive or not	STD: Time since last diagnosis	1:22	Integer	Total number of years since last diagnosis	707
Intra-uterine device (IUD)(years)	0:19	Integer	117	The number of years since the patient is using IUD’s	STD: Time since first diagnosis	1:22	Integer	Total number of years since the first diagnosis	0
IUD	0:1	Boolean	117	It indicates whether the patient used IUD’s or not	Dx	0:1	Boolean	It indicates whether HPV, Cervical intraepithelial neoplasia (CIN) or cancer are present	0
Sexually transmitted diseases (STD’s)	0:1	Boolean	105	It indicates whether the patient has STD.	Dx: HPV	0:1	Boolean	It indicates the presence of this disease	0
Number of STD’s	0–4	Integer	105	The number of STD’s present.	Dx: Cin	0:1	Boolean	It indicates the presence of this disease	0
STD- Condylomatosis	0:1	Boolean	105	It indicates whether the patient has Condylomatosis	Dx: Cancer	0:1	Boolean	It indicates the presence of this disease	0

Figure 2. (a) Bar graph showing the percentage of null values in the dataset (b) The percentage of biopsy positive and biopsy negative results.

Figure 3. Count plots and density plots (a) The number of patients who smoke (b) The number of patients who use hormonal contraceptive (c) The number of patients who use intra-uterine devices (d) The number of patients who have sexually transmitted diseases (e) Patient age distribution (f) The number of pregnancies (g) The number of sexual partners.

Figure 4. Multivariate analysis using box plots.

Figure 5. The impact of STD’s on the biopsy result.

Figure 6. The presence of outliers in data. (a) Outliers before IQR treatment (b) After IQR treatment (Outliers removed).

Figure 7. Mutual Information which describes the relationship among various attributes which diagnose cervical cancer.

Figure 8. Pearson’s correlation heatmap which describes the relationship among various attributes which diagnose cervical cancer.

Figure 9. Biopsy results before and after balancing. (a) Initial unbalanced data (b) Balanced data after using Borderline-SMOTE.

Figure 10. Various steps followed to predict biopsy results using machine learning.

Figure 11. Custom stacking architecture to predict cervical cancer biopsy results.

Table

Algorithm 1. Feature selection using ANOVA.
Input:	M: Size Sn * H feature matrix; where Sn-number of samples, H- number of features
Output:	Best N Marchkers
1:	Read the dataset, Replace the missing values.
2:	For every gi in the feature set do
3:	j = 1,2,3, … .H
4:	The value of MS can be calculated by equation (3)(3) $MS=SE/DF$(3)
5:	Calculate MSL using equation (6)(6) $MSL=ICR/DFC$(6)
6:	Find the value of F-statistic (F) by equation (7)(7) $F=MS/MSL\hspace{0.17em}$(7)
7:	Use the F-distribution table and calculate the p-value(pj) for all F-statistics
8:	If pj < 0.001 then
9:	Choose feature gj
10:	Concatenate gj to feature matrix Hm
11:	Else
12:	Feature gj is removed
13:	End if
14:	Using p value, arrange all the features based on their importance
15:	If feature_size (Hm) > 18 then
16:	Only choose the best 18 Marchkers
17:	Else
18:	Keep all the Marchkers in Hm
19:	End if
20:	End for
21:	Return Hm

Table 4. Performance evaluation of classifiers without using feature selection and borderline-SMOTE (Unbalanced dataset)

Classifier	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)
Logistic regression	78	59	67	62
Decision tree	47	0	0	0
K Nearest Neighors	75	53	53	53
Support vector machine	78	57	87	68
Naïve Bayes	62	24	93	38
STACKA (Logistic regression + Decision tree + K Nearest Neighbors + support vector machine + Naïve Bayes)	75	53	67	59
Random forest	79	60	60	60
Multi-layer perceptron	62	24	93	38
Adaboost	79	58	93	72
Catboost	74	50	67	57
Lightgbm	72	55	80	65
Xgboost	77	55	80	65
Extra trees	47	0	0	0
StackB (Random forest + Multi-layer perceptron + Adaboost + Catboost + Lightgbm + xgboost + Extratrees)	74	50	60	55
STACKC (STACKA + STACK B)	74	50	60	55

Figure 12. ROC curves obtained by the classifiers. (a) Logistic regression (b) Decision tree (c) KNN (d) SVM (e) Naïve Bayes (f) STACK A.

Figure 13. Precision-recall curves of the initial set of classifiers.

Table 5. Performance evaluation of the initial set of classifiers after data balancing and hyperparameter tuning

Classifier	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	AUC (%)	AP (%)	Best hyperparameters
Logistic regression	95	95	95	95	98	96	{“C”: 1, “penalty”: “l2”}
Decision Tree	97	97	97	97	98	97	{“criterion”: “entropy”, “max_depth”: 8, “max_features”: “log2”, “min_samples_leaf”: 1, “min_samples_split”: 10, “splitter”: “best”}
KNN	98	98	99	98	98	97	{“n_neighbors”: 1}
Support vector machine	96	96	96	96	97	94	{“kernel”:’linear’, “probability”:’True’}
Naïve Bayes	94	92	97	95	98	97	-
STACKA (Logistic regression + Decision tree + K Nearest Neighbors + support vector machine + Naïve Bayes)	98	98	98	98	100	100	{“Meta classifier”:’logistic regression’}

Figure 14. AUC’s of the bagging, boosting and stacking classifiers. (a) Random forest (b) MLP (c) Adaboost (d) Catboost (e) Lightgbm (f) Xgboost (g) Extratrees (h)STACKB (i) STACKC.

Figure 15. Precision-recall curves of the bagging, boosting and stacking classifiers.

Figure 16. Confusion matrices: (a) STACK A (b) STACK B (c) STACK C.

Table 6. Performance evaluation of bagging and boosting classifiers after data balancing and hyperparameter tuning

Classifier	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	AUC (%)	AP (%)	Best hyperparameters
Random forest	96	96	97	96	99	100	{“bootstrap”: True, “max_depth”: 100, “max_features”: 2, “min_samples_leaf”: 3, “min_samples_split”: 8, “n_estimators”: 200}
Multi-layer perceptron	94	92	96	94	97	96	{“activation”: “relu”, “alpha”: 0.05, “hidden_layer_sizes”: (100,), “learning_rate”: “constant”, “max_iter”: 50, “solver”: “lbfgs”}
Adaboost	98	97	98	98	99	99	{“learning_rate”: 1.0, “n_estimators”: 300}
Catboost	97	96	99	98	99	99	{“border_count”: 10, “depth”: 3, “iterations”: 250, “l2_leaf_reg”: 1, “learning_rate”: 0.03}
Lightgbm	98	97	98	98	99	99	-
Xgboost	98	96	100	98	99	100
Extratrees	94	96	93	95	98	97
STACK B ((Random forest + Multi-layer perceptron + Adaboost + Catboost + Lightgbm + xgboost + Extratrees)	98	98	98	98	100	100	{“Meta classifier”:’logistic regression’}
STACK C (STACKA +STACKB)	98	97	99	98	100	100	{“Meta classifier”:’logistic regression’}

Figure 17. Feature importance using SHAP. (a) Bee swarm plot (b) Mean SHAP values.

Figure 18. Feature importance using random forest.

Figure 19. Feature importance using LIME (Positive Biopsy result).

Figure 20. Feature importance using LIME (Negative Biopsy result).

Figure 21. Feature importance using LIME graphs. (a) Positive biopsy result (b) negative biopsy result.

Figure 22. Feature importance using ELI5 for cervical cancer positive biopsy result.

Figure 23. User interface of the prediction model deployed using “Gradio” to classify cervical cancer results.

Table 7. Comparison of various researches

Reference	Year	Models used	Number of Classes	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	AUC (%)
(Chauhan & Singh, 2022)	2022	Various ML models	2	98	98	100	97	99
(Curia, 2021)	2021	Various ML models	2	94.5	-	-	-	98
(Jahan et al., 2021)	2021	Various ML models	2	96	95	96	95	-
(Nithya & Ilango, 2019)	2019	Various ML models	2	100	-	-	-	91
(Fernandes, Chicco, et al., 2018)	2018	Various ML models	2	98	98	98	98	96
[42	2022	Various ML models	2	97	-	-	-	-
(Aloss et al., 2022)	2022	Various ML models	2	97	-	-	-	-
(Tanimu et al., 2022)	2022	Various ML models	2	98	-	-	-	-
(Nsugbe, 2022)	2022	Various ML models	2	91.5	-	96	-	95
(Nagadeepa et al., 2022)	2022	Various ML models	2	97%	-	-	-	-
Proposed study	-	Custom multi-layer Stacking models	2	98	97	99	98	100

Nithya, B., & Ilango, V. (2019, Jun). Evaluation of machine learning based optimized feature selection approaches and classification methods for cervical cancer prediction. SN Applied Sciences, 1(6), 1–6. https://doi.org/10.1007/s42452-019-0645-7

 Web of Science ®Google Scholar

Fernandes, K., Chicco, D., Cardoso, J. S., & Fernandes, J. (2018, May 14). Supervised deep learning embeddings for the prediction of cervical cancer diagnosis. PeerJ Computer Science, 4, e154. https://doi.org/10.7717/peerj-cs.154

 PubMedGoogle Scholar

Ijaz, M. F., Attique, M., & Son, Y. (2020, January). Data-driven cervical cancer prediction model with outlier detection and over-sampling methods. Sensors, 20(10), 2809. https://doi.org/10.3390/s20102809

 PubMed Web of Science ®Google Scholar

Suman, S. K., & Hooda, N. (2019, May). Predicting risk of Cervical Cancer: A case study of machine learning. Journal of Statistics and Management Systems, 22(4), 689–696. https://doi.org/10.1080/09720510.2019.1611227

 Web of Science ®Google Scholar

Lee, C. K., Tse, Y. K., Ho, G. T., & Chung, S. H. (2021, January 1). Uncovering insights from healthcare archives to improve operations: An association analysis for cervical cancer screening. Technological Forecasting and Social Change, 162, 120375. https://doi.org/10.1016/j.techfore.2020.120375

 Web of Science ®Google Scholar

Soğukkuyu, D. Y., & Ata, O. (2022). Diagnosing Cervical Cancer Using Machine Learning Methods. In2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), Jun 9, (pp. 1–3). IEEE.

 Google Scholar

Aloss, A., Sahu, B., Deeb, H., & Mishra, D. (2022). A crow search algorithm-based machine learning model for heart disease and cervical cancer diagnosis. In InElectronic systems and intelligent computing (pp. 303–311). Springer. https://doi.org/10.1007/978-981-16-9488-2_27

 Google Scholar

Chauhan, N. K., & Singh, K. (2022, Jan). Performance assessment of machine learning classifiers using selective feature approaches for cervical cancer detection. Wireless Personal Communications, 12, 1–32. https://doi.org/10.1007/s11277-022-09467-7

 Google Scholar

Curia, F. (2021, July). Cervical cancer risk prediction with robust ensemble and explainable black boxes method. Health and Technology, 11(4), 875–885. https://doi.org/10.1007/s12553-021-00554-6

 Web of Science ®Google Scholar

Jahan, S., Islam, M. D., Islam, L., Rashme, T. Y., Prova, A. A., Paul, B. K., Islam, M. D., & Mosharof, M. K. (2021, Oct). Automated invasive cervical cancer disease detection at early stage through suitable machine learning model. SN Applied Sciences, 3(10), 1–7. https://doi.org/10.1007/s42452-021-04786-z

 Web of Science ®Google Scholar

Tanimu, J. J., Hamada, M., Hassan, M., Kakudi, H., & Abiodun, J. O. (2022). A machine learning method for classification of cervical cancer. Electronics, 11(3), 463. https://doi.org/10.3390/electronics11030463

 Web of Science ®Google Scholar

Nsugbe, E. (2022, April). Towards the use of cybernetics for an enhanced cervical cancer care strategy. Intelligent Medicine, 2(3), 117–126. https://doi.org/10.1016/j.imed.2022.02.001

 Google Scholar

Nagadeepa, C. H., Sai, P. P., Madhuri, G., Reddy, K. S., & Reddy, D. V. (2022). Artificial Intelligence based Cervical Cancer Risk Prediction Using M1 Algorithms. In2022 International Conference on Emerging SMarcht Computing and Informatics (ESCI), March 9, (pp. 1–6). IEEE. https://doi.org/10.1109/ESCI53509.2022.9758241

 Google Scholar