The Deep Learning ResNet101 and Ensemble XGBoost Algorithm with Hyperparameters Optimization Accurately Predict the Lung Cancer

Figures & data

Figure 1. Schematic diagram to reflect the flow of work to detect lung cancer using XGBoost algorithm and deep learning ResNet101 with hyperparameter tuning.

Table

Machine Learning		Deep learning
Model	Library	Model	Library
XGBoost	Sklearn, xgboost, pandas, numpy, matplotlib	ResNet101	Sklearn, keras, pandas, numpy, matplotlib

Table

Model	Hyperparameters	Tunned parameters
XGBoost	‘booster:’[‘gbtree,’‘gblinear’], ‘colsample_bytree:’[0.4,0.6,0.8,1], ‘learning_rate:’[0.01,0.1,0.2,0.4] ‘max_depth:’[2,3,4,6], ‘n_estimators:’[200,300,400,500], ‘subsample:’[0.4,0.6,0.8,1]}	{‘subsample:’ 0.8, ‘n_estimators:’ 200, ‘max_depth:’ 6, ‘learning_rate:’ 0.1, ‘colsample_bytree:’ 1, ‘booster:’ ‘gbtree’}

Figure 2. General architecture of XGBoost algorithm.

Table

Input	Memory	Parameters
Pool 3x3 conv, 512	memory: 14*14*512=100K	params: (3*3*512) *512 = 2,359,296
FC 1000	memory: 1000	params: 4096*1000 = 4,096,000

Figure 3. Lung cancer NSCLC and SCLC original image samples along with heatmaps.

Table 1. Binary (NSCLC, SCLC) classification using traditional ML SVM algorithm with 10-fold cross validation.

Class	Precision	Recall	F1-score	Support
NSCLC	83.0%	69.0%	75.0%	71
SCLC	82.0%	91.0%	87.0%	113
Accuracy			83.0%	184
Macro Avg	83.0%	80.0%	81.0%	184
Weighted Avg	83.0%	83.0%	82.0%	184

Table 2. Binary (NSCLC, SCLC) classification using traditional ML KNN algorithm with 10-fold cross validation.

Class	Precision	Recall	F1-score	Support
NSCLC	97.0%	94.0%	96.0%	71
SCLC	97.0%	98.0%	97.0%	113
Accuracy			97.0%	184
Macro Avg	97.0%	96.0%	97.0%	184
Weighted Avg	97.0%	97.0%	97.0%	184

Table 3. Lung cancer detection performance utilizing XGBoost with default parameters.

Class	Precision	Recall	F1-score	Support
NSCLC	100%	99.0%	99.0%	71
SCLC	99.0%	100%	100%	113
Micro Avg	100%	99.0%	99.0%	184

Table 4. Lung cancer detection performance utilizing XGBoost with hyperparameters optimization.

Class	Precision	Recall	F1-score	Support
NSCLC	100%	100%	100%	71
SCLC	100%	100%	100%	113
Micro Avg	100%	100%	100%	184

Figure 4. AUC-ROC and error graph to detect the lung cancer using XGBoost.

Figure 5. ROC and PR curves to detect lung cancer.

Figure 6. Accuracy-loss graph for binary (NSCLC, SCLC) classification: (a) accuracy graph, (b) loss graph using ResNet101 model with 10-fold cross validation.

Table 5. Binary-class (NSCLC, SCLC) classification using ResNet101 model with 10-fold cross validation.

Class	Precision	Recall	F1-score	Support
NSCLC	100%	100%	100%	71
SCLC	100%	100.0%	100.0%	113
Accuracy			100%	184
Macro Avg	100%	100%	100%	184
Weighted Avg	100%	100%	100%	184

Table 6. Binary (NSCLC, SCLC) classification using ResNet101 model at selected twofold cross validation.

Class	Precision	Recall	F1-score	Support
NSCLC	100.0%	100%	100%	30
SCLC	100.0%	100.0%	100.0%	48
Accuracy			100.0%	78
Macro Avg	100.0%	100.0%	100.0%	78
Weighted Avg	100.0%	100.0%	100.0%	78

Table 7. Binary (NSCLC, SCLC) classification using ResNet101 model at selected fourfold cross validation.

Class	Precision	Recall	F1-score	Support
NSCLC	100.0%	100%	100%	31
SCLC	100.0%	100.0%	100.0%	46
Accuracy			100.0%	77
Macro Avg	100.0%	100.0%	100.0%	77
Weighted Avg	100.0%	100.0%	100.0%	77

Table 8. Binary (NSCLC, SCLC) classification using ResNet101 model at selected eightfold cross validation.

Class	Precision	Recall	F1-score	Support
NSCLC	100.0%	96.0%	98.0%	25
SCLC	98.0%	100.0%	99.0%	52
Accuracy			99.0%	77
Macro Avg	99.0%	98.0%	99.0%	77
Weighted Avg	99.0%	99.0%	99.0%	77

Table 9. Bacterial lung cancer detection performance utilizing XGBoost using optimized hyperparameters.

Class	Precision	Recall	F1-score	Support
Bacterial	98.0%	97.0%	98.0%	505
Normal	96.0%	96.0%	96.0%	305
Macro Avg	97.0%	97.0%	97.00%	810

Figure 7. Confusion matrix for binary (NSCLC, SCLC) classification using ResNet101 model at different folds from 1 to 10 k-fold cross validation.

Figure 8. AUC-ROC and error graph to detect the bacterial lung cancer using XGBoost.

Figure 9. ROC and PR curves to detect bacterial lung cancer.

Table 10. Viral lung cancer detection performance utilizing XGBoost using optimized hyperparameters.

Class	Precision	Recall	F1-score	Support
Viral	95.0%	97.0%	96.0%	305
Normal	97.0%	95.0%	96.0%	269
Macro Avg	96.0%	96.0%	96.00%	574

Figure 10. ROC and PR curves to detect viral lung cancer.

Table 11. Viral lung cancer detection performance utilizing ResNet101 with 10-fold cross validation.

Class	Precision	Recall	F1-score	Support
Viral	100%	100%	100%	504
Normal	100%	100%	1000%	505
Accuracy			100%	1009
Macro Avg	100%	100%	1000%	1009
Macro Avg	100%	100%	1000%	1009

Figure 11. Accuracy-loss graph for binary (viral, normal) classification: (a) accuracy graph, (b) loss graph using ResNet101 model with 10-fold cross validation.

Table 12. Comparison of findings from previous studies.

Author	Features used	Performance
Guo et al. (2009)	1. Texture 2. Shape	Sensitivity = 94%,
da Silva Sousa et al. (2010)	1. Gradient 2. Histogram 3. Spatial	Sensitivity = 84%, Specificity = 96% Accuracy=95%
Teramoto et al. (2014)	1. Shape 2. Intensity	Sensitivity = 83%,
Dandıl (2018)	1. GLCM 2. Shape 3. Statistical 4. Energy	Sensitivity = 97%, Specificity = 94% Accuracy=95%
Hussain et al. (2019)	Lung cancer detection based on multimodal features such as texture, morphological, and EFDs Texture features using MFE with standard deviation, Morphological features using RCMFE with mean EFDs features using MFE	P-value (1.95E–50) P-value (3.01E–14) P-value (1.04E–13)
Hussain et al. (2021)	RICA features and SVM	Accuracy = 99.77%
Nasrullah et al. (2019)	1. Statistical	Sens. = 94%, Spec. = 90% AUC = 99%
This study	GLCM features with XGBoost default parameters NSCLC SCLC	Precision=99.0%, Accuracy =99.0% Recall =99.0%, Accuracy =99.0%
	GLCM features with XGBoost optimized parameters NSCLC vs SCLC	Precision = 100.0%, Recall = 100.0% Accuracy = 100% F-measure = 100%
	ResNet101with optimized parameters NSCLC vs SCLC	Precision = 100.0%, Recall = 100.0% Accuracy = 100% F-measure = 100%

Guo, W., Y. Wei, H. Zhou, and D. Xue. 2009. An adaptive lung nodule detection algorithm. 2009 Chinese Control and Decision Conference, 2361–65. 10.1109/CCDC.2009.5192686

 Google Scholar

da Silva Sousa, J. R. F., A. C. Silva, A. C. de Paiva, and R. A. Nunes. 2010. Methodology for automatic detection of lung nodules in computerized tomography images. Computer Methods and Programs in Biomedicine 98 (1):1–14. doi:10.1016/j.cmpb.2009.07.006.

 PubMed Web of Science ®Google Scholar

Teramoto, A., H. Fujita, K. Takahashi, O. Yamamuro, T. Tamaki, M. Nishio, and T. Kobayashi. 2014. Hybrid method for the detection of pulmonary nodules using positron emission tomography/computed tomography: A preliminary study. International Journal of Computer Assisted Radiology and Surgery 9 (1):59–69. doi:10.1007/s11548-013-0910-y.

 PubMed Web of Science ®Google Scholar

Dandıl, E. 2018. A computer-aided pipeline for automatic lung cancer classification on computed tomography scans. Journal of Healthcare Engineering 2018:1–12. doi:10.1155/2018/9409267.

 Web of Science ®Google Scholar

Hussain, L., W. Aziz, A. A. A. Alshdadi, M. S. Ahmed Nadeem, I. R. Khan, and Q. -U. -A. Chaudhry. 2019. Analyzing the dynamics of lung cancer imaging data using refined fuzzy entropy methods by extracting different features. IEEE Access 7:64704–21. doi:10.1109/ACCESS.2019.2917303.

 Web of Science ®Google Scholar

Hussain, L., M. S. Almaraashi, W. Aziz, N. Habib, and S. -U. -R. Saif Abbasi. 2021. Machine learning-based lungs cancer detection using reconstruction independent component analysis and sparse filter features. Waves in Random and Complex Media 1–26. doi:10.1080/17455030.2021.1905912.

 Web of Science ®Google Scholar

Nasrullah, N., J. Sang, M. S. Alam, and H. Xiang. 2019. Automated detection and classification for early stage lung cancer on CT images using deep learning. In Pattern recognition and tracking XXX, ed. M. S. Alam, 27. SPIE. doi: 10.1117/12.2520333.

 Google Scholar