Detection and classification of gastrointestinal disease using convolutional neural network and SVM

Figures & data

Figure 1. Image samples of the various labeled classes.

Figure 2. Original and resultant image after noise removal.

Table 1. Parameters of the proposed VGGNet architecture of the CNN model

Operation	Kernel/stride	Filter size	Output shape	Parameter
2D convolution 1	3 × 3/1	32	32 x 32 x 32	896
2D convolution 2	3 × 3/1	32	32 x 32 x 32	9248
Batch normalization	-	-	32 x 32 x 32	128
2D maxpooling1	2 × 2/-	-	16 x 16 x 32	0
Dropout 1	-	-	16 x16 x 32	0
2D convolution 3	3 × 3/1	64	16 x 16 x 64	18,496
2D convolution 4	3 × 3/1	64	16 x 16 x 64	36,928
Batch normalization	-	-	16 x 16 x 64	256
2D MaxPooling 2	2 × 2/-	-	8 x 8 x 64	0
Dropout 3	-	-	8 x 8 x 64	0
2D convolution 6	3 × 3/1	128	8 x 8 x 128	73,856
2D convolution 7	3 × 3/1	128	8 x 8 x 128	147,584
2D MaxPooling 3	2 × 2/-	-	4 x 4 x 128	0
Dropout 5	-	-	4 x 4 x 128	0
Flatten_1	-	2048	2048	0
Dense_1	-	512	512	1,049,088
Dense_2	-	1024	1024	525,312
Dropout 4	-	-	1024	0
Dense_3	-	23	23	23,575

Figure 3. The proposed VGGNet CNN model architecture.

Table 2. Description of the parameters of the proposed InceptionNet architecture of the CNN model

Operation	Kernel /stride	Filter size	Output shape	Parameter
2D convolution 1	1x1	64	32 x 32 x 96	384
2D convolution 2	1x1	96	32 x 32 x 16	64
2D convolution 3	3x3	128	32 x 32 x 64	256
2D convolution 4	1x1	16	32 x 32 x 128	110,720
2D convolution 5	5x5	32	32 x 32 x 32	12,832
2D maxpooling1	3x3	-	32 x 32 x 3	0
2D convolution 6	1x1	32	32 x 32 x 32	128
Concatenate	-	-	32 x 32 x 256	0
Batch normalization	-	-	32 x 32 x 256	1024
Global average pooling	-	-	256	0
Dropout	-	-	256	0
Flatten	-	256	256	0
Dense		524	524	134,668
Dropout	-	-	524	0
Dense	-	524	23	12,075

Figure 4. The proposed InceptionNet CNN model.

Figure 5. The proposed gastrointestinal disease classification model.

Table 3. Accuracy of the proposed model with resized data using noise filtering and segmentation techniques

Experiment Number	Model	Noise filtering technique	Segmentation technique	Classifier	Test accuracy (%)
Experiment 1	VGGNet-InceptionNet	No	No	Softmax	71.8
Experiment 2	VGGNet-InceptionNet	Gaussian filter	No	Softmax	70.8
Experiment 3	VGGNet-InceptionNet	Median filter	No	Softmax	68.6
Experiment 4	VGGNet-Inception Net	Adaptive median filter	No	Softmax	72.7
Experiment 5	VGGNet-InceptionNet	Adaptive median filter	Otsu	Softmax	54.7
Experiment 6	VGGNet-InceptionNet	Adaptive median filter	Thresholding	Softmax	52.3
Experiment 7	VGGNet-InceptionNet	Adaptive median filter	Kmeans	Softmax	56.3

Table 4. Comparison of classifiers based on the proposed feature extraction methods

Experiment Number	Model	Noise filtering technique	Classifier	Test accuracy (%)
Experiment 8	VGGNet	Adaptive median filter	Softmax	93
Experiment 9	VGGNet	Adaptive median filter	KNN	93.2
Experiment 10	VGGNet	Adaptive median filter	RF	93.3
Experiment 11	VGGNet	Adaptive median filter	SVM	93.4
Experiment 12	InceptionNet	Adaptive median filter	Softmax	84
Experiment 13	InceptionNet	Adaptive median filter	KNN	84.5
Experiment 14	InceptionNet	Adaptive median filter	RF	84.7
Experiment 15	InceptionNet	Adaptive median filter	SVM	85
Experiment 16	VGGNet-InceptionNet	Adaptive median filter	Softmax	94
Experiment 17	VGGNet-InceptionNet	Adaptive median filter	KNN	95.4
Experiment 18	VGGNet-InceptionNet	Adaptive median filter	RF	95.7
Experiment 19	VGGNet-InceptionNet	Adaptive median filter	SVM	98
Experiment 20	Pre- trained VGGNet16	Adaptive median filter	-	89
Experiment 21	Pre-trained Inceptionv3	Adaptive median filter	-	91

Table 5. The precision, recall, training accuracy, validation accuracy, training loss, validation loss, Matthews correlation coefficient, and f1-score of the proposed concatenated CNN model with SVM classifier

Class	Precision	Recall	F1-score
Barrett’s	0.98	0.94	0.96
Barrett’s short-segment	0.97	0.98	0.98
Bbps 0–1	0.99	0.99	0.99
Bbps 2–3	0.99	0.99	0.99
Dyed-lifted-polyps	0.99	0.93	0.96
Dyed rection margins	0.98	0.98	0.98
Esophagitis a	0.98	0.97	0.97
Esophagitis b-d	0.92	0.99	0.95
Hemorrhoids	0.99	1.00	1.00
Ileum	1.00	0.99	0.99
Impacted-stool	0.99	1.00	1.00
Cecum	0.97	0.97	0.97
Pylorus	0.95	0.99	0.97
Z-line	0.92	0.99	0.95
Polyps	0.98	0.99	0.99
Retroflex-rectum	1.00	0.98	0.99
Retroflex-stomach	0.99	0.98	0.98
Ulcerative-colitis-grade-0–1	0.99	0.98	0.99
Ulcerative-colitis-grade-1	0.98	0.98	0.98
Ulcerative colitis-grade-1–2	0.99	1.00	1.00
Ulcerative-colitis-grade-2	0.98	0.97	0.97
Ulcerative-colitis-grade-2–3	1.00	1.00	1.00
Ulcerative colitis-grade-3	1.00	0.94	0.97
Training accuracy	0.987
Testing accuracy	0.98
Validation accuracy	0.982
Training loss	0.107
Validation loss	0.071
Matthews_correlation coefficient	0.978

Figure 6. The training and validation accuracy of the proposed ensemble- CNN model with SVM classifier with epoch size of 75.

Figure 7. The training and validation loss of the proposed ensemble- CNN model with Softmax classifier for epoch size of 75.

Table 6. Comparison of proposed study with existing researches related to gastrointestinal disease

Author	Method	End Goal	Classifier	Strength	Weakness	Accuracy/MCC
Borgli et al. (2020)	Averaged ResNet-152 + DenseNet-161	Classification of GI diseases	Multiclass	They use the power of two models	Misclassification between class	MCC = 0.902
Khan, Khan et al. (2020)	(RNN,pre-trained ResNet101) +SVM	detection of ulcers and classification of gastrointestinal diseases	Multiclass	Accuracy is high	fail for the segmentation of polyp and bleeding regions and not rely on the correctness of ground truth data Limited number of classes	99.13%
Gamage et al. (2019)	combining pre-trained DenseNet-201, ResNet-18, and VGG-16 CNN+ANN	predicts anomalies of the digestive tract diseases	Multiclass	combining pre-trained models as the Feature extractors	predicts eight-class anomalies of the digestive tract diseases only	97%
Takiyama et al. (2018)	GoogleNet	Anatomical classification of GI images	Multiclass	High classification performance Computationally efficient	The Limited number of classes Only used for anatomical classification	97%
Owais et al. (2019)	CNN (ResNet) + LSTM	Classification of multiple GI diseases	Multiclass	Computationally efficient High classification performance Cascaded training of CNN and LSTM requires more time	Cascaded training of CNN and LSTM requires more time	92.57%
Sharif et al. (2019)	(VGG16, VGG19)+K-Nearest Neighbor	gastrointestinal tract diseases detection and classification	Multiclass	merging deep convolutional neural networks (CNN)	Limited dataset and number of classes	99.42%
Shichijo et al. (2017)	CNN	H. pylori infection detection	Multiclass	Comparable performance of second CNN with the clinical diagnosis reference standard	CAD performance should be enhanced. A limited number of classes	83.1%
Segu et al. (2016)	CNN	Small intestine movement Characterization	Multiclass	High classification performance	the limited number of classes	96%
Li et al. (2012)	K-nearest neighbor classifier	Distinguish lymph node metastasis from non-lymph node metastasis	Binary	combines several feature selection algorithms and metric learning methods to get high efficiency	insufficient number of clinical cases	96.33%
Wang et al. (2015)	Edge cross-section profile (ECSP) features	detecting polyps in colonoscopy	-	Performance is high	They did not classify polyps into different types	97.7%
Souaidi et al. (2019)	texture patterns (LBP, Laplacian) + SVM	detection of ulcers	Binary	conduct several experiments to show comparison with the state-of-the-art methods	Limited dataset and number of classes	93.88%
Li & Meng (2012)	Texture features (LBP, wavelet) + SVM	Tumor recognition in the digestive tract	Binary	Invariant to illumination change and extract multi-resolution features	Limited dataset and number of classes	92.4%
Yuan & Meng (2014)	Texture features (SIFT, Saliency) + SVM	Polyp classification	Binary	Extract scale-invariant features	Limited dataset and number of classes	92%
Karargyris & Bourbakis (2011)	Log Gabor filter, SUSAN edge det- ection and SVM	Small bowel polyps and ulcers detection	Binary	Computationally efficient	Limited dataset, number of classes, and Low detection performance	-
Li & Meng (2012)	Texture features (ULBP, wavelet) + SVM	Polyp detection in GI tract	Binary	Robust to illumination change and scale-invariant	Limited dataset and number of classes	91.6%
Hirasawa et al. (2018)	CNN	Gastric cancer detection	Binary	High sensitivity Computationally efficient The overall low positive prediction value	Limited dataset and number of classes	-
Proposed method	VGGNet-InceptionNet + SVM	Classification of GI diseases	Multiclass	We concatenate the feature of two model	It uses an average Training Time	Accuracy = 98%, and MCC = 0.978

Borgli, H., Vajira Thambawita, V., Smedsrud, P. H., Hicks, S., Jha, D., Sigrun L. Eskeland,S. L., Randel, K. R., Pogorelov, K., Lux, M., Nguyen,D. T. D., Johansen, D., Griwodz, C., Stensland, H. K., Garcia-Ceja, E., Schmidt, P. T., Hammer, H. L., Riegler, M. A., Halvorsen, P., Thomas de Lange. (2020). HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Scientific Data, 7(1), 1–14. https://doi.org/10.1038/s41597-020-00622-y

 PubMed Web of Science ®Google Scholar

Khan, M. A., Khan, M. A., Ahmed, F., Mittal, M., Goyal, L. M., Jude Hemanth, D., & Satapathy, S. C. (2020). Gastrointestinal diseases segmentation and classification based on duo-deep architectures. Pattern Recognition Letters, 131(1), 193–204. https://doi.org/10.1016/j.patrec.2019.12.024

 Web of Science ®Google Scholar

Gamage, C., Wijesinghe, I., Chitraranjan, C., & Perera, I. (2019). GI-Net: Anomalies classification in gastrointestinal tract through endoscopic imagery with deep learning,” MERCon 2019 - Proceedings, 5th International Multidisciplinary Moratuwa Engineering Research Conference, 66–71, https://doi.org/10.1109/MERCon.2019.8818929

 Google Scholar

Takiyama, H., Ozawa, T., Ishihara, S., Fujishiro, M., Shichijo, S., Nomura, S., Miura, M., & Tada, T. (2018). Automatic anatomical classification of esophagogastrodenoscopy images using deep convolutional neural networks. Scientific Reports, 8(1), 1–8. https://doi.org/10.1038/s41598-018-25842-6

 PubMedGoogle Scholar

Owais, M., Arsalan, M., Choi, J., Mahmood, T., & Park, K. R. (2019). Artificial intelligence-based classification of multiple gastrointestinal diseases using endoscopy videos for clinical diagnosis. Journal of Clinical Medicine, 8(7), 986. https://doi.org/10.3390/jcm8070986

 PubMed Web of Science ®Google Scholar

Sharif, M., Attique Khan, M., Rashid, M., Yasmin, M., Afza, F., & Tanik, U. J. (2019). Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images. Journal of Experimental and Theoretical Artificial Intelligence, 1–23. https://doi.org/10.1080/0952813X.2019.1572657

 Google Scholar

Shichijo, S., Nomura, S., Aoyama, K., Nishikawa, Y., Miura, M., Shinagawa, T., Takiyama, H., Tanimoto, T., Ishihara, S., Matsuo, K., & Tada, T. (2017). EBiomedicine application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine, 25(1), 106–111. https://doi.org/10.1016/j.ebiom.2017.10.014

 PubMed Web of Science ®Google Scholar

Segu, S., Drozdzal, M., Pascual, G., Radeva, P., Malagelada, C., Azpiroz, F., & Vitrià, J. (2016). Generic feature learning for wireless capsule endoscopy analysis. Computers in Biology and Medicine, 79(1), 163–172. https://doi.org/10.1016/j.compbiomed.2016.10

 PubMed Web of Science ®Google Scholar

Li, C., Zhang, S., Zhang, H., Pang, L., Lam, K., Hui, C., & Zhang, S. (2012). Using the K-Nearest neighbor algorithm for the classification of lymph node metastasis in gastric cancer. Computational and Mathematical Methods in Medicine, 2012, 1–11. https://doi.org/10.1155/2012/876545

 Web of Science ®Google Scholar

Wang, Y., Tavanapong, W., Wong, J., Hwan, J., & De Groen, P. C. (2015). Polyp-Alert : Near real-time feedback during colonoscopy. Computer Methods and Programs in Biomedicine, 120(3), 164–179. https://doi.org/10.1016/j.cmpb.2015.04.002

 PubMed Web of Science ®Google Scholar

Souaidi, M., Ait, A., & El Ansari, M. (2019). Multi-scale completed local binary patterns for ulcer detection in wireless capsule endoscopy images. Multimedia Tools and Applications, 78(10), 13091–13108. https://doi.org/10.1007/s11042-018-6086-2

 Web of Science ®Google Scholar

Li, B., & Meng, M. Q. H. (2012). Tumor recognition in wireless capsule endoscopy images using textural features and SVM-based feature selection. IEEE Transactions on Information Technology in Biomedicine, 16(3), 323–329. https://doi.org/10.1109/TITB.2012.2185807

 PubMedGoogle Scholar

Yuan, Y., & Meng, M. Q. (2014). Polyp classification based on bag of features and saliency in wireless capsule endoscopy. IEEE International Conference on Robotics and Automation (ICRA), 3930–3935.

 Google Scholar

Karargyris, A., & Bourbakis, N. (2011). Detection of small bowel polyps and ulcers in wireless capsule endoscopy videos. IEEE Transactions on Biomedical Engineering, 58(10 PART 1), 2777–2786. https://doi.org/10.1109/TBME.2011.2155064

 PubMed Web of Science ®Google Scholar

Li, B., & Meng, M. Q. (2012). Expert systems with applications automatic polyp detection for wireless capsule endoscopy images. Expert Systems With Applications, 39(12), 10952–10958. https://doi.org/10.1016/j.eswa.2012.03.029

 Web of Science ®Google Scholar

Hirasawa, T., Aoyama, K., Tanimoto, T., Ishihara, S., Shichijo, S., Ozawa, T., Ohnishi, T., Fujishiro, M., Matsuo, K., Fujisaki, J., & Tada, T. (2018). Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer: Official Journal of the International Gastric Cancer Association and the Japanese Gastric Cancer Association, 21(4), 653–660. https://doi.org/10.1007/s10120-018-0793-2

 PubMed Web of Science ®Google Scholar