Artificial intelligence methods for the diagnosis of breast cancer by image processing: a review

Figures & data

Figure 1 Stages of systematic review.

Table 1 Advantages and disadvantages of various imaging techniques in breast cancer

Imaging method	Application	Advantage	Disadvantage
Mammography	Golden standard imaging and diagnosis of breast cancer early stages	• It uses low levels of X-rays for imaging • This method is good for detecting DCIS and calcifications • Mammography is the gold standard method to detect early-stage breast cancer before the lesions become clinically palpable	• Radiation risk and other risks • Risk of false alarm • It is difficult for the radiologist to interpret the results from mammograms as mammograms generally have low contrast • Double reading of mammogram leads to increase in the cost of detection • Mammography alone misses many cancers in dense-breasted women
Ultrasound	Suitable for dense and soft tissues	• Widely available and accessible • Noninvasive • Quick • Highly sensitive • Suitable for women with dense breasts	• Quality and interpretation of the image depends highly on the skill of the person doing the scan
Thermography	Suitable for muscle tissue	• Noninvasive	• Physicians can have difficulty interpreting the images because of the low quality and low resolution of the images taken by the first generation of the medical infrared imaging cameras

Note: Data adapted from Kerlikowske et al,²³ Heywang-Köbrunner et al,³³ and Qi and Diakides.³⁴

Abbreviation: DCIS, ductal carcinoma in situ.

Figure 2 Stages of cancer detection by image processing.

Note: Data from Pradeep et al¹⁶ and Lin et al.¹⁹

Table 2 Preprocessing techniques

Preprocessing image techniques
FPN	FPN is the result of differences in responsivity of the detectors to incoming irradiance. It is a common problem when working with FPA. FPN for a particular configuration can be recovered from a blackbody image for later subtraction from the thermogram sequence.
Badpixels	A badpixel can be defined as an anomalous pixel behaving differently from the rest of the array. For instance, a dead pixel remains unlit (black), while a hot pixel is permanently lit (white). In any case, badpixels do not provide any useful information and only contribute to deteriorate the image contrast. A map of badpixels is generally known from the FPA manufacturer or they can be detected manually or automatically; the value at badpixel locations is then replaced by the average value of neighboring pixels.
Vignetting	Vignetting is another source of noise on thermograms that cause a darkening of the image corners with respect to the image center due to limited exposure. It depends on both pixel location and temperature difference with respect to the ambient. A correction procedure has been proposed.^Citation3
Temperature calibration	A transformation function is used to convert the grayscale values g provided by the infrared camera into a linearly incrementing physical quantity, eg, temperature. The procedure^Citation3 is used to position the IR camera in front of a reference temperature source (such as a blackbody and a thick copper plate) brought to various known temperatures. As the reference temperature source is varied, the IR images are recorded. Average of the central pixels in the field of view allows getting the calibration curve through a polynomial fit
Noise smoothing	One of the most useful preprocessing (and postprocessing) techniques is noise smoothing. For instance, neighbor processing can be performed bypassing a mask or kernel through the image. More elaborate noise removal techniques are available.

Note: Data adapted from Ibarra-Castanedo et al.¹⁷

Abbreviations: FPA, focal plane array; FPN, fixed pattern noise; IR, infrared.

Table 3 Image segmentation methods

Name of segmentation method	Description of method	Benefits	Limitations
Edge detection method	Depends on discontinuity detection; generally aims to situate points with less or more rapid gray-level changes	• A human-like approach • Appropriate methods to get the best picture disproportionate area	• Not good with images where edges are unclearly defined • Not suitable for images whose edges are not completely clear • Not suitable for images with a large number of edges • It has the lowest noise compared to other methods
Thresholding method	Requires images with diffident sharp edges, each of which fits a single area	• No need for prior knowledge about the image • Minimum complexity of computation	• Not good for image with no clear peaks • Not suitable for images that do not have any clear edge • Do not consider distances, so there is no guarantee for the division of continuous areas
Region-dependent method	• The same pixel in nearby areas are similar • Counting the growing area, separating, assembling or reversing them	• Does well if region homogeneity norm is painless to define • Extra noise resistant as compared to edge detection method	• The amount of memory and calculation time is very costly • Region growing relies on seed region selection and sequence by which regions, pixels are inspected • Output segments by region splitting emerge too square because of splitting format
Fuzzy method	Uses operators, mathematics, properties and rules of fuzzy inference	• Fuzzy membership function can be used to represent the degree of low properties or linguistic phrases	• Determining the fuzzy membership function is not easy • The calculations used in the fuzzy method can be difficult
Neural network method	Uses for clustering or categorization	• Does not require writing tedious programs • Could entirely exploit the parallel nature of neural net	• Need a lot of time to train • Initialization might affect the outcome

Note: Data adapted from Pradeep et al.¹⁶

Table 4 The most important texture features^16,19

Texture features
Mean
Skewness
Entropy
SD
Line likeness
Homogeneity
Smoothness
Correlation
Sum average
Sum entropy
Variance
Kurtosis
Energy
LBP
Regularity
Contrast
Coarseness
Kurtosis
Sum variance
Difference variance

Note: Data adapted from Pradeep et al¹⁶ and Lin et al.¹⁹

Abbreviation: LBP, local binary pattern.

Table 5 Artificial intelligence techniques for classifying breast cancer

Methods	Images	Attributes	Advantages	Disadvantages	Reported performance	Reference
SVM	Medical infrared thermal imaging	• Energy, contrast, correlation, sum of squares: variance, inverse difference moment, sum variance, sum entropy, entropy, difference variance, difference entropy, information measures of correlation 1, information measures of correlation 2	• Use of curve let before feature extraction	–	Accuracy: 90.91 Sensitivity: 81.82 Specificity: 100	^Citation35
SVM	Ultrasound	• Histogram features • MRF • Tamura features • GLRLM features • GLCM features • Morphological features (RS, aspect ratio, C, NRV, Pratio)	• Combined textural and morphological features • Being independent of settings of US systems or different US machines in the diagnosis	–	Accuracy: 95.85 Sensitivity: 96 Specificity: 91.46	^Citation36
Fuzzy SVM	Ultrasound	• One pixel distance in the mean of the information measure of correlation (d=1) • One pixel distance in the mean maximum correlation coefficient (d=1) • One pixel distance in the range of different entropies (d=1) • Two pixels distance in the variance ranges (d=2) • Two pixels distance in the range of the information measure of correlation (d=2) • Four pixels distance in the mean maximum correlation coefficient (d=4) • Eight pixels distance in the mean of the information measure of correlation (d=8) • The mean of inverse different moment with the distance of 16 pixels (d=16)	• Using of fractal dimensions • Powerful in solving two-class classification problem • The importance of the training sample can be measured by the fuzzy membership	• In practical classification problems, the effects of the samples in training dataset may be different	Accuracy: 94.25 Sensitivity: 91.67 Specificity: 96.08	^Citation37
Cascade-forward back-propagation artificial neural network	Mammography	• Autocorrelation • Correlation • Maximum probability • Dissimilarity energy • Entropy • Inverse difference normalized	• Features, extracted from the known mammogram images, are stored in the knowledge base	–	Accuracy: 67.8	^Citation38
Feed-forward back-propagation artificial neural network	Mammography	• Autocorrelation • Correlation • Maximum probability • Dissimilarity energy • Entropy • Inverse difference normalized	• Using image registration techniques	–	Accuracy: 92.8%	^Citation39
k-NN	Mammography	• GLCM features (angular second moment, entropy, contrast, local homogeneity, correlation, shade, prominence, variance, sum average, sum entropy, difference entropy, sum variance, and difference variance) • GLRLM features (SRE, LRE, GLNU, RLNU, and RPERC) • LTEM features (mean, SD, range, skewness, and kurtosis)	• Texture analysis of tissue surrounding shows promising results in computer-aided diagnosis of breast cancer and may contribute to the reduction in unnecessary biopsies	• Completion of the proposed method should include the investigation of additional classification schemes and textural features, as well as validation over a larger dataset	Accuracy: 64.0–89.0 Sensitivity: 72.2–92.6 Specificity: 54.4–87.0	^Citation40
SVM	Mammography	• Wavelet features • Statistical intensity-based features (mean value, SD, smoothness, entropy, skewness, kurtosis, uniformity, shape features, compactness, aspect ratio, area to background percentage, perimeter ratio)	• Intensity- and shape-driven features have the greatest role on the later classification stage • Normal–abnormal classification stage before benign–malignant classification can be effective and reduce false positives • Find out a specific hyperplane among many • Do not require independent assumptions	• Obvious since increasing the sensitivity will yield in a system that identifies more regions as abnormal in order to achieve higher sensitivity • The classification accuracy is slightly reduced after performing benign– malignant classification	Accuracy: 93.06 Sensitivity: 97.82 Specificity: 89.09	^Citation41
Genetic algorithm and back-propagation neural network	Medical infrared thermal imaging	• Age • Mean • Differences among the two breasts • Variance • Skewness • Kurtosis • Entropy • Thermal pattern of breasts	• This method is beneficial to patients with and without symptoms	• Still need to improve the methods	Accuracy: 70 Sensitivity: 50 Specificity: 75	^Citation9
SVM	Medical infrared thermal imaging	• Kurtosis • Skewness • Contrast • Inverse difference moment • Correlation	• Generates input–output mapping functions which can be used for classification or for regression • Use of four texture features, namely, moment 1, moment 3, run percentage, and gray-level nonuniformity for representing the thermograms	• Improved by extracting better texture features and by using a larger sample size	Accuracy: 88.10 Sensitivity: 85.71 Specificity: 90.48	^Citation42
k-NN	Medical infrared thermal imaging	• Angular second moment (energy) • Contrast • Correlation • Sum of squares: variance • Inverse difference moment (homogeneity) • Sum average • Sum variance • Sum entropy • Entropy • Difference variance • Difference entropy • Information measure of correlation 1 • Information measure of correlation 2 • Autocorrelation • Dissimilarity • Cluster shade • Cluster prominence • Maximum probability • Inverse difference normalized • Inverse difference moment normalized	• Detection using texture and features and minimum variance quantization is based on the data acquisition protocol parameter and the image statistics	–	Accuracy: 92.5 Sensitivity: 100 Specificity: 78.6	^Citation43
Naive Bayes	Medical infrared thermal imaging	• Same above	• Requires a small amount of training data to estimate the parameters necessary	• SVM classification results are better than the naive Bayes classification results	Accuracy: 80.0 Sensitivity: 76.9 Specificity: 85.7	^Citation44
Deep learning	Mammogram	• Density • Margin	• Classifying between malignant and benign masses • Enable to segment the diverse lesions • Avoid the image feature errors introduced from the error or inconsistency of lesion segmentation. • Can be directly applied to all testing cases • Do not need to human intervention to design	• Heavily depends on the distribution and/or diversity of the ROIs (or cases) in the specific training and testing datasets	ROC curve: 0.790±0.019	^Citation45

Abbreviations:C, circularity (the ratio of square of the perimeter to the tumor area); GLCM, gray-level co-occurrence matrix; GLNU, gray-level no uniformity; GLRLM, gray-level run length matrix; k-NN, k-nearest neighbor; LRE, long-run emphasis; LTEM, laws texture energy measures; MRF, Markov Random Fields; NRV, normalized residual value; RLNU, run length non-uniformity; ROC, receiver operating characteristic; ROI, region of interest; RPERC, run percentage; RS, overlap ratio; SRE, short-run emphasis; SVM, support vector machine; US, ultrasound.

KerlikowskeKGradyDRubinSMSandrockCErnsterVLEfficacy of screening mammography. A meta-analysisJAMA199527321491547799496

 PubMed Web of Science ®Google Scholar

Heywang-KöbrunnerSHHackerASedlacekSAdvantages and Disadvantages of Mammography ScreeningBreast Care2011632207

 Web of Science ®Google Scholar

QiHDiakidesNAThermal infrared imaging in early breast cancer detectionAugmented Vision Perception in InfraredLondonSpringer2009139152

 Google Scholar

PradeepNGirishaHSreepathiBKaribasappaKFeature extraction of mammogramsInt J Bioinform Res201241241244

 Google Scholar

LinHCChiuCYYangSNFinding textures by textual descriptions, visual examples, and relevance feedbacksPattern Recognit Lett2003241422552267

 Web of Science ®Google Scholar

GhonchehMPournamdarZSalehiniyaHIncidence and Mortality and Epidemiology of Breast Cancer in the WorldAsian Pac J Cancer Prev201617S34346

 PubMedGoogle Scholar

Ibarra-CastanedoCBendadaAMaldagueXThermographic image processing for NDTPaper presented at: IV Conferencia Panamericana de END2007Buenos Aires, Argentina

 Google Scholar

FrancisSVSasikalaMSaranyaSDetection of breast abnormality from thermograms using curvelet transform based feature extractionJ Med Syst20143842324659445

 PubMed Web of Science ®Google Scholar

PrabusankarlalKMThirumoorthyPManavalanRAssessment of combined textural and morphological features for diagnosis of breast masses in ultrasoundHuman Centr Comput Inform Sci20155112

 Web of Science ®Google Scholar

ShiXChengHDHuLMass detection and classification in breast ultrasound images using fuzzy SVMProceedings of the 2006 Joint Conference on Information Sciences, JCIS 2006Kaohsiung, Taiwan, ROCOctober 8–11, 2006ParisAtlantis Press200617591777

 Google Scholar

SainiSVijayRMammogram analysis using feed-forward back propagation and cascade-forward back propagation artificial neural networkProceedings; 2015 Fifth International Conference on Communication Systems and Network TechnologiesNew YorkIEEE201511771180

 Google Scholar

SainiSVijayROptimization of Artificial Neural Network Breast Cancer Detection System Based on Image Registration TechniquesOptimization2014105142629

 Google Scholar

KarahaliouASkiadopoulosSBoniatisITexture analysis of tissue surrounding microcalcifications on mammograms for breast cancer diagnosisBr J Radiol20078095664865617621604

 PubMed Web of Science ®Google Scholar

BasheerNMMohammedMHClassification of breast masses in digital mammograms using support vector machinesInt J Adv Res Comput Sci Software Eng201322775763

 Google Scholar

HaddadniaJHashemianMHassanpourKDiagnosis of breast cancer using a combination of genetic algorithm and artificial neural network in medical infrared thermal imagingIran J Med Phys201294265274

 Google Scholar

AcharyaURNgEYTanJHSreeSVThermography based breast cancer detection using texture features and Support Vector MachineJ Med Syst20123631503151020957511

 PubMed Web of Science ®Google Scholar

AliMASayedGIGaberTHassanienAESnaselVSilvaLFDetection of breast abnormalities of thermograms based on a new segmentation methodProceedings of the 2015 Federated Conference on Computer Science and Information SystemsNew YorkIEEE20152015255261

 Google Scholar

MilosevicMJankovicDPeulicAThermography based breast cancer detection using texture features and minimum variance quantizationExcli J201413120426417334

 PubMed Web of Science ®Google Scholar

QiuYYanSGundreddyRRA new approach to develop computer-aided diagnosis scheme of breast mass classification using deep learning technologyJ Xray Sci Technol201725575176328436410

 PubMed Web of Science ®Google Scholar