Deep learning model to improve melanoma detection in people of color

Abstract

Melanoma is a type of skin cancer that is particularly dangerous to people with dark skin. This is due to the disease's late diagnosis and detection in people with dark skin. When melanoma is detected, the prognosis is often poor. Advancement in Artificial Intelligence (AI) technology and image classification has brought about tremendous progress and applications in medicine and diagnosis of skin cancer. Albeit, these techniques continue to produce unsatisfactory results when applied to people with dark skin. This work considered the trend in cancer detection using AI techniques. Dark skin melanoma clinical images were acquired and pre-processed to remove illumination and noise to aid the other stages such as segmentation and data augmentation. The acquired images were combined with a curated and augmented version of the Human Against Machines 10000 image (HAM10000) dataset and split into two classes: melanoma and non-melanoma. A pre-trained DenseNet121 was used as a base model for training and in addition, a transfer learning process was performed to exclude the top layer and fine-tune all the layers in the final Dense Block of the model pre-trained on ImageNet. The model achieved an accuracy of 99% for melanoma detection in white skin color and 98% for dark skin. The results show that the proposed model is effective in detecting melanoma in skin.

Keywords:

• Artificial intelligence
• cancer detection
• deep learning
• dermatology
• melanoma
• transfer learning

1. Introduction

Skin cancer is less common in people with darker skin, but when it does occur, the morbidity and mortality rates are higher than in Caucasians (Abdulhak & Moiin, 2020). This is due to the lack of diagnosis and treatment in the earlier stages of its development. It is usually discovered in the late stages of development in people of color (Gloster & Neal, 2006; Ramnani, 2020). Authors in (Gloster & Neal, 2006) recommended that clinicians focus on the preventive measures and examinations that reduce the significant threat posed by these cancers.

Basal cell carcinoma (BCC), Kaposi sarcoma, squamous cell carcinoma (SCC), and malignant melanoma (MM) are some of the cancer types that are most common in people with dark skin tone, especially in black people and people of African descent (Nthumba, Cavadas, & Landin, 2011). Also, they are some of the most prevalent skin problems in sub‐Saharan Africa (Gohara, 2015; Nthumba et al., 2011).

This work is primarily concerned with melanoma in black skin. Melanoma has the potential to spread at an alarming rate. This makes it a very disturbing form of cancer (Khan et al., 2019). Melanoma can occur anywhere on the body but it most commonly develops on the soles of the feet.

Melanoma incidence has increased steadily over the past 30 years amongst Africans, Asian Indians, Chinese Asians, and African Americans. The melanoma case analysis carried out in South Africa on dark-skinned people revealed that a majority of the cases were acral lentiginous melanoma (ALM), with the foot and hand being the most frequent location of occurrence (de Giorgi et al., 2006; Mikhail, 2020; Swerdlow, 1990). Although ALM accounts for less than 5% of all melanomas, making it extremely rare among white people, it is disproportionately represented in black people (de Giorgi et al., 2006; Mahendraraj et al., 2017). The diagnosis of melanoma is usually delayed, and hence the prognosis is worse than in people with lighter skin.

Artificial intelligence (AI) is a field in computer science in which machines are designed to imitate human intelligence and thinking (Jeffcock, 2018). Deep Learning (DL) is a technique in artificial intelligence that involves mimicking how the brain works to make machines process data, interpret patterns, and make decisions from perceived patterns. Usually, in deep learning, training is nonlinear and unsupervised, i.e. the data used for training a DL model is unlabelled.

In the past, DL models have generated huge successes in skin cancer diagnosis for people with lighter skin color. This is a result of the huge amount of computer vision studies in the area of dermatology.

Models have been known to outperform dermatology experts and professionals in the area of skin cancer classification which used datasets or controlled settings (Codella et al., 2017; Esteva et al., 2017; Haenssle et al., 2018).

Concerns have been raised over the lack of representation and data availability for clinical research, and in the field of dermatology with regards to darker skin (Nelson, 2020). In (Wong, Lin, Hocker, & Burgin, 2019), there were discussions about the potential for artificial intelligence to wrongly detect or classify skin cancer types in darker skin tone.

As with other forms of cancer, early detection of melanoma is essential to fast recovery. According to literature, the rate of death from melanoma can be reduced by 90% if it is detected at its initial stages. Therefore, it is of importance to carry out research to improve the artificial intelligence classification and detection of melanoma cancer in skin of color.

For this study, it is noteworthy to state that there are few differences between the dermatoscopic melanoma characteristics in the skin of white and dark people (Wong et al., 2019). Therefore, we included dark skin images of melanoma cancer with white skin images while training our model.

Advancement in artificial intelligence has equipped dermatologists and scientists with the tools for the early detection of cancer. Albeit, this has not translated across all ethnicities or races. A lack of technological development to address the threat it poses to the darker-skinned population could jeopardize AI's achievements in the fight against cancer.

There is a need to proffer techniques that will address the inadequacies of these technologies for the detection of skin cancer. For example, a network that would detect melanoma in white skin should have the same accuracy and correctness when detecting the same stage of cancer in a darker-skinned patient. People of color (POC) are less likely to develop skin cancer, but they are much more likely to die from it due to delay in detection or prevention. In POC, skin cancer is frequently discovered at a more advanced stage, making treatment challenging (Gupta et al., 2016).

The results of medical AI technologies should reflect robustness across ethnicity in order to be considered complete. Early detection of cancer improves the likelihood of recovery from the disease. In this work, the proposed model improved the prognosis of melanoma among patients with darker skin tones using medical technology that does not discriminate based on ethnicity.

The remaining parts of this paper are organized as follows. Section 2 presents a review of related works. Section 3 describes the methodology used in the development of the proposed DenseNet121 model, while Section 4 presents the results and discussion. Finally the paper is concluded in Section 5.

2. Related works

In (Brinker et al., 2019), a ResNet50 CNN model was trained with clinical images. The authors aimed to classify melanoma and Atypical nevi. Their work involved training the model with 402 melanomas and 402 nevi, as the previous sample size of 20 melanoma and 100 nevi initially used was too small. Their model showed remarkable results when compared with the classification performed by 157 board-certified dermatologists. Tests carried out in (Brinker et al., 2019) with the trained model shows that it has the same accuracy as 145 dermatologists, but with an improved sensitivity. In (Haenssle et al., 2018, 2020), an Inception V4 model was trained on a dataset of more than a hundred thousand melanoma and benign skin lesions. The results of the DL model, when compared to dermatologists, show that the model achieved a comparably high mean specificity without large amounts of information available.

The ResNet-152 deep learning technique was used for the classification of twelve skin malignancies from clinical images belonging to the Asan dataset (Han et al., 2018). Their model performs superbly for the classification of BCC, a common cancer type in darker skin. Their training dataset covers skin of people with Asian descent only. A similar approach was adopted in (Fujisawa et al., 2019), whose deep learning method was used to classify 140 clinical images. Authors in (Tschandl et al., 2019) trained the ResNet50 and the InceptionV3 DL architectures on a dataset consisting of 13,724 images. Their models performance resulted in an impressive diagnosis of cancer types of skin that wasn’t pigmented. Authors in Codella et al. (2017) trained an ensemble network for the recognition of melanoma using the ISIC-2016 dataset.

Authors in (Goyal et al. 2020) investigated the challenges found in transferability of learned features across races, noting that authors in (Han et al., 2018) tested their model, trained on a predominantly Asian skin race, and received poorer results on white skin. Authors in (Sagar & Dheeba, 2020) utilized the concept of transfer learning in the classification of melanoma cancer. Their research shows that retraining a pre-trained model is not only more convenient and time-efficient, but it also results in a better performance. In their experiment, various top-of-the-shelf ConvNets were compared to establish the best for melanoma cancer detection. In (Adegun & Viriri, 2021), it was determined that ensemble learning and training ensemble models with the best performing DL models like SENet, Xception, InceptionV3 provides excellent results in classifying skin lesions and detecting cancer. However, the technique is computationally demanding. The research performed in (Akkoca-Gazioğlu & Kamasak, 2020) shows that of the four models, ResNet50, DenseNet121, VGG16, and AlexNet, DenseNet121 gives the best accuracy for melanoma classification, while ResNet50 performed relatively closer to DenseNet121. This information was useful in selecting a model for our research.

Most of the previous works used deep learning techniques to detect and classify cancer, and achieved high performances. Albeit, only authors in (Han et al., 2018) considered training their model to detect cancer in other skin types asides white skin. None of these techniques considered skin of color during training and evaluation. The method described in (Jojoa Acosta, et al., 2021) consists of two stages, the first which automatically crops the region of interest within a dermatoscopic image using the Mask and Region-based Convolutional Neural Network technique, and the second which is based on a ResNet152 structure and classifies skin lesions as either "benign" or "malignant." In (Dildar et al., 2021), the authors presented a thorough analysis of deep learning methods for early detection of skin cancer. In order to enhance medical professionals’ visual perception and diagnostic abilities to distinguish benign from malignant lesions, the authors in (Thapar, Rakhra, Cazzato, & Hossain, 2022) presented a trustworthy method for diagnosing skin cancer using dermoscopy images. Skin lesion region of interest (RoI) segmentation from dermoscopy images was achieved using swarm intelligence algorithms, and feature extraction from the RoI. The best segmentation result was achieved using the Grasshopper Optimization Algorithm.

The work of authors in (Brinker et al., 2019) is related to ours in the sense that we trained our model with most of the images obtained from the HAM10000 dataset. In this paper, knowledge obtained from the concepts outlined in (Sagar & Dheeba, 2020) and (Akkoca-Gazioğlu & Kamasak, 2020) were harnessed to develop a model with high accuracy in detecting and classifying melanoma in people of color. The developed DenseNet121 model was trained and a transfer learning process was used to fine-tune all the layers in the final Dense Block of the model that was already trained on ImageNet while excluding the top layer. In order to close the current gap in the underrepresentation of darker skin tones, this work is the first step toward increasing skin tone diversity in existing image databases. In (Rezket al., 2022a) and (Rezket al., 2022b), the authors worked on dark skin color and white skin color and proposed a deep learning method in their paper. The difference of the proposed work with (Rezk et al., 2022a) and (Rezk et al., 2022b) which are relatively similar are: In (Rezk et al., 2022a), the authors used two deep learning methods, style transfer (ST) and deep blending (DB) and used metrics such as accuracy and area under the curve (AUC), while we have used a pretrained DenseNet121 and used metrics such as Accuracy, Precision, Specificity, Recall, and F1 score. In (Rezk et al., 2022b), the authors presented a segmented model and also used metrics such accuracy, area under the curve (AUC), Precision, and Specificity for evaluation of their model. The proposed pretrained Dense Convolutional Network 121 (DenseNet121) offers a number of compelling benefits, including the elimination of the vanishing-gradient issue, improved feature propagation, promotes feature reuse, and significantly fewer parameters. DenseNets are used to keep increasing the depth of deep convolutional networks but CNNs have issues when they delve deeper. In (Rezk et al., 2022a, 2022b), DenseNet was not used, while in our study DenseNet121 was used and the proposed DenseNet121 model’s accuracy was increased by using the concept of early stopping.

3. Methodology

In this section, we describe how we generated our model, and showed the procedure performed in the three stages involved in our work. Figure 1 shows the block diagram of the entire setup involved in our work. Details of the three stages are explained in Sections 3.1 to 3.4.

Figure 1. Block diagram of proposed system.

Firstly, we acquired data to create our dataset. This was aimed at generating a unique dataset of dark skin melanoma cancer clinical cases. It included data from different online medical sources which is discussed in section 3.1.

Next, we used pre-processing techniques to reduce noise and unnecessary detail in the acquired clinical images. The dataset acquired from Kaggle, an online data resource platform, consisted of already curated and augmented dermatoscopic images. Hence, we didnt use this data in the pre-processing stage.

After the basic pre-processing processes were performed, we then performed data augmentation on the generated data. Data Augmentation is a technique in deep learning whereby copies of original images are slightly modified or new data is synthesized from the original. This creates variations of the original images which increases the dataset and also helps DL models to generalize better.

Finally, we combined the images from both datasets and fed them into our pre-trained convolutional neural network. We applied the concepts of transfer learning in this stage to fine-tune our network.

3.1. Data acquisition

As a result of the challenges associated with melanoma in darker skin which were discussed earlier, a dataset containing melanoma in dark skin types is extremely difficult to come by. Public image datasets available for cancer research such as the acquired dataset from Kaggle, are primarily filled with white skin dermoscopic images. So, we needed to manually acquire a more diverse dataset to avoid the limitations highlighted in (Goyal et al., 2020). Consequently, we began our task by performing a data collection to acquire certified melanoma cancer clinical images of dark skin cases.

The data used for this study was obtained from journal publications such as (Gloster & Neal, 2006; Gohara, 2015; Tschandl et al., 2018), medical documents (Dildar et al., 2021; Rezk et al., 2022a; Thapar et al., 2022), a newspaper publication (Rezk et al., 2022b), and an online medical image gallery (Cancer A-Z, 2022). About 100 clinical images were acquired from various certified health sources. These consist of 37 melanoma cancer images and 63 images of other common skin lesions (e.g. basal, and squamous cell carcinoma, cutaneous T-cell lymphoma, etc.) associated with darker-skinned people.

3.1.1. Ham10000

Some of the data used consisted of curated and normalized images from the HAM10k dermoscopic image dataset (Tschandl et al., 2018). Half of the diagnoses were validated by histopathology and the other half was based on a consensus by dermatologists. Instead of classifying seven different skin lesions, the dataset was split into two classes, melanoma and non-melanoma.

The groups contained 1,113 melanoma images, and 8902 images not affected by melanoma.

Data augmentation techniques were performed on the melanoma group to mitigate the effects of highly imbalanced data. For augmentation, Brightness range, rotation range, zoom range, width shift range, horizontal flip, vertical flip, height shift range operations were used. The result was a balanced dataset of 17,800 images consisting of 8,903 melanoma images and 8,902 non-melanoma images.

The data collected needed to be pre-processed to achieve an optimal performance. Clinical images in some cases are taken from low-quality cameras, different resolutions, poor and/or varying illumination, etc. For this reason, it was beneficial to perform some processes to segment lesion edges, remove unnecessary artifacts, remove noise, etc., from our images. The various pre-processing steps performed are explained in Section 3.2.

3.2. Data pre-processing

To ensure that our model receives correct information for the training stage, we performed the following stages of pre-processing such as cropping, denoising, illumination correction, segmentation, and data augumentation. These stages are discussed in Sections 3.2.1 to 3.2.5.

3.2.1. Cropping

In this stage, we eliminated unnecessary artifacts such as tapes, clothing items, hospital materials from our acquired images. For our task, the information we require in the images is the skin lesion region and the skin surrounding the lesion. Hence, we ensured that all the irrelevant information in the images were removed without affecting the important parts.

3.2.2. Denoising

Gaussian blur is an image processing application that results in the blurring of an image to reduce noise. We took the cropped images and performed an image smoothing process, where we blurred the images using a Gaussian filter (sigma = 2).

3.2.3. Illumination correction

Illumination correction was performed to reduce the effect of lighting sources on the images acquired. Clinical images are taken in different lighting conditions, and with different camera types. As a result, some of these images have light reflecting off their surfaces, and some areas of the images are exposed to more light than other areas. Therefore, it was necessary to correct the illumination, contrast, and sharpness of the images. To extract these features, the Hue, Saturation, and Value (HSV) percentages of the images was determined. Afterwards, the images were converted to HSV color space and sharp SV channel changes were detected similar to the procedure in (Giotis et al., 2015). We performed edge correction to ensure that the lesion edges are not lost during pre-processing.

3.2.4. Segmentation

Adaptive thresholding was used to segment the images, especially to detect and remove objects on the skin. Thereafter, the images were then converted back to the RGB color space and the output was superimposed bitwise with the original image to provide the CNN with images which have reduced noise and better-segmented lesion areas. With this, our network could better discriminate between normal skin areas and cancer-affected areas in the images.

3.2.5. Augmentation

Data augmentation was performed on the pre-processed images to generate similar images to assit the network during training. At the end of this stage, we had a total of 569 pre-processed clinical images consisting of 282 melanoma and 287 non-melanoma.

Therefore, in total 18374 images consisting of 9185 melanoma (8903 light skin, 282 colored skin), and 9189 (8902 light skin, 287 colored skin) non-melanoma images were used. 11,081 random images from this set was used in training the proposed model, while 3,647 images and 3646 images was used for validating and testing the model respectively. The combined dataset used for training the proposed model is presented in Table 1.

Table 1. Combined dataset used for training the proposed model.

Dataset	Melanoma	Non-melanoma	Total (images)
Curated HAM10000 (before augmentation)	1113	8902
Pre-processed images (before augmentation)	37	63
Curated HAM10000 (after augmentation)	8903	8902	17805
Pre-processed images (after augmentation)	282	287	569
Total (after augmentation)	9185	9189	18374

3.3. Proposed CNN architecture

In this section, present the proposed DenseNet, a convolutional network architecture that was improved upon and deployed in this work. Extraction of a useful and discriminative feature set is a challenging task (Huang et al., 2017; Salau, 2021; Salau & Jain, 2019). As a result of this, we used the strength of CNN for feature extraction and for automatic melanoma detection. CNN examines various structures of images and extracts the appropriate aspects of input images for classification. In this work, a pretrained DenseNet121 model was proposed for the classification task due to its superior performance in blurred and noisy image detection as compared to the RESNet50. Authors in (Huang et al., 2017) developed the Dense Convolutional Network (DenseNet). One of the major characteristics of this ConvNet is that it does not need as much feature maps as other networks because it does not relearn the feature maps. DenseNets are narrow and reuse features learned from previous layers of the network to maximize the potential of the trained network. A major benefit of DenseNet is that it allows for gradient flow and information passage which ensures the ease of training across the network. All layers have access to the loss function and initial input’s gradient, which results in an implicit deep control that helps the training of deeper network architectures. The proposed DenseNet121 architecture including the DenseLayer, Transition Blocks, and Dense Blocks is shown in Figure 2.

Figure 2. Architecture of the proposed DenseNet121 containing Dense Block (Dx), Transition Block (Tx), Dense Layer (DLx).

The DenseNet consists of 4 Dense Blocks with a varying number of layers. DenseNet improves the flow of information between the layers, and the connectivity is such that each layer receives feature maps from all previous layers as input. Consequently, the $l^{th}$ layer in the network receives input from all the previous layers, $x_0,\dots ,x_{l-1}:$ (1) $$x_l=\mathrm{ }{H}_l\left(\left[x_0,x_1,\dots ,x_{l-1}\right]\right)$$(1) where [$x_0,x_1,\dots ,x_{l-1}$] is the channel-wise concatenating operation of feature maps in previous layers, $0, \dots ,l-1.$ The initial $H_l(.)$ convolution, which combines the multiple inputs in Equation (1)(1) $$x_l=\mathrm{ }{H}_l\left(\left[x_0,x_1,\dots ,x_{l-1}\right]\right)$$(1) into a single tensor, is not viable since feature maps are constantly increasing. It was replaced by a combined operation that involves a trio of batch normalization (BN), Rectified Linear Unit (ReLu), and a 3×3 convolution (Conv). Down-sampling was performed by splitting the network into Dense Blocks separated by an intermediary layer where convolution and average pooling were performed. This intermediary is known as the transition layers.

3.3.1. Growth rate

The growth rate k is a hyperparameter found in the DenseNet. It represents the number of feature maps generated by the operation $H_l.$ This means that, for the $l^{th}$ in a network, we would expect the input to that layer to be $k_0+\mathit{\text{k x }}(l-1)$ number of feature maps. This requires very heavy computation. DenseNets have very narrow layers as compared to other network architecture.

3.3.2. Densenet-B

A 1×1 bottleneck layer was introduced within the transition layer (before the 3×3 Conv) so that the input of the feature maps to the next layer is reduced to $4k.$ The new variant network with a bottleneck in the transition layer is referred to as a DenseNet-B. The function [BN-ReLU-Conv (1×1) − BN- ReLu-Conv (3×3)] replaces the composite function $H_l$ in a DenseNet-B.

3.3.3. Densenet-C

For DenseNet-C model compactness, the feature map input number is reduced in the transition layer. A transition layer, directly following a dense block containing a map of m number of features, generates $\left[{\theta }_m\right]$ output feature maps, where $0<\theta \le 1,$ whereas authors in (Huang et al., 2017; Saraiya, 2022) set $\mathit{\text{θ at }}0.5$ in their experiment.

3.3.4. Densenet-BC

When both bottlenecking and compression are combined, the model is referred to as a DenseNet-BC. For the ImageNet, a variant called DenseNet-BC was used. The proposed model is built from the DenseNet121 pre-trained on ImageNet. So essentially, our implementation consists mainly of a DenseNet-BC architecture with the classification layer removed and replaced with an additional custom layer. The proposed architectures parameters are presented in Table 2.

Table 2. The architecture of the proposed DenseNet model used for ImageNet, all with a fixed growth rate of 32. The “conv” shows the BN-ReLU-Conv sequence in the layer.

Layers	Output Size	DenseNet-121	DenseNet-169	DenseNet-201	DenseNet-262
Convolution	$112x112$	$7x7 \mathit{conv}, \mathit{stride} 2$
Pooling	$56x56$	$3x3\max \hspace{0.17em}\mathit{pool}, \mathit{stride} 2$
Dense Block (1)	$56x56$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}6$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}6$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}6$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}6$
Transition Layer (1)(1) $$x_l=\mathrm{ }{H}_l\left(\left[x_0,x_1,\dots ,x_{l-1}\right]\right)$$(1)	$56x56$	$1x1 \mathit{conv}$
	$28x28$	$2x2 \mathit{average} \mathit{pool}, \mathit{stride} 2$
Dense Block (2)(2) $$\mathit{Precision}=\mathrm{ }\frac{\mathit{true}\mathrm{ }\mathit{positive}\mathrm{ }\left(TP\right)}{\mathit{true}\mathrm{ }\mathit{positive}\mathrm{ }\left(TP\right)+\mathit{false}\mathrm{ }\mathit{positive}\left(FP\right)}$$(2)	$28x28$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}12$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}12$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}12$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}12$
Transition Layer (2)(2) $$\mathit{Precision}=\mathrm{ }\frac{\mathit{true}\mathrm{ }\mathit{positive}\mathrm{ }\left(TP\right)}{\mathit{true}\mathrm{ }\mathit{positive}\mathrm{ }\left(TP\right)+\mathit{false}\mathrm{ }\mathit{positive}\left(FP\right)}$$(2)	$28x28$	$1x1 \mathit{conv}$
	$14x14$	$2x2 \mathit{average} \mathit{pool}, \mathit{stride} 2$
Dense Block (3)(3) $$\mathit{Recall}=\mathrm{ }\frac{TP}{TP\mathrm{ }+fa\mathrm{l}se\mathrm{ }\mathit{negative}\mathrm{ }\left(FN\right)}$$(3)	$14x14$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}24$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}32$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}48$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}64$
Transition Layer (3)(3) $$\mathit{Recall}=\mathrm{ }\frac{TP}{TP\mathrm{ }+fa\mathrm{l}se\mathrm{ }\mathit{negative}\mathrm{ }\left(FN\right)}$$(3)	$14x14$	$1x1 \mathit{conv}$
	$7x7$	$2x2 \mathit{average} \mathit{pool}, \mathit{stride} 2$
Dense Block (4)	$7x7$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}16$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}32$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}32$	$\left[\begin{array}{c}1x1conv\\ 3x3conv\end{array}\right]\hspace{0.17em}x\hspace{0.17em}48$
Classification Layer	$1x1$	$7x7$ global average pool
		1000D fully-connected, softmax

3.4. Transfer learning

Transfer learning is a method of learning where the skills learned from training one model are applied to the training of another model. In widely used deep learning technique, models that have already been trained on one task are used to start training new models for tasks involving vision or language. Transfer learning in deep learning is done to enable people to use their knowledge of tasks learned for one domain to solve new problems more quickly or more effectively. The common deep learning strategies for transfer learning are:

1. The use of pre-trained models as feature extractors.

2. Fine-tuning the pre-trained models.

In the first strategy, the final layer of the pre-trained model, which is used for prediction, is removed and replaced with a new layer for the task. The weights of the pre-trained model are used as feature extractors, and the layers in the pre-trained model are frozen (i.e. they are not allowed to learn) during training.

In the second strategy, the final is also replaced with a new layer, and this layer is then trained by freezing some of the initial layers and leaving others to learn during the training.

The reason for this is that the lower layers within the pre-trained network learn the features which are not task-specific from the pre-trained model, so we don’t update their weights. On the other hand, higher layers in the network learn more task-specific features, so we unfreeze these higher layers to ensure they update their weights and learn during training. This is called fine-tuning the pre-trained model.

We performed transfer learning on the DenseNet121 model pre-trained on ImageNet, a dataset consisting of 14 million images. We freeze and load the pre-trained weights into our base model (DenseNet121) and connect a global average pooling, dropout, and dense layer to the output. Next, we ran our dataset through the model. We ensured that we avoided overfitting by monitoring the valuation losses during training. After training the new model, we fine-tuned the network of our model by unfreezing the final 113 (out of 426) layers of the DenseNet121 and training the entire model on the dataset. We trained the network with a learning rate of 1e-2. The final classifier in the network decides the class of input based on the feature maps on the network.

3.4.1. Training hyperparameters

The training hyperparameters are discussed in this section and presented in Table 3.

Table 3. Hyperparameters used in the proposed model.

Parameter	Value
Train-test split ratio	3:1
Learning rate	1e-2
Epoch	15
Batch size	20
Optimization algorithm	rmsprop
Momentum	0.9

1. Batch size: The batch size is defined as the number of samples worked through before updating the internal model parameters during training. We performed our training using a batch size of 20.

2. Epoch: This is the amount of work the training process undergoes before it covers the whole training set. It consists of one or more batches. For training the final layer, we set the number of epochs to 10. We set an epoch of 15 while finetuning the entire model. We achieved the best performance at epochs 8 and 12 respectively.

3. Learning rate: Learning rate, also referred to as step size, is the rate at which training weights are updated. Smaller rates mean the training weights are updated slower and the model requires more epochs. On the other hand, a higher learning rate means a faster model convergence but could cause suboptimal convergence.

4. Momentum: Momentum can be used to smoothen the progression of a learning algorithm, which, in turn can accelerate the learning process. Using high momentum during training helps the training process. To achieve similar results of high momentum, it is important to increase the epochs if momentum is not used. We used a momentum of 0.9 in our training stage.

5. Adaptive Optimisers: Adaptive optimizers are algorithms that update the learning rate while the model is training. This helps the model to generalize better and balance during training. Some examples are Adagrad, Adadelta, Adam, rmsprop, etc. We used the rmsprop for training.

3.5. Model evaluation parameters

The proposed model was evaluated using metrics such as accuracy, precision, recall, specificity, and f-measure.

Precision—This is the ratio of the actual positive in the data to the predicted positives by the model. It is a measure of how precise a model is. (2) $$\mathit{Precision}=\mathrm{ }\frac{\mathit{true}\mathrm{ }\mathit{positive}\mathrm{ }\left(TP\right)}{\mathit{true}\mathrm{ }\mathit{positive}\mathrm{ }\left(TP\right)+\mathit{false}\mathrm{ }\mathit{positive}\left(FP\right)}$$(2)

Recall—This is the measure of how well the model can determine all the appropriate cases in the dataset. It is the number of true positives divided by the total actual positives. (3) $$\mathit{Recall}=\mathrm{ }\frac{TP}{TP\mathrm{ }+fa\mathrm{l}se\mathrm{ }\mathit{negative}\mathrm{ }\left(FN\right)}$$(3)

F measure (F1-score) is a metric that is used to check the balance between precision and recall. It is a function of the other two metrics. (4) $$f1\mathrm{ }\mathit{score}=2\mathrm{ }x\mathrm{ }\frac{\mathit{precision}\mathrm{ }x\mathrm{ }\mathit{recall}}{\mathit{precision}+\mathit{recall}}$$(4)

Specificity (Spec) represents the prediction of the Normal class (non-melanoma) as expressed in Equation (5)(5) $$\mathit{Spec}=\frac{TN}{TN+FP}$$(5) . (5) $$\mathit{Spec}=\frac{TN}{TN+FP}$$(5)

4. Results and discussion

Figure 3 shows the result of the illumination correction step during pre-processing. It was observed that the melanoma edges are more defined as a result of this correction. This suggests that we would have a performance upgrade if we compared the model performance when using raw images. The pre-processed image of melanoma is shown on the right in Figure 4 as compared to our output on the right-hand side of the graph. Our model performed very well with an accuracy of 99%. We trained the new model up to an epoch of 9. From the graph in Figure 5, we observe that our model was a perfect fit for the data. It shows that the model stabilized its training losses and validation losses just beyond an epoch of 8.

Figure 3. Adaptive threshold output for performing illumination correction.

Figure 4. Clinical image of melanoma before (left) and after (right) the preprocessing stage.

Figure 5. Training and testing losses during the development of the custom melanoma classification model.

4.1. Model evaluation

The proposed method was tested through simulation using the MATLAB Programming Language and toolboxes for image processing, neural networks, and optimization on an Intel Core i5 3.4 GHz processor.

The CNN models were evaluated using other metrics including accuracy. Evaluation parameters such as precision, recall, specificity, and f-measure as given by Equations (1)–(5)(5) $$\mathit{Spec}=\frac{TN}{TN+FP}$$(5) were used to assess the models performance.

We evaluated our model using 3646 images of size 124 × 124 which were randomly split from the dataset before training the model. The proposed model achieved the same values for precision, F1-score, and recall of 0.92 for melanoma detection and specificity of 0.98 in white skin color, and achieved 0.96 for precision, 0.93 for F1-score, recall of 0.90, specificity of 0.99, and an accuracy of 98% for dark skin color. This means the number of false positives our model predicts in the test data and those of the false negatives it predicts is the same for white skin color. The proposed DenseNet model’s accuracy was increased by using the idea of early stopping. The results for the models performance are presented in Table 4.

Table 4. Models performance on white and black skin.

Evaluation metrics for dark skin		Evaluation metrics for white skin
Accuracy	0.98	Accuracy	0.99
Precision	0.96	Precision	0.92
Specificity	0.99	Specificity	0.98
Recall	0.90	Recall	0.92
F1 score	0.93	F1 score	0.92

We used the International Skin Imaging Collaboration (ISIC) archive, where the majority of our initial data was acquired, to compare our model to other melanoma detection works, such as (Hasan et al., 2019; Hosny et al., 2019; Sarkar et al., 2019). The results show that our model performed similarly to the work presented in (Hosny et al., 2019) which also used a custom model based on fine-tuning the AlexNet. The model in (Hosny et al., 2019) was trained on white skin images only. Thus, the goal of creating a model that performs better or similarly to existing models while predicting melanoma in dark skin was accomplished.

Transfer learning is a method in computer vision that enables us to build accurate models faster. We leveraged on this to use the resources and features learned from the DenseNet121 model to train our model. This is more efficient than starting the learning process from scratch. Also, with a smaller dataset, we were able to achieve excellent results with transfer learning. It was observed that having a larger dataset can improve the performance of AI in detecting melanoma and other skin cancer types in darker skin. In (Salau & Jain, 2019; Wei et al., 2019; Yi, Walia, & Babyn, 2019), the authors stated that Generative Adversarial Networks (GANs) can be used to generate high-quality replicas of an image and can also be used to create imaging data that appear real. We have established that GANs can be beneficial when generating synthetic melanoma images that look original to solve the issue of lack of data (Bissoto et al., 2018; Brennard & Buster, 2021). In Table 5, a comparative analysis was presented with existing related works. Its is noteworthy to state that our suggested strategy outperforms earlier work in numerous performance metrics. The proposed model was able to correctly categorize melanoma cancer in people with white skin faster than that of black skin achieving 1.2s and 2.3s respectively.

Table 5. Proposed model’s performance compared to other related works.

Authors	Accuracy (%)	Precision (%)
Thapar et al. Thapar et al., (2022)	98.4	97.7
Rezk et al. (2022a)	76.0	Not mentioned
Rezk et al. (2022b)	94.0	Not mentioned
Sarkar et al. Sarkar, Chatterjee, and Hazra, (2019)	99.6	99.5
Hasan et al. Hasan, Barman, Islam, and Reza, (2019)	83.2	93.0
Hosny et al. Hosny, Kassem, and Foaud, (2019)	92.3	95.9
Jain et al. Jain et al., (2021)	92.0	91.0
Bi, Kim, Ahn, Feng, and Fulham (2016)	92.0	87.5
Abbes and Sellami (2019)	87.5	90.1
Proposed approach (for white skin and black skin)	99.0 and 98.0	92.0 and 0.96

The dataset used for training the model in this work consists of images obtained as secondary data. The source of the data is a custom generated dataset that includes low-quality clinical images from online medical sources. As a result, this experiment, if reproduced, may achieve a different result with higher-quality images or a standard dataset.

5. Conclusion

One of the most prevalent diseases that can be initially identified by visual observation and further identified with the aid of dermoscopic examination and other tests is skin cancer. Melanoma detection in people with darker skin tones requires attention due to the identified gap in the literature. In this paper, we used cutting-edge computer vision and artificial intelligence methods for skin cancer image classification. We leveraged on the knowledge from previous works to create a robust AI solution for classifying melanoma. In this paper, we developed a model that can correctly categorize melanoma cancer in people with darker skin. With the suggested model, 99% accuracy rate was achieved. Data augmentation was used to increase the number of images needed for the experiment as deep learning methods require large amounts of data. As it is widely known, having enough data aids in improving a deep learning models performance. In the future, interested academics can conduct further research to ascertain how other data augmentation techniques, like generative adversarial networks, will enhance model performance to make up for the lack of clinical data for cancer in people of color.

Competing interests

The authors declare that they have no competing interests.

Funding

Authors declare no funding for this research.

Disclosure statement

No potential conflict of interest was reported by the authors.

Availability of data

The dataset generated during and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request.