Intelligent System for Diagnosis of Erythema Migrans

Abstract

Lyme borreliosis is the most common human tick-borne infectious disease of the Northern Hemisphere. One of the first signs of the disease is erythema migrans, a skin lesion that appears within days to weeks after an infected tick bite. In this article, a novel intelligent system for erythema migrans recognition is presented based on image and text information, applicable for individual and clinical web-based use. Novelties of our approach include a combination of visual and textual attributes, a new combination of visual attributes (geometrical, color, and Gabor-filter based), and a new algorithm for calculation of color-based attributes. Procedurally, the intelligent system for erythema migrans recognition integrated in a web-based application facilitates provisional diagnosis of erythema migrans in the general population and assists general medical practitioners in their decisions. Several classification methods—Naïve Bayes, Support Vector Machine, Adaboost, and Random forest—were tested in order to achieve improved performance.

INTRODUCTION

Lyme borreliosis (LB) is a multisystem disease (Steere 2001; Steere, Coburn, and Glickstein 2004), caused by Borrelia burgdorferi sensu lato and transmitted by a tick bite. Early course of LB is characterized by an expanding skin lesion named erythema migrans (EM). The lesion typically appears after a tick bite as a small redness at the site of the bite and expands over a period of days to weeks, often to an oval lesion with a central clearing (Strle and Stanek 2009; Stanek et al. 2011). The EM may be accompanied by fatigue, fever, headache, mild stiff neck, arthralgia, and myalgia, but such symptoms are not indicative of LB if they occur in the absence of EM (Steere, Coburn, and Glickstein 2004). Without treatment, EM resolves within several weeks to months but the infection may progress and affect skin, nervous system, and/or joints, and less frequently eyes and/or heart (Steere 2001; Steere, Coburn, and Glickstein 2004; Stanek et al. 2012). Early detection of the disease and proper antibiotic treatment successfully prevent later, more harmful manifestations of LB.

Intelligent systems help to improve the diagnosis of various diseases due to growing computing power and advances in artificial intelligence. When properly used, intelligent systems can help physicians with diagnosis (Avci 2012; Übeyli and Doğdu 2008; Zolnoori et al. 2012; Vila-Francés et al. 2013, Barbosa, DeVito, and Felippe Filho 2009; De Toro, Aroba, and Ros 2011; Lavrač et al. 1998). However, for some diseases, such as LB, the design of an intelligent system presents a particular challenge because physicians combine various sources of information and knowledge.

Most of the related approaches for computer-aided diagnosis of skin diseases use image processing techniques. Intensive researches are made in the field of skin cancer (Celebi et al. 2007; Blum et al. 2004; Burroni et al. 2005; Patwardhan, Dhawan, and Relue 2003; Abbas et al. 2013; Maglogiannis and Charalampos 2009; LeAnder et al. 2010) computer-aided diagnosis systems, but not in the field of infectious diseases, including early LB. Safi et al. (2012) proposed a machine learning approach to classify melanocytic lesions into malignant and benign skin lesions. They combined color, shape, and texture information for the classification made with a Support Vector Machine (SVM) algorithm with polynomial kernel. An alternative computer-aided diagnosis of melanoma skin lesion was proposed by Garnavi, Aldeen, and Bailey (2012). The novelty of their system lies in the optimized selection and integration of textural, border-based, and geometrical properties of skin lesions. Classification was performed with the use of SVM, Random forest, Logistic model tree, and Hidden Naïve Bayes classifier. Furthermore, Abbas et al. (2012) developed a novel pattern-classification system based on the extraction of color, architectural order, symmetry of pattern, and homogeneity. These attributes were applied to the multiclass SVM classifier.

Another related detection system is classification of color and texture attributes of the human tongue that reflects the patient health status (Ning et al. 2012; Zhang, Xu, and Cai 2009). For the classification of the patient health status, a classification transductive SVM method was proposed by Zhang, Xu, and Cai (2009).

Certain similarities to our work are also found in the field of allergic lesion recognition (Roullot et al. 2005; Huttunen et al. 2011).

Because analysis of the course and outlook of EM and description of the disease symptoms represent the bases of the clinical diagnosis of this early manifestation of LB, we decided to design an intelligent system that would incorporate both visual and textual information. Our objectives were to facilitate the decision for or against the need of medical help in the general population and to assist general medical practitioners in their decisions.

Recognition of EM is a challenging task because the color of the skin lesion typically varies from very pale to very intense red, less frequently blue or brown shades, or even a combination of several colors. Furthermore, the diameters of skin lesions in our database vary from 2 centimeters to 30 centimeters or even larger. In addition, according to the database that we collected, the differences between symptoms associated with EM and symptoms associated with non-EM are small, differing only in details. In this study, an intelligent system for EM (ISEM) recognition that combines several sources of information is presented. It describes the materials and methods used in the intelligent system, gives experimental results, and provides discussion. The conclusion section summarizes our findings and introduces further research.

DESIGN AND STRUCTURE

The ISEM in Figure 1 is designed for web diagnosis to advise whether medical help is needed. Web users are asked by the ISEM to provide image and textual information in the form of a questionnaire concerning the course and outlook of EM. If the decision to see a medical practitioner is reached, the ISEM can assist the practitioner by enabling the access to previous images and text information of the patient using the unique patient identity key (UPIDK), and the line of the reasoning of how the system reached its opinion.

FIGURE 1 Flowchart of the proposed ISEM use.

For the obtained input, ISEM first processes the image and questionnaire text responses. The information from both sources is integrated in two phases: the edge detection of the skin lesion and the recognition of potential EM.

Because the edge detection affects the results of the aforementioned subsequent step, the process of the edge detection is of main importance (Abbas, Fondón, and Rashid 2011). This process is divided into three steps. The first step is called image scaling, wherein images are resized to the same size. In the second step, the images are processed to reduce noise and improve image contrast (Celebi, Iyatomi, and Schaefer 2009). The third step in edge detection is segmentation. Most commonly used segmentation methods in medical imaging (Couprie, Najman, and Talbot 2011) and, therefore, suitable for segmentation of skin lesions are contour tracking methods, region growing method (Roullot et al. 2005; Iyatomi et al. 2008), GrowCut method (Ayoub, Hajdu, and Nagy 2012), Random Walker method (Wighton et al. 2009), active contours (Kass, Witkin, and Terzopoulos 1988; Zhang, Song, and Zhang 2010), level sets, geodesic active contours, Graph Cut method (Felzenszwalb and Huttenlocher 2004; Peng, Zhang, and Yang 2009), watershed algorithms (Vincent and Soille 1991), scale multiplication algorithms (Zhang and Paul 2002; Paul, Zhang, and Wu 2005), generic segmentation models, and deformable segmentation models (McInerney and Terzopoulos 1996; He et al. 2008). Similar to edge detection, the recognition of EM is also divided into steps. In the first step, calculation of visual attributes gained from edges of skin lesion images is performed. These attributes are then combined with textual attributes and are used together for the second step, named classification (Garnavi, Aldeen, and Bailey 2012), to class EM.

Figure 2 illustrates the algorithmic structure of the proposed ISEM system. The touchscreen input skin lesion marker (LM; the user draws a curve around the skin lesion) defines the position of the skin lesion on the image and is used further on for calculation of a label matrix, an input matrix needed for the functioning of GrowCut (Vezhnevets and Konouchine 2005) segmentation. Furthermore, the text input requires that the user provides additional answers to the questions related to EM.

FIGURE 2 ISEM system. i) User input: skin lesion image, LM, web-based questionnaire. ii) Edge detection: a) image scaling and preprocessing, b) segmentation. iii) Recognition: c) visual and textual attributes, d) classification.

In the following sections, the database and methods used in the ISEM are described.

DATABASE

In 2012 and 2013, an image database was acquired by taking photographs of skin lesions of patients with suspected EM, examined at the Lyme borreliosis Outpatients Clinic of the Department of Infectious Diseases, University Medical Center Ljubljana, Slovenia, using a commercial camera Canon EOS 600D. Each skin lesion image from the database is accompanied with textual description of the lesion’s course and outlook as well as information on associated symptoms related to the early LB, obtained at the medical examination of the patient. Out of 181 skin lesions located on different parts of the human body, 113 cases of EM have been determined by a physician experienced in the diagnosis of LB. The remaining 68 skin lesions represent negative close-match cases of EM skin lesions.

GROWCUT-LM SEGMENTATION

Our LM was used to improve the functioning of the GrowCut method (Vezhnevets and Konouchine 2005). The calculation of LM first requires the user to draw a curve around the skin lesion. Second, from the outlier curve or the outlier points, we automatically calculate inlier points by uniformly decreasing the outlier curve using the MATLAB function maketform (MathWorks 2012a) and affine transformation for scaling, which is defined as follows:

(1)

where experimentally obtained scaling factor values for our image database are After the scaling of the outlier curve or outlier points, the inlier curve or the inlier points are translated to the center of the outlier curve or the outlier points.

RECOGNITION OF EM

Visual Attributes

After the extraction of skin lesion edges with the GrowCut-LM segmentation method, the parameters eccentricity, small axis b, large axis a, orientation o, and center c were obtained using the MATLAB regionprops (MathWorks 2012b) function. Based on the geometrical, color, and textural properties of the skin lesion image, the following visual attributes were defined:

• eccentricity e: MATLAB regionprops function

• axis ratio r:

(2)

• ellipse focus f:

  (3)

• Color: standard deviation (Sc) and median (Mc):

  For the calculation of new color band attributes, let A(i,j,k) with dimension M*N*3 be the skin lesion RGB image, which is first converted to HSV color space described by Issac Niwas et al. (2012). The H component of the HSV color space was experimentally chosen out of H, S, and V for further investigation and is denoted as H(i,j). Vectors X(α) and Y(α) represent the x and y (respectively) coordinates of points on the ellipse that lie just outside the skin lesion edge represented in the parametric form, while X_n(α) and Y_n(α) represent the points on the ellipse n lying inside the skin lesion edge. Figure 3 shows the ellipses represented with dots and the skin lesion is represented by the curve.

  FIGURE 3 The outer ellipse f (Y(α), X(α)) and the inner ellipse n f(Y_n(α), X_n(α)).

  

  The procedure for calculation of points on the ellipses in parametric form in Figure 3 is as follows:

  (4)
  (5)
  where x_c and y_c represent the center of the segmented skin lesion and are calculated with the MATLAB regionprops function. Parameter α increases with equal step from 0 to 2π, z is a factor that sets the outlier ellipse radius, and β is the angle between the y axis of the image coordinate system and the ellipse a axis. Furthermore, a_n and b_n define the size of inlier ellipse n. The size of inlier ellipses increases with equal step to form an appropriate number of the inner ellipses.

  Afterward, matrices C and C_n are formed as zero matrices and are filled with values of 1 to define a binary matrix representing an ellipse contour:

  (6)
  (7)

  The matrices O(i,j) and O_n(i,j) represent the bands of H component values for each ellipse and are computed by multiplying the H(i,j) with the matrices C(i,j) and C_n(i,j) as defined by Equation (8) and Equation (9):

  (8)
  (9)
  where is an element by element product. The mean value L of nonzero pixels is calculated from O(i,j), while the O_n(i,j) is used for calculation of the mean values L._n for each inner ellipse separately. The differences l_n in the mean color of inlier ellipse n versus the mean color of the outlier ellipse are computed by subtracting L from L._n. Figure 4 shows the inner ellipses O₁(i,j), O₂(i,j), O_n(i,j) and outer ellipse O(i,j), represented with dots whereas the skin lesion is represented by the curve. The inner and outer ellipses are then used for the computation of l_n with the corresponding Equation (10):
  (10)
  and are used for the final calculation of attributes Sc and Mc.

  FIGURE 4 Computation of difference l_n.

  

• Gabor: magnitude mean value (Mρ) and standard deviation (Sρ):

  For the calculation of the mean value and standard deviation of the normalized response magnitude, a two-dimensional normalized Gabor filter ψ(u,v) (Kamarainen, Kyrki, and Kalviainen 2006) was used. For the calculation of normalized response S(u,v) of ψ(u,v) for the previously mentioned H color space component of segmented skin lesion image φ(u,v), the first step is to calculate a two-dimensional fast Fourier transform (MathWorks 2013a) Λ(u′, v′) of image φ(u,v):

  (11)
  where 0 ≤ u′ ≤ M−1 and 0 ≤ v′ ≤ N − 1. Furthermore, a two-dimensional inverse fast Fourier transform (MathWorks 2013b) of element-by-element product of Λ(u′, v′) and φ(u,v) is as follows:
  (12)
  where 0 ≤ u ≤ M − 1 and 0 ≤ v ≤ N − 1.

  From the S(u,v) the magnitude ρ(u,v) was calculated by the following equation:

  (13)
  where ρ(u,v) was used further on for the calculation of Mρ and Sρ.

Textual Attributes

The first part of textual attributes used in the system depicts the course of EM symptoms related to the early LB and includes answers to the following questions:

• Is the tick bite (TB) on the site of skin lesion (binary attribute zero or one)?

• What is the duration (from 0 onward) of skin lesion in weeks (DW)?

• What is the number of local problems (LP), including itching, burning, and pain (possible answers are: from 1 to 3)?

• What is the number of general problems (GP), including fatigue, malaise, headache, muscle pain, joint pain, fever, shiver, sleepiness, insomnia (possible answers are: from 1 to 9)?

• What is the spread (S) of the skin lesion (binary attribute zero or one)?

The second part of the textual attributes describes the additional visual properties of the skin lesion:

• Does the skin lesion have a form of a ring shape (RS) (binary attribute zero or one)?

• What is the size of the minor diameter (MI) of the skin lesion in centimeters at the time of image capture?

• What is the size of the major diameter (MA) of the skin lesion in centimeters at the time of image capture?

Visual and Textual Attribute Ranking Method

After the calculation of visual attributes and integration with textual attributes, the attribute-quality ranking assessment was performed using the Relieff method. The method was verified with 10-fold cross validation.

The Relieff algorithm is used to estimate the attribute quality with respect to the two neighbors nearest to a randomly chosen instance, a nearest hit or miss. The algorithm is capable of estimating the quality of attributes with strong dependencies between attributes and is not limited to two-class problems. Furthermore, the algorithm is robust to incomplete and noisy data and provides a unified view on the attribute estimation in regression and classification (Kononenko, Šimec, and Robnik-Šikonja 1997; Robnik-Šikonja and Kononenko 1997; Robnik-Šikonja and Kononenko 2003).

Classification

The attributes described in the previous visual and textual attribute subsections were used in the experiment in which the goal was to recognize EM and thus provide the decision for the diagnosis of potential early LB and refer the web user to seek professional help. Table 1 shows the number of positive and negative cases for the experiment.

TABLE 1 Positive and Negative Cases of EM Class for all 181 Cases

Class	Positive	Negative
EM	113	68

All 181 cases of visual and textual attributes were used as inputs to MATLAB implementations of classification algorithms Naïve Bayes, SVM, Adaboost and Random forest. Several measures—accuracy, sensitivity, and F measure (F₁; Kononenko and Kukar 2007)—were calculated to estimate the performance of these classifiers. Equations for these measures are defined as follows:

(14)

(15)

(16)

where true positive (TP) is the number of cases correctly classified as EM. True negative (TN) is the number of cases correctly classified as not related to EM, whereas false positive (FP) is the number of incorrectly classified cases of EM, and false negative (FN) is the number of incorrectly classified cases not related to the EM.

Furthermore, the results of the classification were evaluated with 10-fold cross validation. Additionally, standard deviation results for the accuracy (σAcc), sensitivity (σSens), and F1 (σF1) were also calculated.

RESULTS AND DISCUSSION

The computation of mean ranks, using the Relieff method for all 181 cases, was evaluated with 10-fold cross validation. Table 2 shows the mean ranks for visual attributes, Table 3 shows the mean ranks for the textual attributes, and Table 4 the mean ranks for visual and textual attributes together. The highest mean rank in Table 2, Table 3, and Table 4 has a value of 1. From Table 2, the attributes e and Sρ have the highest mean rank, however, for Table 3 and Table 4 the attribute S has the highest mean rank.

Table 5 shows accuracy, sensitivity, and F₁ results for the classification of all 181 cases to class EM. All the results in Table 5 were evaluated with 10-fold cross validation. Table 5 also shows that the Random forest gave slightly better results for all the calculated measures to all 181 cases.

TABLE 2 Visual Attribute Mean Ranks

Attribute number	1	2	3	4	5	6	7
Attribute name	e	r	f	Mρ	Sρ	Sc	Mc
Attribute rank	1.7	3.5	2.9	5.8	1.9	5.2	7

TABLE 3 Textual Attribute Mean Ranks

Attribute number	8	9	10	11	12	13	14	15
Attribute name	TB	S	DW	RS	MI	MA	LP	GP
Attribute rank	2	1	6.8	4.3	4.7	3	6.2	8

TABLE 4 Visual and Textual Attribute Mean Ranks

Attribute number	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
Attribute name	e	r	f	Mρ	Sρ	Sc	Mc	TB	S	DW	RS	MI	MA	LP	GP
Attribute rank	6.2	8	6.8	11.3	5.8	7.9	13.7	2	1	12.7	10	4.7	3	11.9	15

TABLE 5 Classification to Class EM for all 181 Cases

Method	Accuracy (%)	σAcc (%)	Sensitivity (%)	σSens (%)	F1 (%)	σF1 (%)
Naïve Bayes	87.24	8.04	92.20	10.44	89.97	6.51
SVM	91.13	7.52	92.88	11.15	92.58	7.21
Adaboost	92.28	4.64	94.77	6.22	93.82	3.92
Random forest	93.91	3.16	98.18	3.83	95.21	2.63

Combining textual and visual attributes enabled over 93% classification accuracy. ISEM seems suitable for real-world applications, because it successfully emulates the first-level medical expert examination.

For general public tests, it is most important to classify correctly as many infected people as possible. For this purpose, a sensitivity (see Equation 15) measure was calculated. The higher the sensitivity is, the better the classification of infected people. In addition, the accuracy (see Equation 14) and F₁ (see Equation 16) measures were also calculated as additional verification of the classification model performance.

Because it was envisaged that the system will be used also on mobile devices with touchscreen display, the display was simulated using the laptop touchpad, allowing the user to draw an LM input and choose the answers related to disease symptoms. For older mobile devices, there might be problems with lens quality as well as with absence of touch-sensitive display.

CONCLUSION AND FUTURE WORK

We investigated the methodology needed for the development of the intelligent system for erythema migrans (ISEM) recognition and, thus, provide a means to make the decision for the diagnosis of potential early LB and refer a person with skin redness to seek medical help.

The study indicates the importance of attributes based on the attribute ranking (see Tables 2, 3, and 4) with the Relieff algorithm. The highest ranking attributes were the spread of the skin lesion (S), the tick bite on the site of the skin lesion (TB), the size of the major diameter in centimeters (MA), the size of the minor diameter in centimeters (MI), eccentricity (e), and the magnitude standard deviation (Sρ); see Table 4.

The proposed combination of textual and visual attributes gave satisfactory results, emulating the medical expert examination and indicating its potential usefulness for the general population. The Random forest classifier gave slightly better results for all the measures calculated (see Table 5). The decision to combine image analysis with textual analysis is based on medical practice used for the diagnosis of early LB.

In addition to the expectations of rapid development of mobile devices, we intend to perform tests on mobile devices with touchscreen display and reduce the number of attributes as much as possible in order to minimize user interaction to the point where the system performance will not decrease.