Land Usage Identification with Fusion of Thepade SBTC and Sauvola Thresholding Features of Aerial Images Using Ensemble of Machine Learning Algorithms

ABSTRACT

Detecting the appropriate usage of a piece of land is known as Land Usage Mining. The key resource to detect the Land Usages is aerial Images. The advancement in technology in form of satellites, drones, unmanned aerial vehicles do capture the tons of wide land cover images. Aerial images are used for various purposes alias getting an overview to put up the settlement by making construction, extracting mineral deposits, disaster mitigation planning, disaster recovery, and surveillance. Automated land usage identification with help of modern machine learning algorithms may be a great boon to mankind. Different feature extraction methods are being explored to represent aerial image content in the signature form, these features are mainly taken as a global or local content description of the image. Feature Level Fusion of both the global and the local content description features may give a more accurate capability for identification of land usage. The paper proposes feature level fusion of global features extracted using Thepade’s Sorted Block Truncation Coding (Thepade SBTC) and local features extracted using Sauvola Thresholding for land usage identification. Consideration of more than one Machine Learning classifiers as an ensemble has proven better than individual Machine Learning classifiers. Thepade SBTC is explored in aerial image feature extraction with nine variations as TSBTC 2-ary, TSBTC 3-ary, TSBTC 4-ary, TSBTC 5-ary, TSBTC 6-ary, TSBTC 7-ary, TSBTC 8-ary, TSBTC 9-ary, and TSBTC 10-ary. The experimentation is done on UC Land Merced Dataset having 2100 images spread across 21 land usage types. Here the land usage identification accuracy, Matthews Correlation Coefficient (MCC), and F Measure have shown better performance in TSBTC 10-ary global feature extraction method. Further, the TSBTC 10-ary global features are concatenated with Sauvola thresholding-based local features for feature level fusion, which show the performance improvement of the proposed land usage identification technique. Also, the ensembles of machine learning algorithms are deployed for performance assessment along with the individual nine machine learning algorithms for the proposed land usage identification technique. The majority voting-based ensemble of ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’ has resulted in better accuracy of land usage identification.

Introduction

An entity whose consumption rate is zero but without which none of the production is attainable is a land, which is used for multiple purposes and personified by humans as well as by Mother Nature. Satellites, Unmanned Aerial Vehicle (UAV), Drones are enabling the availability of such aerial images in digital form. Such aerial images obtained as the bird’s eye view may help in understanding the purpose for which the specific region or piece of land is being utilized. Automated understanding of land usage would be very interesting in various fields of study of umpteen applications. The aerial images are very rich in contents like colors, shapes, textures. These contents are taken into consideration for the identification of land usage. The aerial image contents are extracted and represented in form of the global features or local features. These extracted features of aerial images are utilized for training of the machine learning algorithms for automatic land usage identification. The machine learning algorithms help in giving the artificial intelligence support and understanding of land usage based on extracted aerial image features and discovery of the purpose for which the land is being utilized. Selecting proper content of aerial images to form the feature vectors and fusion of global and local content descriptors for better identification of land usages is explored in the paper here. The paper also explores and presents experimentation of ensemble of multiple machine learning algorithms for decision-making as a better option than individual machine learning algorithms for land usage identification using aerial images.

The paper is formed as per the trailing sections elaborated herewith. Section II gives a literature review. The proposed methodology for land usage identification is given in Section III. Section IV comprises of the experiment environment being set up for the proposed methodology. Results obtained for the proposed technique are analyzed in Section V and the conclusion is in Section VI.

Literature Review

The majority of researchers have attempted deploying various Machine Learning Algorithms with different image content descriptors for automatic identification of the land usage, from aerial digital image repository. Mainly the development happened till date in land usage identification of aerial images can be briefly categorized into two types as the extraction of features to represent image content with the selection of appropriate machine learning algorithms and devising the architecture of deep convolutional neural networks.

In deep learning, the main advantage is feature extraction is not explicitly done, but the enormous training time and need for huge training data are the limitations.

In (Yansheng et al. 2016) authors have considered multiple features like local binary pattern (LBP), gray level co-occurrence matrix (GLCM), maximal response 8 (MR8) and scale-invariant feature transform (SIFT) with help of four different neural networks for remote sensing image retrieval (Yansheng et al. 2016) using proposed collaborative affinity fusion approach. This work proposes the generation of the aerial image descriptors to be used in various applications. The generation of features is a much tedious and complex process here.

The framework to annotate satellite using proposed multifeature joint sparse coding (MFJSC) is proposed in (Zheng et al. 2013). These annotations are further used in spatial relation constrained classifier (SRCC). The method is not rotation invariant as it is constrained by spatial relations.

The convolutional neural network is used for feature extraction and XGBoost is used as a classifier in (Jiang et al. 2018) for satellite image land usage classification. Here the feature extraction is dependent on CNN.

CNN needs training for the same. In (Castelluccio et al. 2015) the pertained CNNs named CaffeNet and GoogLeNet are used with the transfer learning concept of land usage classification.

The scaling invariant feature transform (SIFT) based bag of visual words (BoVW) features of remote sensing satellite images are used with deep convolutional neural network (DCNN) VGG19 for classification of high-resolution remote sensing scenes is proposed in (Gong, Yuanyuan, and Xie 2018).

The method proposed in (JingboChen et al. 2018) uses the computational human attention visual model for feature extraction which needs human intervention and attention for feature extraction. Further, these features are used with CNN for classification. The need for human attention is a limitation of this method.

While the features are extracted mainly in global or in local perception, while the combination of both perceptions needs to be explored. Also, individual machine learning classifiers are deployed for land usage identification, the more interesting results could be observed using ensembles of these machine learning algorithms.

Proposed Aerial Imagery Land Usage Identification Technique

The proposed method of land usage identification may be visualized in two phases as the training phase and testing phase as shown in Figure 1, where the training of machine learning algorithms is done using the extracted features of aerial images taken from the dataset. For each aerial image the feature extraction unit extracts the features using Thepade SBTC N-ary as well as Sauvola thresholding logic. The fusion of Thepade SBTC and Sauvola features is considered together in ‘features fusion of TSBTC N-ary + SAUVOLA’. The individual machine learning classifiers and ensembles are trained using these features of aerial images.

Figure 1. Block Diagram depicting two phases of proposed land usage identification technique using aerial images with feature level fusion of TSBTC and Sauvola thresholding with ensemble of machine learning algorithms

In the testing phase, a query aerial image is supplied to the feature extraction module. The extracted features of the query aerial image are given to the already trained machine learning algorithms or an ensemble for final identification of the land usage.

The global feature extraction method Thepade’s Sorted N -ary Block Truncation Coding (TSBTC N-ary) and local feature extraction method using Sauvola thresholding are elaborated in subsections A and B herewith. The sub sections C and D respectively are elaborating on the feature level fusion in the proposed method and ensemble of machine learning algorithms with majority voting logic.

Thepade’s Sorted N-ary Block Truncation Coding (Badre and Thepade 2016) Based Aerial Image Feature Vector Generation

Let the aerial image be AE (v,w) of size ‘rxc’, with three color planes as R, G, and B. The feature vector of TSBTC N-ary may be considered as [TR₁, TR₂, TR_n, TG₁, TG₂, TG_n, TB₁, TB₂, TB_n]. Here the TR_i, TG_i, and TB_i are an indication the i’th cluster centroids of individual color planes computed using TSBTC N-ary.

In TSBTC-2 ary, for aerial image AE (v,w) of size ‘rxc’ pixels; the color planes R, G and B be converted in one dimensional array and be sorted as sortR, sortG, and sortB. Using these sorted one-dimensional color planes, the TSBTC-2ary feature vector is computed as [TR₁, TR₂, TG₁, TG₂, TB₁, TB₂] as shown in equations 1(1) $T{R}_1=\frac{2}{rxc}\sum _{p=1}^{\frac{rxc}{2}}sortR(p)$(1) –6.

(1) $T{R}_1=\frac{2}{rxc}\sum _{p=1}^{\frac{rxc}{2}}sortR(p)$(1)

(2) $T{R}_2=\frac{2}{rxc}\sum _{p=1+\frac{rxc}{2}}^{rxc}sortR(p)$(2)

(3) $T{G}_1=\frac{2}{rxc}\sum _{p=1}^{\frac{rxc}{2}}sortG(p)$(3)

(4) $T{G}_2=\frac{2}{rxc}\sum _{p=1+\frac{rxc}{2}}^{rxc}sortG(p)$(4)

(5) $T{B}_1=\frac{2}{rxc}\sum _{p=1}^{\frac{rxc}{2}}sortB(p)$(5)

(6) $T{B}_2=\frac{2}{rxc}\sum _{p=1+\frac{rxc}{2}}^{rxc}sortB(p)$(6)

Here in proposed aerial image land usage identification, in all ten variations of TSBTC are experimented as TSBTC 2-ary, TSBTC 3-ary, TSBTC 4-ary, TSBTC 5-ary, TSBTC 6-ary, TSBTC 7-ary, TSBTC 8-ary, TSBTC 9-ary, and TSBTC 10-ary.

Sauvola Thresholding-Based Aerial Image Feature Vector Generation

Let the aerial image be AE (v,w) of size ‘rxc’, with three color planes as R, G, and B. The feature vector of Sauvola Thresholding may be considered as [SR₁, SR₂, SG₁, SG₂, SB₁, SB₂] as given equations 20(20) $S{R}_1=\frac{1}{\sum _{b=1}^c\sum _{a=1}^{r}BR(a,b)}\left\{\sum _{b=1}^c\sum _{a=1}^r\left[R(a,b)\ast BR(a,b)\right]\right\}$(20) –25.

Let k be the constant value as given in equation 7(7) $k=0.34$(7) . Let for each pixel window of size 3 × 3 centered at pixel position (p,q); the Sauvola Thresholds for red, green and blue color planes be computed as TR_(p,q), TG_(p,q), and TB_(p,q) as given in equations 8(8) $T{R}_{(p,q)}=Mean{R}_{(p,q)}\ast \left[1+k\ast \frac{dev{R}_{(p,q)}}{M{R}_{(3,3)}-1}\right]$(8) , 9(9) $T{G}_{(p,q)}=Mean{G}_{(p,q)}\ast \left[1+k\ast \frac{dev{G}_{(p,q)}}{M{G}_{(3,3)}-1}\right]$(9) and 10(10) $T{B}_{(p,q)}=Mean{B}_{(p,q)}\ast \left[1+k\ast \frac{dev{B}_{(p,q)}}{M{B}_{(3,3)}-1}\right]$(10) . Here the devR_(p,q), devG_(p,q), and devB_(p,q) denote the red, green, and blue deviations of the respective pixel window centered at (p,q) as shown in equations 11(11) $dev{R}_{(p,q)}=\sqrt{{(\sum _{x=1}^3\sum _{y=1}^{3}IR(x,y))}^2-{\left[(\sum _{x=1}^3\sum _{y=1}^{3}IR(x,y))\right]}^2}$(11) , 12(12) $dev{G}_{(p,q)}=\sqrt{{(\sum _{x=1}^3\sum _{y=1}^{3}IG(x,y))}^2-{\left[(\sum _{x=1}^3\sum _{y=1}^{3}IG(x,y))\right]}^2}$(12) , and 13(13) $dev{B}_{(p,q)}=\sqrt{{(\sum _{x=1}^3\sum _{y=1}^{3}IB(x,y))}^2-{\left[(\sum _{x=1}^3\sum _{y=1}^{3}IB(x,y))\right]}^2}$(13) with a maximum of window wise deviation values computed in equations 14(14) $M{R}_{(p,q)}=max(dev{R}_{(p,q)})$(14) , 15(15) $M{G}_{(p,q)}=max(dev{G}_{(p,q)})$(15) , and 16(16) $M{B}_{(p,q)}=max(dev{B}_{(p,q)})$(16) as MR_(p,q), MG_(p,q), and MB_(p,q)

(7) $k=0.34$(7)

(8) $T{R}_{(p,q)}=Mean{R}_{(p,q)}\ast \left[1+k\ast \frac{dev{R}_{(p,q)}}{M{R}_{(3,3)}-1}\right]$(8)

(9) $T{G}_{(p,q)}=Mean{G}_{(p,q)}\ast \left[1+k\ast \frac{dev{G}_{(p,q)}}{M{G}_{(3,3)}-1}\right]$(9)

(10) $T{B}_{(p,q)}=Mean{B}_{(p,q)}\ast \left[1+k\ast \frac{dev{B}_{(p,q)}}{M{B}_{(3,3)}-1}\right]$(10)

(11) $dev{R}_{(p,q)}=\sqrt{{(\sum _{x=1}^3\sum _{y=1}^{3}IR(x,y))}^2-{\left[(\sum _{x=1}^3\sum _{y=1}^{3}IR(x,y))\right]}^2}$(11)

(12) $dev{G}_{(p,q)}=\sqrt{{(\sum _{x=1}^3\sum _{y=1}^{3}IG(x,y))}^2-{\left[(\sum _{x=1}^3\sum _{y=1}^{3}IG(x,y))\right]}^2}$(12)

(13) $dev{B}_{(p,q)}=\sqrt{{(\sum _{x=1}^3\sum _{y=1}^{3}IB(x,y))}^2-{\left[(\sum _{x=1}^3\sum _{y=1}^{3}IB(x,y))\right]}^2}$(13)

(14) $M{R}_{(p,q)}=max(dev{R}_{(p,q)})$(14)

(15) $M{G}_{(p,q)}=max(dev{G}_{(p,q)})$(15)

(16) $M{B}_{(p,q)}=max(dev{B}_{(p,q)})$(16)

For all pixel 3 × 3 windows of the aerial image AE (v,w) the pixel values can be compared with the individual threshold values to obtain the binary maps as BR, BG, and BB as given in equations 17(17) $BR(m,n)=\begin{array}{cc}\left\{\begin{array}{c}1,\\ 0,\end{array}\right.& \begin{array}{c}\begin{array}{cc}if& AE(m.n)>=T{R}_{(p.q)}\end{array}\\ \begin{array}{cc}if& AE(m.n)<T{R}_{(p.q)}\end{array}\end{array}\end{array}$(17) , 18s(18) $BG(m,n)=\begin{array}{cc}\left\{\begin{array}{c}1,\\ 0,\end{array}\right.& \begin{array}{c}\begin{array}{cc}if& AE(m.n)>=T{G}_{(p.q)}\end{array}\\ \begin{array}{cc}if& AE(m.n)<T{G}_{(p.q)}\end{array}\end{array}\end{array}$(18) and 19(19) $BB(m,n)=\begin{array}{cc}\left\{\begin{array}{c}1,\\ 0,\end{array}\right.& \begin{array}{c}\begin{array}{cc}if& AE(m.n)>=T{B}_{(p.q)}\end{array}\\ \begin{array}{cc}if& AE(m.n)<T{B}_{(p.q)}\end{array}\end{array}\end{array}$(19) .

(17) $BR(m,n)=\begin{array}{cc}\left\{\begin{array}{c}1,\\ 0,\end{array}\right.& \begin{array}{c}\begin{array}{cc}if& AE(m.n)>=T{R}_{(p.q)}\end{array}\\ \begin{array}{cc}if& AE(m.n)<T{R}_{(p.q)}\end{array}\end{array}\end{array}$(17)

(18) $BG(m,n)=\begin{array}{cc}\left\{\begin{array}{c}1,\\ 0,\end{array}\right.& \begin{array}{c}\begin{array}{cc}if& AE(m.n)>=T{G}_{(p.q)}\end{array}\\ \begin{array}{cc}if& AE(m.n)<T{G}_{(p.q)}\end{array}\end{array}\end{array}$(18)

(19) $BB(m,n)=\begin{array}{cc}\left\{\begin{array}{c}1,\\ 0,\end{array}\right.& \begin{array}{c}\begin{array}{cc}if& AE(m.n)>=T{B}_{(p.q)}\end{array}\\ \begin{array}{cc}if& AE(m.n)<T{B}_{(p.q)}\end{array}\end{array}\end{array}$(19)

(20) $S{R}_1=\frac{1}{\sum _{b=1}^c\sum _{a=1}^{r}BR(a,b)}\left\{\sum _{b=1}^c\sum _{a=1}^r\left[R(a,b)\ast BR(a,b)\right]\right\}$(20)

(21) $S{R}_2=\frac{1}{\sum _{b=1}^c\sum _{a=1}^r\left[1-BR(a,b)\right]}\left\{\sum _{b=1}^c\sum _{a=1}^r\left\{R(a,b)\ast \left[1-BR(a,b)\right]\right\}\right\}$(21)

(22) $S{G}_1=\frac{1}{\sum _{b=1}^c\sum _{a=1}^{r}BG(a,b)}\left\{\sum _{b=1}^c\sum _{a=1}^r\left[G(a,b)\ast BG(a,b)\right]\right\}$(22)

(23) $S{G}_2=\frac{1}{\sum _{b=1}^c\sum _{a=1}^r\left[1-BG(a,b)\right]}\left\{\sum _{b=1}^c\sum _{a=1}^r\left\{G(a,b)\ast \left[1-BG(a,b)\right]\right\}\right\}$(23)

(24) $S{B}_1=\frac{1}{\sum _{b=1}^c\sum _{a=1}^{r}BB(a,b)}\left\{\sum _{b=1}^c\sum _{a=1}^r\left[B(a,b)\ast BB(a,b)\right]\right\}$(24)

(25) $S{B}_2=\frac{1}{\sum _{b=1}^c\sum _{a=1}^r\left[1-BB(a,b)\right]}\left\{\sum _{b=1}^c\sum _{a=1}^r\left\{B(a,b)\ast \left[1-BB(a,b)\right]\right\}\right\}$(25)

Fusion of Thepade SBTC and Sauvola Thresholding Features of Aerial Images

The TSBTC 10-ary (Thepade et al. 2018) and Sauvola Thresholding features are concatenated to get the feature level fusion for land usage identification. Let the aerial image be AE (v,w) of size ‘rxc’, with three color planes as R, G, and B. The feature vector of fusion of TSBTC 10-ary and Sauvola thresholding can be represented as [TR₁, TR₂, TR₁₀, TG₁, TG₂, TG₁₀, TB₁, TB₂, TB_10, SR₁, SR₂, SG₁, SG₂, SB₁, SB₂]. The individual feature vector elements can be extracted as per the mathematical model elaborated in section A and section B herewith.

Ensemble of Machine Learning Algorithms (Badre and Thepade 2016)

The machine learning algorithms from Bayes, Rule, Lazy, and Tree families are considered here with an ensemble of multiple algorithms with a majority voting concept.

From Bayes family the BayesNet; from Rule family SMO and Simple Logistic; from Lazy family Kstar and IB1; from Tree family J48, Random Forest, random Tree, and REPTree are explored here in proposed land usage identification methods. The ensemble with majority voting of IB1, Random Forest, and Simple Logistic is also explored.

The experimentation environment setup to explore the effectiveness of the proposed technique and possible variations is given in the next section.

Experimentation Environment

The experimentation for the proposed land usage identification technique is done in MATLAB with help of the Weka tool. The UC Merced land use dataset (Yang and Newsam 2010) is taken for experimentation. The dataset has a total of 2100 images depicting 21 different categories of land usage. These images are taken from the images of the Urban Area Imagery collection of USGS National Map. Every image is of size 256 × 256 with a one-foot pixel resolution. The types of land usage are Agricultural, Baseball Diamond, Airplane Field, Beach, Chaparral, Buildings, Forest, Dense Residential, Harbor, Freeway, Golf Course, Medium Residential, Intersection, Mobile Home Park, Parking Lot, Overpass, River, Sparse Residential, Runway, Storage Tanks, and Tennis Court. The sample aerial images of each of these categories are shown in Figure 2.

Figure 2. Sample aerial images of 21 categories of land usage from UC Merced land use dataset (Yang and Newsam 2010)

The performance appraise of all variations of the proposed land usage identification technique is done using the percentage accuracy of land usage identification, the Matthews Correlation Coefficient (MCC) and F Measure, which can be computed as given in equations 26(26) $\mathrm{L}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{U}\mathrm{s}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{I}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{(\mathrm{T}\mathrm{p}+\mathrm{T}\mathrm{n})}{(\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n}+\mathrm{T}\mathrm{n}+\mathrm{F}\mathrm{p})}$(26) , 27(27) $\mathrm{M}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{w}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{C}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}(\mathrm{M}\mathrm{C}\mathrm{C})=\frac{(\mathrm{T}\mathrm{p}\ast \mathrm{T}\mathrm{n})-(\mathrm{F}\mathrm{p}\ast \mathrm{F}\mathrm{n})}{\sqrt{(\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{p})\ast (\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n})\ast (\mathrm{T}\mathrm{n}+\mathrm{F}\mathrm{p})\ast (\mathrm{T}\mathrm{n}+\mathrm{F}\mathrm{n})}}$(27) , and 28(28) $\mathrm{F}\mathrm{M}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}=2\ast \frac{\left[\left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{p}}\right)\ast \left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n}}\right)\right]}{\left[\left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{p}}\right)+\left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n}}\right)\right]}$(28) , respectively.

(26) $\mathrm{L}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{U}\mathrm{s}\mathrm{a}\mathrm{g}\mathrm{e}\mathrm{I}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{(\mathrm{T}\mathrm{p}+\mathrm{T}\mathrm{n})}{(\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n}+\mathrm{T}\mathrm{n}+\mathrm{F}\mathrm{p})}$(26)

(27) $\mathrm{M}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{w}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{C}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}(\mathrm{M}\mathrm{C}\mathrm{C})=\frac{(\mathrm{T}\mathrm{p}\ast \mathrm{T}\mathrm{n})-(\mathrm{F}\mathrm{p}\ast \mathrm{F}\mathrm{n})}{\sqrt{(\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{p})\ast (\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n})\ast (\mathrm{T}\mathrm{n}+\mathrm{F}\mathrm{p})\ast (\mathrm{T}\mathrm{n}+\mathrm{F}\mathrm{n})}}$(27)

(28) $\mathrm{F}\mathrm{M}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}=2\ast \frac{\left[\left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{p}}\right)\ast \left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n}}\right)\right]}{\left[\left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{p}}\right)+\left(\frac{\mathrm{T}\mathrm{p}}{\mathrm{T}\mathrm{p}+\mathrm{F}\mathrm{n}}\right)\right]}$(28)

where,

Tp = True Positive = True Positive is an end result where the aerial image land usage type gets correctly identified

Fp = False Positive = False Positive is an end result where the aerial image land usage gets inaccurately identified

Tn = True Negative = True Negative is an end result where the technique correctly identifies the aerial image not belonging to the considered land usage type

Fn = False Negative = False Negative is an end result where the technique inaccurately identifies the aerial image not belonging to the considered land usage type to which it actually belongs

Results and Discussion

The proposed land usage identification technique is experimented on UC merced land use dataset with 228 variations with nine machine learning algorithms and three ensemble combinations, nine variations of global feature extraction using TSBTC N-ary, local feature extraction using Sauvola thresholding and nine variations with feature fusion of global and local features.

The performances appraise using percentage accuracy of land usage identification of the TSBTC N-ary global features with considered variations is given in Figure 3 for relevant machine learning algorithms and ensembles. Here the performance improves with the increasing level of TSBTC N-ary for each machine learning algorithm. The percentage improvement is quite impressive initially from TSBTC 2-ary till TSBTC 8-ary. The performance improvement is just marginal from TSBTC 8ary to TSBTC 10-ary. Overall the accuracy given by TSBTC 10-ary is the best in the case of all machine learning algorithms as well as the ensembles considered of ‘IB1+ Random Forest+ Simple Logistic’, ‘IB1+ Random Forest+ KStar’, and ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’.

Figure 3. Percentage accuracy-based performance appraise of variations of TSBTC N-ary global features for respective machine learning algorithms and ensembles in proposed land usage identification technique

The percentage accuracy-based performance comparison of the nine machine learning algorithms (Bayes Network, Simple Logistic, Binary SMO, KStar, IB1, J48, Random Forest, Random Tree, REPTree) and three ensembles (‘IB1+ Random Forest + Simple Logistic’, ‘IB1+ Random Forest+ KStar’ and ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’) is given in Figure 4 for respective TSBTC N-ary global feature vector extraction method variants in proposed land usage identification technique. Here the ensembles do perform better than consideration of individual machine learning algorithms almost for all variants of TSBTC N-ary global feature extraction methods. Overall best performance is observed by an ensemble ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’ and second best by the ensemble ‘IB1+ Random Forest+ Simple Logistic’.

Figure 4. Percentage accuracy-based performance appraise of considered machine learning algorithms and ensembles for respective TSBTC N-ary global feature vector extraction method variants in proposed land usage identification technique

Figure 5 shows the percentage accuracy-based performance appraise of machine learning algorithms and ensembles for Sauvola thresholding-based local feature vector extraction method in the proposed land usage identification technique. Here the ensembles have given higher percentage accuracy of land usage identification as compared to individual machine learning algorithms. Overall best performance is observed in the ensemble ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’ immediately seconded by the ensemble ‘IB1+ Random Forest+ Simple Logistic’.

Figure 5. Percentage accuracy-based performance appraise of machine learning algorithms and ensembles for Sauvola thresholding based local feature vector extraction method in proposed land usage identification technique

Figure 6 gives the Performance Appraise of undertaken ensembles of machine learning algorithms for relevant Feature Level Fusion combinations of local features of Sauvola and global features of TSBTC N-ary in proposed land usage identification technique. The ensemble ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’ has given the best performance immediately seconded by the ensemble ‘IB1+ Random Forest+ Simple Logistic’.

Figure 6. Performance Appraise of considered ensembles of machine learning algorithms for respective Feature Level Fusion combinations of local features of Sauvola and global features in proposed land usage identification technique

Figure 7 shows the Performance Appraise of Feature Level Fusion of local features of Sauvola and global features of respective TSBTC N-ary for considered ensembles of machine learning algorithms in the proposed land usage identification technique. Here the performance is getting gradually improved with increasing level of TSBTC from TSBTC 2-ary till TSBTC 8-ary. From TSBTC 8-ary to TSBTC 10-ary the marginal improvement is observed. Overall the fusion of Sauvola and TSBTC 10-ary has shown the best performance across all considered ensembles.

Figure 7. Performance Appraise of Feature Level Fusion of local features of Sauvola and global features of respective TSBTC N-ary for considered ensembles of machine learning algorithms in proposed land usage identification technique

Figure 8 represents the percentage accuracy-based performance appraise of various feature extraction methods considered as global features with TSBTC N-ary, local features with Sauvola and feature level fusion of TSBTC N-ary and Sauvola for individual machine learning algorithms and respective ensembles in proposed land usage identification technique. Here for all machine learning algorithms, the fusion of local and global features taken as ‘Sauvola+ TSBTC-10 ary’ gives best performance as indicated by the highest accuracy of land usage identification. The global features taken with TSBTC 10-ary are better than the local features computed using Sauvola thresholding. Overall the ensembles have given better percentage accuracy than individual machine learning algorithms.

Figure 8. Percentage accuracy-based performance appraise of various feature extraction methods considered as TSBTC N-ary, Sauvola and feature level fusion of TSBTC N-ary and Sauvola for individual machine learning algorithms and respective ensembles in proposed land usage identification technique

Figure 9 represents the Matthews Correlation Coefficient (MCC)-based performance appraise of various feature extraction methods considered as global features with TSBTC N-ary, local features with Sauvola and feature level fusion of TSBTC N-ary and Sauvola for individual machine learning algorithms and respective ensembles in proposed land usage identification technique. Here for all machine learning algorithms, the fusion of local and global features taken as ‘Sauvola+ TSBTC-10 ary’ gives the best performance as indicated by higher MCC value. The global features taken with TSBTC 10-ary are better than the local features computed using Sauvola thresholding. Overall the ensembles have given better MCC values than individual machine learning algorithms.

Figure 9. Matthews Correlation Coefficient (MCC) based performance appraise of various feature extraction methods considered as TSBTC N-ary, Sauvola and feature level fusion of TSBTC N-ary and Sauvola for individual machine learning algorithms and respective ensembles in proposed land usage identification technique

Figure 10 represents the F Measure-based performance appraise of various feature extraction methods considered as global features with TSBTC N-ary, local features with Sauvola and feature level fusion of TSBTC N-ary and Sauvola for individual machine learning algorithms and respective ensembles in proposed land usage identification technique. Here for all machine learning algorithms, the fusion of local and global features taken as ‘Sauvola+ TSBTC-10 ary’ gives the best performance as indicated by higher F Measure value. The global features taken with TSBTC 10-ary are better than the local features computed using Sauvola thresholding. Overall the ensembles have given better F Measure values than individual machine learning algorithms.

Figure 10. F Measure based performance appraise of various feature extraction methods considered as TSBTC N-ary, Sauvola and feature level fusion of TSBTC N-ary and Sauvola for individual machine learning algorithms and respective ensembles in proposed land usage identification technique

Conclusion

Due to advancements in technology the aerial images are being easily accessible these days. Various applications do demand the automated analysis and understanding of these aerial images. Land usage identification from aerial images is one of such tasks gaining momentum these days. Machine learning algorithms used with extracted aerial image features may help in automated land usage identification. The paper proposes feature level fusion of global features extracted using Thepade SBTCN-ary and local features extracted using Sauvola thresholding with the ensemble of the machine learning algorithms for better land usage identification. Experimentation done on 2100 images of UC merced land usage of aerial images dataset shows higher identification accuracy, higher Matthews Correlation Coefficient (MCC) and higher F Measure values in feature level fusion as compared to individual features proving the worth of proposed technique. Also identification accuracy of the ensemble of machine learning algorithms is observed to be better than the deployment of individual machine learning algorithms. Overall best performance is observed in the fusion of Thepade SBTC10-ary and Sauvola thresholding features for an ensemble of ‘IB1+ Random Forest+ Simple Logistic+ SMO+ KStar’. Further, it would be interesting to see the fusion of different features extracted using various methods for land usage identification from aerial images.