A Novel Classification Method for PolSAR Image Combining the Deep Learning Model and Adaptive Boosting of Shallow Classifiers

Abstract

Polarimetric synthetic aperture radar (PolSAR) images are classified mainly according to the backscattering information of ground objects. For regions with complex backscattering information, misclassification is easy to occur, which leads to challenges in improving the classification accuracy of the PolSAR image. Given this situation, this paper combines the Deep Learning Model and traditional classifiers to classify PolSAR image. First, the Convolution Neural Network (CNN) was used to classify the PolSAR image and according to the category prediction probability of pixels, the key pixels easily misclassified are located. Then, the adaptive boosting (AdaBoost) algorithm combined the three shallow classifiers (the Support Vector Machine (SVM), the Wishart and the Decision Tree classifier) into strong classifiers to reclassify the key pixels. Finally, the labels of key pixels and other pixels are output as the final classification result. Experiments on two PolSAR images show that the proposed method can improve classification performance and obtain better classification results.

Résumé

Les images d’un radar à synthèse d’ouverture polarimétrique (PolSAR) sont classées principalement en fonction des informations de rétrodiffusion des objets au sol. Pour les régions où l’information de rétrodiffusion est complexe, il est facile de produire des erreurs de classification, ce qui pose des défis pour l’amélioration de la précision des classifications des images PolSAR. Dans ce contexte, cet article combine un modèle d’apprentissage profond et des algorithmes traditionnels pour classer l’image PolSAR. Tout d’abord, un réseau neuronal de convolution (CNN) est utilisé pour classer l’image PolSAR et les pixels clés, facilement mal classés, sont localisés selon la probabilité de bonne classification d'une classe donnée. Ensuite, l’algorithme d’optimisation adaptatif (AdaBoost) a combiné les trois algorithmes traditionnels (la machine à vecteurs de support (SVM), Wishart et l’arbre de décision) en des algorithmes puissants pour reclasser les pixels clés. Enfin, les étiquettes des pixels clés et des autres pixels de l’image sont extraites en tant que résultat final de la classification. Des expériences sur deux images PolSAR montrent que la méthode proposée peut améliorer les performances de la classification et obtenir des résultats plus précis.

Introduction

Synthetic aperture radar actively transmits electromagnetic waves and receives echoes to obtain ground information (Lee and Pottier 2009). It is not blocked by light and clouds and can work all the time and all the weather (Hong et al. 2015). PolSAR has 4 polarization channels. It can receive both horizontal and vertical electromagnetic waves after horizontal transmission and vertical transmission to obtain fully polarized backscattering information of ground objects. It has more advantages in representing the polarization characteristics of ground objects (Wang et al. 2022). Therefore, the classification of PolSAR images is one of its important applications.

Traditional PolSAR image classification methods can be divided into unsupervised polarization classification and supervised polarization classification according to whether training samples are needed (Zhang et al. 2022). In the absence of prior knowledge, unsupervised classification refers to the method of clustering statistical analysis of images by computers based on cluster theory and establishment of decision rules for classification according to statistical characteristics of characteristic parameters of samples to be classified. For example, Van Zyl (Van Zyl 1989) proposed an unsupervised classification method based on the relationship between phase and rotation between the incident wave and the scattered wave. The k-means algorithm (Gadhiya and Roy 2020) initializes the clustering center by calculating the maximum likelihood estimate of each cluster. H/α unsupervised classification (Cloude and Pottier 1997) classifies targets according to scattering entropy H and mean scattering angle feature parameters. Supervised classification is a process in which training samples with known attribute categories are first used to train classifiers to master the statistical characteristics of each category and then classification recognition is carried out according to classification decision rules. The supervised polarization classification for PolSAR image mainly includes the maximum likelihood (Cheng et al. 2013; Shokrollahi and Ebadi 2016), the SVM (Aghababaee et al. 2013; Zhao et al. 2023) and the decision tree (Qi et al. 2012; Deng et al. 2015). The use of samples makes the precision of the supervised classification method higher than that of the unsupervised classification method to some extent (Deng et al. 2015; Santana-Cedres et al. 2019).

With the development of machine learning, researchers have found that when the target object has rich meaning, the above classification methods of shallow structures have obvious shortcomings in terms of feature extraction and generalization ability. In recent years, due to the strong learning ability of the deep learning model (Hinton et al. 2006), it can directly learn rich features of images, solve complex problems, have good portability and greatly improve the accuracy of image classification, showing good performance in PolSAR image classification (Dong et al. 2020; Liu et al. 2021). At present, deep learning models such as deep Boltzmann machine, stacked auto-encoder, CNN and capsule networks have been applied in PolSAR image classification. For example, Hua et al. (Hua and Guo 2020) proposed a multi-layer Wishart constrained Boltzmann machine model for PolSAR image classification based on the fact that PolSAR image were subject to Wishart distribution to improve classification results. Shang et al. (Shang et al. 2019) added the relation between local pixels as the classification feature and proposed a new method of PolSAR classification combining scattering power and the stack Sparse Self-encoder. To greatly reduce the annotation cost and improve the classification performance, Bi et al. (Bi et al. 2019) proposed an active deep learning method that combined active learning and CNN to classify minimum supervised PolSAR images. Cheng et al. (Cheng et al. 2021) argued that a single neuron in CNN could not represent multiple polarization attributes of land cover, while a capsule network could use a vector instead of a single neuron to represent polarization attributes and proposed a layered capsule network for the classification of PolSAR images. Although the classification accuracy of the deep learning method is high, deep learning requires a large number of calculation, has high requirements on hardware, and the model design is very complex.

From the above analysis, it can be found that any classifier has its advantages and disadvantages. Existing studies have found that the advantages of multiple classifiers can be obtained by integrating multiple classifiers (Jun-Feng and Luo 2009; Doğan and Akay 2010; Mangai et al. 2010). The classification results of multiple classifiers are integrated to obtain better classification results than that of a single classifier (Breiman 2001; Polikar 2006; Ghimire et al. 2012; Maghsoudi et al. 2012). For example, Qin et al. (Qin et al. 2017) took the layer module Constrained Boltzmann machine of deep belief net as the classifier and built the integrated boosting model with adaptive boosting(AdaBoost), which improved the classification performance of PolSAR image and avoided the requirement of large data volume. He et al. (He et al. 2020) proposed a PolSAR image classification method combining nonlinear manifold learning and a full convolutional network. Zhu et al. (Zhu et al. 2021) combined deep learning technology with traditional classifiers based on scattering features to propose a new semi-supervised PolSAR image classification method to solve the deficiency of the labeling training data set. Jiao et al. (Jiao and Liu 2016) realized the rapid realization of Wishart distance through a special linear transformation, which accelerated the classification speed of the POLSAR image and made the polarized information available for subsequent neural networks. Jamali et al. (Jamali et al. 2022) used the Haar wavelet transform to carry out effective feature extraction in deep CNN, to improve the classification accuracy of the PolSAR image.

Some pixels located in regions with complex backscattering information may have features of several land covers at the same time. If these pixels are treated the same as other pixels by the classifier, they are prone to be misclassified. The above classification methods, no matter the traditional shallow classification method, deep learning classification method, or the classification method integrating multiple classifiers, do not focus on the pixels that are easy to be misclassified, affecting the final classification results. To solve this problem, this paper reclassifies pixels that are easy to be misclassified to get the final classification result. In this paper, pixels that are easy to be misclassified are key pixels, and others are general pixels. First, CNN is used to classify the PolSAR image and divides all pixels into key pixels and general pixels according to the category prediction probabilities of each pixel. The labels of general pixels are retained. Then, the AdaBoost algorithm is used to form a strong classifier consisting of the SVM classifier, the Wishart classifier and the decision tree classifier to reclassify key pixels. Finally, the labels of all pixels are combined into the final result.

Materials and methods

Materials

The first PolSAR image was obtained by the NASA/JPL AIRSAR system on August 16, 1989, from the L-band fully polarized 4-view data over Flevoland, Central Netherlands. The image size is 750 pixels ×1024 pixels, the azimuth-oriented resolution is 12.1 m, and the range-oriented resolution is 6.7 m, including 11 land covers, namely bare land, beet, grass, lucerne, pea, potato, rape, soybean, water, wheat and wood.

Figure 1a is the Pauli RGB image of the PolSAR image. JPL Laboratory conducted a detailed survey of this area during the imaging period and obtained the ground truth reference map (Radman et al. 2022) shown in Figure 1b, which provides a basis for evaluating classification accuracy.

Figure 1. The first PolSAR image: (a) Pauli RGB image; (b) ground truth reference map.

The second experimental data of this paper is the PolSAR image collected by the EMISAR system of Denmark in Foloum on April 17, 1998. The image has a size of 1100 pixels × 876 pixels, a range resolution of 0.75 m and an azimuth resolution of 1.5 m. The scene is an L-band full PolSAR image containing five lands covers (except the background): Bare land, Broad Leaves Crop, Fine Stem Crop, Forest and Town. Figure 2a shows the Pauli RGB image and the corresponding ground truth reference map (He et al. 2020) is shown in Figure 2b.

Figure 2. The second PolSAR image: (a) Pauli RGB image; (b) ground truth reference map.

Methods

The polarimetric scattering matrix S of PolSAR is obtained by measuring the scattering echo in each resolution unit on the ground (1) $$\left[\mathit{S}\right]=\left[\begin{array}{cc}{S}_{HH}& S_{HV}\\ S_{VH}& S_{VV}\end{array}\right]$$(1)

The scattering matrix S can only describe the so-called coherent or pure scatterers, and cannot describe the so-called distributed scatterers. These scatterers can only be described statistically due to speckle noise. To reduce the influence of speckle noise, only the second-order polarization expression can be used to analyze the distributed scatterers. Coherence matrix T is one of the second-order descriptor factors. (2) $$\left[\mathit{T}\right]=\frac{1}{2}\left[\begin{array}{lll}〈{\left|S_{HH}+S_{VV}\right|}^2〉& 〈\left(S_{HH}+S_{VV}\right){\left(S_{HH}-S_{VV}\right)}^{\mathrm{*}}〉& 〈2\left(S_{HH}+S_{VV}\right)S_{HV}^{\mathrm{*}}〉\\ 〈{\left(S_{HH}+S_{VV}\right)}^{\mathrm{*}}\left(S_{HH}-S_{VV}\right)〉& 〈{\left|S_{HH}-S_{VV}\right|}^2〉& 〈2\left(S_{\mathrm{H}H}-S_{VV}\right)S_{HV}^{\mathrm{*}}〉\\ 〈{\left(S_{HH}+S_{VV}\right)}^{\mathrm{*}}{S}_{HV}〉& 〈{\left(S_{HH}-S_{VV}\right)}^{\mathrm{*}}{S}_{HV}〉& 〈{\left|S_{HV}\right|}^2〉\end{array}\right]$$(2)

Previous studies have shown that (Duan et al. 2019) backscattering information of ground objects is mainly concentrated in the coherence matrix T. Therefore, the 9 elements of coherence matrix T, namely T11, T12_real, T12_imaginary, T13_real, T13_imaginary, T22, T23_real, T23_imaginary and T33, are chosen as the classification features of the method in this paper.

The flowchart of the proposed method is shown in Figure 3. Firstly, training samples are selected by the random stratified sampling method, and CNN is trained by patches which are centered on the training samples to obtain a classification model. Next, the PolSAR image is clipped into multiple patches through sliding the window. Then the patches are fed into the CNN classification model to divide all pixels into general pixels and key pixels. Subsequently, the AdaBoost algorithm is used to compose a strong classifier based on an SVM classifier, a Wishart classifier and a decision tree classifier to reclassify key pixels. Finally, the labels of key pixels and general pixels are combined into the final classification result.

Figure 3. The flowchart of the proposed method.

Selecting training samples

The training samples of the proposed method in this paper come from the ground truth reference map corresponding to each PolSAR image, and the number of pixels of each land cover varies greatly. To avoid the adverse impact of the imbalance of the number of training samples of each land cover on the final result, this method adopts the stratified random sampling method (Tessier et al. 2023) to select training samples for each land cover. Stratified random sampling is applicable to survey objects with a large number of total units and large internal differences, and has a small sampling error.

The total number of training samples was determined according to the number of land covers. There are 11 land covers in first PolSAR image, so the total number of training samples is 11,000. There are 5 land covers in second PolSAR image, so the total number of training samples is 5000.

Finding key pixels

CNN is a kind of feed forward neural network with deep structure including convolution computation, and is one of the representative algorithms of deep learning. In this paper, CNN is used to locate key pixels. The structure of CNN adopted in the method of this paper is shown in Figure 4. The probability of dropout is 0.4, the learning rate is 0.001 and the number of iterations is 100.

Figure 4. The structure of CNN used in the proposed method.

With the training samples selected in Section “Selecting training samples” as the central pixels, the corresponding patches with size of 15 pixels × 15 pixels were constructed to train the CNN, and the classification model was obtained. Pad 7 zeros around the PolSAR image to add 14 pixels to both the rows and columns of the image. By sliding through the window, the PolSAR image was clipped to obtain 15 pixels × 15 pixels patches with each pixel as the center pixel. The patches were fed into the CNN classification model to obtain the output of the last fully connected (FC) layer and the final classification result. FC layers can complete the further fusion of features, so that the features finally seen by the neural network are global. The FC layer outputs a matrix with the size of N × C, where N is the number of pixels in the PolSAR image and C is the number of land covers. The row of this matrix represents the C label prediction probabilities of a pixel. Find out the maximum prediction probability and the second-largest prediction probability of each pixel, and the non-negative difference between the two is the probability difference. If the probability difference of a pixel is less than the given threshold, it is considered as a key pixel; otherwise, it is a general pixel. At the same time, labels of general pixels are reserved. This process is shown in Figure 5.

Figure 5. The flowchart of locating the key pixels. C is the number of land cover and P1, …, PC are the C prediction probabilities of each pixel.

Reclassifying

The SVM classifier can usually obtain higher classification accuracy (Mountrakis et al. 2011), the decision tree classifier is good at mining potential relationships between data (Hong et al. 2017), and the Wishart classifier is based on polarization coherence matrix for SAR multi-view cases (Lee et al. 1994). These three shallow classifiers widely used in the field of PolSAR classification are taken as weak classifiers to generate a strong classifier through the AdaBoost algorithm to reclassify key pixels.

Firstly, the SVM classifier, Wishart classifier and decision tree classifier were trained respectively to obtain three classification models.

SVM is a binary model (Khosravi et al. 2021). The basic idea of SVM learning is to solve the separation hyperplane that can correctly partition the training data set and has the largest geometric spacing. The separation hyperplane is ${\mathrm{w}}^{\mathrm{T}}x+b=0,$ where $x$ is a value in the hyperspace, $\mathrm{w}$ is the hyperplane normal vector, and $b$ is the distance from the hyperplane to the origin. LIBSVM open source library (Chang and Lin 2011) were used in the proposed method to realize SVM multi-class classification.

Lee et al. (Lee et al. 1994) extended the maximum likelihood (Entezari et al. 2012) rule to SAR multi-view cases and developed a supervised classification algorithm of polarimetric coherence matrix based on complex Wishart distribution. The decision rule is $\mathrm{p}\in {\omega }_i,\mathit{\text{ if }}{\omega }_i=\mathit{ArgmaxL}\left(\left[T\right]|\left[{\widehat{\mathrm{\Sigma }}}_i\right]\right),$ where ${\omega }_i$ is a class; $\mathit{Argmaxf}(x)$ returns the x value corresponding to the maximum value of $f();$ $\left[{\widehat{\mathrm{\Sigma }}}_i\right]$ is the maximum likelihood estimation of the coherence matrix. A new decision rule $\mathrm{p}\in {\omega }_i,\mathit{\text{ if }}{\omega }_i=\mathit{Argmind}\left(\left[T\right]|\left[{\widehat{\mathrm{\Sigma }}}_i\right]\right)$ can be obtained by taking the minus sign of the above formula and removing the terms unrelated to the research clustering, where $d\left(\left[T\right]\left[{\widehat{\mathrm{\Sigma }}}_i\right]\right)=ln\left|\left[{\widehat{\mathrm{\Sigma }}}_i\right]\right|+Tr\left({\left[{\widehat{\mathrm{\Sigma }}}_i\right]}^{-1}\left[T\right]\right),$ the $Tr$ is the matrix trace. Pixels are assigned to the class ${\omega }_i$ with the smallest distance.

The decision tree classifier (Yin et al. 2020) selects the best feature recursively, and divides the training samples according to the feature. The decision tree classifier measures the selection of features by information gain and selects the features with the maximum information gain after splitting. Assumes that the sample set is X and the proportion of the samples with label k is the $P_k,$ information entropy $\mathrm{H}\left(X\right)$ of X is defined as $\mathrm{H}\left(X\right)=-\sum _{k=1}{P}_k{\log }_2{P}_k.$ The smaller the value of $\mathrm{H}\left(X\right),$ the higher the purity of X is. $\mathrm{H}\left(X|Y\right)=\sum _{k=1}P\left(y_k\right)H\left(X=x_k|Y\right)$ is the conditional entropy and represents the information entropy. $P\left(y_k\right)$ represents the proportion of a feature. The information gain $\mathrm{I}\left(X,Y\right)=H\left(X\right)-H\left(X|Y\right)$ refers to the degree to which the uncertainty of the whole sample features is reduced after a certain feature is known. Finally, the decision tree classification model is obtained by pruning the generated tree.

AdaBoost (Dou et al. 2018) is an iterative algorithm, whose core idea is to use the same training set to train different weak classifiers, and then combine these weak classifiers to form a strong classifier. In this process, the weight of the samples misclassified by the previous weak classifier will increase, while the weight of the samples correctly classified will decrease.

$\left(x_1,y_1\right),\dots ,\left(x_N,y_N\right)$ is the training sample set, where $y_i\in \left\{1,-1\right\},i=1,\dots ,N$ is used to represent the label of the training sample. Initialize the weight distribution $D_1$ of the training sample set. Each training sample is assigned the same weight $w_i=1/N,i=1,\dots ,N,$ then the initial weight distribution of the training set is: (3) $$D_1=\left(w_1,\dots ,w_N\right)=\left(1/N,\dots ,1/N\right)$$(3) where $w_i$ is the weight of each training sample, and $N$ is the number of training samples.

Then the t-th iteration is implemented, t = 1, 2, 3.

A weak classifier $h$ with the lowest error rate is selected as the $H_t$ basic classifier, and the error of the basic classifier on the distribution $D_t$ is calculated (4) $$e_{\mathrm{t}}=P\left(H_t\left(x_i\right)\ne y_i\right)=\sum _{i=1}^N{w}_{ti}I\left(H_t\left(x_i\right)\ne y_i\right)$$(4) (5) $$I=\{\begin{array}{c}1,{\mathrm{ }H}_t\left(x_i\right)\ne y_i\\ 0,{\mathrm{ }H}_t\left(x_i\right)=y_i\end{array}$$(5) where $D_t=\left(w_1,\dots ,w_N\right)$ is the weight of the training sample set at the t-th iteration; $e_t$ is the error rate.

Calculate the weight of the basic classifier in the final strong classifier (6) $${\alpha }_t=\frac{1}{2}ln\left(\frac{1-e_t}{e_t}\right)$$(6) where, ${\alpha }_t$ is the weight of the weak classifier in the t-th iteration.

Update the weight distribution $D_{t+1}$ of the training samples: (7) $$D_{t+1}=\frac{D_t\left(i\right)\hspace{0.17em}\exp \hspace{0.17em}\left(-{\alpha }_t{y}_i{H}_t\left(x_i\right)\right)}{Z_t}$$(7) where $Z_t=2\sqrt{e_t\left(1-e_t\right)}$ is the normalization constant; when samples are misclassified, $y_i{H}_t\left(x_i\right)=-1;$ when samples are correctly classified, $y_i{H}_t\left(x_i\right)=-1.$

Finally, each weak classifier is combined according to its weight: (8) $$f\left(\mathrm{x}\right)=\sum _{t=1}^3{\alpha }_t{H}_t\left(x\right)$$(8)

Through the function sign, a strong classifier is obtained as follows: (9) $$H_{\mathit{final}}=\mathit{sign}\left(f\left(x\right)\right)=\mathit{sign}\left(\sum _{t=1}^3{\alpha }_t{H}_t\left(x\right)\right)$$(9)

The AdaBoost algorithm is aimed at the binary classification problem. Since the PolSAR image used in this paper contains C land covers, the proposed method in this paper realizes classification by iterating the AdaBoost algorithm C-1 times. In each iteration, one land cover was taken as 1 and other land covers were taken as −1, and a strong classifier was constructed to classify key pixels. Key pixels labeled 1 did not participate in the next iteration and their land covers were recorded. After C-1 iterations, new land covers of all key pixels were obtained.

Finally, the land covers of general pixels and key pixels were combined into the classification results of PolSAR.

Results and discussion

The method in this paper was implemented using MATLAB2022b programming, and the computer was configured with 48 GB memory and NVIDIA GeForce RTX 2080TiGPU. To verify the superiority of the proposed method, the following three aspects of comparative experiments were carried out.

Precision comparison before and after reclassification of key pixels

For the first PolSAR image, when the probability difference was 0.7, the overall classification accuracy reached the maximum of 92.10%. For the second PolSAR image, when the probability difference was 0.5, the overall classification accuracy reached the maximum of 91.02%. The key pixels shown in Figure 6 were located. It can be found that most of the key pixels are located at the edge of each land cover, that is, the area with complex backscattering information.

Figure 6. (a) Key pixels of the first PolSAR image when the probability difference is 0.7; (b) key pixels of the second PolSAR image when the probability difference is 0.5.

The number of key pixels and pixels on the ground truth reference map is shown in Tables 1 and 2.

Table 1. The number of key pixels and pixels in the ground reference map for the first PolSAR image.

Land cover	The number of key pixels	The number of pixels	Ratio (%)
Bare land	4041	4797	84.24
Beet	3221	8453	38.10
Grass	2656	16771	15.84
Lucerne	1985	10078	19.70
Pea	2841	12980	21.89
Potato	3192	16969	18.81
Rape	9065	20253	44.76
Soybean	1157	7877	14.69
Water	27	9202	0.29
Wheat	5559	38680	14.37
Wood	665	21605	3.08

Table 2. The number of key pixels and pixels in the ground reference map for the second PolSAR image.

Land cover	The number of key pixels	The number of pixels	Ratio (%)
Bare land	5903	84483	6.99
Broad leaved crop	7780	66291	11.74
Fine stem crop	13568	46703	29.05
Forest	4827	65662	7.35
Town	6310	44612	14.14

As shown in Figures 7 and 8, labels of the key pixels in two PolSAR images before and after reclassification has changed.

Figure 7. The labels of key pixels in the first PolSAR image: (a) before reclassification; (b) after reclassification.

Figure 8. The labels of key pixels in the second PolSAR image: (a) before reclassification; (b) after reclassification.

Since the ground truth reference map does not cover the entire PolSAR image, the analysis that follows in this section focuses only on the key pixels in the ground truth reference map, rather than the entire image.

According to the ground truth reference map of the first experimental data shown in Figure 1b and the number of key pixels of each land cover in the ground truth reference map shown in Table 1, the classification accuracy of key pixels before and after reclassification is statistically analyzed, and the results are shown in Table 3. Among the 11 land covers, the classification accuracy of 10 land covers is improved. The accuracy of wood increases by 0.3%, which is the smallest gain. The accuracy of soybean increases by 38.29%, which is the biggest gain. That the average classification accuracy of key pixels in the first experimental data is improved by 9.44% indicates the method in this paper is effective in improving the classification accuracy of key pixels. However, the classification accuracy of water decreases by 29.63% from 88.89% to 59.26%, and the reasons for this result will be analyzed.

Table 3. Before and after reclassification, the statistics of key pixels in the ground reference map of the first PolSAR image.

Land cover	The number of key pixels correctly classified		Accuracy		Average improvement of accuracy
	Before reclassification	After reclassification	Before reclassification (%)	After reclassification (%)
Bare land	3346	3794	82.80	93.89	9.44%
Beet	2150	2294	66.75	71.22
Grass	1516	2369	57.08	89.19
Lucerne	1309	1626	65.94	81.91
Pea	1626	1787	57.23	62.90
Potato	1811	2288	56.74	71.68
Rape	5720	6499	63.10	71.69
Soybean	473	916	40.88	79.17
Water	24	16	88.89	59.26
Wheat	3329	3442	59.88	61.92
Wood	370	372	55.64	55.94

As can be seen from Table 1, there are only 27 key pixels of water in the ground truth reference map. These pixels are located in the rectangular box of Figure 9a, c, namely 8 pixels in the upper right corner of the rectangular box and 19 pixels in the lower left corner. As is shown in Figure 9b, before reclassification, eight pixels in the upper right corner are correctly labeled as water; 16 pixels in the lower left corner are correctly labeled as water and three pixels in the lower left corner are incorrectly labeled as Bare Land. As is shown in Figure 9d, after reclassification, six pixels in the upper right corner are correctly labeled as water and two pixels in the upper right are incorrectly labeled as potatoes; 10 pixels in the lower left corner are correctly labeled as water and nine pixels in the lower left corner are incorrectly labeled as Bare Land.

Figure 9. (a) and (c) the labels of key pixels in the ground truth reference map before and after reclassification, and the label of the pixels in the top right black rectangle box is water; (b) and (d) enlarged area inside the black rectangle box in the upper right corner of (a) and (c); (e) the Pauli RGB image of the first PolSAR image; (f) enlarged area inside the red rectangle box in the upper right corner of (e).

As can be seen from Figure 1b, the land cover in the rectangle box shown in Figure 9e is water. However, the upper right corner of the rectangle box is an abnormal light color area and the lower left corner is a slightly larger abnormal dark blue area, which is shown in Figure 9f. Before reclassification, the CNN classification method was used, and the input data were patches with a size of 15 pixels ×15 pixels. The feature value of the central pixel in a patch was easily affected by the feature values of the surrounding pixels. Because the upper right corner area was small, all pixels in this area were greatly affected by the feature values of pixels outside the region and easy to be labeled as the land cover of surrounding pixels. While the lower left corner area was large, so some pixels in this area were greatly affected by the feature values of the pixels outside the region. As a result, some pixels were easily labeled as the land cover of the surrounding pixels. When reclassifying, the combination of shallow classifiers was used, and the input data was pixels, which were less affected by the feature values of surrounding pixels. The land cover of pixels depended largely on their features. So, more pixels in the two regions were labeled as water when CNN classification is adopted than when the proposed method was adopted. Although the two areas were obviously different from the surrounding pixels from the visual effect, the ground truth reference map shown in Figure 1b was drawn by hand and the two very small anomalous land covers in the top right rectangular box of Figure 9e had to be labeled water. Besides, there are only 27 key pixels of water, and even a slight change in the number makes the accuracy change before and after the reclassification appears drastic. That is what caused water’s accuracy to drop after reclassification. The above analysis also shows that CNN is inferior to the proposed method in describing the details of land covers.

According to the ground reference map of the second experimental data shown in Figure 2b, the classification accuracy of key pixels before and after reclassification is statistically analyzed. The results are shown in Table 4. The classification accuracy of three out of five land covers is improved. The accuracy of Fine Stem Crop increases the least, with 10.96% and that of Forest increases the most, with 30.72%. The average classification accuracy of key pixels in the second experimental data is improved by 9.61%, which also proves the effectiveness of the proposed method in improving the classification accuracy of key pixels. But Broad Leaves Crop’s accuracy dropped slightly from 58.16 to 57.52%, and Town’s dropped sharply from 60.08 to 42.38%. The accuracy of Broad Leaves Crop changes a little before and after reclassification, so next, the reasons for the decline in the accuracy of Town will be analyzed.

Table 4. Before and after reclassification, the statistics of key pixels in the ground reference map of the second PolSAR image.

Land cover	The number of key pixels correctly classified		Accuracy		Average improvement of accuracy
	Before reclassification	After reclassification	Before reclassification (%)	After reclassification (%)
Bare land	3009	4468	50.97	75.69	9.61%
Broad leaves crop	4525	4475	58.16	57.52
Fine stem crop	7426	8913	54.73	65.69
Forest	2819	4302	58.40	89.12
Town	3791	2674	60.08	42.38

In the ground reference map, the number of key pixels labeled as Town is 6310 shown in Table 2. Before and after reclassification, the labels of these key pixels are recorded, as shown in Table 5. After reclassification, the labels of some key pixels are changed from Bare Land and Town to Fine Stem Crop, Broad Leaves Crop and Forest, which can be found in Table 5 and Figure 10g and h. As shown in Figure 10c–f, Town is fragmented and mixed with Fine Stem Crop, Broad Leaves Crop and Forest. Similar to the case of water in the first PolSAR image, when CNN was used for classification, the fewer land covers were easily ignored, while the proposed method was more likely to find them. However, the hand-painted ground truth reference map shown in Figure 2b could not separate the small land covers from the largest number of land covers and had to ignore these small land covers. This makes the accuracy of Town before reclassification greater than that after classification. The above analysis also shows that the proposed method has more advantages in describing the details of ground objects than CNN (Figure 10).

Figure 10. (a) Pauli RGB image of the second PolSAR image; (b) the ground truth reference map in the four rectangular boxes; (c) – (f) the four rectangular boxes with 3 times magnification; (g) – (h) the labels of key pixels labeled as Town in the ground truth reference map before and after reclassification.

Table 5. The labels of key pixels labeled as town in the ground reference map before and after reclassification.

Land cover	Before reclassification	Ratio (%)	After reclassification	Ratio (%)
Bare land	430	6.81	249	3.95
Broad leaves crop	558	8.84	728	11.54
Fine stem crop	214	3.39	688	10.90
Forest	1317	20.87	1971	31.24
Town	3791	60.08	2674	42.38
Total	6310	100.00	6310	100.00

Comparison of visual effects of various methods

To evaluate the visual effect of the proposed method, five groups of comparison experiments were set up, i.e., the SVM classifier, the Wishart classifier, the decision tree classifier, the CNN and the AdaBoost algorithm (a SVM classifier, a Wishart classifier and a decision tree classifier were taken as weak classifiers).

Figure 11 shows the classification results of the methods described above. With the ground truth map shown in Figure 1b, four elliptical regions were selected for comparison on each classification result. It can be found that, compared with the classification results of the SVM classifier, the Wishart classifier, the decision tree classifier and the AdaBoost algorithm, the classification results of the CNN are purer and less spotted, but the classification results of the CNN are fuzzier and the texture is lost. The texture of the result of the proposed method is clearer than that of the CNN. There are fewer spots in the classification results of the proposed method than in the shallow classifiers and the AdaBoost algorithm.

Figure 11. Classification results of different methods for the first PolSAR image: (a) CNN; (b) SVM; (c) Wishart; (d) decision tree; (e) AdaBoost; (f) the proposed method.

As shown in Figure 12, these are the classification results of the second experimental data respectively using the CNN, the SVM classifier, the Wishart classifier, the decision tree classifier, the AdaBoost algorithm and the proposed method. With the ground truth map shown in Figure 2b, four elliptical regions were selected for comparison on each classification result. It can also be found that, compared with the SVM classifier, the Wishart classifier, the decision tree classifier and the AdaBoost algorithm, the classification results of the CNN are with fewer spots but fuzzier and the texture is lost. The texture of the results of the proposed method is clearer than that of the CNN, while there are much fewer spots in the classification results of the proposed method than those of the shallow classifiers and the AdaBoost algorithm.

Figure 12. Classification results of different methods for the second PolSAR image: (a) CNN; (b) SVM; (c) Wishart; (d) decision tree; (e) AdaBoost; (f) the proposed method.

Comparison of classification accuracy of various methods

With the ground reference maps shown in Figures 1b and 2b as references, the classification results of the two experimental data were analyzed from four aspects of overall accuracy (OA), Kappa coefficient (KC), producer accuracy and user accuracy by using a confusion matrix. The classification results of the proposed method are compared with those of the CNN, the SVM classifier, the Wishart classifier, the decision tree classifier and the AdaBoost algorithm.

The OA and KC of the classification results of the first experimental data are shown in Table 6. In terms of accuracy, compared with the other 5 classification methods, the OA of the proposed method is improved by 2.22–20.38%, and the KC is increased by 0.03–0.24.

Table 6. The overall accuracy and Kappa coefficient of the first PolSAR image.

Method	OA (%)	KC
CNN	89.88	0.88
SVM	88.90	0.87
Wishart	83.63	0.81
Decision tree	71.72	0.67
AdaBoost	89.61	0.88
The proposed method	92.10	0.91

The producer accuracy of the classification results of each land cover using different methods for the first experimental data are shown in Table 7. The standardized differences (STD) of the producer accuracy of each land cover using these methods are adopted in the last row of the table. They are used for the local and global evaluation respectively. The producer accuracy of each land cover using the proposed method is larger than the minimum and close to the maximum or even greater than the maximum. The STD is used to judge the equilibrium of the producer accuracy of all land covers using each method. The smaller the value, the more balanced the data. As can be seen from the data in Table 7, the STD of the proposed method is 0.0569, which is 0.0012 larger than the minimum value. It shows that the producer accuracy of the proposed method is relatively balanced.

Table 7. The producer accuracy of the first PolSAR image.

	CNN	SVM	Wishart	Decision tree	AdaBoost	The proposed method
Bare land	75.32%	95.31%	97.64%	0.00%	93.35%	84.66%
Beet	84.57%	80.97%	91.38%	64.18%	80.75%	86.28%
Grass	88.62%	96.27%	89.38%	79.91%	92.66%	93.70%
Lucerne	91.60%	92.64%	92.10%	41.58%	91.83%	94.74%
Pea	88.27%	69.95%	90.15%	83.52%	85.79%	89.51%
Potato	88.70%	83.83%	92.14%	80.55%	83.83%	91.51%
Rape	77.82%	82.40%	83.40%	60.42%	82.40%	81.66%
Soybean	89.78%	93.13%	93.41%	85.15%	93.13%	95.40%
Water	99.97%	98.35%	78.81%	12.13%	98.35%	99.88%
Wheat	93.34%	92.48%	77.68%	87.85%	92.48%	93.63%
Wood	98.09%	92.61%	67.79%	86.47%	92.27%	98.10%
STD	0.0749	0.0856	0.0881	0.3112	0.0557	0.0569

The user accuracy of each land cover using different methods for the first experimental data is shown in Table 8. The STD of the user accuracy of all land cover using each method is listed in the last row of the table. The user accuracy of the proposed method is larger than the minimum value and close to the maximum value. The STD of the proposed method is 0.0482, which is 0.0047 larger than the minimum value and very close to the minimum value.

Table 8. The user accuracy of the first PolSAR image.

	CNN	SVM	Wishart	Decision tree	AdaBoost	The proposed method
Bare land	90.64%	85.54%	69.59%	0.00%	93.53%	96.83%
Beet	88.07%	89.14%	78.17%	85.43%	90.27%	92.95%
Grass	90.30%	89.27%	94.80%	63.67%	94.50%	95.62%
Lucerne	87.95%	89.16%	88.41%	98.29%	90.09%	95.08%
Pea	72.49%	82.84%	70.14%	53.69%	82.02%	81.37%
Potato	92.54%	93.35%	71.79%	89.36%	93.35%	96.98%
Rape	86.95%	90.10%	73.97%	55.57%	90.10%	89.33%
Soybean	94.02%	95.30%	90.78%	91.26%	95.30%	95.98%
Water	98.60%	97.21%	99.97%	98.76%	97.21%	97.74%
Wheat	91.60%	87.32%	93.63%	76.04%	87.32%	89.71%
Wood	95.69%	85.55%	91.55%	73.74%	85.61%	93.20%
STD	0.0675	0.0435	0.1127	0.2852	0.0454	0.0482

Therefore, the proposed method is superior to the other five classification methods in terms of the producer accuracy and the user accuracy, both locally and globally.

The OA and KC of the classification results of the second experimental data are shown in Table 9. Compared with the other five classification methods, the OA of the proposed method increases by 1.06–11.53%, and the KC increases by 0.02–0.15.

Table 9. The overall accuracy and Kappa coefficient of the second PolSAR image.

Method	OA (%)	KC
CNN	89.96	0.87
SVM	83.50	0.79
Wishart	79.49	0.74
Decision tree	80.80	0.76
AdaBoost	82.61	0.78
The proposed method	91.02	0.89

In the second experimental data, the producer accuracy of each land cover using different classification methods is shown in Table 10. The last row of the table lists the STD of the user accuracy using each method. It can be found that the producer accuracy of Fine Stem Crop and Forest using the proposed method is the maximum, while that of Bare Land, Broad Leaves Crop and Town using the proposed method is close to the maximum, with the difference values of 0.95, 0.08 and 2.51%, respectively. Moreover, the STD of the proposed method is the smallest, which is 0.0703, indicating that the producer accuracy using the proposed method is the most balanced.

Table 10. The producer accuracy of the second PolSAR image.

	CNN	SVM	Wishart	Decision tree	AdaBoost	The proposed method
Bare land	93.21%	95.89%	94.85%	93.54%	92.96%	94.94%
Broad leaved crop	92.60%	84.20%	85.97%	76.89%	84.83%	92.52%
Fine stem crop	75.98%	71.36%	68.34%	67.64%	74.35%	79.16%
Forest	93.79%	90.07%	73.28%	91.05%	94.21%	96.05%
Town	88.89%	62.03%	61.56%	61.17%	51.31%	86.38%
STD	0.0747	0.1384	0.1347	0.1418	0.1766	0.0703

The user accuracy of each land cover using different classification methods is shown in Table 11 for the second experimental data. The last row shows the STD of the user accuracy using each method. It can be found that the user accuracy of Bare land, Fine Stem Crop and Town using the proposed method is the highest, while those of Broad Leaves Crop and Forest are close to the maximum accuracy, with a difference of 0.49 and 0.81%. Moreover, the STD of the proposed method is the smallest, which is 0.0527, indicating that the user accuracy using the proposed method is the most balanced.

Table 11. The user accuracy for the second PolSAR image.

	CNN	SVM	Wishart	Decision tree	AdaBoost	The proposed method
Bare land	93.76%	88.94%	91.19%	90.59%	93.06%	94.77%
Broad leaved crop	88.26%	80.24%	75.12%	79.38%	77.45%	87.77%
Fine stem crop	80.16%	75.55%	76.30%	74.00%	71.97%	82.51%
Forest	95.86%	86.67%	85.96%	82.97%	81.99%	95.04%
Town	86.76%	79.64%	59.45%	65.66%	84.60%	91.60%
STD	0.0619	0.0548	0.1216	0.0938	0.0791	0.0527

Conclusion

The proposed method in this paper combines the advantages of the deep learning model and shallow classification method to reclassify pixels that are easy to be misclassified and obtain better classification results. In terms of local classification accuracy, overall classification accuracy and visual effect, the proposed classification method performs better than the other methods listed above.

Acknowledgement

The authors would like to thank the anonymous reviewers and the editor for their constructive comments and suggestions.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Science and Technology Research Project of Hubei Province, Department of Education and Excellent Young and Middle-aged Science and Technology Innovation Team Program for Colleges and Universities in Hubei Province under [Grant Q20203006]; University-level research start-up fund under [Grant ESRC20220046]; and Natural Science Foundation of Hubei Province, [2019CFB827].