A high-precision image classification network model based on a voting mechanism

ABSTRACT

With the development of satellite remote sensing technology, image classification task, as the basis of remote sensing data interpretation, has received wide attention to improving accuracy and robustness. At the same time, in-depth learning technology has been widely used in remote sensing and has a far-reaching impact. Since the existing image classification methods ignore the feature that the general image semantics are the same as the semantics of a single pixel, this paper presents an algorithm that uses the semantics of an image to achieve high-precision image classification. Based on the idea of partial substitution for global, this algorithm designs a split result voting mechanism and builds a Vgg-Vote network model. This mechanism votes on the semantically segmented result of an image and uses the maximum filtering function to select the category containing the most significant number of pixels as the prediction category of the image. Experiments on UC Merced Land-User complete datasets and five types of incomplete datasets with varying degrees of interference, including noise, data occlusion and loss, show that the Vote mechanism dramatically improves the classification accuracy, robustness and anti-jamming capability of Vgg-Vote.

KEYWORDS:

• Deep learning
• Image classification
• Robustness
• Remote sensing image
• Vote mechanism

1. Introduction

In recent years, image classification has been an important research hotspot in the field of computer vision. Its purpose is to assign different kinds of labels to images according to scene semantics, and it is widely used in unmanned driving (Qayyum, Malik, Saad 2019), face recognition (Wu et al. 2021), and medical assistant diagnosis (Manzo and Pellino 2021). With the development of satellite remote sensing imaging technology, the acquisition of remote sensing images becomes more convenient and fast, and the accuracy and automation of image classification are increasingly required in the field of remote sensing (Cheng et al. 2018). Because remote sensing images contain rich texture information and spatial semantic information, scene classification of remote sensing images plays an increasingly important role in smart agriculture (Li et al. 2018), disaster detection, and urban planning (Zhou et al. 2020), which improves the efficiency and accuracy of remote sensing data interpretation.

Traditional remote sensing image classification is mainly based on low-level features such as color (Van De Sande, Gevers, and Snoek 2010), direction gradient (Zhou et al. 2013), density feature (Zhang et al. 2014), feature point (Zhu et al. 2012), and texture in the transform domain (Qian et al. 2014). Such methods need to be based on prior knowledge, and the degree of automation and classification accuracy are low. Remote sensing image classification methods based on statistical analysis include neighborhood classification algorithm (KNN[Zhao et al. 2021]), decision tree algorithm (Zhang et al. 2022), support vector machine (Qu et al. 2022), etc. These methods are difficult to show a good classification effect in the face of complex data (Li et al. 2005). Image classification based on deep learning (Zheng et al. 2019) can mine deep-seated features of images, and combine feature extractors with classifiers to form an end-to-end network model, so as to obtain higher classification accuracy and automation level. Classical network models include AlexNet (Han et al. 2017), VggNet (Simonyan and Zisserman 2014), ResNet (He et al. 2016). When facing image classification, these networks show excellent hierarchical feature learning ability and mobility far better than traditional classification methods (Tian et al. 2019). Researchers have improved the classical network and made further research on remote sensing image classification. Qiao et al. (2021) improved ResNet and applied it to the classification of remote sensing images and established an attention module in the two dimensions of channel and space to effectively classify remote sensing images. The ResNeXt proposed by Xie et al. (2017) use a layered convolution to replace the standard convolution, which improves the network performance without significantly increasing the order of parameters. Gómez and Meoni (2021) Proposed an MSMatch network based on small sample data, which greatly reduces the demand for labeled data and achieves good classification results. Li et al.(2021) found that two factors, sparse connectivity and dynamic activation function, are effective to improve the accuracy, so arrive at a family of networks, named MicroNet. MobileNet V3, proposed by Howard et al.(2019), uses a network search algorithm and introduces the SE module with H-Swish activation function.

The existing classification network model ignores the same characteristics of the whole image semantics and the semantics of a single image pixel, resulting in low classification accuracy and poor robustness to external interference (Aksoy, Ravanbakhsh, and Demir 2020). This paper uses the same characteristics of the whole image semantics and the semantics of a single pixel of the image, designs the screening voting mechanism of recognition results, and integrates it with the Vgg network to build a Vgg-Vote network model. Through the comparative experiment on UC merged land use public dataset and incomplete dataset, and taking Vgg, MicroNet, MobileNet V3 and other networks as the comparison group, the experimental results prove that the voting mechanism improves the accuracy of image classification tasks and the robustness to external interference.

2. Network principle and design

In this paper, the semantic category of a single image and the exact characteristics of each pixel semantic category are used as a priori knowledge to construct a voting mechanism. The mechanism is integrated with Vgg 16 network to design a Vgg-Vote network model for image high-precision classification. Vgg-Vote network model takes a Fully Convolutional Network (Evan, Long, Darrelll 2017) and voting mechanism as the main body and uses the cross-layer connection to fuse global and local information to realize high-precision image classification. The structure of the Vgg-Vote network model is shown in Figure 1.

Figure 1. Vgg-Vote network structure.

As can be seen from Figure 1, the Vgg-Vote network model is mainly composed of an image semantic segmentation module and a category voting mechanism module. The image semantic segmentation module first extracts multi-dimensional features through five convolution layers using the downsampling method, then uses deconvolution to restore the size of the feature image to the size of the original input feature image, and finally uses the 1 * 1 single convolution layer to realize the pixel by pixel semantic category judgment of the input image. The voting mechanism counts the semantic categories of all pixels, calculates the number of pixels contained in each semantic category, and takes the pixel category with the most significant number as the final prediction category of the network model for the input image.

2.1. Image semantic segmentation module

The image semantic segmentation module of the Vgg-Vote network model is mainly composed of a lower sampling layer, cross-layer connection structure, upsampling layer and pixel-level semantic category prediction layer. The network structure is shown in Figure 2.

Figure 2. Semantic segmentation module.

The lower sampling layer is composed of five convolution modules. Each convolution module includes three convolution structures and a maximum pooling structure. Different convolution modules extract the features of the feature map in different dimensions through convolution and pooling, which will also reduce the size of the feature map. The function of the deconvolution layer is mainly to restore the size of the characteristic image to the size of the original image. The deconvolution formula is as follows. (1) $\underset{m\times m}{I}\to \underset{m^2\times 1}{I^{\mathrm{\prime }}}$(1) (2) $\underset{n^2\times m^2}{W^T}\cdot \underset{m^2\times 1}{I^{\mathrm{\prime }}}=\underset{n^2\times 1}{O^{\mathrm{\prime }}}$(2) (3) $\underset{n^2\times 1}{O^{\mathrm{\prime }}}\to \underset{n\times n}{O}$(3)

Among them,$\underset{m\times m}{I}$represents the small-size feature map to be sampled, and $\underset{m\times m}{I}$is reshaped to obtain $\underset{m^2\times 1}{I^{\mathrm{\prime }}}$. $\underset{n^2\times m^2}{W^T}$represents transposed convolution matrix, which is the core parameter matrix of deconvolution and is constantly updated in the process of model training. After remodelling$\underset{n^2\times 1}{O^{\mathrm{\prime }}}$, $\underset{n\times n}{O}$is obtained, the large-scale feature map obtained after deconvolution.

In addition, to facilitate the extraction of global features and avoid the loss of detailed features, the semantic segmentation module adopts the strategy of cross-layer connection of different dimensional features. The same size can be found in the feature maps during process of the upsampling and downsampling. Therefore, the feature map features extracted by the second convolution module and the feature map features extracted by the second deconvolution module are feature stitched. The feature map features extracted by the fourth convolution module and the feature map features extracted by the first deconvolution module are spliced together to restore more detailed features of the original feature map in the process of upsampling. Through the skip structure connection of the features of different depth layers, the local information and global information are fused, which greatly retains the characteristics of the input feature map and ensures the performance of the model.

The function of 1 * 1 single convolution layer is to predict the category pixel by pixel. While keeping the size of the feature map unchanged, the convolution layer acts as a classifier to output each pixel in the feature map to channel 21, which corresponds to the prediction probability of the image category to which each pixel belongs. Finally, the category probability eigenvector of each pixel is normalized by the Softmax function. The sum of all vector values of the probability matrix is 1. The Softmax function formula is as follows. (4) $\sigma (z{)}_j={\displaystyle \frac{e^{z_j}}{\sum _{k=1}^K{e}^{z_k}},j=1,\dots ,K}$(4) Where represents the prediction result of the j-th category, which is expressed in the form of numbers. It should be pointed out here that the semantic segmentation module can correctly predict most of the pixel categories represented by the scene in the image, laying the foundation for the voting mechanism to play a role.

2.2. Image semantic segmentation module

The Vote mechanism uses the idea of replacing the global with the local one, considering that the scene category of a single image is the same as the semantic category of each pixel point, and fuses this prior knowledge into a deep neural network, which improves the accuracy of scene classification of an image. The Vote mechanism works as shown in Figure 3.

Figure 3. Vote mechanism structure.

The semantic segmentation module of Vgg-Vote judges the category of each pixel in the image according to its trained feature parameters, obtains the category information of each pixel, and transmits the information to the voting mechanism. Since the Semantic Segmentation Module can theoretically predict most of the pixels contained in a scene as true categories, the idea of Vote mechanism replacing global with local is possible.The voting mechanism counts the number of pixels in each category by the proportion function (formula 5). It uses the maximum value filtering function (formula 6) to select the category with the largest number of pixels as the final prediction category (K) of the image. The formula is as follows. (5) $L_{label-i}={\displaystyle \frac{N_{label-i}}{N_{all}},N_{all}=\sum _{i=1}^n{N}_{label-i}}$(5) (6) $K=\max (L_1{L}_2\dots \dots L_n)$(6) Where n is the total number of categories, and $N_{label-i}$ represents the number of pixels whose category is label-i.

The advantage of the Vote mechanism is that it uses a local instead of global strategy, taking into account that the categories of images are the same as the semantic categories of the effective pixels in the image, and the network model is not required to accurately judge each pixel in the image to achieve the correct image classification. When the correct number of pixels is more than 50% in the result of the semantics segmentation module, the Vote mechanism will use the correct category as the category prediction of the image as a whole, ignoring the remaining misjudged pixel points, to achieve the correct classification of the image. Theoretically, with the assistance of the Vote mechanism, when the average accuracy of the Vgg-Vote semantics segmentation module exceeds 50%, the image classification accuracy of Vgg-Vote can reach 100%, which significantly improves the robustness of the network.

2.3. Accuracy evaluation index

Accuracy, Precision, Recall, and F1-score are often used as precision indicators in multi-class tasks for in-depth learning. In order to evaluate the performance of the classification model as a whole from the classification of all categories, a comprehensive set of evaluation indicators is used to evaluate the model. The macro method is used to obtain Macro-Precision, Macro-Recall and Maco-F1-score. The macro method calculates the arithmetic mean for different categories of indicators with the following formula. (7) $A={\displaystyle \frac{1}{K}\sum _{i=1}^k{A}_i={\displaystyle \frac{T{P}_i+T{N}_i}{T{P}_i+F{P}_i+T{N}_i+F{N}_i}}}$(7) Among them, accuracy (A) represents the proportion of samples that are predicted correctly to the total number of samples, $A_i$ represents the precision of a certain type of sample, TP represents the number of samples that are correctly determined to be positive, FP represents the number of samples that are incorrectly determined to be positive, FN represents the number of samples that are incorrectly determined to be negative. K represents the total number of categories of samples. (8) $P_{macro}={\displaystyle \frac{1}{K}\sum _{i=1}^k{P}_i={\displaystyle \frac{1}{K}\sum _{i=1}^k{\displaystyle \frac{T{P}_i}{T{P}_i+F{P}_i}}}}$(8) (9) $R_{macro}={\displaystyle \frac{1}{K}\sum _{i=1}^k{R}_i={\displaystyle \frac{1}{K}\sum _{i=1}^k{\displaystyle \frac{T{P}_i}{T{P}_i+F{N}_i}}}}$(9) (10) $F_{macro}={\displaystyle \frac{1}{K}\sum _{i=1}^k{F}_i={\displaystyle \frac{2}{K}\sum _{i=1}^k{\displaystyle \frac{P_i\times R_i}{P_i+R_i}}}}$(10)

Precision (P) represents the ratio of the number of correctly predicted positive samples to the total number of predicted positive samples, and $P_i$ represents the precision of the i-th sample. Recall (R) represents the ratio of the number of samples correctly predicted to be positive samples to the total number of actually positive samples and, the $R_i$represents the recall of the i-th sample.. F1 score is an index used to measure the accuracy of two classification models in statistics, which considers the accuracy and recall of classification models.

3. Experimental results and analysis

This article uses the UCMerced_LandUse dataset to evaluate the accuracy and robustness of Vgg-Vote networks. The environment of the image classification experiment in this paper: Win10 system, Python 3.7, CUDA 10.1, CuDnn 7.6.5, uses PyTorch 1.7.1 deep learning framework. In the experiment of image classification, the batch of Vgg-Vote network training and testing is set to 16; Epoch is set to 128; The ratio of the training set to test set is 7:3; Adam optimizer is adopted; The learning rate is set to 0.001; The learning rate decay rate is set to 0.7.

3.1. Overview of dataset

In this paper, the dataset used to verify the accuracy of the Vgg-Vote network is UCMerced_LandUse public dataset. All images are extracted from the USGS National Map Urban Area Imagery series, including 21 types of land use image data such as agriculture, airplanes, buildings, forests, highways, etc. Each type of image data is 100 pieces, and the size of each image is 256 × 256 pixels. The pixel resolution of public domain images is 1 foot. The dataset is shown in Figure 4.

Figure 4. UCMerced LandUse dataset.

Due to the easy interference in the acquisition process of remote sensing images, there are noise, occlusion, and data loss in remote sensing images. In view of this, this paper discusses UCMerced_Landuse public dataset adds different ways of external interference, mainly including the following five ways: randomly adding 30% noise to the image; Randomly adding 50% noise to the image; Block the image in multiple small areas; Multiple small areas of the image are lost; In order to facilitate visualization, the occluded part is represented by pure black area, and the missing part is represented by the pure white area. The comparison effect between the finished dataset and the original dataset is shown in Figure 5:

Figure 5. The incomplete dataset.

Therefore, the data set used in this experiment is UC Merced_LandUse Complete Dataset and UC Merced_LandUses incomplete dataset with Five Interferences whose characteristics are summarized in Table 1.

Table 1. Dataset summary table.

Data set	Processing method	Image example	Characteristic
UC Merced_ LandUses complete data set	No interference added		Clear color and texture;Local feature and global featureinteg-rity
UC Merced_ LandUses Incomplete Data set	Randomly add 30% more noise		A small amount of noise causes loss of color information and changes local characteristics.
	Randomly add 50% more noise		A large amount of noise leads to the loss of color information of more pixels, changes in local and global characteristics,and a significant decrease in image quality.
	Multiple small area occlusion		Several local areas of the image are obscured (lost), important pixel information is lost, local features are seriously damaged, pixel topological feature information is lost, global features are changed, and feature information is poorly characterized.
	Multiple small areas are missing.
	Single large area is missing		Local and global features chan-ge dramatically and cannot accurately express image information.

3.2. Analysis of experimental results

This paper uses the original dataset and five kinds of incomplete datasets to carry out classification experiments, using the Vgg-Vote network as the experimental group and Vgg, MicroNet and MobileNet V3 as the control group, to verify the model accuracy and robustness of the Vgg-Vote network. The difference between Vgg-Vote and Vgg is whether or not the Vote mechanism is used, so comparative experiments on all six datasets can be considered ablation studies. For easy comparison of the results, the red curve represents the Vgg-Vote network, the green curve represents the Vgg network, the purple represents the MicroNet, and the orange represents the MoblieNet V3.

3.2.1. verification of model accuracy

This section uses the complete dataset to compare the classification accuracy of Vgg-Vote, Vgg, MicroNet and MobileNet V3. The comparison between Vgg-Vote and Vgg on the complete dataset is a control variable experiment with the Vote mechanism as a variable, so it can be used as an ablation study to prove that the Vote mechanism can improve the classification accuracy of the model. The classification accuracy change diagram of the above network on the complete dataset is shown in Figure 6:

Figure 6. Variation diagram of accuracy of different networks in the complete dataset.

The accuracy of the Vgg-Vote network keeps increasing, and the accuracy of the Vgg-Vote network is always higher than other networks during the whole training process. When the number of iterations reaches 128, the accuracy reaches the highest, which is 95.12%; Although the accuracy of networks such as Vgg, MicroNet and MobileNet V3 is also increasing, the rise is gradually slower than that of Vgg-Vote. The optimal accuracy table for each of the above networks on the original dataset is shown in Table 2.

Table 2. Comparison of classification indices for different networks on complete datasets.

network model	Best-Acc(%)	Macro-Recall(%)	Macro-Precision(%)	Macro-F1(%)
Vgg	66.67	67.96	70.73	68.44
MicroNet	85.28	83.81	84.10	83.29
MobileNet V3	57.49	60.03	59.00	57.98
Vgg-Vote	95.12	96.42	94.85	95.67

In the complete data set, the optimal accuracy of the Vgg-Vote network is up to 95.12%, and the optimal accuracy of the Vgg network is only 66.67%, 28.45% lower than Vgg-Vote, which fully demonstrates that the Vote mechanism can significantly improve the classification accuracy of Vgg-Vote network. The optimal classification accuracy of MicroNet and MobleNet V3 is 85.28% and 57.49%, respectively, which is 9.84% and 37.63% lower than that of the Vgg-Vote network. According to Macro-Recall, Macro-Precision and Macro-F1, Vgg-Vote also performs better than the other three networks, which proves that the Vgg-Vote network performs better in classification tasks.

In summary, the Vgg-Vote network model built by introducing the Vote mechanism into the Vgg network shows better recognition ability, and higher classification accuracy than the existing mainstream image scene classification network.

3.2.2. Verification of model robustness

Due to the weather, equipment itself, and other reasons, remote sensing image acquisition is prone to noise, data loss, occlusion, and other interference, resulting in the classification accuracy of the classification network being affected by varying degrees. Therefore, this section compares the classification accuracy of Vgg-Vote, Vgg, MicroNet, and MobileNet V3 on five types of incomplete datasets and then verifies the robustness of the Vgg-Vote network model to common disturbances such as noise, data loss, and occlusion.The comparison of Vgg-Vote and Vgg on five incomplete datasets uses the control variable method with the Vote mechanism as a variable. Hence, ablation research proves that the Vote mechanism can enhance the robustness of the model.

3.2.2.1. Comparison of results under noise interference

Noise is common in images, and the noise volume is usually different, so this paper adds 30% and 50% noise to the dataset and tests it on four networks, such as Vgg-Vote, to prove the robustness of Vgg-Vote to noise. Put the results of two levels of noise interference together for comparison. The classification accuracy curve of each network in the two groups of experiments is shown in Figure 7:

Figure 7. Variation diagram of accuracy of each network under 30% and 50% noise interference.

The optimal accuracy of Vgg is only 38.52% and 4.29% under both noise disturbances, and the network failure of Vgg on the 50% noise disturbance dataset is not represented in Figure 7. The optimal accuracy of the Vgg-Vote network is 90.05% under the interference of 30% normal noise. The optimal accuracy of MicroNet and MobleNet V3 is only 84.21%, 65.54%. The optimal accuracy of the Vgg-Vote network is 51.53%, 5.84%, and 25.51% higher than that of the Vgg network, MicroNet and MobleNet V3, respectively. The optimal accuracy of the Vgg-Vote network is 83.82% under the interference of 50% severe noise, while that of MicroNet and MobleNet V3 is only 76.03%, 62.16%. Compared with Vgg, MicroNet and MobleNet V3, the optimal accuracy of the Vgg-Vote network is improved by 79.53%, 7.79%, and 21.66%, respectively. According to accuracy indicators such as Macro-Recall, Macro-Precision and Macro-F1, Vgg-Vote performs better than the other three comparison networks.

The above experimental results show that the classification accuracy of the Vgg-Vote network is still at a high level in the case of different noise interference. Compared with the other three networks, the Vgg-Vote network also has stronger robustness to noise, which proves that the voting mechanism greatly enhances the robustness of the Vgg-Vote network model in the case of noise interference.

3.2.2.2. Comparison of results under occlusion and random area loss interference

Due to the mutual occlusion and loss of geometric elements in the scene during image data acquisition, local features are lost, and global features are changed. Therefore, this paper deals with occlusion and random area loss of data, including two cases: one is large area loss in a single area, and the other is slight area loss in multiple areas. The former will cause global feature changes and partial local feature changes; The latter will lead to local feature changes in multiple regions and then affect the accuracy of global features, thus affecting the integrity of the extracted features to varying degrees. Through the comparative experiment with the control variable method, the accuracy change of each network in the case of data loss is shown in Figure 8:

Figure 8. Accuracy variation diagram of each network with data loss interference.

Through the experimental results, it can be found that the impact of small area loss in multiple areas is more significant because the classification accuracy of the Vgg network under the interference of small area loss in multiple areas is only 3.33%. The failure phenomenon occurs again, indicating that the network has poor robustness to noise, so it is not shown in Figure 8.

The optimal accuracy of the Vgg-Vote network can reach 94.30% under the interference of large area loss in a single area. The optimal accuracy of Vgg, MicroNet and MobleNet V3 is only 37.86%, 80.24%, and 65.02%. The optimal accuracy of the Vgg-Vote network is 56.44%, 14.06%, and 29.28% higher than that of Vgg, MicroNet and MobleNet V3, respectively. Under the interference of small area loss in multiple regions, the optimal accuracy of the Vgg-Vote network can reach 94.80%. The optimal accuracy of Vgg, MicroNet and MobleNet V3 is only 18.33%, 76.33%, and 65.18%. The optimal accuracy of the Vgg-Vote network is 76.48%, 76.35%, and 65.18% higher than that of Vgg, MicroNet and MobleNet V3, respectively. According to Macro-Recall, Macro-Precision and Macro-F1, the Vgg-Vote table is still better than the three comparison networks. From the above results, it is known that the Vgg-Vote network is robust to data loss. (Tables 3–7)

Table 3. Comparison of classification indexes of different networks in 30% noise dataset.

network model	Best-Acc(%)	Macro-Recall(%)	Macro-Precision(%)	Macro-F1(%)
Vgg	38.52	38.34	41.48	38.64
MicroNet	84.21	83.67	83.52	83.10
MobileNet V3	64.54	62.98	64.07	62.41
Vgg-Vote	90.05	89.68	90.73	90.14

Table 4. Comparison of classification indexes of different networks in 50% noise dataset.

network model	Best-Acc(%)	Macro-Recall(%)	Macro-Precision(%)	Macro-F1(%)
Vgg	4.29(Invalid)	–	–	–
MicroNet	76.03	73.90	75.43	73.67
MobileNet V3	62.16	60.31	60.92	59.32
Vgg-Vote	83.82	84.17	83.66	83.91

Table 5. Comparison of classification indexes of different networks on large and single area lost dataset.

network model	Best-Acc(%)	Macro-Recall(%)	Macro-Precision(%)	Macro-F1(%)
Vgg	37.86	38.21	41.02	36.62
MicroNet	80.24	77.29	77.19	76.52
MobileNet V3	65.02	64.48	62.95	62.38
Vgg-Vote	94.30	93.07	95.55	93.24

Table 6. Comparison of classification indexes of different networks on small and multi area lost dataset.

network model	Best-Acc(%)	Macro-Recall(%)	Macro-Precision(%)	Macro-F1(%)
Vgg	18.33	16.94	23.52	24.79
MicroNet	76.35	75.10	74.71	74.05
MobileNet V3	65.18	63.65	62.99	62.13
Vgg-Vote	94.80	93.13	93.83	95.96

Table 7. Comparison of classification indexes of different networks on small and multi area lost dataset.

network model	Best-Acc(%)	Macro-Recall(%)	Macro-Precision(%)	Macro-F1(%)
Vgg	42.86	43.45	45.46	43.10
MicroNet	85.13	83.86	83.90	83.43
MobileNet V3	60.73	60.03	59.00	57.98
Vgg-Vote	93.83	94.57	92.40	94.64

Because the occlusion between scene geometry elements is easy to occur during image acquisition, this paper also obscures the dataset, adds 6–10 random obscure areas to each image data, and performs classification experiments on this dataset. The accuracy change curve of each network is shown in Figure 9.

Figure 9. Accuracy variation diagram of each network with occlusion interference.

Under the interference of regional occlusion, the optimal accuracy of Vgg, MicroNet and MobleNet V3 is only 42.86%, 85.13%, and 60.73%. The optimal accuracy of the Vgg-Vote network can reach 93.83%, increasing by 50.97%, 8.70%, and 33.10% year on year. In addition, the curve graph shows that the accuracy of the Vgg-Vote network in the 25th epoch classification has reached 80%, while the accuracy of Vgg and MobleNet V3 is consistently below 80%. The accuracy of the Vgg-Vote network in the 70th epoch classification is already 90%, while that of MicroNet is always less than 90%. According to Macro-Recall, Macro-Precision and Macro-F1, the Vgg-Vote table is better than the three comparison networks. The above results show that with the Vote mechanism, the Vgg-Vote network can achieve higher accuracy with fewer iterations, which is more conducive to avoiding the occurrence of over-fitting phenomena and is robust to occlusion interference.

It can be seen from the above results that in the case of regional loss, the accuracy of the Vgg-Vote network is almost the same as that of the complete dataset, which proves that the network has strong resistance to regional loss. Vgg and MobileNet V3 have weak resistance to occlusion interference. In contrast, Micronet has slightly stronger resistance to occlusion interference, while the classification accuracy of the Vgg-Vote network is almost not affected by occlusion interference, and there is little difference between the accuracy under the complete dataset. To sum up, it shows that the voting mechanism greatly enhances the robustness of the Vgg-Vote network model in the case of data loss and data occlusion.

This section also analyzes the classification accuracy of each network on five types of incomplete datasets. The comparison experimental accuracy results are shown in Table 8:

Table 8. Comparison of classification accuracy of each network on incomplete dataset.

dataset processing	30% Noise	50% Noise	Occlusion	Single large area is missing	Multiple small areas are missing.
Vgg	38.52% (↑51.53%)	4.29%(Invalid) (↑79.53%)	42.86% (↑50.97%)	37.86% (↑56.44%)	18.33% (↑76.47%)
MicroNet	84.21% (↑5.84%)	76.03% (↑7.79%)	85.13% (↑8.70%)	80.24% (↑14.06%)	76.35% (↑18.45%)
MoblieNet V3	64.54% (↑25.51%)	62.16% (↑21.66%)	60.73% (↑33.10%)	65.02% (↑29.28%)	65.18% (↑29.62%)
Vgg-Vote	90.05%	83.82%	93.83%	94.30%	94.80%

In Table 8, each comparative experiment adopts the strict control variable method, and the experimental environment is consistent. It can be seen from Table 8 that for a lot of noise interference and large area loss interference, the Vgg network has failed; In the case of five kinds of incomplete data, the accuracy of the Vgg-Vote network is at a high level. Except for solid noise interference, the accuracy is 83.82%, and the classification accuracy under other interference is more than 90%. Compared with Vgg, the accuracy of the Vgg-Vote network is improved by about 50% to 80%, 5% to 15% compared with MicroNet, and 20% to 30% compared with MobileNet v3. The above experimental results show that the voting machine enhances the robustness of the Vgg-Vote network model under the interference of noise, data occlusion, and data loss.

3.2.3. Verification of the migration capability of voite mechanism

Because the pre-training weight parameter is a common feature derived from a large number of public datasets, it makes the model less burdensome to learn specific tasks. As a result, network migration is often used, but network models using pre-training weight parameters may not fit well in the face of new data, resulting in poor migration ability. To further verify the migration capability of the Vote mechanism, this section compares the classification accuracy of Vgg-Vote and Vgg using pre-training weight parameters on five types of incomplete datasets. The experimental results are compared as shown in Table 9:

Table 9. Comparison of classification accuracy between Vgg(pre) and Vgg-Vote on incomplete datasets.

dataset processing	30% Noise	50% Noise	Occlusion	Large area loss in single area	Small areas lost in multiple areas
Vgg(pre)	80.24%	73.81%	80.48%	83.81%	81.90%
Vgg-Vote	90.05%	83.82%	93.83%	94.30%	94.80%
Increase	9.81%	10.01%	13.35%	10.49%	12.90%

In Table 9, the accuracy of Vgg-Vote loaded with vote module on five types of incomplete datasets is 90.05%, 83.82%, 93.83%, 94.30%, and 94.80%, respectively. In contrast, the accuracy after using pre-training weight parameters is only 80.24%, 73.81%, 80.48%, 83.81%, and 81.90%, with a year-on-year increase of 9.81%, 10.01%, 13.35%, 10.49%, and 12.90%. When remote sensing images are disturbed by noise, data occlusion, and data loss, Vgg-Vote not only has a classification accuracy much higher than Vgg but also has a higher classification accuracy even when Vgg uses pre-training weight parameters. The experimental results show that the voting mechanism enhances the migration ability of the Vgg-Vote network so that the classification network still maintains high classification accuracy on the new dataset.

4. Conclusion

Aiming at the problem that the existing remote sensing image classification methods ignore the characteristics that the overall semantic category of the image is the same as the semantic category of a single pixel, this paper designs a recognition result screening voting mechanism and integrates this mechanism with Vgg network to build a Vgg-Vote network model, and then verifies the accuracy and robustness of the Vgg-Vote network model.

In the classification experiment on the original dataset, the classification accuracy of Vgg-Vote is improved by 28.45%, 9.84%, and 37.63%, respectively, compared with Vgg, MicroNet, MobileNet V3, and other networks, indicating that the voting mechanism dramatically improves the accuracy of Vgg-Vote network model in classification tasks. In the five kinds of comparative experiments in which noise, data occlusion, and loss are introduced, the accuracy of the Vgg-Vote network is improved by about 50% to 80% compared with Vgg, 5% to 20% compared with MicroNet, and 20% to 30% compared with mobileNet V3, indicating that voting mechanism greatly enhances the robustness of Vgg-Vote network model under the interference of noise, data occlusion, data loss and so on. In the migration ability verification experiment of the voting mechanism, the classification accuracy of Vgg-Vote on five types of interference datasets is about 10% higher than that of Vgg, indicating that the voting mechanism enhances the migration ability of the Vgg-Vote network.

In the follow-up research, we will consider the migration of the Vgg-Vote network, apply the voting mechanism to the sound spectrum, three-dimensional point cloud, and other data, and verify the mobility and universality of the voting mechanism. At the same time, we try to verify the possibility of applying the Vgg-Vote network in the field of target detection and target recognition.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study were derived from the following resources available in the public domain: [UC Merced Land-Use, http://weegee.vision.ucmerced.edu/datasets/landuse.html]

Additional information

Funding

This work was supported by the Project of the Natural Science Foundation of Beijing [8172016]; National Natural Science Foundation Project [41601409,41971350]; Open Fund Project of State Key Laboratory of Surveying and Remote Sensing Information Engineering of Wuhan University [19E01]; Open Fund Project of State Key Laboratory of Geographic Information Engineering [SKLGIE2019-Z-3-1]; Special fund project for basic scientific research business expenses of municipal colleges and universities of Beijing Jianzhu University [X18063]; National Key R&D Program Project [2018YFC0807806]; Digital Mapping and Open Research Foundation Project of the Key Laboratory for Land Information Applications of the Ministry of Natural Resources [ZRZYBWD202102]; the Soft Science Research Project of Ministry of Housing and Urban-Rural Development of China (R20200287). Major Decision Consulting Project of the Beijing Social Science Foundation (21JCA004)