A deep 2D/3D Feature-Level fusion for classification of UAV multispectral imagery in urban areas

Abstract

In this paper, a deep convolutional neural network (CNN) is developed to classify the Unmanned Aerial Vehicle (UAV) derived multispectral imagery and normalized digital surface model (DSM) data in urban areas. For this purpose, a multi-input deep CNN (MIDCNN) architecture is designed using 11 parallel CNNs; 10 deep CNNs to extract the features from all possible triple combinations of spectral bands as well as one deep CNN dedicated to the normalized DSM data. The proposed method is compared with the traditional single-input (SI) and double-input (DI) deep CNN designations and random forest (RF) classifier, and evaluated using two independent test datasets. The results indicate that increasing the CNN layers parallelly augmented the classifier’s generalization and reduced overfitting risk. The overall accuracy and kappa value of the proposed method are 95% and 0.93, respectively, for the first test dataset, and 96% and 0.94, respectively, for the second test data set.

Keywords:

• UAV
• multispectral imagery
• nDSM
• Deep CNN

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Structural, textural, Spectral

2 Light Detection and Ranging

3 Scale Invariant Feature Transform

4 Digital Elevation Model

5 Digital Number

6 Double Input Deep CNN

7 Single Input Deep CNN