A Novel Multispectral Maritime Target classification based on ThermalGAN (RGB-to-Thermal Image Translation)

ABSTRACT

Convolutional Neural Networks (CNN) for ship classification in multi-spectral images (RGB, IR, etc.) is proposed in this paper. Recent developments in deep learning have significantly advanced the field of ship recognition. However, since maritime light intensity is frequently disturbed, multispectral imaging is considered a more robust substitute for RGB imaging. The proposed architectures were fine-tuned after being trained from scratch on the publicly available dataset VAIS (RGB-IR pairs). Unfortunately, the classification results plateaued at 59.74% accuracy, which is unsatisfactory for most real-life applications. Such an accuracy wall was due to the small number of training images. In order to overcome the scarcity of IR ship images, we proposed a novel image data augmentation strategy that translates RGB images to IR. A Pix2Pix model and a Generative Adversarial Network (GAN) network were modified to carry out the generation process as an RGB to IR translator. The KAIST general-purpose RGB-IR image pairs dataset was used to train our RGB-to-IR image translator, whereas the VAIS dataset was held aside for validation purposes. Our proposed network improved the accuracy of the native network by 8% (from 59.74% to 67.74%), which is fairly satisfactory in the field of ship recognition.

KEYWORDS:

• ThermalGAN
• Classification
• VAIS
• KAIST
• Multi-spectral

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/0952813X.2023.2165723

Notes

1. http://www.shipspotting.com/.