Parallel interactive delayed attention network for pansharpening

ABSTRACT

As deep learning has gained wide attention in computer vision tasks, researchers have also started to explore the application of deep learning in panchromatic sharpening. In recent years, various convolutional neural network-based methods for panchromatic sharpening have been proposed, and these methods have achieved good results. However, shallower networks are unable to learn complex mapping relationships. Networks that are too deep are prone to overfitting, which leads to the degradation of the fusion effect. Moreover, since the convolutional operations are concentrated in local regions, it is difficult to obtain the association information with other regions even in deep networks. Therefore, more accurate feature selection and representation with limited network depth is needed. In this paper, a parallel interactive delayed attention network for panchromatic sharpening (PIDAN) is proposed. The network consists of multiple resolution subnetworks with parallel inputs to improve the representation of high-resolution feature maps by multiscale low-resolution features of the same depth and similar level. In addition, we propose a delayed channel attention module. This module obtains correlations between low-frequency and high-frequency information through adaptive learning, making the network more flexible in processing different types of information. The depth feature reweighting restriction combined with the residual unit can further avoid the overfitting representation of features as the network deepens. Experimental results on Gaofen-2 and WorldView-2 datasets show that the proposed method is competitive in both quality assessment and visual perception.

KEYWORDS:

• Multiresolution subnets
• interactive fusion
• channel attention
• pansharpening

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Some or all data, models, or code generated or used during the study are available from the corresponding author by request.

Additional information

Funding

The work was supported by the National Natural Science Foundation of China [61972060,62027827,U1713213]; National Key Research and Development Program of China [2019YFE0110800]; the Natural Science Foundation of Chongqing, China [cstc2019cxcyljrc-td0270,cstc2020jcyj-zdxmX0025].