Transformer-based dual path cross fusion for pansharpening remote sensing images

References

• Aiazzi, B., L. Alparone, S. Baronti, A. Garzelli, and M. Selva. 2003. An mtf-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas. In 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas, Berlin, Germany, 90–94. IEEE.
   Google Scholar
• Aiazzi, B., S. Baronti, and M. Selva. 2007. “Improving Component Substitution Pansharpening Through Multivariate Regression of Ms + Pan Data.” IEEE Transactions on Geoscience and Remote Sensing 45 (10): 3230–3239. https://doi.org/10.1109/TGRS.2007.901007.
   Web of Science ®Google Scholar
• Alparone, L., B. Aiazzi, S. Baronti, and A. Garzelli. 2003. Sharpening of Very High Resolution Images with Spectral Distortion Minimization. In IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium: Proceedings (IEEE Cat. No. 03CH37477), Toulouse, France, Vol. 1, 458–460. IEEE.
   Google Scholar
• Ballester, C., V. Caselles, L. Igual, J. Verdera, and B. Rougé. 2006. “A Variational Model for P+ Xs Image Fusion.” International Journal of Computer Vision 69 (1): 43–58. https://doi.org/10.1007/s11263-006-6852-x.
   Web of Science ®Google Scholar
• Carper, W., T. Lillesand, and R. Kiefer. 1990. “The Use of Intensity-Hue-Saturation Transformations for Merging Spot Panchromatic and Multispectral Image Data.” Photogrammetric Engineering and Remote Sensing 56 (4): 459–467.
   Web of Science ®Google Scholar
• Chavez, P. S. Jr., S. C. Sides, J. A. Anderson. 1991. “Comparison of Three Different Methods to Merge Multiresolution and Multispectral Data- Landsat Tm and Spot Panchromatic.” Photogrammetric Engineering and Remote Sensing 57 (3): 295–303.
   Web of Science ®Google Scholar
• Chen, C., Y. Li, W. Liu, and J. Huang. 2014. “Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity.” 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2760–2765. https://doi.org/10.1109/CVPR.2014.347.
   Google Scholar
• Fang, F., L. Fang, G. Zhang, and C. Shen. 2013. “A Variational Method for Multisource Remote-Sensing Image Fusion.” International Journal of Remote Sensing 34 (7): 2470–2486. https://doi.org/10.1080/01431161.2012.744882.
   Web of Science ®Google Scholar
• Fei, R., J. Zhang, J. Liu, F. Du, P. Chang, and J. Hu. 2019. “Convolutional Sparse Representation of Injected Details for Pansharpening.” IEEE Geoscience and Remote Sensing Letters 16 (10): 1595–1599. https://doi.org/10.1109/LGRS.2019.2904526.
   Web of Science ®Google Scholar
• Garzelli, A. 2014. “Pansharpening of Multispectral Images Based on Nonlocal Parameter Optimization.” IEEE Transactions on Geoscience and Remote Sensing 53 (4): 2096–2107. https://doi.org/10.1109/TGRS.2014.2354471.
   Web of Science ®Google Scholar
• Garzelli, A. 2016. “A Review of Image Fusion Algorithms Based on the Super-Resolution Paradigm.” Remote Sensing 8 (10): 797. https://doi.org/10.3390/rs8100797.
   Web of Science ®Google Scholar
• Garzelli, A., F. Nencini, and L. Capobianco. 2007. “Optimal Mmse Pan Sharpening of Very High Resolution Multispectral Images.” IEEE Transactions on Geoscience and Remote Sensing 46 (1): 228–236. https://doi.org/10.1109/TGRS.2007.907604.
   Web of Science ®Google Scholar
• Guo, M.-H., T.-X. Xu, J.-J. Liu, Z.-N. Liu, P.-T. Jiang, T.-J. Mu, S.-H. Zhang, R. R. Martin, M.-M. Cheng, and S.-M. Hu. 2022. “Attention Mechanisms in Computer Vision: A Survey.” Computational Visual Media 8 (3): 331–368. https://doi.org/10.1007/s41095-022-0271-y.
   Google Scholar
• Guo, M., H. Zhang, J. Li, L. Zhang, and H. Shen. 2014. “An Online Coupled Dictionary Learning Approach for Remote Sensing Image Fusion.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7 (4): 1284–1294. https://doi.org/10.1109/JSTARS.2014.2310781.
   Web of Science ®Google Scholar
• Hou, Q., L. Zhang, M.-M. Cheng, and J. Feng. 2020. “Strip Pooling: Rethinking Spatial Pooling for Scene Parsing.” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 4002–4011. https://doi.org/10.1109/CVPR42600.2020.00406.
   Google Scholar
• Huang, W., L. Xiao, Z. Wei, H. Liu, and S. Tang. 2015. “A New Pan-Sharpening Method with Deep Neural Networks.” IEEE Geoscience and Remote Sensing Letters 12 (5): 1037–1041. https://doi.org/10.1109/LGRS.2014.2376034.
   Web of Science ®Google Scholar
• Hu, J., L. Shen, and G. Sun. 2018. “Squeeze-And-Excitation Networks[C].” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7132–7141. IEEE. https://doi.org/10.1109/CVPR.2018.00745.
   Google Scholar
• Kwan, C., J. Hee Choi, S. H. Chan, J. Zhou, and B. Budavari. 2018. “A Super-Resolution and Fusion Approach to Enhancing Hyperspectral Images.” Remote Sensing 10 (9): 1416. https://doi.org/10.3390/rs10091416.
   Web of Science ®Google Scholar
• Laben, C. A., and B. V. Brower. Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening. US Patent 6,011,875, January 4, 2000.
   Google Scholar
• Lei, X., Y. Xia, A. Wang, X. Jian, H. Zhong, and L. Sun. 2023. “Mutual Information Based Anomaly Detection of Monitoring Data with Attention Mechanism and Residual Learning.” Mechanical Systems and Signal Processing 182:109607. https://doi.org/10.1016/j.ymssp.2022.109607.
   Web of Science ®Google Scholar
• Li, Z., and H. Leung. 2008. “Fusion of Multispectral and Panchromatic Images Using a Restoration-Based Method.” IEEE Transactions on Geoscience and Remote Sensing 47 (5): 1482–1491. https://doi.org/10.1109/TGRS.2008.2005639.
   Web of Science ®Google Scholar
• Li, G., L. Lingling, H. Zhu, X. Liu, and L. Jiao. 2019. “Adaptive Multiscale Deep Fusion Residual Network for Remote Sensing Image Classification.” IEEE Transactions on Geoscience and Remote Sensing 57 (11): 8506–8521. https://doi.org/10.1109/TGRS.2019.2921342.
   Web of Science ®Google Scholar
• Li, M., Y. Liu, T. Xiao, Y. Huang, and G. Yang. 2023. “Local-Global Transformer Enhanced Unfolding Network for Pan-Sharpening.” The 32nd International Joint Conference on Artificial Intelligence (IJCAI), Macao, S.A.R, 1071–1079. arXiv preprint arXiv:2304.14612.
   Google Scholar
• Liu, Y., X. Chen, Z. Wang, Z. J. Wang, R. K. Ward, and X. Wang. 2018. “Deep Learning for Pixel-Level Image Fusion: Recent Advances and Future Prospects.” Information Fusion 42:158–173. https://doi.org/10.1016/j.inffus.2017.10.007.
   Web of Science ®Google Scholar
• Liu, J., Y. Feng, C. Zhou, and C. Zhang. 2020. “Pwnet: An Adaptive Weight Network for the Fusion of Panchromatic and Multispectral Images.” Remote Sensing 12 (17): 2804. https://doi.org/10.3390/rs12172804.
   Web of Science ®Google Scholar
• Liu, X., X. Liu, H. Dai, X. Kan, A. Plaza, and W. Zu. 2023. “Mun-Gan: A Multi-Scale Unsupervised Network for Remote Sensing Image Pansharpening.” IEEE Transactions on Geoscience and Remote Sensing, Vol. 61, 1–18, Art no. 5404018. https://doi.org/10.1109/TGRS.2023.3288073.
   Google Scholar
• Li, S., and B. Yang. 2010. “A New Pan-Sharpening Method Using a Compressed Sensing Technique.” IEEE Transactions on Geoscience and Remote Sensing 49 (2): 738–746. https://doi.org/10.1109/TGRS.2010.2067219.
   Web of Science ®Google Scholar
• Masi, G., D. Cozzolino, L. Verdoliva, and G. Scarpa. 2016. “Pansharpening by Convolutional Neural Networks.” Remote Sensing 8 (7): 594. https://doi.org/10.3390/rs8070594.
   Web of Science ®Google Scholar
• Meng, X., H. Shen, Q. Yuan, H. Li, L. Zhang, and W. Sun. 2018. “Pansharpening for Cloud-Contaminated Very High-Resolution Remote Sensing Images.” IEEE Transactions on Geoscience and Remote Sensing 57 (5): 2840–2854. https://doi.org/10.1109/TGRS.2018.2878007.
   Web of Science ®Google Scholar
• Meng, X., H. Shen, L. Zhang, Q. Yuan, and L. Huifang. 2015. A Unified Framework for Spatio-Temporal-Spectral Fusion of Remote Sensing Images. In 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy, 2584–2587. IEEE.
   Google Scholar
• Meng, X., Y. Xiong, F. Shao, H. Shen, W. Sun, G. Yang, Q. Yuan, R. Fu, and H. Zhang. 2020. “A Large-Scale Benchmark Data Set for Evaluating Pansharpening Performance: Overview and Implementation.” IEEE Geoscience and Remote Sensing Magazine 9 (1): 18–52. https://doi.org/10.1109/MGRS.2020.2976696.
   Web of Science ®Google Scholar
• Mnih, V., N. Heess, A. Graves. 2014. “Recurrent Models of Visual Attention.” Advances in Neural Information Processing Systems: 27.
   Google Scholar
• Monga, V., Y. Li, and Y. C. Eldar. 2021. “Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing.” IEEE Signal Processing Magazine 38 (2): 18–44. https://doi.org/10.1109/MSP.2020.3016905.
   Web of Science ®Google Scholar
• Nencini, F., A. Garzelli, S. Baronti, and L. Alparone. 2007. “Remote Sensing Image Fusion Using the Curvelet Transform.” Information Fusion 8 (2): 143–156. https://doi.org/10.1016/j.inffus.2006.02.001.
   Web of Science ®Google Scholar
• Otazu, X., M. González-Audícana, O. Fors, and J. Núñez. 2005. “Introduction of Sensor Spectral Response into Image Fusion Methods. Application to Wavelet-Based Methods.” IEEE Transactions on Geoscience and Remote Sensing 43 (10): 2376–2385. https://doi.org/10.1109/TGRS.2005.856106.
   Web of Science ®Google Scholar
• Rockafellar, R. T. 1976. “Monotone Operators and the Proximal Point Algorithm.” SIAM Journal on Control and Optimization 14 (5): 877–898. https://doi.org/10.1137/0314056.
   Web of Science ®Google Scholar
• Rush, A. M., S. Chopra, and J. Weston. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685, 2015.
   Google Scholar
• Scarpa, G., S. Vitale, and D. Cozzolino. 2018. “Target-Adaptive Cnn-Based Pansharpening.” IEEE Transactions on Geoscience and Remote Sensing 56 (9): 5443–5457. https://doi.org/10.1109/TGRS.2018.2817393.
   Web of Science ®Google Scholar
• Shen, H., X. Meng, and L. Zhang. 2016. “An Integrated Framework for the Spatio–Temporal–Spectral Fusion of Remote Sensing Images.” IEEE Transactions on Geoscience and Remote Sensing 54 (12): 7135–7148. https://doi.org/10.1109/TGRS.2016.2596290.
   Web of Science ®Google Scholar
• Su, X., J. Li, and Z. Hua. 2022. “Transformer-Based Regression Network for Pansharpening Remote Sensing Images.” IEEE Transactions on Geoscience and Remote Sensing 60:1–23. https://doi.org/10.1109/TGRS.2022.3152425.
   Web of Science ®Google Scholar
• Ulyanov, D., A. Vedaldi, and V. Lempitsky. 2018. “Deep image prior.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 9446–9454.
   Google Scholar
• Ulyanov, D., F. Wang, M. Jiang, C. Qian, S. Yang, L. Cheng, H. Zhang, X. Wang, and X. Tang. 2017. “Residual Attention Network for Image Classification.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 3156–3164.
   Google Scholar
• Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. 2017. “Attention is All You Need.” Advances in Neural Information Processing Systems 30.
   Google Scholar
• Vivone, G., L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald. 2014. “A Critical Comparison Among Pansharpening Algorithms.” IEEE Transactions on Geoscience and Remote Sensing 53 (5): 2565–2586. https://doi.org/10.1109/TGRS.2014.2361734.
   Web of Science ®Google Scholar
• Vivone, G., S. Marano, and J. Chanussot. 2020. “Pansharpening: Context-Based Generalized Laplacian Pyramids by Robust Regression.” IEEE Transactions on Geoscience & Remote Sensing 58 (9): 6152–6167. https://doi.org/10.1109/TGRS.2020.2974806.
   Web of Science ®Google Scholar
• Wang, Q., W. Banggu, P. Zhu, L. Peihua, W. Zuo, and H. Qinghua. 2020. Supplementary Material for ‘Eca-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, WA, USA, 13–19.
   Google Scholar
• Wang, F., H. Haotian, and C. Shen. Bam: A balanced attention mechanism for single image super resolution. arXiv preprint arXiv:2104.07566, 2021.
   Google Scholar
• Wang, Q., P. Yan, Y. Yuan, and X. Li. 2013. “Multi-spectral saliency detection.” Pattern Recognition Letters 34 (1): 34–41. https://doi.org/10.1016/j.patrec.2012.06.002.
   Web of Science ®Google Scholar
• Wei, Q. Bayesian fusion of multi-band images: A powerful tool for super-resolution. PhD thesis, Institut national polytechnique de Toulouse (INPT), 2015.
   Google Scholar
• Wei, Y., Q. Yuan, H. Shen, and L. Zhang. 2017. “Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network.” IEEE Geoscience and Remote Sensing Letters 14 (10): 1795–1799. https://doi.org/10.1109/LGRS.2017.2736020.
   Web of Science ®Google Scholar
• Yang, J., F. Xueyang, H. Yuwen, Y. Huang, X. Ding, and J. Paisley. 2017. Pannet: A Deep Network Architecture for Pan-Sharpening. In Proceedings of the IEEE international conference on computer vision, Honolulu, Hawaii, 5449–5457.
   Google Scholar
• Yin, W., H. Schütze, B. Xiang, and B. Zhou. 2016. “Abcnn: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs.” Transactions of the Association for Computational Linguistics 4:259–272. https://doi.org/10.1162/tacl_a_00097.
   Google Scholar
• Zhang, L., H. Shen, W. Gong, and H. Zhang. 2012. “Adjustable Model-Based Fusion Method for Multispectral and Panchromatic Images.” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42 (6): 1693–1704. https://doi.org/10.1109/TSMCB.2012.2198810.
   PubMed Web of Science ®Google Scholar
• Zhang, H., H. Wang, X. Tian, and J. Ma. 2023. “P2sharpen: A Progressive Pansharpening Network with Deep Spectral Transformation.” Information Fusion 91:103–122. https://doi.org/10.1016/j.inffus.2022.10.010.
   Web of Science ®Google Scholar
• Zhang, K., W. Zuo, G. Shuhang, and L. Zhang. 2017. “Learning deep cnn denoiser prior for image restoration.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 3929–3938.
   Google Scholar
• Zhao, X., J. Guo, Y. Zhang, and Y. Wu. 2023. Asymmetric Bidirectional Fusion Network for Remote Sensing Pansharpening. IEEE Transactions on Geoscience and Remote Sensing, Vol. 61, 1–16, Art no. 5404816. https://doi.org/10.1109/TGRS.2023.3296510.
   Google Scholar
• Zhu, X. X., and R. Bamler. 2012. “A Sparse Image Fusion Algorithm with Application to Pan-Sharpening.” IEEE Transactions on Geoscience and Remote Sensing 51 (5): 2827–2836. https://doi.org/10.1109/TGRS.2012.2213604.
   Web of Science ®Google Scholar
• Zhu, X. X., C. Grohnfeldt, and R. Bamler. 2015. “Exploiting Joint Sparsity for Pansharpening: The J-Sparsefi Algorithm.” IEEE Transactions on Geoscience and Remote Sensing 54 (5): 2664–2681. https://doi.org/10.1109/TGRS.2015.2504261.
   Web of Science ®Google Scholar
• Zhu, W., J. Li, Z. An, and Z. Hua. 2023. “Multiscale Hybrid Attention Transformer for Remote Sensing Image Pansharpening.” IEEE Transactions on Geoscience and Remote Sensing 61:1–16. https://doi.org/10.1109/TGRS.2023.3239013.
   Web of Science ®Google Scholar