Hyperspectral and multispectral image fusion via residual selective kernel attention-based U-net

References

• Akbari, H., Y. Kosugi, K. Kojima, and N. Tanaka. 2010. “Detection and Analysis of the Intestinal Ischemia Using Visible and Invisible Hyperspectral Imaging.” IEEE Transactions on Biomedical Engineering 57 (8): 2011–2017. https://doi.org/10.1109/tbme.2010.2049110.
   PubMed Web of Science ®Google Scholar
• Akhtar, N., F. Shafait, and A. Mian. 2014. “Sparse Spatio-Spectral Representation for Hyperspectral Image Super-Resolution.” Computer Vision – ECCV 2014 63–78. https://doi.org/10.1007/978-3-319-10584-0_5.
   Google Scholar
• Akhtar, N., F. Shafait, and A. Mian. 2015. “Bayesian Sparse Representation for Hyperspectral Image Super Resolution.” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2015.7298986.
   Google Scholar
• Anul Haq, M. 2022a. “CDLSTM: A Novel Model for Climate Change Forecasting.” Computers Materials & Continua 71 (2): 2363–2381. https://doi.org/10.32604/cmc.2022.023059.
   Web of Science ®Google Scholar
• Anul Haq, M. 2022b. “Planetscope Nanosatellites Image Classification Using Machine Learning.” Computer Systems Science and Engineering 42 (3): 1031–1046. https://doi.org/10.32604/csse.2022.023221.
   Web of Science ®Google Scholar
• Anul Haq, M., S. B. H. Hassine, S. J. Malebary, H. A. Othman, and E. M. Tag-Eldin. 2023. “3D-CNNHSR: A 3-Dimensional Convolutional Neural Network for Hyperspectral Super-Resolution.” Computer Systems Science and Engineering 47 (2): 2689–2705. https://doi.org/10.32604/csse.2023.039904.
   Google Scholar
• Camacho, A., E. Vargas, and H. Arguello. 2022. “Hyperspectral and Multispectral Image Fusion Addressing Spectral Variability by an Augmented Linear Mixing Model.” International Journal of Remote Sensing 43 (5): 1577–1608. https://doi.org/10.1080/01431161.2022.2041762.
   Web of Science ®Google Scholar
• Chen, Z., H. Pu, B. Wang, and G.-M. Jiang. 2014. “Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods.” IEEE Geoscience and Remote Sensing Letters 11 (8): 1418–1422. https://doi.org/10.1109/lgrs.2013.2294476.
   Web of Science ®Google Scholar
• Dell’acqua, F., P. Gamba, A. Ferrari, J. A. Palmason, J. A. Benediktsson, and K. Arnason. 2004. “Exploiting Spectral and Spatial Information in Hyperspectral Urban Data with High Resolution.” IEEE Geoscience & Remote Sensing Letters 1 (4): 322–326. https://doi.org/10.1109/lgrs.2004.837009.
   Web of Science ®Google Scholar
• Deng, S., L. Deng, X. Wu, R. Ran, D. Hong, and G. Vivone. 2023. “PSRT: Pyramid Shuffle-And-Reshuffle Transformer for Multispectral and Hyperspectral Image Fusion.” IEEE Transactions on Geoscience and Remote Sensing 61:1–15. https://doi.org/10.1109/tgrs.2023.3244750.
   Google Scholar
• Deng, L.-J., M. Feng, and X.-C. Tai. 2019. “The Fusion of Panchromatic and Multispectral Remote Sensing Images via Tensor-Based Sparse Modeling and Hyper-Laplacian Prior.” Information Fusion 52:76–89. https://doi.org/10.1016/j.inffus.2018.11.014.
   Web of Science ®Google Scholar
• Dian, R., and S. Li. 2019. “Hyperspectral Image Super-Resolution via Subspace-Based Low Tensor Multi-Rank Regularization.” IEEE Transactions on Image Processing 28 (10): 5135–5146. https://doi.org/10.1109/tip.2019.2916734.
   Web of Science ®Google Scholar
• Dian, R., S. Li, B. Sun, and A. Guo. 2021. “Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion.” Information Fusion 69:40–51. https://doi.org/10.1016/j.inffus.2020.11.001.
   Web of Science ®Google Scholar
• Dong, W., F. Fu, G. Shi, X. Cao, J. Wu, G. Li, and X. Li. 2016. “Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation.” IEEE Transactions on Image Processing 25 (5): 2337–2352. https://doi.org/10.1109/tip.2016.2542360.
   PubMed Web of Science ®Google Scholar
• Dong, X., X. Sun, X. Jia, Z. Xi, L. Gao, and B. Zhang. 2021. “Remote Sensing Image Super-Resolution Using Novel Dense-Sampling Networks.” IEEE Transactions on Geoscience and Remote Sensing 59 (2): 1618–1633. https://doi.org/10.1109/tgrs.2020.2994253.
   Web of Science ®Google Scholar
• Fang, J., J. Yang, A. Khader, and L. Xiao. 2023. “A Multiresolution Details Enhanced Attentive Dual-Unet for Hyperspectral and Multispectral Image Fusion.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 16:638–655. https://doi.org/10.1109/jstars.2022.3228941.
   Web of Science ®Google Scholar
• Gao, H., S. Li, and R. Dian. 2022. “Hyperspectral and Multispectral Image Fusion via Self-Supervised Loss and Separable Loss.” IEEE Transactions on Geoscience and Remote Sensing 60:1–12. https://doi.org/10.1109/tgrs.2022.3204769.
   Web of Science ®Google Scholar
• Han, X.-H., B. Shi, and Y. Zheng. 2018. “SSF-CNN: Spatial and Spectral Fusion with CNN for Hyperspectral Image Super-Resolution.” 2018 25th IEEE International Conference on Image Processing (ICIP). https://doi.org/10.1109/icip.2018.8451142.
   Google Scholar
• Haq, M. A., G. Rahaman, P. Baral, and A. Ghosh. 2020. “Deep Learning Based Supervised Image Classification Using UAV Images for Forest Areas Classification.” Journal of the Indian Society of Remote Sensing 49 (3): 601–606. https://doi.org/10.1007/s12524-020-01231-3.
   Web of Science ®Google Scholar
• Huang, B., H. Song, H. Cui, J. Peng, and Z. Xu. 2014. “Spatial and Spectral Image Fusion Using Sparse Matrix Factorization.” IEEE Transactions on Geoscience and Remote Sensing 52 (3): 1693–1704. https://doi.org/10.1109/tgrs.2013.2253612.
   Web of Science ®Google Scholar
• Hu, J., C. Du, and S. Fan. 2020. “Two-Stage Pansharpening Based on Multi-Level Detail Injection Network.” Institute of Electrical and Electronics Engineers Access 8:156442–156455. https://doi.org/10.1109/access.2020.3019201.
   Google Scholar
• Hu, J., L. Shen, and G. Sun 2018. “Squeeze-and-Excitation Networks.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2018.00745.
   Google Scholar
• Jiang, J., H. Sun, X. Liu, and J. Ma. 2020. “Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery.” IEEE Transactions on Computational Imaging 6:1082–1096. https://doi.org/10.1109/tci.2020.2996075.
   Web of Science ®Google Scholar
• Jiao, C., C. Chen, S. Gou, X. Wang, B. Yang, X. Chen, and L. Jiao. 2023. “₁ Sparsity-Regularized Attention Multiple-Instance Network for Hyperspectral Target Detection.” IEEE Transactions on Cybernetics 53 (1): 124–137. https://doi.org/10.1109/tcyb.2021.3087662.
   Web of Science ®Google Scholar
• Kanatsoulis, C. I., X. Fu, N. D. Sidiropoulos, and W.-K. Ma. 2018. “Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach.” IEEE Transactions on Signal Processing 66 (24): 6503–6517. https://doi.org/10.1109/tsp.2018.2876362.
   Web of Science ®Google Scholar
• Kawakami, R., Y. Matsushita, J. Wright, M. Ben-Ezra, Y.-W. Tai, and K. Ikeuchi. 2011. “High-Resolution Hyperspectral Imaging via Matrix Factorization.” Cvpr 2011. https://doi.org/10.1109/cvpr.2011.5995457.
   Google Scholar
• Khader, A., J. Yang, and L. Xiao. 2023. “Model-Guided Deep Unfolded Fusion Network with Nonlocal Spatial-Spectral Priors for Hyperspectral Image Super-Resolution.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 16:4607–4625. https://doi.org/10.1109/jstars.2023.3272370.
   Web of Science ®Google Scholar
• Kilmer, M. E., K. Braman, N. Hao, and R. C. Hoover. 2013. “Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging.” SIAM Journal on Matrix Analysis and Applications 34 (1): 148–172. https://doi.org/10.1137/110837711.
   Web of Science ®Google Scholar
• Kingma, D. P., and Ba. Jimmy. 2014. “Adam: A Method for Stochastic Optimization.” arXiv.org. Accessed January 30, 2017. https://arxiv.org/abs/1412.6980.
   Google Scholar
• Kruse, F. A., A. B. Lefkoff, J. W. Boardman, K. B. Heidebrecht, A. T. Shapiro, P. J. Barloon, and A. F. Goetz. 1993. “The Spectral Image Processing System (Sips)-Interactive Visualization and Analysis of Imaging Spectrometer Data.” AIP Conference Proceedings. https://doi.org/10.1063/1.44433.
   Google Scholar
• Li, J., M. Ai, S. Wang, and Q. Hu. 2022. “GRF: Guided Residual Fusion for Pansharpening.” International Journal of Remote Sensing 43 (10): 3609–3627. https://doi.org/10.1080/01431161.2022.2100726.
   Web of Science ®Google Scholar
• Li, S., R. Dian, L. Fang, and J. M. Bioucas-Dias. 2018. “Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization.” IEEE Transactions on Image Processing 27 (8): 4118–4130. https://doi.org/10.1109/tip.2018.2836307.
   Web of Science ®Google Scholar
• Li, W., X. Liang, and M. Dong. 2021. “MDECNN: A Multiscale Perception Dense Encoding Convolutional Neural Network for Multispectral Pan-Sharpening.” Remote Sensing 13 (3): 535. https://doi.org/10.3390/rs13030535.
   Web of Science ®Google Scholar
• Liu, X., Q. Liu, and Y. Wang. 2020. “Remote Sensing Image Fusion Based on Two-Stream Fusion Network.” Information Fusion 55:1–15. https://doi.org/10.1016/j.inffus.2019.07.010.
   Web of Science ®Google Scholar
• Liu, D., J. Li, and Q. Yuan. 2021. “A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution.” IEEE Transactions on Geoscience & Remote Sensing 59 (9): 7711–7725. https://doi.org/10.1109/tgrs.2021.3049875.
   Web of Science ®Google Scholar
• Liu, J., Z. Wu, L. Xiao, J. Sun, and H. Yan. 2020. “A Truncated Matrix Decomposition for Hyperspectral Image Super-Resolution.” IEEE Transactions on Image Processing 29:8028–8042. https://doi.org/10.1109/tip.2020.3009830.
   Web of Science ®Google Scholar
• Li, X., W. Wang, X. Hu, and J. Yang. 2019. “Selective Kernel Networks.” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2019.00060.
   Google Scholar
• Li, K., W. Zhang, D. Yu, and X. Tian. 2022. “HyperNet: A Deep Network for Hyperspectral, Multispectral, and Panchromatic Image Fusion.” ISPRS Journal of Photogrammetry & Remote Sensing 188:30–44. https://doi.org/10.1016/j.isprsjprs.2022.04.001.
   Web of Science ®Google Scholar
• Mangan, P., M. Anul Haq, and P. Baral. 2019. “Morphometric Analysis of Watershed Using Remote Sensing and GIS—A Case Study of Nanganji River Basin in Tamil Nadu, India.” Arabian Journal of Geosciences 12 (6). https://doi.org/10.1007/s12517-019-4382-4.
   Web of Science ®Google Scholar
• Oseledets, I. V. 2011. “Tensor-Train Decomposition.” SIAM Journal on Scientific Computing 33 (5): 2295–2317. https://doi.org/10.1137/090752286.
   Web of Science ®Google Scholar
• Ronneberger, O., P. Fischer, and T. Brox. 2015. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” Lecture Notes in Computer Science 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.
   Google Scholar
• Scarpa, G., S. Vitale, and D. Cozzolino. 2018. “Target-Adaptive CNN-Based Pansharpening.” IEEE Transactions on Geoscience and Remote Sensing 56 (9): 5443–5457. https://doi.org/10.1109/tgrs.2018.2817393.
   Web of Science ®Google Scholar
• Selva, M., B. Aiazzi, F. Butera, L. Chiarantini, and S. Baronti. 2015. “Hyper-Sharpening: A First Approach on SIM-GA Data.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 8 (6): 3008–3024. https://doi.org/10.1109/jstars.2015.2440092.
   Web of Science ®Google Scholar
• Tucker, L. R. 1966. “Some mathematical notes on three-mode factor analysis.” Psychometrika 31 (3): 279–311. https://doi.org/10.1007/BF02289464.
   PubMed Web of Science ®Google Scholar
• Wang, Z., and A. C. Bovik. 2002. “A Universal Image Quality Index.” IEEE Signal Processing Letters 9 (3): 81–84. https://doi.org/10.1109/97.995823.
   Web of Science ®Google Scholar
• Wang, Z., A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. “Image Quality Assessment: From Error Visibility to Structural Similarity.” IEEE Transactions on Image Processing 13 (4): 600–612. https://doi.org/10.1109/tip.2003.819861.
   PubMed Web of Science ®Google Scholar
• Wei, Q., J. Bioucas-Dias, N. Dobigeon, and J.-Y. Tourneret. 2015. “Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation.” IEEE Transactions on Geoscience and Remote Sensing 53 (7): 3658–3668. https://doi.org/10.1109/tgrs.2014.2381272.
   Web of Science ®Google Scholar
• Xiao, B., G. Ou, H. Tang, X. Bi, and W. Li. 2020. “Multi-Focus Image Fusion by Hessian Matrix Based Decomposition.” IEEE Transactions on Multimedia 22 (2): 285–297. https://doi.org/10.1109/tmm.2019.2928516.
   Web of Science ®Google Scholar
• Xie, Q., M. Zhou, Q. Zhao, D. Meng, W. Zuo, and Z. Xu. 2019. “Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net.” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2019.00168.
   Google Scholar
• Yang, J., Y.-Q. Zhao, and J. Chan. 2018. “Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network.” Remote Sensing 10 (5): 800. https://doi.org/10.3390/rs10050800.
   Web of Science ®Google Scholar
• Yokoya, N., T. Yairi, and A. Iwasaki. 2011. “Coupled Non-Negative Matrix Factorization (CNMF) for Hyperspectral and Multispectral Data Fusion: Application to Pasture Classification.” 2011 IEEE International Geoscience and Remote Sensing Symposium. https://doi.org/10.1109/igarss.2011.6049465.
   Google Scholar
• You, T., C. Wu, Y. Bai, D. Wang, H. Ge, and Y. Li. 2023. “HMF-Former: Spatio-Spectral Transformer for Hyperspectral and Multispectral Image Fusion.” IEEE Geoscience and Remote Sensing Letters 20:1–5. https://doi.org/10.1109/lgrs.2022.3229692.
   Web of Science ®Google Scholar
• Yuan, Q., Y. Wei, X. Meng, H. Shen, and L. Zhang. 2018. “A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 11 (3): 978–989. https://doi.org/10.1109/jstars.2018.2794888.
   Web of Science ®Google Scholar
• Zhang, X., W. Huang, Q. Wang, and X. Li. 2021. “SSR-Net: Spatial–Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion.” IEEE Transactions on Geoscience and Remote Sensing 59 (7): 5953–5965. https://doi.org/10.1109/tgrs.2020.3018732.
   Web of Science ®Google Scholar
• Zhou, Y., L. Feng, C. Hou, and S.-Y. Kung. 2017. “Hyperspectral and Multispectral Image Fusion Based on Local Low Rank and Coupled Spectral Unmixing.” IEEE Transactions on Geoscience and Remote Sensing 55 (10): 5997–6009. https://doi.org/10.1109/tgrs.2017.2718728.
   Web of Science ®Google Scholar