A novel pansharpening model based on two parallel network architectures

References

• Aiazzi, B., L. Alparone, S. Baronti, and A. Garzelli. 2002. “Context-Driven Fusion of High Spatial and Spectral Resolution Images Based on Oversampled Multiresolution Analysis.” IEEE Transactions on Geoscience & Remote Sensing 40 (10): 2300–2312. https://doi.org/10.1109/TGRS.2002.803623.
   Web of Science ®Google Scholar
• Aiazzi, B., L. Alparone, S. Baronti, A. Garzelli, and M. Selva. 2006. “MTF-Tailored Multiscale Fusion of High-Resolution MS and Pan Imagery.” Photogrammetric Engineering & Remote Sensing 72 (5): 591–596. https://doi.org/10.14358/PERS.72.5.591.
   Web of Science ®Google Scholar
• Alparone, L., B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva. 2008. “Multispectral and Panchromatic Data Fusion Assessment without Reference.” Photogrammetric Engineering & Remote Sensing 74 (2): 193–200. https://doi.org/10.14358/PERS.74.2.193.
   Web of Science ®Google Scholar
• Alparone, L., S. Baronti, A. Garzelli, and F. Nencini. 2004. “A Global Quality Measurement of Pan-Sharpened Multispectral Imagery.” IEEE Geoscience & Remote Sensing Letters 1 (4): 313–317. https://doi.org/10.1109/LGRS.2004.836784.
   Web of Science ®Google Scholar
• Aly, H. A., and G. Sharma. 2014. “A Regularized Model-Based Optimization Framework for Pan-Sharpening.” IEEE Transactions on Image Processing 23 (6): 2596–2608. https://doi.org/10.1109/TIP.2014.2316641.
   PubMed Web of Science ®Google Scholar
• Atkinson, P. M. 2013. “Downscaling in Remote Sensing.” International Journal of Applied Earth Observation and Geoinformation 22:106–114. https://doi.org/10.1016/j.jag.2012.04.012.
   Web of Science ®Google Scholar
• Carleer, A. P., O. Debeir, and E. Wolff. 2005. “Assessment of Very High Spatial Resolution Satellite Image Segmentations.” Photogrammetric Engineering & Remote Sensing 71 (11): 1285–1294. https://doi.org/10.14358/PERS.71.11.1285.
   Web of Science ®Google Scholar
• Choi, J., K. Yu, and Y. Kim. 2010. “A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement.” IEEE Transactions on Geoscience & Remote Sensing 49 (1): 295–309. https://doi.org/10.1109/TGRS.2010.2051674.
   Web of Science ®Google Scholar
• Ciotola, M., S. Vitale, A. Mazza, G. Poggi, and G. Scarpa. 2022. “Pansharpening by Convolutional Neural Networks in the Full Resolution Framework.” IEEE Transactions on Geoscience & Remote Sensing 60:1–17. https://doi.org/10.1109/TGRS.2022.3163887.
   Google Scholar
• Dong, C., C. C. Loy, K. He, X Tang. 2014. “Learning a Deep Convolutional Network for Image Super-Resolution.” European conference on computer vision, 184–199. Cham. Springer.
   Google Scholar
• Dong, C., C. C. Loy, K. He, and X. Tang. 2015 “Image Super-Resolution Using Deep Convolutional Networks”.” IEEE Transactions on Pattern Analysis & Machine Intelligence 38 (2): 295–307. https://doi.org/10.1109/TPAMI.2015.2439281.
   Web of Science ®Google Scholar
• Dong, W., J. Liang, and S. Xiao. 2020. “Saliency Analysis and Gaussian Mixture Model-Based Detail Extraction Algorithm for Hyperspectral Pansharpening.” IEEE Transactions on Geoscience & Remote Sensing 58 (8): 5462–5476. https://doi.org/10.1109/TGRS.2020.2966550.
   Google Scholar
• Dong, W., S. Xiao, and Y. Li. 2018. “Hyperspectral Pansharpening Based on Guided Filter and Gaussian Filter.” Journal of Visual Communication and Image Representation 53:171–179. https://doi.org/10.1016/j.jvcir.2018.03.014.
   Google Scholar
• Du, Q., N. H. Younan, R. King, and V. Shah. 2007. “On the Performance Evaluation of Pan-Sharpening Techniques.” IEEE Geoscience and Remote Sensing Letters 4 (4): 518–522. https://doi.org/10.1109/LGRS.2007.896328.
   Web of Science ®Google Scholar
• Ehlers, M., S. Klonus, P. Johan Åstrand, and P. Rosso. 2012. “Multi-Sensor Image Fusion for Pansharpening in Remote Sensing.” International Journal of Image and Data Fusion 1 (1): 25–45. https://doi.org/10.1080/19479830903561985.
   Google Scholar
• Fang, F., F. Li, C. Shen, and G. Zhang. 2013. “A Variational Approach for Pan-Sharpening.” IEEE Transactions on Image Processing 22 (7): 2822–2834. https://doi.org/10.1109/TIP.2013.2258355.
   PubMed Web of Science ®Google Scholar
• Fang, S., X. Wang, J. Zhang, Y. Cao. 2020. “Pan-Sharpening Based on Parallel Pyramid Convolutional Neural Network.” In IEEE International Conference on Image Processing, Alaska, USA, 453–457.
   Google Scholar
• Fang, Y., Y. Cai, and L. Fan. 2023. “SDRCNN: A Single-Scale Dense Residual Connected Convolutional Neural Network for Pansharpening.”IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing 16:6325–6338. https://doi.org/10.1109/JSTARS.2023.3292320.
   Google Scholar
• Gao, Y., M. Zhang, J. Wang, and W. Li. 2023. “Cross-Scale Mixing Attention for Multisource Remote Sensing Data Fusion and Classification.” IEEE Transactions on Geoscience & Remote Sensing 61:1–15. https://doi.org/10.1109/TGRS.2023.3263362.
   Web of Science ®Google Scholar
• Garzelli, A., F. Nencini, and L. Capobianco. 2007. “Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images.” IEEE Transactions on Geoscience & Remote Sensing 46 (1): 228–236. https://doi.org/10.1109/TGRS.2007.907604.
   Google Scholar
• He, X., L. Condat, B.-D. J. M, J. Chanussot, and J. Xia. 2014. “A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors.” IEEE Transactions on Image Processing 23 (9): 4160–4174. https://doi.org/10.1109/TIP.2014.2333661.
   PubMed Web of Science ®Google Scholar
• Hore, A., and D. Ziou. 2010. “Image Quality Metrics: PSNR Vs. SSIM.” In IEEE 2010 20th international conference on pattern recognition, Washington, USA, 2366–2369.
   Google Scholar
• Irsoy, O., and C. Cardie. 2014. “Deep Recursive Neural Networks for Compositionality in Language.” Advances in Neural Information Processing Systems 27 (3): 2096–2104.
   Google Scholar
• Jiang, C., H. Zhang, H. Shen, and L. Zhang. 2013. “Two-Step Sparse Coding for the Pan-Sharpening of Remote Sensing Images.” IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing 7:1792–1805. https://doi.org/10.1109/JSTARS.2013.2283236.
   Web of Science ®Google Scholar
• Kim, J., J. Kwon Lee, and K. Mu Lee. 2016. “Accurate Image Super-Resolution Using Very Deep Convolutional Networks.” In Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, USA, 1646–1654.
   Google Scholar
• Kingma, D. P., and J. Ba. 2015. “Adam: A Method for Stochastic Optimization.” In The 3rd International Conference for Learning Representations, CA, USA, 1–13.
   Google Scholar
• Kwarteng, P., and A. Chavez. 1989. “Extracting Spectral Contrast in Landsat Thematic Mapper Image Data Using Selective Principal Component Analysis.” Photogrammetric Engineering & Remote Sensing 55 (1): 339–348.
   Google Scholar
• Laben, C. A., and B. V. Brower. 2000. Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening: U.S. Patent 6,011,875.
   Google Scholar
• Ledig, C., L. Theis, F. Huszár, J Caballero, A Cunningham, A Acosta, A Aitken, A Tejani, J Totz, Z Wang, W Shi. 2017. “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network.” In Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, USA, 4681–4690.
   Google Scholar
• Li, F., R. Cong, H. Bai, Y. He. 2020. “Deep Interleaved Network for Image Super-Resolution with Asymmetric Co-Attention” In Proceedings of the 29th International Joint Conference on Artificial Intelligence, Yokohama, Japan, 537–543.
   Google Scholar
• Li, S., Q. Guo, and A. Li. 2022. “Pan-Sharpening Based on CNN+ Pyramid Transformer by Using No-Reference Loss.” Remote Sensing 14 (3): 624. https://doi.org/10.3390/rs14030624.
   Google Scholar
• Li, S., and B. Yang. 2010. “A New Pan-Sharpening Method Using a Compressed Sensing Technique.” IEEE Transactions on Geoscience & Remote Sensing 49 (2): 738–746. https://doi.org/10.1109/TGRS.2010.2067219.
   Google Scholar
• Liang, Y., P. Zhang, Y. Mei, and T. Wang. 2022. “PMACNet: Parallel Multiscale Attention Constraint Network for Pan-Sharpening.” IEEE Geoscience & Remote Sensing Letters 19:1–5. https://doi.org/10.1109/LGRS.2022.3170904.
   Google Scholar
• Liu, J. G. 2000. “Smoothing Filter-Based Intensity Modulation: A Spectral Preserve Image Fusion Technique for Improving Spatial Details.” International Journal of Remote Sensing 21 (18): 3461–3472. https://doi.org/10.1080/014311600750037499.
   Web of Science ®Google Scholar
• Lorenzo, B., and L. Carlin. 2006. “A Multilevel Context-Based System for Classification of Very High Spatial Resolution Images.” IEEE Transactions on Geoscience & Remote Sensing 44 (9): 2587–2600. https://doi.org/10.1109/TGRS.2006.875360.
   Google Scholar
• M, T. T., S. S. C, H. C. Shyu, and P. S. Huang. 2001. “A New Look at IHS-Like Image Fusion Methods.” Information Fusion 2 (3): 177–186. https://doi.org/10.1016/S1566-2535(01)00036-7.
   Google Scholar
• Ma, J., L. Tang, F. Fan, J. Huang, X. Mei, and Y. Ma. 2022. “SwinFusion: Cross-Domain Long-Range Learning for General Image Fusion via Swin Transformer.” IEEE/CAA Journal of Automatica Sinica 9 (7): 1200–1217. https://doi.org/10.1109/JAS.2022.105686.
   Google Scholar
• Myint, S. W., P. Gober, A. Brazel, S. Grossman-Clarke, and Q. Weng. 2011. “Per-Pixel Vs. Object-Based Classification of Urban Land Cover Extraction Using High Spatial Resolution Imagery.” Remote Sensing of Environment 115 (5): 1145–1161. https://doi.org/10.1016/j.rse.2010.12.017.
   Web of Science ®Google Scholar
• Neidell, N. S., and M. T. Taner. 1971. “Semblance and Other Coherency Measures for Multichannel Data.” Geophysics 36 (3): 482–497. https://doi.org/10.1190/1.1440186.
   Google Scholar
• Otazu, X., M. González-Audícana, O. Fors, and J. Nunez. 2005. “Introduction of Sensor Spectral Response into Image Fusion Methods. Application to Wavelet-Based Methods.” IEEE Transactions on Geoscience & Remote Sensing 43 (10): 2376–2385. https://doi.org/10.1109/TGRS.2005.856106.
   Web of Science ®Google Scholar
• Q, D. S., D. L. J, X. Wu, R. Ran, D. Hong, and G. Vivone. 2023. “PSRT: Pyramid Shuffle-And-Reshuffle Transformer for Multispectral and Hyperspectral Image Fusion.” IEEE Transactions on Geoscience & Remote Sensing 61:1–15. https://doi.org/10.1109/TGRS.2023.3244750.
   Google Scholar
• Que, Y., H. Xiong, X. Xia, You J, Yang Y. 2024. “Integrating Spectral and Spatial Bilateral Pyramid Networks for Pansharpening.” IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing 17:3985–3998. https://doi.org/10.1109/JSTARS.2024.3356513.
   Google Scholar
• R, V. M., R. Restaino, G. Vivone, M. Dalla Mura, and J. Chanussot. 2014. “A Pansharpening Method Based on the Sparse Representation of Injected Details.” IEEE Geoscience & Remote Sensing Letters 12 (1): 180–184. https://doi.org/10.1109/LGRS.2014.2331291.
   Google Scholar
• Ranchin, T., and L. Wald. 2000. “Fusion of High Spatial and Spectral Resolution Images: The ARSIS Concept and Its Implementation.” Photogrammetric Engineering & Remote Sensing 66 (1): 49–61.
   Web of Science ®Google Scholar
• Restaino, R., M. Dalla Mura, G. Vivone, and J. Chanussot. 2016. “Context-Adaptive Pansharpening Based on Image Segmentation.” IEEE Transactions on Geoscience & Remote Sensing 55 (2): 753–766. https://doi.org/10.1109/TGRS.2016.2614367.
   Google Scholar
• Rodriguez-Galiano, V., E. Pardo-Iguzquiza, M. Sanchez-Castillo, M. Chica-Olmo, and M. Chica-Rivas. 2012. “Downscaling Landsat 7 ETM+ Thermal Imagery Using Land Surface Temperature and NDVI Images.” International Journal of Applied Earth Observation and Geoinformation 18:515–527. https://doi.org/10.1016/j.jag.2011.10.002.
   Web of Science ®Google Scholar
• Sahu, D.K., and M. P. Parsai. 2012. “Different Image Fusion Techniques–A Critical Review.” International Journal of Modern Engineering Research 2 (5): 4298–4301.
   Google Scholar
• Scarpa, G., S. Vitale, and D. Cozzolino. 2018. “Target-Adaptive CNN-Based Pansharpening.” IEEE Transactions on Geoscience & Remote Sensing 56 (9): 5443–5457. https://doi.org/10.1109/TGRS.2018.2817393.
   Web of Science ®Google Scholar
• Shahdoosti, H. R., and A. Mehrabi. 2018. “Multimodal image fusion using sparse representation classification in tetrolet domain.” Digital Signal Processing 79:9–22. https://doi.org/10.1016/j.dsp.2018.04.002.
   Google Scholar
• Tai, Y., J. Yang, and X. Liu. 2017. “Image Super-Resolution via Deep Recursive Residual Network.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 3147–3155.
   Google Scholar
• Tang, W., F. He, Y. Liu, and Y. Duan. 2022. “MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer.” IEEE Transactions on Image Processing 31:5134–5149. https://doi.org/10.1109/TIP.2022.3193288.
   PubMedGoogle Scholar
• Tang, W., F. He, Y. Liu, Y. Duan, and T. Si. 2023. “DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer.” IEEE Transactions on Circuits and Systems for Video Technology 33 (7): 3159–3172. https://doi.org/10.1109/TCSVT.2023.3234340.
   Google Scholar
• Vivone, G., L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald. 2014a. “A Critical Comparison Among Pansharpening Algorithms.” IEEE Transactions on Geoscience & Remote Sensing 53 (5): 2565–2586. https://doi.org/10.1109/TGRS.2014.2361734.
   Web of Science ®Google Scholar
• Vivone, G., M. Dalla Mura, A. Garzelli, and F. Pacifici. 2021. “A Benchmarking Protocol for Pansharpening: Dataset, Preprocessing, and Quality Assessment.” IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing 14:6102–6118. https://doi.org/10.1109/JSTARS.2021.3086877.
   Web of Science ®Google Scholar
• Vivone, G., R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot. 2013. “Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening.” IEEE Geoscience & Remote Sensing Letters 11 (5): 930–934. https://doi.org/10.1109/LGRS.2013.2281996.
   Web of Science ®Google Scholar
• Vivone, G., M. Simões, M. Dalla Mura, R. Restaino, J. M. Bioucas-Dias, G. A. Licciardi, and J. Chanussot. 2014b. “Pansharpening Based on Semiblind Deconvolution.” IEEE Transactions on Geoscience & Remote Sensing 53 (4): 1997–2010. https://doi.org/10.1109/TGRS.2014.2351754.
   Google Scholar
• Wald, L., T. Ranchin, and M. Mangolini. 1997. “Fusion of Satellite Images of Different Spatial Resolutions: Assessing the Quality of Resulting Images.” Photogrammetric Engineering & Remote Sensing 63 (6): 691–699.
   Web of Science ®Google Scholar
• Wang, Q., W. Shi, A. P. M, and E. Pardo-Iguzquiza. 2015. “A New Geostatistical Solution to Remote Sensing Image Downscaling.” IEEE Transactions on Geoscience & Remote Sensing 54 (1): 386–396. https://doi.org/10.1109/TGRS.2015.2457672.
   Google Scholar
• Wang, T., F. Fang, F. Li, and G. Zhang. 2018. “High-Quality Bayesian Pansharpening.” IEEE Transactions on Image Processing 28 (1): 227–239. https://doi.org/10.1109/TIP.2018.2866954.
   PubMedGoogle Scholar
• Wang, Z., B. A. C, S. H. R, and E. P. Simoncelli. 2004. “Image Quality Assessment: From Error Visibility to Structural Similarity.” IEEE Transactions on Image Processing 13 (4): 600–612. https://doi.org/10.1109/TIP.2003.819861.
   PubMed Web of Science ®Google Scholar
• Wei, Y., Q. Yuan, H. Shen, and L. Zhang. 2017. “Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network.” IEEE Geoscience & Remote Sensing Letters 14 (10): 1795–1799. https://doi.org/10.1109/LGRS.2017.2736020.
   Web of Science ®Google Scholar
• Wu, Z. C, Huang T. Z, Deng L. J, J. Huang, J. Chanussot, and G. Vivone. 2023. “LRTCFPan: Low-Rank Tensor Completion Based Framework for Pansharpening.” IEEE Transactions on Image Processing 32:1640–1655. https://doi.org/10.1109/TIP.2023.3247165.
   Web of Science ®Google Scholar
• Xu, Q., Y. Zhang, B. Li, and L. Ding. 2014. “Pansharpening Using Regression of Classified MS and Pan Images to Reduce Color Distortion.” IEEE Geoscience & Remote Sensing Letters 12 (1): 28–32. https://doi.org/10.1109/LGRS.2014.2324817.
   Google Scholar
• Yang, J., X. Fu, Y. Hu, Huang Y, Ding X, Paisley J. 2017. “PanNet: A Deep Network Architecture for Pan-Sharpening.” In Proceedings of the IEEE international conference on computer vision, Venice, Italy, 5449–5457.
   Google Scholar
• Yang, Y., W. Tu, S. Huang, H. Lu, W. Wan, and L. Gan. 2021. “Dual-Stream Convolutional Neural Network with Residual Information Enhancement for Pansharpening.” IEEE Transactions on Geoscience & Remote Sensing 60:1–16. https://doi.org/10.1109/TGRS.2021.3098752.
   Web of Science ®Google Scholar
• Yuan, Q., Y. Wei, X. Meng, H. Shen, and L. Zhang. 2018. “A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening.” IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing 11 (3): 978–989. https://doi.org/10.1109/JSTARS.2018.2794888.
   Web of Science ®Google Scholar
• Yuhas, R. H., A. F. H. Goetz, and J. W. Boardman. 1992. “Discrimination Among Semi-Arid Landscape Endmembers Using the Spectral Angle Mapper (SAM) Algorithm.” JPL Airborne Geoscience Workshop Jet Propulsion Laboratory (US) 1:147–149.
   Google Scholar
• Zamir, Z. S., A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, L. Shao, et al. 2022. “Learning Enriched Features for Fast Image Restoration and Enhancement.” IEEE Transactions on Pattern Analysis & Machine Intelligence 45 (2): 1934–1948. https://doi.org/10.1109/TPAMI.2022.3167175.
   Google Scholar
• Zhang, Y., C. Liu, M. Sun, and Y. Ou. 2019. “Pan-Sharpening Using an Efficient Bidirectional Pyramid Network.” IEEE Transactions on Geoscience & Remote Sensing 57 (8): 5549–5563. https://doi.org/10.1109/TGRS.2019.2900419.
   Google Scholar
• Zhu, X. X., and R. Bamler. 2012. “A Sparse Image Fusion Algorithm with Application to Pan-Sharpening.” IEEE Transactions on Geoscience & Remote Sensing 51 (5): 2827–2836. https://doi.org/10.1109/TGRS.2012.2213604.
   Google Scholar