SiamixFormer: a fully-transformer Siamese network with temporal Fusion for accurate building detection and change detection in bi-temporal remote sensing images

References

• Awrangjeb, M., C. Zhang, and C. S. Fraser. 2011. “Improved Building Detection Using Texture Information.” The International Archives of Photogrammetry, Remote Sensing & Spatial Information Sciences 38:143–148. https://doi.org/10.5194/isprsarchives-XXXVIII-3-W22-143-2011.
   Google Scholar
• Bandara, W. G. C., and V. M. Patel. 2022. “A Transformer-Based Siamese Network for Change Detection.” arXiv preprint arXiv:2201.01293.
   Google Scholar
• Bem, D., P. Pozzobon, C. J. Osmar Ablio de, R. Fontes Guimarães, and R. Arnaldo Trancoso Gomes. 2020. “Change Detection of Deforestation in the Brazilian Amazon Using Landsat Data and Convolutional Neural Networks.” Remote Sensing 12 (6): 901. https://doi.org/10.3390/rs12060901.
   Web of Science ®Google Scholar
• Chen, H., E. Nemni, S. Vallecorsa, X. Li, W. Chen, and L. Bromley. 2022. “Dual-Tasks Siamese Transformer Framework for Building Damage Assessment.“ IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 1600–1603. https://doi.org/10.1109/IGARSS46834.2022.9883139.
   Google Scholar
• Chen, H., and Z. Shi. 2020. “A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection.” Remote Sensing 12 (10): 1662. https://doi.org/10.3390/rs12101662.
   Web of Science ®Google Scholar
• Chen, H., Q. Zipeng, and Z. Shi. 2021. “Remote Sensing Image Change Detection with Transformers.” IEEE Transactions on Geoscience and Remote Sensing 60, 1–14
   Web of Science ®Google Scholar
• Chen, K., Z. Zou, and Z. Shi. 2021. “Building Extraction from Remote Sensing Images with Sparse Token Transformers.” Remote Sensing 13 (21): 4441. https://doi.org/10.3390/rs13214441.
   Web of Science ®Google Scholar
• Contributors, MMSegmentation. 2020. “MMSegmentation: OpenMmlab Semantic Segmentation Toolbox and Benchmark.” https://github.com/open-mmlab/mmsegmentation.
   Google Scholar
• Dosovitskiy, A., L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, et al. 2020. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.“ CoRR abs/2010.11929. https://doi.org/10.48550/arXiv.2010.11929. arXiv preprint arXiv:2010.11929.
   Google Scholar
• Fang, S., L. Kaiyu, J. Shao, and L. Zhe. 2021. “SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images.” IEEE Geoscience and Remote Sensing Letters 19:1–5. https://doi.org/10.1109/LGRS.2021.3056416.
   Web of Science ®Google Scholar
• Ferraioli, G. 2009. “Multichannel InSar Building Edge Detection.” IEEE Transactions on Geoscience and Remote Sensing 48 (3): 1224–1231. https://doi.org/10.1109/TGRS.2009.2029338.
   Web of Science ®Google Scholar
• Gong, M., S. Linzhi, M. Jia, and W. Chen. 2013. “Fuzzy Clustering with a Modified MRF Energy Function for Change Detection in Synthetic Aperture Radar Images.” IEEE Transactions on Fuzzy Systems 22 (1): 98–109. https://doi.org/10.1109/TFUZZ.2013.2249072.
   Web of Science ®Google Scholar
• Gupta, R., B. Goodman, N. Patel, R. Hosfelt, S. Sajeev, E. Heim, J. Doshi, K. Lucas, H. Choset, and M. Gaston. 2019. “Creating xBD: A Dataset for Assessing Building Damage from Satellite Imagery.” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, Long Beach, California, 10–17.
   Google Scholar
• Hendrycks, D., and K. Gimpel. 2016. “Gaussian Error Linear Units (Gelus).“ CoRR abs/1606.08415. https://doi.org/10.48550/arXiv.1606.08415. arXiv preprint arXiv:1606.08415.
   Google Scholar
• Lebedev, M. A., Y. V. Vizilter, O. V. Vygolov, V. A. Knyaz, and A. Yu Rubis. 2018. “Change Detection in Remote Sensing Images Using Conditional Adversarial Networks.” International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences 42 (2): 565–571. https://doi.org/10.5194/isprs-archives-XLII-2-565-2018.
   Google Scholar
• Lin, T.-Y., P. Goyal, R. Girshick, H. Kaiming, and P. Dollár. 2017. “Focal Loss for Dense Object Detection.” In Proceedings of the IEEE international conference on computer vision, Venice, Italy, 2980–2988.
   Google Scholar
• Liu, Y., D. Chen, M. Ailong, Y. Zhong, F. Fang, and X. Kai. 2020. “Multiscale U-Shaped CNN Building Instance Extraction Framework with Edge Constraint for High-Spatial-Resolution Remote Sensing Imagery.” IEEE Transactions on Geoscience and Remote Sensing 59 (7): 6106–6120. https://doi.org/10.1109/TGRS.2020.3022410.
   Web of Science ®Google Scholar
• Liu, R., M. Kuffer, and C. Persello. 2019. “The Temporal Dynamics of Slums Employing a CNN-Based Change Detection Approach.” Remote Sensing 11 (23): 2844. https://doi.org/10.3390/rs11232844.
   Web of Science ®Google Scholar
• Liu, Y., C. Pang, Z. Zhan, X. Zhang, and X. Yang. 2020. “Building Change Detection for Remote Sensing Images Using a Dual-Task Constrained Deep Siamese Convolutional Network Model.” IEEE Geoscience and Remote Sensing Letters 18 (5): 811–815. https://doi.org/10.1109/LGRS.2020.2988032.
   Web of Science ®Google Scholar
• Liu, L., S. Yuan, J. Liu, X. Guo, Y. Yan, and Q. Tian. 2022. “SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-Trained Siamese Transformers.” In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, 1747–1755.
   Google Scholar
• Minghong, H., L. Jiatian, Y. Zhao, L. Mei, L. Wen, and W. Li. 2023. “FSAU-Net: A Network for Extracting Buildings from Remote Sensing Imagery Using Feature Self-Attention.” International Journal of Remote Sensing 44 (5): 1643–1664. https://doi.org/10.1080/01431161.2023.2177125.
   Web of Science ®Google Scholar
• Peng, X., R. Zhong, L. Zhen, and L. Qingyang. 2020. “Optical Remote Sensing Image Change Detection Based on Attention Mechanism and Image Difference.” IEEE Transactions on Geoscience and Remote Sensing 59 (9): 7296–7307. https://doi.org/10.1109/TGRS.2020.3033009.
   Web of Science ®Google Scholar
• Pihur, V., S. Datta, and S. Datta. 2007. “Weighted Rank Aggregation of Cluster Validation Measures: A Monte Carlo Cross-Entropy Approach.” Bioinformatics 23 (13): 1607–1615. https://doi.org/10.1093/bioinformatics/btm158.
   PubMed Web of Science ®Google Scholar
• Rolnick, D., P. L. Donti, L. H. Kaack, K. Kochanski, A. Lacoste, K. Sankaran, A. Slavin Ross, et al. 2022. “Tackling Climate Change with Machine Learning.” ACM Computing Surveys 55 (2): 1–96. https://doi.org/10.1145/3485128.
   Web of Science ®Google Scholar
• Shen, Y., S. Zhu, T. Yang, C. Chen, D. Pan, J. Chen, L. Xiao, and D. Qian. 2021. “BDANet: Multiscale Convolutional Neural Network with Cross-Directional Attention for Building Damage Assessment from Satellite Images.” IEEE Transactions on Geoscience and Remote Sensing 60:1–14. https://doi.org/10.1109/TGRS.2021.3080580.
   Web of Science ®Google Scholar
• Shi, Q., M. Liu, L. Shengchen, X. Liu, F. Wang, and L. Zhang. 2021. “A Deeply Supervised Attention Metric-Based Network and an Open Aerial Image Dataset for Remote Sensing Change Detection.” IEEE Transactions on Geoscience and Remote Sensing 60:1–16. https://doi.org/10.1109/TGRS.2021.3085870.
   Web of Science ®Google Scholar
• Shunping, J., Y. Shen, L. Meng, and Y. Zhang. 2019. “Building Instance Change Detection from Large-Scale Aerial Images Using Convolutional Neural Networks and Simulated Samples.” Remote Sensing 11 (11): 1343. https://doi.org/10.3390/rs11111343.
   Web of Science ®Google Scholar
• Shunping, J., S. Wei, and L. Meng. 2018. “Fully Convolutional Networks for Multisource Building Extraction from an Open Aerial and Satellite Imagery Data Set.” IEEE Transactions on Geoscience and Remote Sensing 57 (1): 574–586. https://doi.org/10.1109/TGRS.2018.2858817.
   Web of Science ®Google Scholar
• Sirmacek, B., and C. Unsalan. 2008. “Building Detection from Aerial Images Using Invariant Color Features and Shadow Information.” In 2008 23rd International Symposium on Computer and Information Sciences, Istanbul, Turkey, 1–5. IEEE.
   Google Scholar
• Sudre, C. H., L. Wenqi, T. Vercauteren, S. Ourselin, and M. Jorge Cardoso. 2017. “Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations.” In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 240–248. Springer. https://doi.org/10.1007/978-3-319-67558-9_28.
   Google Scholar
• Tian, Q., Y. Zhao, L. Yao, J. Chen, X. Chen, and K. Qin. 2021. “Multiscale Building Extraction with Refined Attention Pyramid Networks.” IEEE Geoscience & Remote Sensing Letters 19:1–5. https://doi.org/10.1109/LGRS.2021.3075436.
   Web of Science ®Google Scholar
• Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. 2017. “Attention is All You Need.“ Neural Information Processing Systems (NIPS2017), Long Beach, CA, USA, 1–11.
   Google Scholar
• Wang, G., L. Bin, T. Zhang, and S. Zhang. 2022. “A Network Combining a Transformer and a Convolutional Neural Network for Remote Sensing Image Change Detection.” Remote Sensing 14 (9): 2228. https://doi.org/10.3390/rs14092228.
   Web of Science ®Google Scholar
• Wang, W., E. Xie, L. Xiang, D.-P. Fan, K. Song, D. Liang, L. Tong, P. Luo, and L. Shao. 2021. “Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions.” In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 568–578.
   Google Scholar
• Wang, D., J. Zhang, B. Du, G.-S. Xia, and D. Tao. 2022. “An Empirical Study of Remote Sensing Pretraining.” IEEE Transactions on Geoscience & Remote Sensing 61:1–20. https://doi.org/10.1109/TGRS.2022.3176603.
   Web of Science ®Google Scholar
• Woo, S., J. Park, J.-Y. Lee, and I. So Kweon. 2018. “Cbam: Convolutional Block Attention Module.” In Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 3–19.
   Google Scholar
• Xiao, X., W. Guo, R. Chen, Y. Hui, J. Wang, and H. Zhao. 2022. “A Swin Transformer-Based Encoding Booster Integrated in U-Shaped Network for Building Extraction.” Remote Sensing 14 (11): 2611. https://doi.org/10.3390/rs14112611.
   Web of Science ®Google Scholar
• Xie, E., W. Wang, Y. Zhiding, A. Anandkumar, J. M. Alvarez, and P. Luo. 2021. “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers.” Advances in Neural Information Processing Systems abs/2105.15203: 34. https://doi.org/10.48550/arXiv.2105.15203.
   Google Scholar
• Yan, T., Z. Wan, and P. Zhang. 2022. “Fully Transformer Network for Change Detection of Remote Sensing Images.” In Proceedings of the Asian Conference on Computer Vision, Macau SAR, China, 1691–1708.
   Google Scholar
• Yuanxin, Y., L. Zhou, B. Zhu, C. Yang, M. Sun, J. Fan, and F. Zhitao. 2022. “Feature Decomposition-Optimization-Reorganization Network for Building Change Detection in Remote Sensing Images.” Remote Sensing 14 (3): 722. https://doi.org/10.3390/rs14030722.
   Web of Science ®Google Scholar
• Zhang, H., Y. Liao, H. Yang, G. Yang, and L. Zhang. 2022. “A Local-Global Dual-Stream Network for Building Extraction from Very-High-Resolution Remote Sensing Images.” IEEE Transactions on Neural Networks and Learning Systems 33: 1269–1283 .
   PubMed Web of Science ®Google Scholar
• Zhao, W., X. Chen, X. Ge, and J. Chen. 2020. “Using Adversarial Network for Multiple Change Detection in Bitemporal Remote Sensing Imagery.” IEEE Geoscience & Remote Sensing Letters 19. https://doi.org/10.1109/LGRS.2020.3040435.
   PubMed Web of Science ®Google Scholar