Building panoptic change segmentation with the use of uncertainty estimation in squeeze-and-attention CNN and remote sensing observations

References

• Amin, A. M., E. Q. Liu, and Y. Wang. 2017. “Zoom Out CNNs Features for Optical Remote Sensing Change Detection.” In 2017 2nd International Conference on Image, Vision and Computing (ICIVC), Chengdu, China. 812–817. doi:https://doi.org/10.1109/ICIVC.2017.7984667.
   Google Scholar
• Arnab, A., O. Miksik, and P. H. S. Torr. 2018. “On the Robustness of Semantic Segmentation Models to Adversarial Attacks.” In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. 888–897. doi:https://doi.org/10.1109/CVPR.2018.00099.
   Google Scholar
• Bodla, N., B. Singh, R. Chellappa, and L. S. Davis. 2017. “Soft-NMS – Improving object detection with one line of code.” ArXiv:1704.04503 [Cs], August. http://arxiv.org/abs/1704.04503
   Google Scholar
• Bourdis, N., D. Marraud, and H. Sahbi. 2011. “Constrained Optical Flow for Aerial Image Change Detection.” In 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada. 4176–4179. doi:https://doi.org/10.1109/IGARSS.2011.6050150.
   Google Scholar
• Champion, N., D. Boldo, M. Pierrot-Deseilligny, and G. Stamon. 2010. “2D Building Change Detection from High Resolution Satelliteimagery: A Two-Step Hierarchical Method Based on 3D Invariant Primitives.” Pattern Recognition Letters, Pattern Recognition in Remote Sensing 31 (10): 1138–1147. doi:https://doi.org/10.1016/j.patrec.2009.10.012.
   Web of Science ®Google Scholar
• Chen, H., and Z. Shi. 2020. “A Spatial-Temporal Attention-Based Method and A New Dataset for Remote Sensing Image Change Detection”. Remote Sensing 12(10). Multidisciplinary Digital Publishing Institute: 1662. doi:https://doi.org/10.3390/rs12101662.
   Web of Science ®Google Scholar
• Chen, L.-C., and L.-J. Lin. 2010. “Detection of Building Changes from Aerial Images and Light Detection and Ranging (LIDAR) Data”. Journal of Applied Remote Sensing 4(1). International Society for Optics and Photonics: 041870. doi:https://doi.org/10.1117/1.3525560.
   Google Scholar
• Cheng, B., M. D. Collins, Y. Zhu, T. S. Ting Liu, H. A. Huang, and L.-C. Chen. 2020. “Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation.” In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12472–12482. Seattle, WA, USA: IEEE. doi:https://doi.org/10.1109/CVPR42600.2020.01249.
   Google Scholar
• Daudt, R. C., S. Bertrand Le, and A. Boulch. 2018. “Fully Convolutional Siamese Networks for Change Detection.” ArXiv:1810.08462 [Cs], October. http://arxiv.org/abs/1810.08462
   Google Scholar
• Du, S., Y. Zhang, R. Qin, Z. Yang, Z. Zou, Y. Tang, and C. Fan. 2016. “Building Change Detection Using Old Aerial Images and New LiDAR Data.” Remote Sensing 8 (12): 1030. Multidisciplinary Digital Publishing Institute: 1030. doihttps://doi.org/10.3390/rs8121030.
   Google Scholar
• Gstaiger, V., J. Tian, R. Kiefl, and F. Kurz. 2018. “2D Vs. 3D Change Detection Using Aerial Imagery to Support Crisis Management of Large-Scale Events.” Remote Sensing 10 (12): 2054. Multidisciplinary Digital Publishing Institute: 2054. doihttps://doi.org/10.3390/rs10122054.
   Google Scholar
• He, K., G. Gkioxari, P. Dollár, and R. Girshick. 2017. “Mask R-CNN.” In 2017 IEEE International Conference on Computer Vision (ICCV), 2980–2988. doi:https://doi.org/10.1109/ICCV.2017.322.
   Google Scholar
• He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep Residual Learning for Image Recognition.” In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. Las Vegas, NV, USA: IEEE. doi:https://doi.org/10.1109/CVPR.2016.90.
   Google Scholar
• Hu, S., D. Worrall, S. Knegt, B. Veeling, H. Huisman, and M. Welling. 2019. “Supervised Uncertainty Quantification for Segmentation with Multiple Annotations.” ArXiv:1907.01949 [Cs, Stat], July. http://arxiv.org/abs/1907.01949
   Google Scholar
• Huang, P.-Y., W.-T. Hsu, C.-Y. Chiu, W. Ting-Fan, and M. Sun. 2018. “Efficient Uncertainty Estimation for Semantic Segmentation in Videos.” In Computer Vision – ECCV 2018, edited by V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, 536–552. Lecture Notes in Computer Science. Cham: Springer International Publishing. doi:https://doi.org/10.1007/978-3-030-01246-5_32.
   Google Scholar
• Javadi, S., M. Dahl, and M. I. Pettersson. 2020. “Change Detection in Aerial Images Using Three-Dimensional Feature Maps.” Remote Sensing 12 (9): 1404. Multidisciplinary Digital Publishing Institute: 1404. doihttps://doi.org/10.3390/rs12091404.
   Google Scholar
• Ji, S., Y. Shen, L. Meng, and Y. Zhang. 2019b. “Building Instance Change Detection from Large-Scale Aerial Images Using Convolutional Neural Networks and Simulated Samples.” Remote Sensing 11 (11): 1343. Multidisciplinary Digital Publishing Institute: 1343. doi10.3390/rs11111343
   Web of Science ®Google Scholar
• Ji, S., S. Wei, and M. Lu. 2019a. “Fully Convolutional Networks for Multisource Building Extraction from an Open Aerial and Satellite Imagery Data Set.” IEEE Transactions on Geoscience and Remote Sensing 57 (1): 574–586. doi:https://doi.org/10.1109/TGRS.2018.2858817.
   Web of Science ®Google Scholar
• Jiang, H., H. Xiangyun, L. Kun, J. Zhang, J. Gong, and M. Zhang. 2020. “PGA-SiamNet: Pyramid Feature-Based Attention-Guided Siamese Network for Remote Sensing Orthoimagery Building Change Detection”. Remote Sensing 12(3). Multidisciplinary Digital Publishing Institute: 484. doi:https://doi.org/10.3390/rs12030484.
   Web of Science ®Google Scholar
• Jianya, G., M. Guorui, and Q. Zhou,(2008).“A Review of Multi-Temporal Remote Sensing Data Change Detection Algorithms.”Remote Sensing and Spatial Information Sciences, Beijing, China. 37(B7)
   Google Scholar
• Kendall, A., V. Badrinarayanan, and R. Cipolla. 2016. “Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding.” ArXiv:1511.02680 [Cs], October. http://arxiv.org/abs/1511.02680
   Google Scholar
• Khoshboresh Masouleh, M., and M. R. Saradjian. 2019. “Robust Building Footprint Extraction from Big Multi-sensor Data Using Deep Competition Network.” ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-4/W18: 615–621. XLII-4-W18:615–621.Copernicus GmbH. doi:https://doi.org/10.5194/isprs-archives-XLII-4-W18-615-2019.
   Google Scholar
• Khoshboresh-Masouleh, M. 2019. Deploying Superpixel Segmentation and Deep Learning to Improve the Accuracy of the Building Extraction from Remote Sensing Data. Vol. 1. Tehran: University of Tehran.
   Google Scholar
• Khoshboresh-Masouleh, M., F. Alidoost, and H. Arefi. 2020. “Multiscale Building Segmentation Based on Deep Learning for Remote Sensing RGB Images from Different Sensors.” Journal of Applied Remote Sensing 14 (3): 1. International Society for Optics and Photonics: 034503. doihttps://doi.org/10.1117/1.JRS.14.034503.
   Google Scholar
• Kingma, D. P., and B. Jimmy 2017. “Adam: A Method for Stochastic Optimization.” ArXiv:1412.6980 [Cs], January. http://arxiv.org/abs/1412.6980
   Google Scholar
• Kirillov, A., K. He, R. Girshick, C. Rother, and P. Dollár. 2019. “Panoptic Segmentation.” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. 9396–9405. doi:https://doi.org/10.1109/CVPR.2019.00963.
   Google Scholar
• Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2017. “ImageNet Classification with Deep Convolutional Neural Networks.” CACM 60 (6): 84–90. doi:https://doi.org/10.1145/3065386.
   Google Scholar
• Kwon, Y., J.-H. Won, B. J. Kim, and M. C. Paik. 2020. “Uncertainty Quantification Using Bayesian Neural Networks in Classification: Application to Biomedical Image Segmentation.” Computational Statistics & Data Analysis 142 (February): 106816. doi:https://doi.org/10.1016/j.csda.2019.106816.
   Google Scholar
• Li, Y., J. Wan, Q. Miao, S. Escalera, H. Fang, H. Chen, X. Qi, and G. Guo. 2020. “CR-Net: A Deep Classification-Regression Network for Multimodal Apparent Personality Analysis.” International Journal of Computer Vision 128 (12): 2763–2780. doi:https://doi.org/10.1007/s11263-020-01309-y.
   Web of Science ®Google Scholar
• Li, Y., X. Chen, Z. Zhu, L. Xie, G. Huang, D. Du, and X. Wang. 2019. “Attention-Guided Unified Network for Panoptic Segmentation.” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7019–7028. Long Beach, CA, USA: IEEE. doi:https://doi.org/10.1109/CVPR.2019.00719.
   Google Scholar
• Lin, T., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017. “Focal Loss for Dense Object Detection.” In 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy. 2999–3007. doi:https://doi.org/10.1109/ICCV.2017.324.
   Google Scholar
• Liu, H., C. Peng, Y. Changqian, J. Wang, X. Liu, Y. Gang, and W. Jiang. 2019. “An End-To-End Network for Panoptic Segmentation.” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6165–6174. Long Beach, CA, USA: IEEE. doi:https://doi.org/10.1109/CVPR.2019.00633.
   Google Scholar
• Lu, D., P. Mausel, E. Brondízio, and E. Moran. 2004. “Change Detection Techniques.” International Journal of Remote Sensing 25 (12): 2365–2401. Taylor & Francis. doihttps://doi.org/10.1080/0143116031000139863.
   Web of Science ®Google Scholar
• Maggiori, E., Y. Tarabalka, G. Charpiat, and P. Alliez. 2017. “Can Semantic Labeling Methods Generalize to Any City? the Inria Aerial Image Labeling Benchmark.” In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, Texas, USA. 3226–3229. doi:https://doi.org/10.1109/IGARSS.2017.8127684.
   Google Scholar
• Makuti, S. F., C. Nex, and M. Y. Yang. 2018. “Multi-Temporal Classification And Change Detection Using UAV Images.” 2018 ISPRS TC II Mid-Term Symposium Towards Photogrammetry 2020, 4–7 June 2018, Riva Del Garda, Italy, June. International Society for Photogrammetry and Remote Sensing (ISPRS), 651–658. doi:https://doi.org/10.5194/isprs-archives-XLII-2-651-2018.
   Google Scholar
• Masouleh, M. K., and R. Shah-Hosseini. 2018. “Fusion of Deep Learning with Adaptive Bilateral Filter for Building Outline Extraction from Remote Sensing Imagery.” Journal of Applied Remote Sensing 12 (4): 1. International Society for Optics and Photonics: 046018. doihttps://doi.org/10.1117/1.JRS.12.046018.
   Google Scholar
• Masouleh, M. K., and S. Sadeghian. 2019. “Deep Learning-Based Method for Reconstructing Three-Dimensional Building Cadastre Models from Aerial Images”. Journal of Applied Remote Sensing 13(2). International Society for Optics and Photonics: 024508:1-18. doi:https://doi.org/10.1117/1.JRS.13.024508.
   Google Scholar
• Milletari, F., N. Navab, and S.-A. Ahmadi. 2016. “V-Net: Fully convolutional neural networks for volumetric medical image segmentation.” ArXiv:1606.04797 [Cs], June. http://arxiv.org/abs/1606.04797
   Google Scholar
• Mouat, D. A., G. G. Mahin, and J. Lancaster. 1993. “Remote Sensing Techniques in the Analysis of Change Detection.” Geocarto International 8 (2): 39–50. Taylor & Francis. doihttps://doi.org/10.1080/10106049309354407.
   Google Scholar
• Nemoto, K., R. Hamaguchi, M. Sato, A. Fujita, T. Imaizumi, and S. Hikosaka. 2017. “Building Change Detection via a Combination of CNNs Using Only RGB Aerial Imageries.” Remote Sensing Technologies and Applications in Urban Environments II. 10431: 104310J. International Society for Optics and Photonics. doi:https://doi.org/10.1117/12.2277912.
   Google Scholar
• Paszke, A., S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen 2019. “PyTorch: An Imperative Style, High-Performance Deep Learning Library.” In Advances in Neural Information Processing Systems edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d textquotesingle Alché-Buc, E. Fox, and R. Garnett, 32: 8024–8035. Curran Associates. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
   Google Scholar
• Paul, A. V., M. Chowdary, Y. K. Srivastava, D. Dutta, and J. R. Sharma. 2017. “Change Detection of Linear Features in Temporally Spaced Remotely Sensed Images Using Edge-Based Grid Analysis.” Geocarto International 32 (6): 640–654. Taylor & Francis. doihttps://doi.org/10.1080/10106049.2016.1167966.
   Web of Science ®Google Scholar
• Qizhu, L., A. Arnab, and P. H. S. Torr. 2018. “Weakly- and Semi-Supervised Panoptic Segmentation.” In Computer Vision – ECCV 2018, edited by V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, 106–124. Vol. 11219 Lecture Notes in Computer Science. Cham: Springer International Publishing. doi:https://doi.org/10.1007/978-3-030-01267-0_7.
   Google Scholar
• Růžička, V., S. D’Aronco, J. D. Wegner, and K. Schindler. 2020. “Deep active learning in remote sensing for data efficient change detection.” ArXiv:2008.11201 [Cs], August. http://arxiv.org/abs/2008.11201
   Google Scholar
• Shi, W., M. Zhang, R. Zhang, S. Chen, and Z. Zhan. 2020. “Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges.” Remote Sensing 12 (10): 1688. Multidisciplinary Digital Publishing Institute: 1688. doihttps://doi.org/10.3390/rs12101688.
   Web of Science ®Google Scholar
• Singh, A. 1989. “Review Article Digital Change Detection Techniques Using Remotely-Sensed Data.” International Journal of Remote Sensing 10 (6): 989–1003. doi:https://doi.org/10.1080/01431168908903939.
   Web of Science ®Google Scholar
• Sun, K., B. Xiao, D. Liu, and J. Wang. 2019. “Deep High-Resolution Representation Learning for Human Pose Estimation.” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5686–5696. Long Beach, CA, USA: IEEE. doi:https://doi.org/10.1109/CVPR.2019.00584.
   Google Scholar
• Tan, M., and Q. V. Le. 2020. “EfficientNet: Rethinking model scaling for convolutional neural networks.” ArXiv:1905.11946 [Cs, Stat], September. http://arxiv.org/abs/1905.11946
   Google Scholar
• Tian, J., S. Cui, and P. Reinartz. 2014. “Building Change Detection Based on Satellite Stereo Imagery and Digital Surface Models.” IEEE Transactions on Geoscience and Remote Sensing 52 (1): 406–417. doi:https://doi.org/10.1109/TGRS.2013.2240692.
   Web of Science ®Google Scholar
• Wang, X., R. Zhang, T. Kong, L. Lei, and C. Shen. 2020b. “SOLOv2: Dynamic and Fast Instance Segmentation.” ArXiv:2003.10152 [Cs], October. http://arxiv.org/abs/2003.10152
   Google Scholar
• Wang, X., T. Kong, C. Shen, Y. Jiang, and L. Lei 2020a. “SOLO: Segmenting Objects by Locations.” ArXiv:1912.04488 [Cs], July. http://arxiv.org/abs/1912.04488
   Google Scholar
• Xie, E., P. Sun, X. Song, W. Wang, X. Liu, D. Liang, C. Shen, and P. Luo. 2020. “PolarMask: Single shot instance segmentation with polar representation.” In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12193-12202, Seattle, WA, USA. 12193–12202. https://openaccess.thecvf.com/content_CVPR_2020/html/Xie_PolarMask_Single_Shot_Instance_Segmentation_With_Polar_Representation_CVPR_2020_paper.html
   Google Scholar
• Xiong, Y., R. Liao, H. Zhao, H. Rui, M. Bai, E. Yumer, and R. Urtasun. 2019. “UPSNet: A Unified Panoptic Segmentation Network.” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8810–8818. Long Beach, CA, USA: IEEE. doi:https://doi.org/10.1109/CVPR.2019.00902.
   Google Scholar
• Xu, S. 2015. “Classification and Change Detection in Multi - Epoch Airborne Laser Scanning Point Clouds.” University of Twente, Faculty of Geo-Information Science and Earth Observation (ITC). July. doi:https://doi.org/10.3990/1.97836538350.
   Google Scholar
• Yang, T.-J., M. D. Collins, Y. Zhu, J.-J. Hwang, T. Liu, X. Zhang, V. Sze, G. Papandreou, and L.-C. Chen. 2019. “DeeperLab: Single-Shot image parser.” ArXiv:1902.05093 [Cs], March. http://arxiv.org/abs/1902.05093
   Google Scholar
• Zakeri, F., and M. R. Saradjian. 2020. “Change Detection in Multispectral Images Based on Fusion of Change Vector Analysis in Posterior Probability Space and Posterior Probability Space Angle Mapper.” Geocarto International 1–15. Taylor & Francis. doihttps://doi.org/10.1080/10106049.2020.1768595.
   Google Scholar
• Zhong, Z., Z. Q. Lin, R. Bidart, X. Hu, I. B. Daya, Z. Li, W.-S. Zheng, J. Li, and A. Wong. 2020. “Squeeze-and-Attention Networks for Semantic Segmentation.” In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13062–13071. doi:https://doi.org/10.1109/CVPR42600.2020.01308.
   Google Scholar
• Zong, K., A. Sowmya, and J. Trinder. 2013. “Machine Learning Based Urban Change Detection by Fusing High Resolution Aerial Images and Lidar Data.” In Geo-Informatics in Resource Management and Sustainable Ecosystem, edited by F. Bian, Y. Xie, X. Cui, and Y. Zeng, 522–532. Vol. 398. Communications in Computer and Information Science. Berlin, Heidelberg: Springer Berlin Heidelberg. doi:https://doi.org/10.1007/978-3-642-45025-9_51.
   Google Scholar