471
Views
12
CrossRef citations to date
0
Altmetric
Research Article

MS-CNN: multiscale recognition of building rooftops from high spatial resolution remote sensing imagery

, , &
Pages 270-298 | Received 19 Apr 2021, Accepted 03 Dec 2021, Published online: 10 Jan 2022

References

  • Ahmed, N., R. Bin Mahbub, and R. M. Rahman. 2020. “Learning to Extract Buildings from Ultra-High-Resolution Drone Images and Noisy Labels.” International Journal of Remote Sensing 41 (21): 8216–8237. doi:https://doi.org/10.1080/01431161.2020.1763496.
  • Alex, K., I. Sutskever, and G. E. Hinton. 2012. “ImageNet Classification with Deep Convolutional Neural Networks.” In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, 1097–1105. Lake Tahoe, Nevada: Curran Associates Inc.
  • Alshehhi, R., P. Reddy Marpu, W. Lee Woon, and M. Dalla Mura. 2017. “Simultaneous Extraction of Roads and Buildings in Remote Sensing Imagery with Convolutional Neural Networks.” ISPRS Journal of Photogrammetry and Remote Sensing 130: 139–149. doi:https://doi.org/10.1016/j.isprsjprs.2017.05.002.
  • Blaschke, T. 2010. “Object Based Image Analysis for Remote Sensing.” ISPRS Journal of Photogrammetry and Remote Sensing 65 (1): 2–16. doi:https://doi.org/10.1016/j.isprsjprs.2009.06.004.
  • Blaschke, T., G. J. Hay, M. Kelly, S. Lang, P. Hofmann, E. Addink, R. Queiroz Feitosa et al. 2014. ”Geographic Object-Based Image Analysis – Towards a New Paradigm.” ISPRS Journal of Photogrammetry and Remote Sensing 87: 180–191. doi:https://doi.org/10.1016/j.isprsjprs.2013.09.014.
  • Bui, D., N.-D. Hoang, F. Martínez-Álvarez, P.-T. Thi Ngo, P. V. Viet Hoa, T. D. Dat Pham, P. Samui, and R. Costache. 2020. “A Novel Deep Learning Neural Network Approach for Predicting Flash Flood Susceptibility: A Case Study at a High Frequency Tropical Storm Area.” The Science of the Total Environment 701: 134413. doi:https://doi.org/10.1016/j.scitotenv.2019.134413.
  • Chaudhuri, D., N. K. Kushwaha, A. Samal, and R. C. Agarwal. 2016. “Automatic Building Detection from High-Resolution Satellite Images Based on Morphology and Internal Gray Variance.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 9 (5): 1767–1779. doi:https://doi.org/10.1109/JSTARS.2015.2425655.
  • Chen, K., J. Pang, J. Wang, Y. Xiong, X. Li, S. Sun, W. Feng et al. 2019. “Hybrid Task Cascade for Instance Segmentation. ”Paper presented at the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June.
  • Chen, L., A. Hermans, G. Papandreou, F. Schroff, P. Wang, and H. Adam. 2018. “MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features.” Paper presented at the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18-23 June.
  • Chen, S., T. Yue, and L. Huiguo. 2000. “Studies on Geo-Informatic Tupu and Its Application.” Geographical Research 19 (4): 337–343.
  • Chen, X., R. Girshick, K. He, and P. Dollar. 2019. “TensorMask: A Foundation for Dense Object Segmentation.” Paper presented at the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 27 October–2 November.
  • Dai, J., Y. Li, H. Kaiming, and J. Sun. 2016. “R-Fcn: Object Detection via Region-Based Fully Convolutional Networks.” Paper presented at the Advances in Neural Information Processing Systems.
  • Deng, Z., H. Sun, S. Zhou, J. Zhao, and H. Zou. 2017. “Toward Fast and Accurate Vehicle Detection in Aerial Images Using Coupled Region-Based Convolutional Neural Networks.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10 (8): 3652–3664. doi:https://doi.org/10.1109/JSTARS.2017.2694890.
  • Deng, Z., H. Sun, S. Zhou, J. Zhao, L. Lei, and H. Zou. 2018. “Multi-Scale Object Detection in Remote Sensing Imagery with Convolutional Neural Networks.” ISPRS Journal of Photogrammetry and Remote Sensing 145: 3–22. doi:https://doi.org/10.1016/j.isprsjprs.2018.04.003.
  • Deren, L., L. Zhang, and G. Xia. 2014. “Automatic Analysis and Mining of Remote Sensing Big Data.” Acta Geodaetica Et Cartographica Sinica 43 (12): 1211.
  • Ding, H., X. Jiang, B. Shuai, A. Q. Liu, and G. Wang. 2018. “Context Contrasted Feature and Gated Multi-Scale Aggregation for Scene Segmentation.” Paper presented at the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18-23 June.
  • Duchêne, C., S. Bard, X. Barillot, A. Ruas, J. Trévisan, and F. Holzapfel. 2003. “Quantitative and Qualitative Description of Building Orientation.” Paper presented at the Fifth workshop on progress in automated map generalisation, ICA, commission on map generalisation, 01/01.
  • Feimo, L., L. Shuxiao, C. Zhu, X. Lan, and H. Chang. 2017. “Cost-Effective Class-Imbalance Aware CNN for Vehicle Localization and Categorization in High Resolution Aerial Images.” Remote Sensing 9 (5). doi:https://doi.org/10.3390/rs9050494.
  • Feng, W., H. Sui, L. Hua, C. Xu, G. Ma, and W. Huang. 2020. “Building Extraction from VHR Remote Sensing Imagery by Combining an Improved Deep Convolutional Encoder-Decoder Architecture and Historical Land Use Vector Map.” International Journal of Remote Sensing 41 (17): 6595–6617. doi:https://doi.org/10.1080/01431161.2020.1742944.
  • Fu, K., Z. Chang, Y. Zhang, G. Xu, K. Zhang, and X. Sun. 2020. “Rotation-Aware and Multi-Scale Convolutional Neural Network for Object Detection in Remote Sensing Images.” ISPRS Journal of Photogrammetry and Remote Sensing 161: 294–308. doi:https://doi.org/10.1016/j.isprsjprs.2020.01.025.
  • Gao, S. H., M. M. Cheng, K. Zhao, X. Y. Zhang, M. H. Yang, and P. Torr. 2021. “Res2net: A New Multi-Scale Backbone Architecture.” IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (2): 652–662. doi:https://doi.org/10.1109/TPAMI.2019.2938758.
  • Girshick, R. 2015. “Fast R-CNN.” Paper presented at the 2015 IEEE International Conference on Computer Vision (ICCV), 7-13 December.
  • Girshick, R., J. Donahue, T. Darrell, and J. Malik. 2014. “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation.” Paper presented at the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 23-28 June.
  • Griffiths, D. and J. Boehm. 2019. “Improving Public Data for Building Segmentation from Convolutional Neural Networks (CNNs) for Fused Airborne Lidar and Image Data Using Active Contours.” ISPRS Journal of Photogrammetry and Remote Sensing 154: 70–83. doi:https://doi.org/10.1016/j.isprsjprs.2019.05.013.
  • He, J., Z. Deng, L. Zhou, Y. Wang, and Y. Qiao. 2019. “Adaptive Pyramid Context Network for Semantic Segmentation. ”Paper presented at the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June.
  • He, J., Z. Deng, and Y. Qiao. 2019. “Dynamic Multi-Scale Filters for Semantic Segmentation.” Paper presented at the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 27 October–2 November
  • He, K., G. Gkioxari, P. Dollár, and R. Girshick. 2017. “Mask R-CNN.” Paper presented at the 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 October.
  • He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep Residual Learning for Image Recognition.” Paper presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June.
  • Hoffmann, E. J., Y. Wang, M. Werner, J. Kang, and X. Xiang Zhu. 2019. “Model Fusion for Building Type Classification from Aerial and Street View Images.” Remote Sensing 11 (11): 1259. doi:https://doi.org/10.3390/rs11111259.
  • Huang, G., Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. 2017. “Densely Connected Convolutional Networks.” Paper presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July.
  • Huang, X., H. Liu, and L. Zhang. 2015. “Spatiotemporal Detection and Analysis of Urban Villages in Mega City Regions of China Using High-Resolution Remotely Sensed Imagery.” IEEE Transactions on Geoscience and Remote Sensing 53 (7): 3639–3657. doi:https://doi.org/10.1109/TGRS.2014.2380779.
  • Huang, X., L. Zhang, and T. Zhu. 2014. “Building Change Detection from Multitemporal High-Resolution Remotely Sensed Images Based on a Morphological Building Index.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7 (1): 105–115. doi:https://doi.org/10.1109/JSTARS.2013.2252423.
  • Ji, S., S. Wei, and M. Lu. 2019a. “Fully Convolutional Networks for Multisource Building Extraction from an Open Aerial and Satellite Imagery Data Set.” IEEE Transactions on Geoscience and Remote Sensing 57 (1): 574–586. doi:https://doi.org/10.1109/TGRS.2018.2858817.
  • Ji, S., S. Wei, and M. Lu. 2019b. “A Scale Robust Convolutional Neural Network for Automatic Building Extraction from Aerial and Satellite Imagery.” International Journal of Remote Sensing 40 (9): 3308–3322. doi:https://doi.org/10.1080/01431161.2018.1528024.
  • Lecun, Y., L. Bottou, Y. Bengio, and P. Haffner. 1998. “Gradient-Based Learning Applied to Document Recognition.” Proceedings of the IEEE 86 (11): 2278–2324. doi:https://doi.org/10.1109/5.726791.
  • Lee, Y. and J. Park. 2020. “CenterMask: Real-Time Anchor-Free Instance Segmentation.” Paper presented at the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June.
  • Li, W., H. Liu, Y. Wang, Z. Li, Y. Jia, and G. Gui. 2019. “Deep Learning-Based Classification Methods for Remote Sensing Images in Urban Built-Up Areas.” IEEE Access 7: 36274–36284. doi:https://doi.org/10.1109/ACCESS.2019.2903127.
  • Liang, J., N. Homayounfar, W. C. Ma, Y. Xiong, R. Hu, and R. Urtasun. 2020. “PolyTransform: Deep Polygon Transformer for Instance Segmentation.” Paper presented at the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June.
  • Liasis, G. and S. Stavrou. 2016. “Building Extraction in Satellite Images Using Active Contours and Colour Features.” International Journal of Remote Sensing 37 (5): 1127–1153. doi:https://doi.org/10.1080/01431161.2016.1148283.
  • Lin, T., P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017. “Feature Pyramid Networks for Object Detection.” Paper presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July.
  • Liu, H., J. Luo, B. Huang, H. Xiaodong, Y. Sun, Y. Yang, X. Nan, and N. Zhou. 2019. “DE-Net: Deep Encoding Network for Building Extraction from High-Resolution Remote Sensing Imagery.” Remote Sensing 11 (20): 2380. doi:https://doi.org/10.3390/rs11202380.
  • Liu, J., H. Pu, S. Song, and M. Du. 2018. “An Adaptive Scale Estimating Method of Multiscale Image Segmentation Based on Vector Edge and Spectral Statistics Information.” International Journal of Remote Sensing 39 (20): 6826–6845. doi:https://doi.org/10.1080/01431161.2018.1466077.
  • Liu, J., J. Zhang, F. Xu, Z. Huang, and Y. Li. 2014. “Adaptive Algorithm for Automated Polygonal Approximation of High Spatial Resolution Remote Sensing Imagery Segmentation Contours.” IEEE Transactions on Geoscience and Remote Sensing 52 (2): 1099–1106. doi:https://doi.org/10.1109/TGRS.2013.2247407.
  • Liu, J., M. Du, and Z. Mao. 2017. “Scale Computation on High Spatial Resolution Remotely Sensed Imagery Multi-Scale Segmentation.” International Journal of Remote Sensing 38 (18): 5186–5214. doi:https://doi.org/10.1080/01431161.2017.1325536.
  • Liu, J., S. Wang, X. Hou, and W. Song. 2020. “A Deep Residual Learning Serial Segmentation Network for Extracting Buildings from Remote Sensing Imagery.” International Journal of Remote Sensing 41 (14): 5573–5587. doi:https://doi.org/10.1080/01431161.2020.1734251.
  • Liu, S., L. Qi, H. Qin, J. Shi, and J. Jia. 2018. “Path Aggregation Network for Instance Segmentation.” Paper presented at the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18-23 June.
  • Liu, W., A. Rabinovich, and A. C. Berg. 2015. “ParseNet: Looking Wider to See Better.” Computer Science.
  • Long, J., E. Shelhamer, and T. Darrell. 2015. “Fully Convolutional Networks for Semantic Segmentation.” Paper presented at the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June.
  • Maggiori, E., Y. Tarabalka, G. Charpiat, and P. Alliez. 2017. “Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification.” IEEE Transactions on Geoscience and Remote Sensing 55 (2): 645–657. doi:https://doi.org/10.1109/TGRS.2016.2612821.
  • Martins, V. S., A. L. Kaleita, B. K. Gelder, H. L. F. da Silveira, and C. A. Abe. 2020. “Exploring Multiscale Object-Based Convolutional Neural Network (Multi-OCNN) for Remote Sensing Image Classification at High Spatial Resolution.” ISPRS Journal of Photogrammetry and Remote Sensing 168: 56–73. doi:https://doi.org/10.1016/j.isprsjprs.2020.08.004.
  • Mobile Geospatial Big Data Cloud Service Innovation Team. http://www.dxkjs.com/Public/about/html/research_rs.html.
  • Mura, M., G. V. Dalla, R. Restaino, P. Addesso, and J. Chanussot. 2015. “Global and Local Gram-Schmidt Methods for Hyperspectral Pansharpening.” Paper presented at the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 26-31 July.
  • Pham, T. D., D. Tien Bui, K. Yoshino, and N. N. Le. 2018. “Optimized Rule-Based Logistic Model Tree Algorithm for Mapping Mangrove Species Using ALOS PALSAR Imagery and GIS in the Tropical Region.” Environmental Earth Sciences 77 (5): 159. doi:https://doi.org/10.1007/s12665-018-7373-y.
  • Pham, T. D., K. Yoshino, and D. Tien Bui. 2017. “Biomass Estimation of Sonneratia Caseolaris (L.) Engler at a Coastal Area of Hai Phong City (Vietnam) Using ALOS-2 PALSAR Imagery and GIS-Based Multi-Layer Perceptron Neural Networks.” GIScience & Remote Sensing 54 (3): 329–353. doi:https://doi.org/10.1080/15481603.2016.1269869.
  • Raschka, S. 2018. “Mlxtend: Providing Machine Learning and Data Science Utilities and Extensions to Python’s Scientific Computing Stack.” Journal of Open Source Software 3 (24): 638. doi:https://doi.org/10.21105/joss.00638.
  • Ren, S., K. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (6): 1137–1149. doi:https://doi.org/10.1109/TPAMI.2016.2577031.
  • Sabour, S., N. Frosst, and G. E. Hinton. 2017. “Dynamic Routing Between Capsules.”
  • Senaras, C., M. Ozay, and F. T. Yarman Vural. 2013. “Building Detection with Decision Fusion.” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 6 (3): 1295–1304. doi:https://doi.org/10.1109/JSTARS.2013.2249498.
  • Shao, Z., P. Tang, Z. Wang, N. Saleem, S. Yam, and C. Sommai. 2020. “Brrnet: A Fully Convolutional Neural Network for Automatic Building Extraction from High-Resolution Remote Sensing Images.” Remote Sensing 12 (6): 1050. doi:https://doi.org/10.3390/rs12061050.
  • Shao, Z., Z. Zhou, X. Huang, and Y. Zhang. 2021. “Mrenet: Simultaneous Extraction of Road Surface and Road Centerline in Complex Urban Scenes from Very High-Resolution Images.” Remote Sensing 13 (2): 239. doi:https://doi.org/10.3390/rs13020239.
  • Song, S., J. Liu, H. Pu, Y. Liu, and J. Luo. 2019. “The Comparison of Fusion Methods for HSRRSI Considering the Effectiveness of Land Cover (Features) Object Recognition Based on Deep Learning.” Remote Sensing 11 (12): 1435. doi:https://doi.org/10.3390/rs11121435.
  • Song, S., J. Liu, Y. Liu, G. Feng, H. Han, Y. Yao, and M. Du. 2020. “Intelligent Object Recognition of Urban Water Bodies Based on Deep Learning for Multi-Source and Multi-Temporal High Spatial Resolution Remote Sensing Imagery.” Sensors (Basel) 20 (2): 397–422. doi:https://doi.org/10.3390/s20020397.
  • Szegedy, C., L. Wei, P. S. Jia Yangqing, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2015. “Going Deeper with Convolutions.” Paper presented at the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June.
  • Tang, Y., X. Huang, and L. Zhang. 2013. “Fault-Tolerant Building Change Detection from Urban High-Resolution Remote Sensing Imagery.” IEEE Geoscience and Remote Sensing Letters 10 (5): 1060–1064. doi:https://doi.org/10.1109/LGRS.2012.2228626.
  • Tao, C., Y. Tan, Z. Zou, and J. Tian. 2013. “Unsupervised Detection of Built-Up Areas from Multiple High-Resolution Remote Sensing Images.” IEEE Geoscience and Remote Sensing Letters 10 (6): 1300–1304. doi:https://doi.org/10.1109/LGRS.2013.2237751.
  • Thomas, C., T. Ranchin, L. Wald, and J. Chanussot. 2008. “Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics.” IEEE Transactions on Geoscience and Remote Sensing 46 (5): 1301–1312. doi:https://doi.org/10.1109/TGRS.2007.912448.
  • Wang, Z., D. Ziou, C. Armenakis, D. Li, and Q. Li. 2005. “A Comparative Analysis of Image Fusion Methods.” IEEE Transactions on Geoscience and Remote Sensing 43 (6): 1391–1402. doi:https://doi.org/10.1109/TGRS.2005.846874.
  • Wentz, E. A., D. Nelson, A. Rahman, W. L. Stefanov, and S. Sen Roy. 2008. “Expert System Classification of Urban Land Use/cover for Delhi, India.” International Journal of Remote Sensing 29 (15): 4405–4427. doi:https://doi.org/10.1080/01431160801905497.
  • Xie, E., P. Sun, X. Song, W. Wang, X. Liu, D. Liang, C. Shen, and P. Luo. 2020. “PolarMask: Single Shot Instance Segmentation with Polar Representation.” Paper presented at the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June.
  • Zhang, C., P. A. Harrison, X. Pan, H. Li, I. Sargent, and P. M. Atkinson. 2020. “Scale Sequence Joint Deep Learning (SS-JDL) for Land Use and Land Cover Classification.” Remote Sensing of Environment 237: 111593. doi:https://doi.org/10.1016/j.rse.2019.111593.
  • Zhang, H., C. Wu, Z. Zhang, Y. Zhu, Z. Zhang, H. Lin, Y. Sun, T. He, J. Mueller, and R. Manmatha. 2020. “ResNest: Split-Attention Networks.”
  • Zhang, R., Z. Shao, X. Huang, J. Wang, and D. Li. 2020. “Object Detection in UAV Images via Global Density Fused Convolutional Network.” Remote Sensing 12 (19): 3140. doi:https://doi.org/10.3390/rs12193140.
  • Zhao, H., J. Shi, X. Qi, X. Wang, and J. Jia. 2017. “Pyramid Scene Parsing Network.” Paper presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July.
  • Zhao, W., C. Persello, and A. Stein. 2021. “Building Outline Delineation: From Aerial Images to Polygons with an Improved End-to-End Learning Framework.” ISPRS Journal of Photogrammetry and Remote Sensing 175: 119–131. doi:https://doi.org/10.1016/j.isprsjprs.2021.02.014.
  • Zhou, W., D. Ming, X. Lv, K. Zhou, H. Bao, and Z. Hong. 2020. “So–CNN–CNN Based Urban Functional Zone Fine Division with VHR Remote Sensing Image.” Remote Sensing of Environment 236: 111458. doi:https://doi.org/10.1016/j.rse.2019.111458.
  • Zhu, Q., S. Huang, H. Han, H. Li, M. Chen, and R. Zhong. 2021. “Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images.” ISPRS Journal of Photogrammetry and Remote Sensing 174: 105–116. doi:https://doi.org/10.1016/j.isprsjprs.2021.01.025.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.