GPU based building footprint identification utilising self-attention multiresolution analysis

Figures & data

Figure 1. Proposed architecture of self-attention multiresolution analysis.

Figure 2. Dual attention mechanism.

Figure 3. Column-wise (a) Input image, (b) ground truth, and segmentation results of (c) SVM, (d) UNet, (e) ResUNet, (f) Proposed method of MRA-SA on the WHU building dataset with four samples (row-wise).

Table 1. Performance comparison for the WHU dataset.

Model	IoU		mIoU	Precision	Recall	F1-Score	Accuracy (%)
	Building	Non-Building
U-Net (Ronneberger et al., 2015)	0.870	0.914	0.890	0.853	0.941	0.895	93.70
Deep ResUnet (Z. Zhang et al., 2018)	0.715	0.905	0.811	0.944	0.730	0.823	92.30
ScattNet (Li et al. 2021)	0.904	0.938	0.920	0.933	0.919	0.927	94.03
Self-enhanced SVM (Q. Zhang & Guo, 2007)	0.793	0.856	0.824	0.841	0.911	0.874	89.80
FCN-32 (Long et al., 2015)	0.838	0.862	0.883	0.893	0.929	0.921	90.36
UNetFormer (Wang et al., 2022)	0.903	0.858	0.881	0.900	0.917	0.908	91.96
Buildformer (Wang et al., 2022)	0.944	0.891	0.917	0.948	0.947	0.946	95.18
CG-Swin (Meng et al., 2022)	0.937	0.889	0.913	0.902	0.938	0.919	94.61
ASF-Net (Chen et al., 2022)	0.932	0.901	0.917	0.916	0.941	0.928	94.89
Proposed with Bior MRA	0.904	0.952	0.928	0.966	0.936	0.951	96.51
Proposed with Daub MRA	0.928	0.985	0.956	0.962	0.968	0.965	97.58

Figure 4. Column-wise (a) Input image, (b) ground truth, and segmentation results of (c) SVM, (d) UNet, (e) ResUNet, (f) Proposed method of MRA-SA on the OpenCities building dataset with four sample images (row-wise).

Table 2. Performance comparison for the OpenCities dataset.

Model	IoU		mIoU	Precision	Recall	F1-Score	Accuracy (%)
	Building	Non-Building
U-Net (Ronneberger et al., 2015)	0.925	0.805	0.865	0.893	0.889	0.891	94.13
Deep ResUnet (Z. Zhang et al., 2018)	0.884	0.683	0.783	0.746	0.888	0.811	90.75
ScattNet (Li et al. 2021)	0.937	0.878	0.907	0.878	0.965	0.919	95.77
Self-enhanced SVM (Q. Zhang & Guo, 2007)	0.806	0.844	0.825	0.834	0.876	0.855	84.81
FCN-32 (Long et al., 2015)	0.838	0.880	0.859	0.796	0.853	0.824	86.17
UNetFormer (Wang et al., 2022)	0.893	0.925	0.909	0.905	0.928	0.916	92.38
Buildformer (Wang et al., 2022)	0.924	0.887	0.906	0.948	0.949	0.951	95.07
CG-Swin (Meng et al., 2022)	0.925	0.883	0.904	0.898	0.932	0.915	93.92
ASF-Net (Chen et al., 2022)	0.878	0.864	0.871	0.879	0.902	0.890	91.15
Proposed with Bior MRA	0.931	0.892	0.912	0.906	0.939	0.922	94.41
Proposed with Daub MRA	0.956	0.904	0.931	0.927	0.969	0.948	96.32

Table 3. Ablation results.

Model	mIoU	Precision	Recall	F1-Score	Accuracy (%)
Baseline	0.771	0.733	0.760	0.746	76.57
Baseline + MRA	0.886	0.874	0.902	0.897	93.70
Baseline + Patch Attention	0.826	0.853	0.805	0.828	86.67
Baseline + Channel Attention	0.844	0.881	0.839	0.859	88.94
Baseline + with Dual Self Attention	0.851	0.891	0.837	0.863	90.51
Proposed (Baseline + MRA + Dual Self Attention)	0.956	0.962	0.968	0.965	97.58
Proposed without Progressive Generation	0.947	0.948	0.929	0.938	95.73

Figure 5. (A) Input image, (b) ground truth, and segmentation results of (c) SVM, (d) UNet, (e) ResUNet, (f) Only MRA features, (g) Only SA features (h) Proposed method of MRA-DA on the WHU building dataset.

Table 4. Compute engine comparison.

Compute Engine	Performance (fps)	Execution time (ms)
Intel i5 2GHz CPU	0.23	4347.82
Tesla P100 GPU	28.57	35.01
Jetson Nano 4GB	10.54	94.80

Table 5. Quantitative network efficiency comparison on the Jetson Nano (4GB) on an input of size 256 × 256 with select methods from Table 1.

Method	Memory (MB)	Parameters (in Millions)	Performance (FPS)
UNet (Ronneberger et al., 2015)	1163.2	22.4	3.28
ScattNet (Li et al. 2021)	835.8	18.9	7.53
UNetFormer (Wang et al., 2022)	1007.6	11.7	7.63
Proposed with Daub MRA	960.4	8.4	10.54

Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional net- works for biomedical image segmentation. In N. Navab, H. Joachim, M. W. William, & A. Frangi (Eds.), Proceedings of International Conference Medical Image Computing and Computer-Assisted Intervention (pp. 234–241). Springer.

 Google Scholar

Zhang, Z., Liu, Q., & Wang, Y. (2018, May). Road extraction by deep residual U-Net. IEEE Geoscience and Remote Sensing Letters, 15(5), 749–753. https://doi.org/10.1109/LGRS.2018.2802944

 Web of Science ®Google Scholar

Li, H., Qiu, K., Chen, L., Mei, X., Hong, L., & Tao, C. (2021, May). SCAttNet: Semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images. IEEE Geoscience and Remote Sensing Letters, 18(5), 905–909. https://doi.org/10.1109/LGRS.2020.2988294

 Web of Science ®Google Scholar

Zhang, Q., & Guo, L. (2007). Self-enhanced SVM extraction of building objects from high resolution satellite images. Second International Conference on Innovative Computing, Informatio and Control (ICICIC 2007), 13. https://doi.org/10.1109/ICICIC.2007.511.

 Google Scholar

Long, J., Shelhamer, E., & Darrell, T. (2015, Jun). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, USA (pp. 3431–3440).

 Google Scholar

Wang, L., Fang, S., Meng, X., & Rui, L. (2022). Building extraction with vision transformer. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–11. https://doi.org/10.1109/TGRS.2022.3186634

 Web of Science ®Google Scholar

Meng, X., Yang, Y., Wang, L., Wang, T., Rui, L., & Zhang, C. (2022). Class-guided swin transformer for semantic segmentation of remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 19, 1–5. https://doi.org/10.1109/LGRS.2022.3215200

 Google Scholar

Chen, J., Jiang, Y., Luo, L., & Gong, W. (2022). ASF-Net: Adaptive screening feature network for building footprint extraction from remote-sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–13. https://doi.org/10.1109/TGRS.2022.3165204

 Web of Science ®Google Scholar

Data availability statement

The data that support the findings of this study are openly available in GFDRR at https://doi.org/10.34911/rdnt.f94cxb