Vectorizing planar roof structure from very high resolution remote sensing images using transformers

Figures & data

Figure 1. False detection caused by false positive vertex or edge candidates, which are marked the yellow dotted line.

Figure 2. The overall architecture of Roof-Former, which consists of three steps: (1) Image encoding and edge node initialization (yellow); (2) Image feature fusion with enhanced segmentation refinement branch (blue); and (3) Structural reasoning with Transformer decoders (green).

Figure 3. The corner detection model adapted from edge detection architecture.

Figure 4. Illustration of the proposed AFFM. C and r denote the channel number and the channel reduction ratio, respectively. The refined feature $X^{\prime }$ is enhanced by the extracting local channel context ($L(X)$ in blue box) and global channel context ($G(X)$ in green box) in MS-CAM.

Table 1. Quantitative results on the VWB dataset and the Enschede dataset are provided, where $A{P}^H$ and $F^H$ represent the heatmap-based average precision and F1-score for both vertex and edge primitives.

Datasets	Methods	$msA{P}^V$	$msA{P}^E$	$A{P}_V^H$	$F_V^H$	$A{P}_E^H$	$F_E^H$
VWB	ConvMPN	35.7	34.2	78.0	78.8	57.0	58.1
	HAWP	31.1	31.0	90.9	85.2	76.6	72.1
	RSGNN	34.8	34.6	89.6	85.7	76.4	75.7
	HEAT	41.6	40.3	91.7	87.1	80.6	76.2
	Roof-Former	43.1 ± 0.4	42.4 ± 0.1	92.3 ± 0.2	89.1 ± 0.5	82.3 ± 0.2	78.1 ± 0.3
ENS	ConvMPN	43.0	45.2	63.6	61.3	62.8	61.5
	HAWP	42.1	43.3	62.7	59.6	63.7	62.7
	RSGNN	43.8	46.5	66.1	67.6	66.2	65.0
	HEAT	46.6	48.2	69.3	69.7	65.8	68.8
	Roof-Former	48.1 ± 0.3	49.3 ± 0.2	70.4 ±0.5	71.2 ± 0.2	67.3 ± 0.6	69.5 ± 0.3

Notes: Additionally, $msA{P}^V$ and $msA{P}^E$ denote the mean structural Average Precision for vectorized vertices and edges. Average evaluation metrics were calculated for 5-fold cross-validation as the sensitivity analysis.

Figure 5. Sample results on the Enschede dataset (first two rows) and VWB datasets (the third and fourth row). Roof-Former has been found to produce better reconstruction results for massive and complex buildings, with its output closer to the label in the second column. In comparison, other methods such as ConvMPN and HEAT results contain geometric imperfections such as narrow triangles, self-intersecting edges, and colinear edges.

Table 2. Ablation study for the components of Roof-Former, evaluated with vertex or edge F1 scores.

FPT	Segmentation branch	AFFM	GFLOPs	$F_V^H$	$F_E^H$	Gain
Baseline	–	–	6.3	87.1	76.2	–
✓	–	–	3.8	87.3	76.8	0.2/0.4
✓	✓	–	5.2	88.4	77.6	1.3/1.2
✓(Roof-Former)	✓	✓	5.5	89.1	78.1	1.9/1.7

Figure 6. Sample reconstruction results of building objects based upon our extracted roof structure and nDSM.

Data availability statement

The experiments conducted in this paper are based on two publicly available datasets, which can be accessed at Nauata and Furukawa (2020) and Zhao, Persello, and Stein (2022). Any inquiries regarding the datasets should be directed to the original authors.