Global road extraction using a pseudo-label guided framework: from benchmark dataset to cross-region semi-supervised learning

Figures & data

Figure 1. Examples from the five road datasets (512 × 512 pixels).

Table 1. Details of the five road datasets.

Road Datasets	Resolution (m/pixel)	Width (m)	Size (pixels)	Images (train/test)	Countries	Cities	Types	Area (km^2)
Massachusetts	1	7	1500 × 1500	1108/49	1	>1	Urban	2603.25
RoadTracer	0.6	6	1024 × 1024	2880/1920	6	40	Urban	1811.94
SpaceNet	0.3	2	1300 × 1300	2213/567	4	4	Urban	422.84
DeepGlobe	0.5	–	1024 × 1024	4696/1530	3	>3	Urban/rural	1632.11
GRSet	1	5	1024 × 1024	47,210/-	121	121	Urban/rural	49,503.27

Figure 2. Distribution of the geographical locations of the GRSet dataset.

Figure 3. Annotation examples of the GRSet.

Figure 4. The flowchart of the proposed pseudo-label guided global-scale road extraction framework, which is composed of two parts: (a) GRSet generation, and (b) pseudo-label guided global-scale road extraction network.

Figure 5. Visualizations of the weak and strong data augmentations.

Table

Algorithm 1 Training procedure for the pseudo-label guided global-local road extraction framework.
Model: Encoder, Main decoder, Auxiliary decoder; Model parameters: $\mathit{\rho }$
Weak data augmentation: $\mathit{W}$, Strong data augmentation: $\mathit{S}$
Step1: use GRSet to pre-train Encoder and Main decoder (LinkNet50).
Input: OSM-annotated GRSet ${\left\{X,Y\right\}}_{i=1}^N$.
for j = 1 to epochs do
Get the mini-batch training samples: $\mathit{x},\mathit{y}.$
LinkNet50 forward: $\mathit{p}$= LinkNet50($\mathit{W}$($\mathit{x}$))
Calculate the segmentation loss:$L_s\left(\mathit{p},\mathit{y}\right)$ and update $\mathit{\rho }$.
end
obtain the pre-trained LinkNet50
Step2: update GRNet^SF parameters using target region samples.
Input: target region labeled samples ${\left\{X_l,Y_l\right\}}_{\mathit{i}=1}^{N_l}$ and unlabeled images ${\left\{X_u\right\}}_{\mathit{i}=1}^{N_u}$
for k = 1 to iterations do
Get the mini-batch training samples: $x_l,y_l$, and unlabeled images $x_u$
LinkNet50 forward: $p_l$= LinkNet50($\mathit{W}$($x_l$)), $\hat{p}$ = threshold (LinkNet50($\mathit{W}$($x_u$))), $p_{\mathit{su}}$ = LinkNet50($\mathit{S}$($x_u$)), $p_{\mathit{fu}}$ = Auxiliary decoder (Encoder ($\mathit{W}$($x_u$)))
Calculate the loss:$L_s\left(p_l,y_l\right)$ + $L_u^d\left(p_{\mathit{su}},\hat{p}\right)$ + $L_u^f\left(p_{\mathit{fu}},\hat{p}\right)$ and update $\mathit{\rho }$.
end

Figure 6. Geographical locations of the validation sets. The validation set covers the four continents of Europe, Africa, Asia, and North America.

Table 2. The details of the validation set of global road networks.

Road dataset	Acquisition time	Resolution (m/pixel)	Width (m)	Size (pixels)	Images	Cities	Types	Area (km^2)
Massachusetts test set	2013	1	7	1500×1500	49	>1	urban	110.25
RoadTracer test set	2018	0.6	6	1024×1024	1920	15	urban	724.78
SpaceNet test set	2018	0.3	2	1300×1300	567	4	urban	86.24
DeepGlobe test set	2018	0.5	–	1024×1024	1530	>3	urban/rural	401.08
Boston	2019	0.4411	–	23,104×20,480	1	1	urban	92.06
Birmingham	2019	0.3640	–	22,272×22,464	1	1	urban	66.29
Helsinki	2018	0.5	–	36,864×23,552	1	1	urban	217.06
Kigali	2021	1.2	–	20,480×20,480	1	1	rural	603.98
Coast Province	2021	0.6	–	20,480×20,480	1	1	rural	150.99
Beijing	2019	0.5	–	20,480×20,480	1	1	urban	104.86
Shanghai	2019	0.5	–	16,768×12,544	1	1	urban	52.58
Wuhan	2019	0.5	–	18,272×15,040	1	1	urban	68.70
LoveDA road test set	2016	0.3	–	1024×1024	1669	3	urban/rural	157.51
Total	–				5743	>30	–	2836.38

Figure 7. Illustration of sampling of GRSet.

Table 3. Initialization details of the comparison methods.

Type	Method	Model initialization parameters
		P	I	G
Unsupervised domain adaptation	OSN (Tsai et al. 2018)		✓
	MaxSquare (M. Chen, Xue, and Cai 2019)		✓
	GOAL (Lu, Zhong, Zheng, and Wang 2021)		✓
Semantic segmentation	LinkNet50	✓
	LinkNet50_IN		✓
	LinkNet50_GRSet			✓
Semi-supervised learning	MT (Tarvainen and Valpola 2017)			✓
	CCT (Ouali, Hudelot, and Tami 2020)			✓
	GRNet^S	✓
	GRNet^S_IN		✓
	GRNet^S_GRSet			✓
	GRNet^SF	✓
	GRNet^SF_IN		✓
	GRNet^SF_GRSet			✓

GRNetS means using only strong data augmentation perturbation, and GRNetSF means using strong data augmentation perturbation and feature perturbation.

Figure 8. Quantitative IoU comparison of models trained on the different road datasets.

Figure 9. The visual outputs for the Birmingham image produced by LinkNet50_IN with different training sets.

Figure 10. The histogram illustrates the quantitative results obtained on the SpaceNet and DeepGlobe dataset.

Figure 11. The visualization results of different road datasets, where the first and second rows respectively represent the results obtained on the SpaceNet and DeepGlobe datasets.

Figure 12. The Histogram illustrates the quantitative results of the large-scale images.

Figure 13. The local visualization results of the large-scale images.

Figure 14. The Histogram illustrates the quantitative results obtained on the LoveDA dataset.

Figure 15. The visual urban and rural results obtained on the LoveDA road validation set.

Figure 16. The histogram illustrates the quantitative results obtained on the Birmingham image.

Figure 17. The visual results obtained on the Birmingham image.

Figure 18. The IoU variation with different values of $\mathit{\mu }$.

Tsai, Y.-H., W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang, and M. Chandraker. 2018. “Learning to Adapt Structured Output Space for Semantic Segmentation.” In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7472–7481. Salt Lake City, UT, USA, June 18–22.

 Google Scholar

Chen, M., H. Xue, and D. Cai. 2019. “Domain Adaptation for Semantic Segmentation with Maximum Squares Loss.” In the Proceedings of the IEEE/CVF International Conference on Computer Vision, 2090–2099. Seoul, Korea (South), October–November 27 2.

 Google Scholar

Lu, X., Y. Zhong, Z. Zheng, and J. Wang. 2021. “Cross-Domain Road Detection Based on Global-Local Adversarial Learning Framework from Very High Resolution Satellite Imagery.” Isprs Journal of Photogrammetry & Remote Sensing 180:296–312.

 Web of Science ®Google Scholar

Tarvainen, A., and H. Valpola. 2017. “Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results.” In the Advances in Neural Information Processing Systems, December 4–9, Long Beach, CA, USA.

 Google Scholar

Ouali, Y., C. Hudelot, and M. Tami. 2020. “Semi-Supervised Semantic Segmentation with Cross-Consistency Training.” In the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Online, June 14–19, pp. 12674–12684.

 Google Scholar

Data availability statement

The data that support the findings of this study are available at https://github.com/xiaoyan07/GRNet_GRSet from the author [X. Lu] and the corresponding author [Y. Zhong] ([email protected]), upon reasonable request.