Assessing OSM building completeness for almost 13,000 cities globally

Figures & data

Figure 1. Illustrating the assessment (a, b, c, and d) and evaluation (e and f) approaches with schematic maps.

Table 1. The confusion matrix for comparing between estimated and reference OSM building completeness*.

Type	I'	II'	III'	IV'	UA
I	N1	N2	–	–	UAI
II	N3	N4	–	–	UAII
III	–	–	N5	N6	UAIII
IV	–	–	N7	N8	UAIV
PA	PAI′	PAII′	PAIII′	PAIV'	OA

*Note: N denotes the number of grid cells; UA denotes user accuracy; PA denotes producer accuracy; OA denotes overall accuracy.

Table 2. A description of reference building data.

Study Area(s)	Producer	Year	Website
England	Ordnance Survey	2020	https://osdatahub.os.uk/downloads/open/OpenMapLocal
France	National Institute of Geographic and Forestry Information	2020	https://geoservices.ign.fr/telechargement
New Zealand	Land Information of New Zealand	2020	https://data.linz.govt.nz/layer/101290-nz-building-outlines/
Australia, United States, Canada, Uganda, and Tanzania	Microsoft	2018–2021	https://www.microsoft.com/en-us/maps/building-footprints

Figure 2. The estimated OSM building completeness of 12,975 cities worldwide, in terms of (a) city scale and (b) national scale.

Figure 3. The estimated OSM building completeness of 15 typical cities worldwide.

Figure 4. Relationships between estimated and reference OSM building completeness values for eight different countries.

Figure 5. Distributions of the difference between estimated and reference OSM building completeness for eight different countries.

Figure 6. Confusion matrixes for eight different countries.

Figure 7. Illustrating the reasons for the low accuracy of Type III, by comparing (a, b) Google Earth images, (c, d) OpenStreetMap (OSM) building data, (e, f) reference building data, and (g, h) population grid data.

Figure 8. Relationship between the number of sampled grid cells and overall accuracy.

Table 3. Confusion matrixes for 10,000 sampled grid cells globally.

Type	I'	II'	III'	IV'	UA (%)
I	2,070	587	0	0	77.9
II	933	4,087	0	0	81.4
III	0	0	25	300	7.7
IV	0	0	16	1,982	99.2
PA (%)	68.9	87.4	61.0	86.9	OA (%) = 81.6

Figure 9. Illustrating the flaw of using the count ratio for assessing OSM building completeness.

Table 4. Relationships between the estimated and reference OSM building completeness using the area ratio and count ratio.

Country	Area ratio	Count ratio
England	y = 0.98x + 0.00 (R^2= 0.950)	y = 2.13x − 0.43 (R^2= 0.781)
France	y = −0.05x + 1.02 (R^2= 0.009)	y = −0.24x + 1.20 (R^2= 0.020)
New Zealand	y = 0.92x − 0.03 (R^2= 0.997)	y = 1.55x − 0.47 (R^2= 0.980)
Australia	y = 1.04x − 0.01 (R^2= 0.995)	y = 1.23x − 0.13 (R^2= 0.979)
United States	y = 1.09x − 0.02 (R^2= 0.977)	y = 1.41x − 0.22 (R^2= 0.892)
Canada	y = 1.21x − 0.02 (R^2= 0.968)	y = 1.97x − 0.54 (R^2= 0.836)
Uganda	y = 1.32x − 0.00 (R^2= 0.938)	y = 1.78x − 0.13 (R^2= 0.838)
Tanzania	y = 1.03x + 0.01 (R^2= 0.998)	y = 1.70x − 0.10 (R^2= 0.870)

Figure 10. Flaws of using a smaller threshold (zero) for the population count. “NoData” represents areas that were mapped as unsettled (Bondarenko et al. 2020).

Bondarenko, M., D. Kerr, A. Sorichetta, and A. J. Tatem. 2020. “Data from: Census/projection-disaggregated gridded population datasets for 189 countries in 2020 using Built-Settlement Growth Model (BSGM) outputs” (dataset). WorldPop, University of Southampton, UK. Accessed December 11, 2022. https://hub.worldpop.org/doi/10.5258SOTON/WP00684.

 Google Scholar

Data availability

The data that support the findings of this study are openly available in figshare at https://figshare.com/s/8d4d92388cff90ed7f9f.