An Interactive platform for low-cost 3D building modeling from VGI data using convolutional neural network

Figures & data

Figure 1. Framework of our platform. In the user interaction module, users provide the facades boundaries, roof type, and facade views of the building in images to our web platform. Then, images are fed into facade elements extraction module to automatically obtain the locations of the windows, doors and balconies. In the 3D building modeling step, the locations are transformed from 2D to 3D space. Then locations are rotated according to the information of facades views. Finally, the 3D building model is built and visualized to users

Figure 2. The process of 3D transformation. First, the 2D locations of every facade and its facade elements are extruded using ExtrudeGeometry method from Three.js, to obtain the corresponding 3D locations group of the facade in (b); Then, Every 3D locations group is rotated using Equations (7(7) $An{g}_r\left(D\right)=\left\{\begin{array}{c}0D=front\\ -\begin{array}{c}\pi /2D=left\end{array}\\ \begin{array}{c}+\pi /2D=right\end{array}\\ \begin{array}{c}\pi D=back\end{array}\end{array}\right.$(7) )~(8(8) $M_r=\left[\begin{array}{ccc}cosAn{g}_r\left(D\right)& 0& sinAn{g}_r\left(D\right)\\ 0& 1& 0\\ -sinAn{g}_r\left(D\right)& 0& cosAn{g}_r\left(D\right)\end{array}\right]$(8) ) based on the facade direction from user interaction step; Finally, the 3D building model is obtained

Figure 3. Visualization results for extracting facade elements locations (Containing different views, lighting, and architectural styles)

Figure 4. Qualitative result of 3D building modeling. The first and third columns are the original images of every building. The second and fourth columns are the 3D modeling results of every building through our workflow

Table 1. Statistic modeling results of the 10 buildings

Building No.	1		2		3		4		5
	Pc	Rc	Pc	Rc	Pc	Rc	Pc	Rc	Pc	Rc
window	19/21	19/21	18/19	18/19	18/19	18/18	10/12	10/10	66/66	66/68
balcony	3/3	3/4	0/0	0/1	-	-	-	-	-	-
door	-	-	1/1	1/1	1/1	1/1	1/1	1/2	2/5	2/2
Inc	22/25		19/21		19/19		11/12		68/70
Building No.	6		7		8		9		10
	Pc	Rc	Pc	Rc	Pc	Rc	Pc	Rc	Pc	Rc
window	4/6	4/4	10/12	10/10	2/2	2/3	22/22	22/23	13/16	13/13
balcony	3/3	3/3	10/10	10/10	-	-	19/19	19/20	2/2	2/4
door	0/0	0/1	3/3	3/4	1/1	1/1	1/1	1/1	0/0	0/1
Inc	7/8		23/24		3/4		42/44		15/18

Figure 5. Results of our platform. (a) Interface of our platform. (b) Result of facade elements extraction, which is a middle part of our workflow. (c) Final result of 3D building modeling shown on our website

Data Availability

The FacadeWHU dataset that supports this study is openly available at: https://drive.google.com/drive/folders/1BeKXMuNAaRcwP6lSeiqFOSfscbP9ZKm7?usp=sharing.