An investigation of the visual features of urban street vitality using a convolutional neural network

Figures & data

Figure 1. Study workflow

Figure 2. Study area

Figure 3. Survey sites

Table 1. Table of the CNN layers

Layer	Type	Output shape	Parameter(s)
1	Conv2D(32)	448, 1792, 32	5 × 5, same padding
2	BatchNorm	448, 1792, 32
3	ReLU	448, 1792, 32
4	Conv2D(32)	444, 1788, 32	5 × 5, valid padding
5	BatchNorm	444, 1788, 32
6	ReLU	444, 1788, 32
7	MaxPooling2D	148, 596, 64	3 × 3
8	Dropout	148, 596, 64	0.3
9	Conv2D(64)	148, 596, 64	5 × 5, same padding
10	BatchNorm	148, 596, 64
11	ReLU	148, 596, 64
12	Conv2D(64)	146, 594, 64	5 × 5, valid padding
13	BatchNorm	146, 594, 64
14	ReLU	146, 594, 64
15	MaxPooling2D	48, 198, 128	3 × 3
16	Dropout	48, 198, 128	0.3
17	Conv2D(128)	48, 198, 128	5 × 5, same padding
18	BatchNorm	48, 198, 128
19	ReLU	48, 198, 128
20	Conv2D(128)	46, 196, 128	5 × 5, valid padding
21	BatchNorm	46, 196, 128
22	ReLU	46, 196, 128
23	MaxPooling2D	15, 65, 256	3 × 3
24	Dropout	15, 65, 256	0.3
25	Conv2D(256)	15, 65, 256	5 × 5, same padding
26	BatchNorm	15, 65, 256
27	ReLU	15, 65, 256
28	Conv2D(256)	13, 63, 256	5 × 5, valid padding
29	BatchNorm	13, 63, 256
30	ReLU	13, 63, 256
31	MaxPooling2D	4, 21, 256	3 × 3
32	Dropout	4, 21, 256	0.3
33	Conv2D(128)	4, 21, 128	1 × 1, valid padding
34	BatchNorm	4, 21, 128
35	ReLU	4, 21, 128
36	Dropout	4, 21, 128	0.3
37	Conv2D(16)	4, 21, 16	1 × 1, valid padding
38	BatchNorm	4, 21, 16
39	ReLU	4, 21, 16
40	Dropout	4, 21, 16	0.3
41	Conv2D(1)	4, 21, 1	1 × 1, valid padding
42	AveragePooling2D	1, 1, 1	4 × 21
43	Flatten	1

Figure 4. Output of layer 41 of the CNN as a heat map

Figure 5. Scene image with heat map overlaid

Table 2. Classification statistics for the top 3.5% features

Order	Feature class	Count	Contribution(per-feature average/total)
1	Vehicles	549	6.78/3723.16
2	Trees	337	6.63/2234.41
3	Buildings	306	6.67/2042.17
4	Roadside/walking pavement	286	6.86/961.06
5	Road signs	211	6.71/1416.58
6	Shopfronts	189	6.92/1307.29
7	Construction site	124	7.09/879.63
8	Fence and wall	92	6.79/624.87
9	Bikes and motorbikes	49	6.07/297.28
10	Signs with text	39	6.60/257.35
11	Human	32	7.39/236.38
12	Other	75	6.60/494.98
Total		2289	6.76/15,475.16

Figure 6. Surveyed street vitality scores

Figure 7. Land use of the study area

Figure 8. Training history of the epochs 2–100

Figure 9. Confusion matrix

Figure 10. Model error spatial distribution

Figure 11. Block heat value (output of layer 41) distribution

Figure 12. Extracted high heat feature blocks (sample)

Figure 13. Spatial distributions of vital visual features including vehicles (a), trees (b), buildings (c), pedestrian spaces (d), shop fronts (e), construction sites (f), and humans (g)

Data availability statement

The data that support the findings of this study are available on request from the corresponding author at http://public.dupetrc.qiyi.us:31382/model_simplednn/model_streetvitality. The data are not publicly available due to the uncensored faces and license plates in the street view images taken by our investigators and used in model training may violate individuals’ privacy.