A human-machine adversarial scoring framework for urban perception assessment using street-view images

ABSTRACT

Though global-coverage urban perception datasets have been recently created using machine learning, their efficacy in accurately assessing local urban perceptions for other countries and regions remains a problem. Here we describe a human-machine adversarial scoring framework using a methodology that incorporates deep learning and iterative feedback with recommendation scores, which allows for the rapid and cost-effective assessment of the local urban perceptions for Chinese cities. Using the state-of-the-art Fully Convolutional Network (FCN) and Random Forest (RF) algorithms, the proposed method provides perception estimations with errors less than 10%. The driving factor analysis from both the visual and urban functional aspects demonstrated its feasibility in facilitating local urban perception derivations. With high-throughput and high-accuracy scorings, the proposed human-machine adversarial framework offers an affordable and rapid solution for urban planners and researchers to conduct local urban perception assessments.

KEYWORDS:

• Street view
• urban perception
• deep learning
• urban planning
• human-machine adversarial scoring

Acknowledgments

We fully appreciate Prof. May Yuan, Prof. Robert Weibel and the anonymous reviewers for their helpful comments and suggestions. We are also very grateful to the volunteers who helped us with this work. This study was supported by the National Key R&D Program of China [Grant No. 2017YFB0503800]; the National Natural Science Foundation of China [Grant No. 41801306, 41671408]; the Open Fund of State Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University [Grant No. 18S01] and by the Fundamental Research Founds for National University, China University of Geosciences (Wuhan) [Grant No. CUG190606].

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. The ADE-20K is an open data set that can be downloaded from the MIT website (http://groups.csail.mit.edu/vision/datasets/ADE20K/).

2. Official website of the MIT ADE20K dataset: http://groups.csail.mit.edu/vision/datasets/ADE20K/.

3. The 8 administrative districts are as follows: Wuchang, Hongshan, Jiang’an, Qiaokou, Hanyang, Jianghan, Qingshan and Dongxihu. The residential population of each district is 1,178 million, 1,107 million, 755 million, 723 million, 673 million, 661 million, 502 million and 374 million, respectively.

Additional information

Funding

This work was supported by the National Natural Science Foundation of China [41671408,41801306]; Fundamental Research Founds for National University, China University of Geosciences(Wuhan) [CUG190606]; Open Fund of State Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing,Wuhan University [18S01]; National Key R&D Program of China [2017YFB0503804].

Notes on contributors

Yao Yao

Yao Yao is an Associate Professor at China University of Geosciences (Wuhan) and Senior Algorithm Engineer at Alibaba Group. His research interest is spatio-temporal big data mining and urban comupting.

Zhaotang Liang

Zhaotang Liang obtained his master's degree from the Chinese University of Hong Kong. Now he is a research assistant at the Chinese University of Hong Kong. His research interest is geospatial big data analysis.

Zehao Yuan

Zehao Yuan is a master student at China University of Geosciences (Wuhan). His research interest is urban function analysis using social media data.

Penghua Liu

Penghua Liu is a master student at Sun Yat-sen University. His research interest is geospatial data analysis and modeling.

Yongpan Bie

Yongpan Bie is a master student at China University of Geosciences (Wuhan). His research interest is Spatial data visualization and analysis.

Jinbao Zhang

Jinbao Zhang is a PhD. candidate at Sun Yat-sen University and Visiting scholar at Tencent Technology Inc. His research interest is urban computing and modeling using social media data.

Ruoyu Wang

Ruoyu Wang is a PhD. candidate at University of Edinburgh and research assistant at Sun Yat-sen University. His research interest is health geography.

Jiale Wang

Jiale Wang is a master student at Wuhan University. His major is cartography and geographic information systems.

Qingfeng Guan

Qingfeng Guan is a Professor at China University of Geosciences (Wuhan). His research interest is high-performance spatial intelligence computing and spatio-temporal data mining.