A review of urban physical environment sensing using street view imagery in public health studies

ABSTRACT

Urban physical environments are the physical settings and built environments in neighbourhoods and cities which provide places for human activities. Evidence suggests that there are substantial associations between urban physical environments and various health outcomes, e.g. people’s physical activities might be influenced by surrounding physical environments, thereby affecting their health behaviours; more exposure to urban physical environments may benefit human mental health. Street view imagery enables us to capture the landscape at eye-level, making it a promising data source for observing and analysing the realistic dynamics of urban physical environments. Compared with traditional in-person assessments and field observations, street view imagery-based data collection is relatively time-effective and cost-effective. Researchers from epidemiology, psychology, and geography have used street view imagery to quantify the built environment and understand its impacts on public health. To summarize current research trends, this paper systematically reviews the use of street view images for sensing urban environments in public health studies. Specifically, we describe the characteristics of street view imagery and review the methodology for image processing and semantic understanding. We then summarize the challenges that remain for quantifying urban environments in terms of data and methodology. Several future research directions that would benefit public health research and practices are recommended in urban environment research.

KEYWORDS:

• Street view imagery
• urban environment
• public health

1. Introduction

The past few decades have witnessed the rapid development of cities around the world. More than 50% of the world’s population and 70–80% of economic activities take place in urban areas, which not only brings opportunities to residents’ work and lives but also brings challenges for sustainable urban development (Schneider, Friedl, and Potere 2010; Lu, Sarkar, and Xiao 2018). The urban physical environment plays an important role in affecting residents’ physical activities and social life. People prefer to live in nice places and spend most of their time in interactions in neighbourhood environments (Saelens and Handy 2008; Wang et al. 2019b; Pliakas et al. 2017; Zhang et al. 2020). Effectively measuring and quantifying the urban environment is therefore crucial in improving citizens’ satisfaction with their living environments, promoting sustainable city development, and modelling human-environment interactions (Knox, Marston, and Imort 2016; Su et al. 2017; Wang 2020).

Previous literature has shown a strong connection between the urban physical environment and various health behaviours of its citizens (Naik et al. 2014; Helbich et al. 2019; Svoray et al. 2018; McMichael 2000; Lopez and Hynes 2006; Leslie and Cerin 2008; Leon 2008; Sun et al. 2020). As such, researchers in epidemiology, psychology, and geography have increasingly examined the effects of the built environment on various health outcomes (Ulrich 1979, 1984; Abraham, Sommerhalder, and Abel 2010; Kaplan 1995; Hartig et al. 2014; Subiza-Pérez, Vozmediano, and San Juan 2019). For example, researchers have investigated the associations between urban environment and different human physical behaviours (such as walking and cycling preferences) (Sallis et al. 2012; Lee and Buchner 2008; Li et al. 2005). Urban environment indeed benefits human health (e.g. reduces obesity and chronic diseases) through influencing human physical behaviours. Previous studies have also examined the role of subjective perceptions and feelings about urban environments in mental health, such as whether a safer, greener or more beautiful environment can ease depression and stress (Ulrich et al. 1991; Galea et al. 2005; Berman, Jonides, and Kaplan 2008).

Despite the successes of previous studies, this area of research still faces challenges. A comprehensive observation and representation of the urban environment is a prerequisite in these experiments. By detecting and understanding the elements and scenes of built environments, researchers are able to investigate the relationships between physical environments and health outcomes quantitatively. However, most studies rely on in-person assessments or field observations to collect data about built environment characteristics (Wilson et al. 2012; Rzotkiewicz et al. 2018). Such conventional data collection approaches have several challenges, including high labour intensity, long update period, restriction to small-scale geographic areas, etc. (Golder and Macy 2011; Seresinhe, Preis, and Moat 2015; Lu 2019). New paradigms should be proposed to solve these problems and guide studies on sensing the urban physical environment (Rathore et al. 2016; Janowicz et al. 2019).

Street view images represent real-world scenery including natural scenes and man-made landscapes from a pedestrian perspective and allow users to navigate the realistic streetscape remotely. The emergence of such a big data source provides unprecedented opportunities in digitalizing the world, which enables researchers to investigate the physical urban environment and human activities on a large scale. By sensing the elements and scenes captured by street view images, the urban physical environment can be modelled more accurately and comprehensively, and the connection between the built environment and various health outcomes can be established. In addition, the success of technological advancements such as computer vision and deep learning algorithms enables researchers to understand neighbourhood environments through images (Badland et al. 2010; Ibrahim, Haworth, and Cheng 2020). Existing studies have attempted to use street view images to measure neighbourhood attributes such as safety (Porzi et al. 2015), housing price (Law, Paige, and Russell 2019), greenness (Li et al. 2015c), and population demographics (Gebru et al. 2017). Results of these studies demonstrate that street view imagery can be used as a reliable data source for observing the built environment and for examining the associations between built environment and health outcomes.

In this paper, we present a comprehensive review of existing studies on measuring and quantifying the urban physical environment using street view images for public health. To do so, we performed a keyword search via the Scopus database including the terms ‘street view’, ‘Google street view’, ‘street view images’, and ‘health’, ‘walking behaviors’, etc. Papers with high-relevance were retrieved at first. We examined these papers carefully and tracked other related manuscripts from these papers. Studies that met the following criteria were included in the review: (1) Data sources: street view images are utilized as data sources in sensing urban built environment. (2) Study subject: the observed physical environment pertained to health outcomes (physical or mental aspects), health policy makings, or injury assessments. (3) Time of publication: before 1 January 2020. (4) Language: articles written in English. By doing so, 64 papers in total are included in this work.

In the review, we first categorize methods developed for street view image-based urban environment sensing as being at element level or at scene level. After that, we identify the health study areas that have been intensively examined using street view images in sensing the urban environment. In doing so, we show the potential of street view images as input in examining the relationship between the built environment and human health. The remainder of this review is organized as follows: Section 2 introduces the basic characteristics of street view images. Section 3 examines the quantitative methods used in analysing street view images. Section 4 surveys the recent applications in sensing the physical environment for health research. Section 5 discusses the limitations and opportunities for using street view images in health research. Section 6 presents the conclusions of this paper.

2. Street view images

Street view images refer to the scenery images that describe the urban streetscape with an angle of view similar to that of human vision. This kind of dataset can be accessed from web mapping services. Users are allowed to navigate the realistic scenery remotely through these interactive panoramic images as if they were travelling in the real world (Anguelov et al. 2010; Badland et al. 2010). Google Street View (GSV), launched in 2007, was the first service provider and is currently the most popular one. A great number of other mapping companies around the world now provide similar services, such as Bing maps, Tencent maps, Baidu maps, and Yandex maps.

In order to collect such images, GPS equipped, drive-by sensing vehicles take panorama photos along road networks. By doing so, the coordinates are attached to each panorama photo. Researchers can thereby request and position various angles of view of the physical environment from online map services. For example, Figure 1 shows (a) the images from different angles that were used to construct the panorama image and (b) the panorama image. As shown in the images, the street view images capture objects in the urban streetscape at about eye-level.

Figure 1. Two forms of street view imagery: (a) natural view; (b) panoramic view.

Compared with traditional data sources, street view images have some inherent characteristics that render them as valuable data sources for urban environment auditing: high coverage and volume, relatively low data bias, cost-effectiveness and time effectiveness and eye-level scenery (Wilson, Thomson, and Edwards 2015; Rzotkiewicz et al. 2018; Ibrahim, Haworth, and Cheng 2020). The following paragraphs discuss these advantages, respectively.

First, in terms of the coverage and volume, large-scale street view images have been generated and stored by map service providers around the world. Google street view images have covered most cities in nearly 200 countries; Baidu and Tencent street view images have covered over 400 cities in China. The high coverage of street view imagery not only provides abundant information to characterize the urban environment comprehensively but also enables researchers to evaluate and measure different urban scenery across countries on a global scale (Rundle et al. 2011; Yao et al. 2019; Zhang et al. 2019b).

Second, compared with volunteered geographic information (VGI) (Goodchild 2007) and more broadly user-generated content (UGC) (Krumm, Davies, and Narayanaswami 2008), street view images have relatively higher quality and lower data bias in the urban environment. As a promising data source in the era of big data, all street view images follow the same standard rules and production workflow and have uniform formats with a maximum image size of 2048*2048 pixels,¹ which can guarantee the quality of images to a certain degree. Also, due to the fact that: (1) street view images are collected through uniform sampling along the streets in urban environments; (2) adjacent pictures may overlap in the description of scenery; (3) most regions in urban areas are covered by the street view imagery, there is less data bias when using street view images (Liying Zhang, 2019).

Third, street view images are low cost, easy to access, and easy to download. By using the API services provided by mapping companies, users can download freely available images in a short time. When requesting images, users customize the parameters of the images, including the location, time, camera angle, image size, etc., to collect street view images that fit their needs. Thus, it is possible for researchers to assess neighbourhoods in a wide range of global contexts (Zeng et al. 2018; Li et al. 2015c).

Last but not least, street view images can capture the objective urban scenery from a human perspective. Different from aerial photographs and remote sensing images that take photos from an overlooking view, street view images follow human perception to depict the urban environment by using a view angle similar to that of the human eye and capture detailed objects in built environment more comprehensively (Li et al. 2015c; Wang et al. 2019a; Liying Zhang 2019). In addition, the information contained in street view images enables the exploration of intangible aspects of urban life and people’s perceptions of the environment (Zhang et al. 2018b).

3. Methods for built environments representation

3.1. Built environments representation: elements and scenes

To examine the associations between urban physical environment and various health outcomes, it is necessary to accurately observe, model, and represent the urban environment. To do so, elements and scenes in the urban physical environment should first be sensed and quantified.

Detecting and extracting spatial objects, items, elements, and calculating specific indices of elements from the urban environment is termed as element level observation, as shown in Figure 2. The objects such as trees (Donovan et al. 2015), waterbodies (Prüss 1998), smoke-free signage (Wilson and Thomson 2015; Wilson, Thomson, and Edwards 2015), traffic signs (Hazelhoff, Creusen, and de With 2014), zebra crosswalks (Ahmetovic et al. 2015), etc., have been detected and identified in modelling the urban physical environment. Based on the elements and objects identified, several environmental attributes can be inferred and used to measure neighbourhood attributes. A set of indices about the urban environment are defined as well. For example, researchers have calculated the green view index (Yang et al. 2009; Li et al. 2015c; Long and Liu 2017), sky view factor (Li, Ratti, and Seiferling 2018a), tree view factor (Gong et al. 2018), and street canyons (Chan et al. 2002; Hu et al. 2020) in cities. All these measurements reflect various physical attributes of the urban environment which potentially contribute to diverse health outcomes.

Figure 2. Methods in understanding elements in urban environment.

Figure 3. Methods in understanding scenes in urban environment.

In contrast, having an overall assessment of the entire scene and understanding the underlying semantics is termed as scene level observation, as shown in Figure 3. It is crucial to explore the associations between the environment and health behaviours at the scene level. On one hand, the entire urban physical environment has strong associations with various neighbourhood environment attributes (Figure 3), such as urban land-use type (Dannenberg et al. 2003; Cao et al. 2018; Zhang et al. 2019a), urban environmental quality (Van Kamp et al. 2003; Liu et al. 2017), and environmental aesthetics (Velarde, Fry, and Tveit 2007; Quercia, O’Hare, and Cramer 2014). On the other hand, analysing the scenes of the urban physical environment enables us to assess human response and perception of the built environment. Measuring different aspects of human perception of neighbourhood environments and places, for instance, safe (Vlahov et al. 2007; Li, Zhang, and Li 2015a; Naik et al. 2014), beautiful (Purciel et al. 2009; Dubey et al. 2016; Seresinhe, Preis, and Moat 2015), lively (Zhang et al. 2018b), etc., benefits many health outcomes, especially mental health. Researchers are therefore trying to understand how people perceive the urban environment, which requires a comprehensive description of scenes in the urban physical environment.

In sum, the urban physical environment can be analysed at the element level and at the scene level. The former focuses on objects in the urban environment while the latter pays more attention to the entire scene. Identifying different targets in the urban environment may require different methods. The following two subsections introduce traditional methods and street view imagery-based methods for quantifying and modelling urban physical environment. Researchers are able to examine the relationships between urban physical environment and public health from these two aspects.

3.2. Urban physical environment representation using traditional methods

Traditional studies for environmental health research usually employ self-reports and questionnaires, field surveys, and secondary sources to characterize built environments at element level and at scene level, respectively. Questionnaires and self-reports are arguably the most commonly used sources. With these sources, investigators may ask respondents to evaluate different aspects of neighbourhood attributes (Wang et al. 2019d; De Jesus et al. 2010; Ball et al. 2001). For field observations, researchers are asked to record and describe their urban environment audits by walking or driving around the study areas with pre-defined survey forms (Gullón et al. 2015; Lawlor et al. 2003; Takano, Nakamura, and Watanabe 2002). Secondary sources refer to those pre-defined environmental measures based on spatial analysis and modelling such as spatial accessibility measures (Pliakas et al. 2017; Leslie and Cerin 2008). All three approaches have their strengths and limitations. Questionnaires are the most commonly used method. It might be affected by people’s biases while field-audits are more objective. But both methods are costly, labour-intensive and time-consuming when conducting research with a large sample size or over large regions. Secondary sources might not characterize neighbourhoods in sufficient detail, and might be limited to specific types of environmental exposures (Pliakas et al. 2017).

3.3. Urban physical environment representation using street view images

Apart from the traditional methods for urban physical environment representation at element level and scene level, abundant street view imagery data have enabled researchers to tackle and compute a wide range of spatial elements and scenes in the neighbourhood environment. To do so, two categories of methods are commonly used in urban physical environment representation. The first category employs traditional image processing and computer vision methods to extract shallow visual features which can encode semantic information of street view photos. For example, by iterating pixels, pixel values that represent different colour channels, and colour histogram (Yu and Wang 2016) are calculated to infer environmental attributes (e.g. green view index (Li et al. 2015c)). Shallow visual features such as texture and geometry features (Chacra and Zelek 2016), scale-invariant feature transform (SIFT) (Naik et al. 2014; Kang, Gao, and Liang 2018), are employed for representing urban physical environment features. Several machine learning methods such as mean-shift clustering algorithm and support vector machines (SVM) are used for image classification (Hazelhoff, Creusen, and de With 2014; Liu et al. 2019; Li et al. 2015b). These methods also support derive scene category and calculating attribute of street view imagery and are easy to be employed.

The second category of methods refers to deep learning methods such as deep convolutional neural networks (DCNN). It enables researchers to extract high-dimensional visual features to enhance the understanding of images (LeCun, Bengio, and Hinton 2015). The deep features can be used efficiently to recognize different space categories and scene types (Zhang et al. 2016; Wang et al. 2019e), evaluate place distinctiveness and compare their similarities (Wang et al. 2019e), which provides solid fundamentals for urban physical environment observation. Specifically, compared with traditional methods, DCNN can significantly improve the accuracy of object detection and image classification for street view images. Therefore, it has been widely used for element level observation in urban physical environment. For example, Zhang et al. (2018a) extracted 64-category objects at hierarchical layers from street view images. Lu, Sarkar, and Xiao (2018) have used street view images to assess urban greenness.

In addition, with large-scale pre-trained image data sets (Zhou et al. 2014) and online perception data sets (Li, Zhang, and Li 2015a), it is possible to provide scene level observation to understand the landscape in street view images using deep learning algorithms. Several crowdsourcing databases have been established, for example, online pairwise surveys MIT Place Pulse² (Dubey et al. 2016), StreetSeen³ (Evans-Cowley and Akar 2014). These surveys record and score people’s subjective perceptions of places. Researchers have trained the DCNN model to quantitatively measure different aspects of human perception of the built environment, such as lively (Zhang et al. 2018a), safe (Li, Zhang, and Li 2015a) and aesthetic (Dubey et al. 2016). By using the above-mentioned methods, researchers are able to capture the objects, analyse the scenes, and evaluate related attributes of the urban environment from street view images.

4. Applications of street view imagery for health research

Understanding the relationship between the built environment and various health outcomes has attracted researchers’ attention for decades. Observation and representation of the urban environment is one of the crucial steps in which street view imagery has been successfully applied. By describing and analysing elements and scenes captured by street view images comprehensively, researchers are able to investigate the environment–health relationships quantitatively. For the purposes of this review, we summarize applications that employed street view images to audit the urban physical environment for use in public health research. Specifically, we categorize two types of research based on different health outcomes: physiological-related applications and psychological-related applications.

4.1. Physiological-related applications

Numerous experimental studies have suggested that the built environment can provide substantial physiological benefits. The environment can either affect people’s health indirectly through promoting various forms of health-related physical activities or influence their health directly (Mitchell and Popham 2008; Ulrich 1984; Lee et al. 2012; Mehrabian and Russell 1974). Here, we illustrate how street view images can be used to audit various built environments and assess their suitability for physical activity as well as their direct effect on public health.

4.1.1. Walkable environment

Evidence shows that there are substantial health benefits of walking for adults, as walking behaviours serve as mediators in the relationship between the urban built environment and health outcomes (Lu, Sarkar, and Xiao 2018; Brookfield and Tilley 2016). Many studies have explored the synergistic beneficial effects of physical activities like walking behaviours and the presence of nature (Pretty et al. 2005). Thus, researchers have evaluated the walkability of the urban physical environment using street view images. Existing studies have proved that street view images can identify the presence of walking activity-related objects, and thus can provide effective and reliable measurement of the streetscape context for walking behaviours (Badland et al. 2010; Chudyk et al. 2014). Lu, Sarkar, and Xiao (2018) and Li et al. (2015b) examined the effects of urban greenery on walking behaviours and found that urban green spaces benefit health via walking activities, while Li et al. (2018b) discovered that the street enclosure has different associations with human walking activities among different land-use types. Moreover, street view images are used to evaluate and quantify walkability according to the visual enclosure of a neighbourhood for assessing the health impacts (Wang et al. 2019c; Yin and Wang 2016; Vargo, Stone, and Glanz 2012) on certain groups of people such as older adults (Brookfield and Tilley 2016) and children (Vanwolleghem et al. 2014). Gullón et al. (2015) used street view images to measure residential environments with regards to walking infrastructure and traffic safety.

4.1.2. Cycling behaviours

Cycling is another type of physical activity that has documented health benefits as well as environmental benefits (Lu et al. 2019). Children and adults enjoy incorporating moderate to vigorous physical activity through cycling into their daily routines (Oja et al. 2011). Also, cycling is an environmentally friendly transportation mode with less CO2 emissions and traffic congestion compared to cars (Fraser and Lock 2011). Badland et al. (2010) suggest that street view imagery can audit the built environment for cycling. According to the built environment captured by street view images, environmental factors that influence cycling behaviours and determine the cyclability of roads can be discovered. For example, street greenery is positively associated with cycling behaviours (Lu et al. 2019); Vanwolleghem et al. (2014) used street view images to assess physical environmental characteristics along cycling routes to school. Such studies provide support for decision-making and cycling policies including choosing bicycling locations and building cycling routes (Mertens et al. 2017; Evans-Cowley and Akar 2014).

4.1.3. Obesity and food environment

Existing studies suggest that obesogenic environments related to physical activity and chronic noncommunicable diseases are associated with the built environment of home and work places. (Silva et al. 2015; Ellaway, Macintyre, and Bonnefoy 2005). How obesogenic and food environments are produced or might be dismantled is thereby important for health issues (Lesser et al. 2013; Swinburn et al. 2011). Researchers have used street view images as a promising tool to examine the obesogenic environment that causes weight gain and obesity in people living in cities (Bethlehem et al. 2014; Feuillet et al. 2016), the advertising of unhealthy food and beverages (Egli et al. 2019), and alcohol-related environment (Clews et al. 2016). Roda et al. (2016) investigated the agreement between the perceptions of residents and the objective measurement from street view on obesogenic environmental features and found that agreement varies by different obesogenic environmental features, regions and neighbourhoods. Clarke et al. (2010) used street view images to evaluate recreational facilities and food environment. Silva et al. (2015) used street view images to examine the relationship between obesogenic built environments and physical activities and chronic noncommunicable diseases. Mayne et al. (2018) used street view images to audit physical disorders which might benefit obesogenic research. Yang et al. (2020) used street view images to measure urban greenery and examine their impacts on children’s body weight status.

4.1.4. Physical injury

Analysis of street view images can also provide valuable information for pedestrian and driver safety by revealing the environmental factors associated with car crash data. For instance, Hanson, Noland, and Brown (2013) examined the association of pedestrian and road infrastructure features with crash severity using street view images, which provided suggestions for pedestrian safety. Johnson and Gabler (2015) evaluated real-world crashes with Google Street View photos and found where car drivers may have larger potential for injury.

4.2. Psychology-related applications

Researchers from environmental psychology argue that exposure to nature has psychological benefits (Chen and Chen 2015; Svoray et al. 2018; Berto 2014; Berman, Jonides, and Kaplan 2008). Characteristics of the built environment can either directly affect people’s mental health or indirectly affect it by influencing their perceptions. In this subsection, we introduce how street view images are employed in identifying those characteristics that benefit psychological health.

4.2.1. Mental health

There is empirical evidence that exposure to nature such as greenness and open space has benefits for mental health (Svoray et al. 2018; Kang et al. 2019; James et al. 2015). Researchers have used street view images to examine the relationships between environmental characteristics including walkability (Wang et al. 2019c), greenness (Wang et al. 2019a; Helbich et al. 2019), public open space (Taylor et al. 2011), and human mental health, e.g. depression (Wang et al. 2019c; Helbich et al. 2019), and psychological restorative effects (Quinn et al. 2016). For example, Wang et al. (2019c) explored the association between walkability evaluated from street view images and older adults’ depression and anxiety levels collected from mental health surveys; Wu et al. (2014) reported the association between depressive symptoms and features of built environment related to mental health. Helbich et al. (2019) used street view images to explore the linkage between green and blue spaces and exposure to natural environments, and mental disorders (i.e. depression symptoms) at the street-level among elderly people in China. Similar conclusions are obtained from comparative studies using traditional data collection methods that exposure to nature has positive benefits for mental health. For example, Wang et al. (2019a) discovered that urban greenery as measured by the normalized difference vegetation index (NDVI) has positive impacts on human mental well-being. Hyam (2017) evaluated the psychological restorative effects of exposure to nature by viewing images of nature. Taylor et al. (2011) assessed how the quality of public open spaces might benefit human mental health.

4.2.2. Perception and Sentiment

Human perception and sentiments of the environment reflect how people think about their surroundings. Such perceptions and sentiments, such as safe, beautiful, and lively, may also play important roles in influencing various human mental well-beings (Loukaitou-Sideris and Eck 2007; Parsons 1991; Özgüner, Eraslan, and Yilmaz 2012). Researchers utilized street view images to evaluate and understand how people perceive the built environment. According to Wang et al. (2019b), the perceptions of built environment by older Chinese are significantly related to both of their mental and physical health. Specifically, the perception of neighbourhood safety can affect mental health by reducing psychological stress, improving life satisfaction and physical activities (Won et al. 2016). Porzi et al. (2015) examined human perception and response to the surrounding environment to explore what makes a place look safe. Li, Zhang, and Li (2015a) discovered that the visibility of green vegetation plays an important role in increasing perceived safety in urban areas. Naik et al. (2014) presented a novel computational tool which can accurately predict the safety scores of streetscapes in cities and their impact on its inhabitants. Also, there are extensive studies about how the aesthetics of the environment might influence well-being (Ulrich 1979; Parsons 1991). Suel et al. (2019) quantifies the relationship between environmental aesthetics and human health, as aesthetics of the environment may have quantifiable consequences for our well-being. Seresinhe, Preis, and Moat (2015) demonstrated how we perceive the beauty of environments, and offered insights for policymakers to design and protect built and natural environments. Compernolle et al. (2016) examined the association of perceived and objective measures such as neighbourhood safety, aesthetics, destinations, and functionality, with transport-related, work-related and leisure-time sedentary behaviours.

4.3. Comparison between street view imagery and remote sensing imagery

Before the emergence of street view imagery, aerial photos and remote sensing imagery were the commonly used data sources for observing urban environments. Different from street view images, aerial photos and remote sensing images can provide a bird’s-eye view (overhead view) of the landscape. Researchers have utilized them to analyse urban land use (Herold, Scepan, and Clarke 2002), urban greenness (Gupta et al. 2012), and disaster situations (Peng et al. 2019). A popular index – the normalized difference vegetation index (NDVI) – has been widely used to measure greenery as well (Rouse et al. 1974). As all datasets have pros and cons, we make a comparison between street view images and remote sensing images here to discuss their characteristics in practice.

Both street view imagery and remote sensing imagery have commonalities in describing several specific urban features. For example, Wang et al. (2019a) discovered that streetscape greenery measured using street view images and NDVI measured by remote sensing imagery are associated with positive impacts on human mental well-being; Wang et al. (2018) found both data sources can predict the commercial activeness with satisfactory precision by adding visual patterns.

Street view images may have better performance in some tasks as the profile view captured reflects what people see on the ground in the urban environment (Li et al. 2015c). For instance, the findings of Villeneuve et al. (2018) provide support for the hypothesis that the attributes measured by neighbourhood greenness may play a more important role in promoting participation in recreational physical activity during the summer as compared with the attributes measured by NDVI. Lu et al. (2019) confirmed that eye-level street greenery, but not NDVI, has a positive correlation with a higher probability of cycling. Ben-Joseph et al. (2013) noted that street view images have advantages in measuring fine-grain features such as sidewalks and obstructions. Larkin and Hystad (2019) suggested that street view measurement can better capture dimensions of trees and other types of vegetation from a ground-based view, and therefore can be a potential source for examining health associations and green space.

In addition, several studies argued that street view images and remote sensing images may characterize the urban environment in different aspects. For example, Helbich et al. (2019) found that street view-based and satellite-derived green and blue space measurements represent different aspects of natural environments. According to Larkin and Hystad (2019), the street view-based green space measurement has less correlation with other measurements using remote sensing images, which suggests that street view images capture unique information about green space exposures. Therefore, street view images can supplement remote sensing images for urban land-use classification (Zhang et al. 2017).

5. Limitations and future works

Despite the success of street view images in auditing urban physical environments for health research, several limitations and challenges should be addressed for future work. First, although existing studies have shown the efficiency of using street view images in observing the urban environment (Wu et al. 2014), researchers argue that when identifying details related to certain items including street lights, sidewalk continuity (Chudyk et al. 2014) and specific phenomenon (e.g. alcohol consumption) (Clews et al. 2016), they may be less reliable. Field observations may have superiority in identifying items such as smoke-free signage in streets by allowing more angles to be viewed (Wilson, Thomson, and Edwards 2015). Second, mapping companies do not update their street view images in real time or frequently, and the street view images can only observe the scenery along the streets. Thus, observing changes in urban environments and coverage area may be limited, making it difficult to analyse the variance of urban environment in time series and inside neighbourhoods. In addition, quality issues such as uncleared pixels and the potential of blocked or blurred items may also impact the performance of object detection (e.g. unidentified items, wrong categories) (Egli et al. 2019). Also, it is hard to determine the size of outdoor items (Egli et al. 2019).

Future work with street view imagery is directed to mitigate these issues and is anticipated to facilitate public health on two fronts. First, as the performance of computer vision models gets better, beyond only using static images, it could be possible to capture more detailed human movement and activity information from cameras and videos with high resolution. Second, researchers in computer vision and deep learning are working on parsing and interpreting well-trained neural networks, which are usually considered ‘black boxes’, to figure out what exactly the nets learned and how this information can be transferred to common knowledge. The output will benefit our understanding of how physical environments affect human behaviour, human perception and human health.

6. Conclusion

The emergence of street view images provides a valuable data source for auditing the built environment and enables a better examination of the relationship between the environment and health outcomes. Compared with traditional approaches such as field observation and secondary sources, street view images capture the urban physical environment at eye-level with lower cost and higher efficiency.

In this work, we present a timely review of health research based on street view imagery. The methods are summarized in two aspects, namely the element level and the scene level. Then, the most recent studies about physiological and psychological applications using street view imagery are discussed. We also make a comparison between street view images and remote sensing images with regard to the examination of their associations with health studies. Finally, we discuss the limitations, challenges, and opportunities of this data source and the associated methods. Several future research directions that would benefit public health research and practices are recommended in urban environment research.

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant 41901321, and the Office of Vice Chancellor for Research and Graduate Education at the University of Wisconsin-Madison with funding from the Wisconsin Alumni Research Foundation. Song Gao is partially supported by the National Science Foundation (2027375). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors would like to thank Hanqi Li at Wuhan University for her helpful discussions, as well as Jake Kruse for his help with proofreading and correction for this manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the National Natural Science Foundation of China [41671378; 41901321]; National Science Foundation [2027375]; Wisconsin Alumni Research Foundation [AAC5663].

Notes

1. https://developers.google.com/maps/documentation/streetview/usage-and-billing

2. http://pulse.media.mit.edu/.

3. http://streetseen.osu.edu/studies/bicycling-preferences-in-columbus-ohio/vote