Special issue on dense surveillance systems for urban traffic

Introduction

Traffic congestion is a serious problem all over the world, especially during morning and evening rush hours in modern cities. Traffic volumes, roadwork, and weather events contribute to traffic congestion. More than 20 percent of it can be attributed to individual incidents such as traffic collisions. Even though most collisions in urban areas do not cause injuries to vehicle occupants due to low driving speeds, drivers have to stop their vehicles and wait for law enforcement to arrive and identify property damage liability, which unavoidably causes traffic congestion. Dense video surveillance systems can address this problem by capturing real-time videos from different viewpoints along roads and at intersections. Law enforcement can identify traffic accident liability through video recordings afterwards, which would allow drivers to drive away immediately following a collision and avoid causing traffic congestion.

Themes of this special issue

In this special issue, we focus on the deployment of dense video surveillance systems and their applications in urban traffic. In the last couple of decades, Artificial Intelligence (AI) has greatly contributed to Intelligent Transportation Systems (ITS), and shows great potential to solve hard problems in traffic video analysis. Thousands of journal and conference papers have investigated how deep learning can analyze and trace traffic trajectories. Alleviating traffic congestion through the application of dense video surveillance systems is a more recent direction. In a dense video surveillance system, camera nodes are placed in a way that ensures full coverage at intersections and along roads. These nodes form a large Wireless Sensor Network (WSN), generating an energy-saving issue. Therefore, there are many studies that evaluate how long camera nodes can robustly survive and work stably. Dense video surveillance systems are quickly gaining interest worldwide, with many active academic and industrial research groups. Last year, near 10 published journals and more than 50 conference papers discussed how video surveillance systems can support ITS.

This special issue is intended to provide a highly-recognized international forum for presenting innovative developments in avoiding urban traffic congestion through dense video surveillance systems. The ultimate objective is to bring together well-focused, top-quality research contributions to promote the visibility and relevance of AI and WSN techniques and to provide the general ITS research community an overall view of recent results and the most promising avenues. The intent is to raise collective awareness of dense video surveillance systems as a highly-promising technology that should be pursued by the ITS research community. This issue will provide a highly-recognized international forum to present recent advances through the Journal of Intelligent Transportation Systems. We welcomed papers considering theoretical contributions, interesting applications, and other aspects such as:

• Wireless sensor network

• Wireless communications

• Video capturing, compression, and transmission

• 3D video compression and analysis

• Multi-view video coding

• Power control

• Energy-saving coding

• Low-complexity video coding

• Video analysis

• Vehicular trajectory analysis and tracing

• Deep learning

• Pattern recognition

• Intelligent transportation

A total of 7 papers out of 14 submissions have been accepted for publication in this issue. Among the accepted papers, 7 are original research articles and the other is a review.

Models

The rapid development of the Internet of Vehicles (IoV) has created new challenges in storing large quantities of vehicle network data and in maintaining retrieval efficiency. The IoV generates a great deal of Global Positioning System (GPS) log data and vehicle monitoring data, but only a small proportion of them are required for reading/writing. Accessing too many small files in the conventional Hadoop Distributed File System (HDFS) creates a series of problems such as high occupancy rates, low access efficiency, and low retrieval efficiency, which degrades the IoV’s performance. In “Storage and access optimization scheme based on correlation probabilities in the internet of vehicles,” Bin et al. propose to tackle these bottleneck problems through a small Files Correlation Probability (FCP) model (Bin et al., 2019). The FCP-based Small Files Merge Scheme (SFMS) and the Small File Prefetching and Caching Strategies (SFPCS) proposed in this paper optimize the storage and access performance of HDFS. Experiments show that the proposed optimization solutions alleviate the problems of high occupancy in HDFS name nodes and low access efficiency compared to the native HDFS read-write scheme and Hadoop ARchive (HAR)-based read-write optimization scheme.

AI tools for traffic video analysis have been widely identified as potential solutions for hard problems in ITS. In order to reach its full potential, AI requires dense camera placement along roads and at intersections in order to monitor all traffic. The captured videos need to be back hauled to the control center and act as the inputs for AI tools. In order to bear such a large data traffic load and cover long transmission ranges, directional communication technology concentrates the energy of the wireless signals in a specified direction to provide high data rates and long transmission ranges (up to tens of kilometers). In the contribution by Yan et al., “An efficient multiple access control protocol for directional dense urban traffic surveillance system,” the Communication Time Extension Problem (CTEP) occurs when directional transmissions are applied to dense urban traffic surveillance systems, and the wireless signal propagation time approximates to the data transmission time (Yan et al., 2019). The CTEP is addressed through a Link Distance Division (LDD) based Time Division Multiple Access (TDMA) protocol. First, the directional wireless communication links are classified according to their link distances, i.e., the nodes located in the same communication ring belong to the same category. Then a Link Distance Aware (LDA) based slot allocation algorithm assigns time slots to the links. The protocol derives the optimal radius of communication rings in a closed-form formula and derives the minimum average link distance. Simulation results show that LDD-TDMA outperforms TDMA by 13.37% when the ring number is 4.

Performance improvements

Infrared and visible images play an important role in ITS since they can monitor traffic conditions around the clock. However, the quality of visible images is susceptible to the environment, and infrared images are not rich enough in detail. Infrared and visible image fusion techniques can fuze these two different modal images into a single image that contains more useful information. In the contribution by Li et al., “Infrared and visible images fusion by using sparse representation and guided filter,” the authors propose an effective infrared and visible images fusion method for traffic systems (Li et al., 2019). First, weight maps are measured by utilizing sparse coefficients. The next step is to decompose the infrared and visible pair into High-Frequency Layers (HFLs) and Low-Frequency Layers (LFLs). Since the two layers contain different structures and texture information, a guided filter optimizes the weight maps in accordance with the different characteristics of the infrared and visible pairs in order to extract a representative component. The final step is to reconstruct the two-scale layers according to the weight maps. Experimental results demonstrate that the proposed method outperforms other popular approaches in terms of subjective perception and objective metrics.

The contribution by Rios-Torres et al., “The extent of reliability for vehicle-to-vehicle communication in safety critical applications: an experimental study,” studies Vehicle-to-Vehicle (V2V) and vehicle collisions issues (Rios-Torres et al., 2020). V2V communication using Dedicated Short Range Communications (DSRC) technology has the potential to drastically reduce vehicle collisions. DSRC allows vehicles in a highly mobile and complex network to send and receive safety messages with higher reliability and lower latency compared to other wireless technologies used for automotive communications. However, there are many factors that could cause communication failures in safety-critical automotive applications. While the reliability of V2V communication has been a subject of study by several researchers, there are still open questions regarding the most effective placement (inside or outside the vehicle) of DSRC devices since interior elements and differences in altitude can affect V2V communication.

The contribution by Cui et al., “Convolutional neural network for recognizing highway traffic congestion,” investigates the use of a deep Convolutional Neural Network (CNN) for recognizing highway traffic congestion states in surveillance camera imagery (Cui et al., 2020). Highway cameras usually have an extensive range of perspective and therefore generate images with large backgrounds, which are different from the usual input images in CNNs for object classification (like ImageNet). The authors built a highway imagery dataset using real-life traffic videos covering a range of road configurations, times of day, and weather and lighting conditions. Each image was labeled with a perceived traffic congestion state. This labeled dataset is used to characterize CNNs for the target congestion state recognition problem. AlexNet and GoogLeNet produced an excellent recognition accuracy of 98% on held-out test samples, with many of the misclassified images being borderline cases. More results indicate that scale and perspective could affect recognition.

Applications

InfraRed (IR) imaging sensors are widely employed in urban traffic systems since they are not affected by lighting conditions. They can work stably around the clock. IR images make the subsequent processing of ITS easier. However, due to hardware and imaging environment limitations, it is difficult to obtain IR images at a desired quality. IR images always lack detailed information, which leads to unsatisfying enhancement results with conventional methods. Compared with IR images, VISible (VIS) images contain detailed information, which could help enhance the quality of the corresponding IR images. Chen et al. proposes an effective method to enhance IR images by applying multi-sensors to images in “A novel infrared image enhancement based on correlation measurement of visible image for urban traffic surveillance systems” (Chen et al., 2019). First, the authors adopt an edge-preserving filter that decomposes the IR and VIS images into their illumination and reflectance components according to the Retinex theory. Second, each region in the IR and VIS images is classified into related and non-related regions according to the correlations between the IR and VIS images. Finally, an Adaptive Fuzzy Plateau HE (AFPHE) enhances the illumination component and a VIS-aided strategy is employed to enhance the detail of the IR reflectance component with the help of the VIS image. Experimental results demonstrate that the proposed method can effectively improve the contrast and enhance the detail of the IR image.

The growing number of surveillance cameras imposes a great demand on high efficiency video coding. Although modern video coding standards have improved coding efficiency significantly, they are designed for general video rather than surveillance video. The special characteristics of surveillance video leave a space for further performance improvement. In the contribution by Ding et al., “A deep learning approach for quality enhancement of surveillance video,” the authors leverage a deep learning approach for the reconstructed frame enhancement in surveillance video compression (Ding et al., 2019). More specifically, the authors formulate the frame enhancement problem as a regression problem and address it through a Convolutional Neural Network (CNN), referred to as Residual Squeeze-and-Excitation Network (RSE-Net). The RSE-Net extensively exploits the non-linear mapping between the reconstructed frame and ground truth with a small number of parameters. Moreover, by improving the You Only Look Once (YOLO) network, the system successfully detects grouped vehicles within a frame. A novel model training scheme is then developed through learning from the grouped vehicles. With the proposed scheme, they train a global model for both the foreground and background of surveillance video. Experimental results show that these methods achieve averages of 0.40 dB, 0.22 dB, and 0.24 dB Peak Signal-to-Noise Ratio (PSNR) gains over the H.265/HEVC anchor under the AI, LDP, and RA configurations, respectively, and produce visually pleasing results when applied to compressed surveillance video.

Conclusion

The articles presented in this special issue provide insights in developing urban traffic monitoring techniques using dense surveillance systems. These contributions include models, performance evaluation and improvements, and application developments. We hope readers will benefit from the insights of these papers, and contribute to these rapidly growing areas. We also hope that this special issue will shed light on major developments in the areas of focus for the Journal of Intelligent Transportation Systems and raise interest in the scientific community to pursue further investigations leading to the rapid implementation of these technologies.

Acknowledgements

We would like to express our appreciation to all the authors for their informative contributions and the reviewers for their support and constructive critiques in making this special issue possible. Finally, we would like to express our sincere gratitude to Professor Asad J. Khattak, the Editor in Chief, for providing us with this unique opportunity to present our works in the Journal of Intelligent Transportation Systems.

Disclosure statement

No potential conflict of interest was reported by the authors.