Deep Q learning-based traffic signal control algorithms: Model development and evaluation with field data

Abstract

To contend traffic congestion on urban networks, existing studies have made great efforts to develop traffic-responsive signal timing algorithms in the last decade. More recently, as an alternative to conventional model-based algorithms, machine learning-based methods have been tested on traffic light timing problems and show promising potentials. However, many researchers and practitioners still questioned the feasibility and applicability of adopting machine learning techniques in the ATSC domain. One of the reasons is that these methods assumed flawless detectors and heavily relied on simulators for training and evaluations. To address such a critical concern, this article customizes a Deep Q-learning Learning (DQL) method to optimize traffic signal timings at urban intersections, where the partial observations from identity-based detectors are inputs, and the green splits are outputs. A simulation-free data-driven prediction model is also developed to train the DQL with reduced computational time. Then the machine learning-based methods are evaluated on a real-world case with Automatic Number-Plate Recognition (ANPR) data. Experiments show the proposed data-driven model can predict the traffic state in limited computational time, and the DQL algorithm is 3.9% better than the field experiment performance from the adaptive control system, SCOOT, and 22% better than the time-of-day plan by SYNCHRO. The results indicate the DQL methods can only yield marginal improvement with restrictive input and output settings in congested traffic flow in comparison to the conventional adaptive method.

Keywords:

• deep neural network
• deep reinforcement learning
• performance evaluation
• Q-learning
• traffic signal timing

Acknowledgments

The authors would like to thank the Police Department of Suzhou Industrial Park, China for providing field Automatic Number-Plate Recognition data, and the fourth author would like to gratefully acknowledge the support of NVIDIA Corporation for the donation of the Titan X Pascal GPUs in speeding-up the neural network training in this research.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author, YY. The data are not publicly available due to their containing information that could compromise the privacy of research participants.