Network-wide traffic signal control based on the discovery of critical nodes and deep reinforcement learning

References

• Abdulhai, B., Pringle, R., & Karakoulas, G. J. (2003). Reinforcement learning for true adaptive traffic signal control. Journal of Transportation Engineering, 129(3), 278–285.
   Web of Science ®Google Scholar
• Aziz, H. A., Zhu, F., & Ukkusuri, S. V. (2018). Learning-based traffic signal control algorithms with neighborhood information sharing: An application for sustainable mobility. Journal of Intelligent Transportation Systems, 22(1), 40–52.
   Web of Science ®Google Scholar
• Balaji, P. G., German, X., & Srinivasan, D. (2010). Urban traffic signal control using reinforcement learning agents. IET Intelligent Transport Systems, 4(3), 177–188.
   Web of Science ®Google Scholar
• Batista, R. D. A., & Bazzan, A. L. C. (2015). Identification of central points in road networks using betweenness centrality combined with traffic demand. Polibits, (52), 85–91.
   Google Scholar
• Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. In P. H. Enslow & A. Ellis (Eds.) WWW 1998. Proceedings of the 7th International Conference on World Wide Web (pp. 107–117). Brisbane, Australia: Elsevier Science Publishers B. V.
   Google Scholar
• Casas, N. (2017). Deep deterministic policy gradient for urban traffic light control. arXiv preprint arXiv:1703.09035.
   Google Scholar
• Cools, S. B., Gershenson, C., & D’Hooghe, B. (2013). Self-organizing traffic lights: A realistic simulation. In M. Prokopenko (Ed.) Advances in applied self-organizing systems (pp. 45–55). London: Springer.
   Google Scholar
• El-Tantawy, S., Abdulhai, B., & Abdelgawad, H. (2014). Design of reinforcement learning parameters for seamless application of adaptive traffic signal control. Journal of Intelligent Transportation Systems, 18(3), 227–245.
   Web of Science ®Google Scholar
• Gao, J., Shen, Y., Liu, J., Ito, M., & Shiratori, N. (2017). Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network. arXiv preprint arXiv:1705.02755.
   Google Scholar
• Genders, W., & Razavi, S. (2016). Using a deep reinforcement learning agent for traffic signal control. arXiv preprint arXiv:1611.01142.
   Google Scholar
• Grégoire, P. L., Desjardins, C., Laumônier, J., & Chaib-Draa, B. (2007, September). Urban traffic control based on learning agents. In D. J. Dailey (Ed.), ITSC 2007. Proceedings of the 10th International IEEE Conference on Intelligent Transportation Systems (pp. 916–921). Seattle, WA: IEEE.
   Google Scholar
• Hausknecht, M., & Stone, P. (2015). Deep recurrent q-learning for partially observable MDPS. CoRR, abs/1507.06527.
   Google Scholar
• Jeon, H., Lee, J., & Sohn, K. (2017). Artificial intelligence for traffic signal control based solely on video images. Journal of Intelligent Transportation Systems, 22, 1–13.
   Web of Science ®Google Scholar
• Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2017). Deep learning for video game playing. arXiv preprint arXiv:1708.07902.
   Google Scholar
• Khamis, M. A., & Gomaa, W. (2014). Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework. Engineering Applications of Artificial Intelligence, 29, 134–151.
   Web of Science ®Google Scholar
• Kirkley, A., Barbosa, H., Barthelemy, M., & Ghoshal, G. (2018). From the betweenness centrality in street networks to structural invariants in random planar graphs. Nature Communications, 9(1), 2501.
   PubMedGoogle Scholar
• Kuyer, L., Whiteson, S., Bakker, B., & Vlassis, N. (2008). Multiagent reinforcement learning for urban traffic control using coordination graphs. In W. Daelemans & K. Morik (Eds.), ECML PKDD 2008. Proceedings of the Machine Learning and Knowledge Discovery in Databases 2008 (pp. 656-671). Antwerp, Belgium: Springer.
   Google Scholar
• Lämmer, S., Gehlsen, B., & Helbing, D. (2006). Scaling laws in the spatial structure of urban road networks. Physica A: Statistical Mechanics and Its Applications, 363(1), 89–95.
   Web of Science ®Google Scholar
• Lample, G., & Chaplot, D. S. (2017). Playing FPS games with deep reinforcement learning. In S. P. Singh, & S. Markovitch (Eds.), AAAI-17. Proceedings of the 31th AAAI Conference on Artificial Intelligence (pp. 2140–2146). San Francisco, CA: AAAI.
   Google Scholar
• Lu, S., Liu, X., & Dai, S. (2008, June). Incremental multistep Q-learning for adaptive traffic signal control based on delay minimization strategy. In Y. Sun, X. Guo, J. He, et al. (Eds.), WCICA 2008. Proceedings of the 7th World Congress on Intelligent Control and Automation (pp. 687–691). Chongqing, China: IEEE.
   Google Scholar
• Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., … Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
   PubMed Web of Science ®Google Scholar
• Mousavi, S. S., Schukat, M., Corcoran, P., & Howley, E. (2017). Traffic light control using deep policy-gradient and value-function based reinforcement learning. arXiv preprint arXiv:1704.08883.
   Google Scholar
• Niu, S. Y., Li, B., Niu, W. J., Zhang, J. S., & Liu, W. F. (2015). Evaluation of highway network node importance via node benefit function and weighted node betweenness. In CICTP 2015 (pp. 2182–2191).
   Google Scholar
• Ozan, C., Baskan, O., Haldenbilen, S., & Ceylan, H. (2015). A modified reinforcement learning algorithm for solving coordinated signalized networks. Transportation Research Part C: Emerging Technologies, 54, 40–55.
   Web of Science ®Google Scholar
• Park, K., & Yilmaz, A. (2010, April). A social network analysis approach to analyze road networks. Paper presented at ASPRS 2010 Annual Conference, San Diego, CA.
   Google Scholar
• Pham, T. T., Brys, T., Taylor, M. E., Brys, T., Drugan, M. M., Bosman, P. A., & Steenhoff, D. (2013, May). Learning coordinated traffic light control. Paper presented at the 11th Adaptive and Learning Agents workshop (at AAMAS-13), St. Paul, MN, USA.
   Google Scholar
• Qian, Y., Wang, B., Xue, Y., Zeng, J., & Wang, N. (2015). A simulation of the cascading failure of a complex network model by considering the characteristics of road traffic conditions. Nonlinear Dynamics, 80(1–2), 413–420.
   Web of Science ®Google Scholar
• Richter, S., Aberdeen, D., & Yu, J. (2007). Natural actor-critic for road traffic optimisation. In J. C. Platt, D. Koller, Y. Singer, & S. T. Roweis (Eds.), NIPS 2007. Proceedings of the Advances in neural information processing systems (pp. 1169–1176). Vancouver, B.C., Canada: Curran Associates.
   Google Scholar
• Scardoni, G., & Laudanna, C. (2013). Identifying critical road network areas with node centralities interference and robustness. In In R. Menezes, A. Evsukoff & M. C. González (Eds.), Complex networks (Vol. 424, pp. 245–255). Berlin: Springer.
   Google Scholar
• Steingrover, M., Schouten, R., Peelen, S., Nijhuis, E., & Bakker, B. (2005, October). Reinforcement learning of traffic light controllers adapting to traffic congestion. In K. Verbeeck, K. Tuyls, A. Nowé, B. Manderick & B. Kuijpers (Eds.), BNAIC 2005, Proceedings of the 17th Belgium-Netherlands Conference on Artificial Intelligence (pp. 216–223). Brussels, Belgium: KVAB.
   Google Scholar
• van der Pol, E., & Oliehoek, F. A. (2016). Coordinated deep reinforcement learners for traffic light control. Paper presented at the NIPS 2016 Workshop on Learning, Inference and Control of Multi-Agent Systems, Barcelona, Spain.
   Google Scholar
• Walraven, E., Spaan, M. T., & Bakker, B. (2016). Traffic flow optimization: A reinforcement learning approach. Engineering Applications of Artificial Intelligence, 52, 203–212.
   Web of Science ®Google Scholar
• Li, D., Jiang, Y., Rui, K., & Havlin, S. (2014). Spatial correlation analysis of cascading failures: congestions and blackouts. Scientific Reports, 4(4), 5381.
   Google Scholar
• Li, L., Lv, Y., & Wang, F. Y. (2016). Traffic signal timing via deep reinforcement learning. IEEE/CAA Journal of Automatica Sinica, 3(3), 247–254.
   Google Scholar
• Wu, J. J., Gao, Z. Y., & Sun, H. J. (2007). Effects of the cascading failures on scale-free traffic networks. Physica A: Statistical Mechanics and Its Applications, 378(2), 505–511.
   Web of Science ®Google Scholar
• Wu, J. J., Gao, Z. Y., Sun, H. J., & Huang, H. J. (2006). Congestion in different topologies of traffic networks. Europhysics Letters, 74(3), 560.
   Google Scholar
• Xu, M., Wu, J., Liu, M., Xiao, Y., Wang, H., & Hu, D. (2018). Discovery of critical nodes in road networks through mining from vehicle trajectories. IEEE Transactions on Intelligent Transportation Systems. doi:10.1109/TITS.2018.2817282
   PubMed Web of Science ®Google Scholar
• Zou, Z., Wu, J., Gao, J., & Xu, X. (2014). Cascade defense in urban road network by inserting modular topologies. Kybernetes, 43(5), 750–763.
   Web of Science ®Google Scholar