Cooperative lane-changing in mixed traffic: a deep reinforcement learning approach

References

• Arbis, D., and V. V. Dixit. 2019. “Game Theoretic Model for Lane Changing: Incorporating Conflict Risks.” Accident Analysis & Prevention 125:158–164. https://doi.org/10.1016/j.aap.2019.02.007.
   PubMed Web of Science ®Google Scholar
• Bouadi, M., B. Jia, R. Jiang, X. Li, and Z. Gao. 2022. “Optimizing Sensitivity Parameters of Automated Driving Vehicles in An Open Heterogeneous Traffic Flow System.” Transportmetrica A: Transport Science 18 (3): 762–806. https://doi.org/10.1080/23249935.2021.1896592.
   Web of Science ®Google Scholar
• Calvert, S., and B. van Arem. 2020. “A Generic Multi-level Framework for Microscopic Traffic Simulation with Automated Vehicles in Mixed Traffic.” Transportation Research Part C: Emerging Technologies 110:291–311. https://doi.org/10.1016/j.trc.2019.11.019.
   Web of Science ®Google Scholar
• Deng, J.-H., and H.-H. Feng. 2019. “A Multilane Cellular Automaton Multi-attribute Lane-changing Decision Model.” Physica A: Statistical Mechanics and Its Applications 529:121545. https://doi.org/10.1016/j.physa.2019.121545.
   Web of Science ®Google Scholar
• Dong, C., H. Wang, Y. Li, X. Shi, D. Ni, and W. Wang. 2021. “Application of Machine Learning Algorithms in Lane-changing Model for Intelligent Vehicles Exiting to Off-ramp.” Transportmetrica A: Transport Science 17 (1): 124–150. https://doi.org/10.1080/23249935.2020.1746861.
   Web of Science ®Google Scholar
• Fujimoto, S., H. Hoof, and D. Meger. 2018. “Addressing Function Approximation Error in Actor-Critic Methods.” In International Conference on Machine Learning, 1587–1596. PMLR.
   Google Scholar
• Ha, P., S. Chen, J. Dong, and S. Labi. 2023. “Leveraging Vehicle Connectivity and Autonomy for Highway Bottleneck Congestion Mitigation Using Reinforcement Learning.” Transportmetrica A: Transport Science 19 (1): 1–26. https://doi.org/10.1080/23249935.2023.2215338.
   Google Scholar
• Hou, K., F. Zheng, X. Liu, and Z. Fan. 2023. “Cooperative Vehicle Platoon Control Considering Longitudinal and Lane-changing Dynamics.” Transportmetrica A: Transport Science 20 (3): 1–29. https://doi.org/10.1080/23249935.2023.2182143.
   Web of Science ®Google Scholar
• Jaritz, M., R. De Charette, M. Toromanoff, E. Perot, and F. Nashashibi. 2018. “End-to-End Race Driving with Deep Reinforcement Learning.” In 2018 IEEE International Conference on Robotics and Automation (ICRA), 2070–2075. IEEE.
   Google Scholar
• Ji, J., A. Khajepour, W. W. Melek, and Y. Huang. 2016. “Path Planning and Tracking for Vehicle Collision Avoidance Based on Model Predictive Control with Multiconstraints.” IEEE Transactions on Vehicular Technology 66 (2): 952–964. https://doi.org/10.1109/TVT.2016.2555853.
   Web of Science ®Google Scholar
• Ji, A., and D. Levinson. 2020. “A Review of Game Theory Models of Lane Changing.” Transportmetrica A: Transport Science 16 (3): 1628–1647. https://doi.org/10.1080/23249935.2020.1770368.
   Web of Science ®Google Scholar
• Ji, A., M. Ramezani, and D. Levinson. 2023a. “Joint Modelling of Longitudinal and Lateral Dynamics in Lane-changing Maneuvers.” Transportmetrica B: Transport Dynamics 11 (1): 996–1025.
   Web of Science ®Google Scholar
• Ji, A., M. Ramezani, and D. Levinson. 2023b. “Pricing Lane Changes.” Transportation Research Part C: Emerging Technologies 149:104062. https://doi.org/10.1016/j.trc.2023.104062.
   Web of Science ®Google Scholar
• Jin, J., H. Huang, Y. Li, G. Zhang, Y. Dong, B. Zhou, and H. Xue. 2023. “Variable Speed Limit Modelling to Improve Traffic Safety and Efficiency of Mixed Traffic Flow by a Two-stage Framework.” Transportmetrica A: Transport Science 19 (1): 1–25. https://doi.org/10.1080/23249935.2023.2253476.
   Google Scholar
• Li, L., J. Gan, K. Zhou, X. Qu, and B. Ran. 2020. “A Novel Lane-changing Model of Connected and Automated Vehicles: Using the Safety Potential Field Theory.” Physica A: Statistical Mechanics and Its Applications 559:125039. https://doi.org/10.1016/j.physa.2020.125039.
   Web of Science ®Google Scholar
• Li, Z.-C., H.-J. Huang, and W. H. Lam. 2012. “Modelling Heterogeneous Drivers' Responses to Route Guidance and Parking Information Systems in Stochastic and Time-dependent Networks.” Transportmetrica 8 (2): 105–129. https://doi.org/10.1080/18128600903568570.
   Web of Science ®Google Scholar
• Li, D., and H. Pan. 2022. “Two-lane Two-way Overtaking Decision Model with Driving Style Awareness Based on a Game-theoretic Framework.” Transportmetrica A: Transport Science 19 (3): 1–26. https://doi.org/10.1080/23249935.2022.2076755.
   Web of Science ®Google Scholar
• Li, X., Y. Xu, N. Li, B. Yang, and Y. Lei. 2022. “Remaining Useful Life Prediction with Partial Sensor Malfunctions Using Deep Adversarial Networks.” IEEE/CAA Journal of Automatica Sinica 10 (1): 121–134. https://doi.org/10.1109/JAS.2022.105935.
   Google Scholar
• Li, J., L. Yao, X. Xu, B. Cheng, and J. Ren. 2020. “Deep Reinforcement Learning for Pedestrian Collision Avoidance and Human-machine Cooperative Driving.” Information Sciences 532:110–124. https://doi.org/10.1016/j.ins.2020.03.105.
   Web of Science ®Google Scholar
• Li, X., S. Yu, Y. Lei, N. Li, and B. Yang. 2023. “Intelligent Machinery Fault Diagnosis with Event-Based Camera.” IEEE Transactions on Industrial Informatics 20 (1): 380–389. https://doi.org/10.1109/TII.2023.3262854.
   Web of Science ®Google Scholar
• Liao, Y., G. Yu, P. Chen, B. Zhou, and H. Li. 2022. “Modelling Personalised Car-following Behaviour: A Memory-based Deep Reinforcement Learning Approach.” Transportmetrica A: Transport Science 20 (1): 1–29. https://doi.org/10.1080/23249935.2022.2035846.
   Web of Science ®Google Scholar
• Lin, D., L. Li, and S. E. Jabari. 2019. “Pay to Change Lanes: A Cooperative Lane-changing Strategy for Connected/automated Driving.” Transportation Research Part C: Emerging Technologies 105:550–564. https://doi.org/10.1016/j.trc.2019.06.006.
   Web of Science ®Google Scholar
• Mirchevska, B., C. Pek, M. Werling, M. Althoff, and J. Boedecker. 2018. “High-Level Decision Making for Safe and Reasonable Autonomous Lane Changing Using Reinforcement Learning.” In 2018 IEEE 21st International Conference on Intelligent Transportation Systems (ITSC), 2156–2162. IEEE.
   Google Scholar
• Nguyen, T. T., N. D. Nguyen, and S. Nahavandi. 2020. “Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications.” IEEE Transactions on Cybernetics 50 (9): 3826–3839. https://doi.org/10.1109/TCYB.6221036.
   PubMed Web of Science ®Google Scholar
• Nie, Y. M., and Q. Li. 2013. “An Eco-routing Model Considering Microscopic Vehicle Operating Conditions.” Transportation Research Part B: Methodological 55: 154–170. https://doi.org/10.1016/j.trb.2013.06.004.
   Web of Science ®Google Scholar
• Pan, T., W. H. Lam, A. Sumalee, and R. Zhong. 2021. “Multiclass Multilane Model for Freeway Traffic Mixed with Connected Automated Vehicles and Regular Human-piloted Vehicles.” Transportmetrica A: Transport Science 17 (1): 5–33. https://doi.org/10.1080/23249935.2019.1573858.
   Web of Science ®Google Scholar
• Parada, L., E. Candela, L. Marques, and P. Angeloudis. 2023. “Safe and Efficient Manoeuvring for Emergency Vehicles in Autonomous Traffic Using Multi-agent Proximal Policy Optimisation.” Transportmetrica A: Transport Science 18 (1): 1–29. https://doi.org/10.1080/23249935.2023.2246586.
   Web of Science ®Google Scholar
• Paul, G., N. Raju, S. Arkatkar, and S. Easa. 2021. “Can Segregating Vehicles in Mixed-traffic Stream Improve Safety and Throughput? Implications Using Simulation.” Transportmetrica A: Transport Science 17 (4): 1002–1026. https://doi.org/10.1080/23249935.2020.1826595.
   Web of Science ®Google Scholar
• Rasekhipour, Y., A. Khajepour, S.-K. Chen, and B. Litkouhi. 2016. “A Potential Field-based Model Predictive Path-planning Controller for Autonomous Road Vehicles.” IEEE Transactions on Intelligent Transportation Systems 18 (5): 1255–1267. https://doi.org/10.1109/TITS.2016.2604240.
   Web of Science ®Google Scholar
• Ren, J., Y. Chen, L. Xin, J. Shi, and H. Mahama. 2017. “Detecting and Locating of Traffic Incidents in a Road Segment Based on Lane-changing Characteristics.” Transportmetrica A: Transport Science 13 (10): 853–873. https://doi.org/10.1080/23249935.2017.1348400.
   Web of Science ®Google Scholar
• Shi, T., P. Wang, X. Cheng, C.-Y. Chan, and D. Huang. 2019. “Driving Decision and Control for Automated Lane Change Behavior Based on Deep Reinforcement Learning.” In 2019 IEEE 22nd International Conference on Intelligent Transportation Systems (ITSC), 2895–2900. IEEE.
   Google Scholar
• Sun, Z., J. Huang, A. Ji, R. Zhao, and G. Zheng. “Cooperative Merging for Connected Automated Vehicles in Mixed Traffic: A Multi-Agent Reinforcement Learning Approach.” Available at SSRN 4564695.
   Google Scholar
• Sun, Z., T. Huang, and P. Zhang. 2020. “Cooperative Decision-making for Mixed Traffic: A Ramp Merging Example.” Transportation Research Part C: Emerging Technologies 120:102764. https://doi.org/10.1016/j.trc.2020.102764.
   Web of Science ®Google Scholar
• Sun, Z., R. Liu, H. Hu, D. Liu, and Z. Yan. 2023. “Cyberattacks on Connected Automated Vehicles: A Traffic Impact Analysis.” IET Intelligent Transport Systems 17 (2): 295–311. https://doi.org/10.1049/itr2.v17.2.
   Web of Science ®Google Scholar
• Sun, Z., X. Yao, Z. Qin, P. Zhang, and Z. Yang. 2021. “Modeling Car-following Heterogeneities by Considering Leader–follower Compositions and Driving Style Differences.” Transportation Research Record 2675 (11): 851–864. https://doi.org/10.1177/03611981211020006.
   Web of Science ®Google Scholar
• Sutton, R. S., and A. G. Barto. 2018. Reinforcement Learning: An Introduction. Cambridge, MA: MIT press.
   Google Scholar
• Wang, M., S. P. Hoogendoorn, W. Daamen, B. van Arem, and R. Happee. 2015. “Game Theoretic Approach for Predictive Lane-changing and Car-following Control.” Transportation Research Part C: Emerging Technologies 58:73–92. https://doi.org/10.1016/j.trc.2015.07.009.
   Web of Science ®Google Scholar
• Wang, G., J. Hu, Z. Li, and L. Li. 2021. “Harmonious Lane Changing Via Deep Reinforcement Learning.” IEEE Transactions on Intelligent Transportation Systems 23 (5): 4642–4650. https://doi.org/10.1109/TITS.2020.3047129.
   Web of Science ®Google Scholar
• Wang, P., H. Li, and C.-Y. Chan. 2019. “Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithm.” In 2019 IEEE Intelligent Vehicles Symposium (IV), 1454–1460. IEEE.
   Google Scholar
• Xu, H., Y. Zhang, C. G. Cassandras, L. Li, and S. Feng. 2020. “A Bi-level Cooperative Driving Strategy Allowing Lane Changes.” Transportation Research Part C: Emerging Technologies 120:102773. https://doi.org/10.1016/j.trc.2020.102773.
   Web of Science ®Google Scholar
• Yu, H., H. E. Tseng, and R. Langari. 2018. “A Human-like Game Theory-based Controller for Automatic Lane Changing.” Transportation Research Part C: Emerging Technologies 88:140–158. https://doi.org/10.1016/j.trc.2018.01.016.
   Web of Science ®Google Scholar
• Zhou, J., S. Xue, Y. Xue, Y. Liao, J. Liu, and W. Zhao. 2021. “A Novel Energy Management Strategy of Hybrid Electric Vehicle Via An Improved Td3 Deep Reinforcement Learning.” Energy 224:120118. https://doi.org/10.1016/j.energy.2021.120118.
   Web of Science ®Google Scholar