857
Views
4
CrossRef citations to date
0
Altmetric
Research Article

Can motorcyclist behavior in traffic conflicts be modeled? A deep reinforcement learning approach for motorcycle-pedestrian interactions

, ORCID Icon & ORCID Icon
Pages 396-420 | Received 09 Jun 2021, Accepted 04 Nov 2021, Published online: 25 Nov 2021

REFERENCES

  • Abbeel, Pieter, and Andrew Y. Ng. 2004. “Apprenticeship Learning via Inverse Reinforcement Learning.” In Proceedings, twenty-First International Conference on Machine Learning, ICML, 1–8.
  • Allen, Brian L., B. Tom Shin, and Peter Cooper. 1978. “Analysis of Traffic Conflicts and Collisions.” Transportation Research Record 667: 67–74.
  • Alsaleh, Rushdi, Mohamed Hussein, and Tarek Sayed. 2020. “Microscopic Behavioural Analysis of Cyclist and Pedestrian Interactions in Shared Spaces.” Canadian Journal of Civil Engineering 47 (1): 50–62.
  • Alsaleh, Rushdi, and Tarek Sayed. 2020. “Modeling Pedestrian-Cyclist Interactions in Shared Space Using Inverse Reinforcement Learning.” Transportation Research Part F: Traffic Psychology and Behaviour 70: 37–57.
  • Alsaleh, Rushdi, and Tarek Sayed. 2021a. “Markov-Game Modeling of Cyclist-Pedestrian Interactions in Shared Spaces: A Multi-Agent Adversarial Inverse Reinforcement Learning Approach.” Transportation Research Part C: Emerging Technologies 128: 103191.
  • Alsaleh, Rushdi, and Tarek Sayed. 2021b. “Microscopic Modeling of Cyclists Interactions with Pedestrians in Shared Spaces: A Gaussian Process Inverse Reinforcement Learning Approach.” Transportmetrica A: Transport Science: 1–27. doi:10.1080/23249935.2021.1898487.
  • Alsaleh, Rushdi, Tarek Sayed, and Mohamed H. Zaki. 2018. “Assessing the Effect of Pedestrians’ Use of Cell Phones on Their Walking Behavior: A Study Based on Automated Video Analysis.” Transportation Research Record 2672 (35): 46–57.
  • Alvarez, Walter Morales, Miguel Angel De Miguel, Fernando Garcia, and Cristina Olaverri-Monreal. 2019. “Response of Vulnerable Road Users to Visual Information from Autonomous Vehicles in Shared Spaces.” 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, October 27–30.
  • Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. “Concrete Problems in AI Safety,” 1–29. http://arxiv.org/abs/1606.06565.
  • Amundsen, F. H., and C. Hydén. 1977. “Proceedings of the 1st Workshop on Traffic Conflicts.” In . Oslo, Norway.
  • Autey, Jarvis, Tarek Sayed, and Mohamed H. Zaki. 2012. “Safety Evaluation of Right-Turn Smart Channels Using Automated Traffic Conflict Analysis.” Accident Analysis and Prevention 45: 120–130.
  • Ayres, Greg, Bruce Wilson, and Jon LeBlanc. 2004. “Method for Identifying Vehicle Movements for Analysis of Field Operational Test Data.” Transportation Research Record 1886: 92–100.
  • Busoniu, Lucian, Robert Babuska, Bart De Schutter, and Damien Ernst. 2009. Reinforcement Learning and Dynamic Programming Using Function Approximators. Boca Raton, FL: CRC Press.
  • Chen, Sikai, Jiqian Dong, Paul Ha, Yujie Li, and Samuel Labi. 2021. “Graph Neural Network and Reinforcement Learning for Multi-Agent Cooperative Control of Connected Autonomous Vehicles.” Computer-Aided Civil and Infrastructure Engineering 36 (7): 838–857.
  • Cunto, Flávio, and Frank F. Saccomanno. 2008. “Calibration and Validation of Simulated Vehicle Safety Performance at Signalized Intersections.” Accident Analysis and Prevention 40 (3): 1171–1179.
  • Das, Sanhita, and Akhilesh Kumar Maurya. 2018. “Modelling of Motorised Two-Wheelers: A Review of the Literature.” Transport Reviews 38 (2): 209–231.
  • El-Basyouny, Karim, and Tarek Sayed. 2013. “Safety Performance Functions Using Traffic Conflicts.” Safety Science 51 (1): 160–164.
  • Essa, Mohamed, and Tarek Sayed. 2015a. “Simulated Traffic Conflicts: Do They Accurately Represent Field-Measured Conflicts?” Transportation Research Record 2514: 48–57.
  • Essa, Mohamed, and Tarek Sayed. 2015b. “Transferability of Calibrated Microsimulation Model Parameters for Safety Assessment Using Simulated Conflicts.” Accident Analysis and Prevention 84: 41–53.
  • Essa, Mohamed, and Tarek Sayed. 2016. “A Comparison Between PARAMICS and VISSIM in Estimating Automated Field-Measured Traffic Conflicts at Signalized Intersections.” Journal of Advanced Transportation 50: 897–917.
  • Fan, Rong, Hao Yu, Pan Liu, and Wei Wang. 2013. “Using VISSIM Simulation Model and Surrogate Safety Assessment Model for Estimating Field Measured Traffic Conflicts at Freeway Merge Areas.” IET Intelligent Transport Systems 7 (1): 68–77.
  • Fu, Ting, Luis Miranda-Moreno, and Nicolas Saunier. 2016. “Pedestrian Crosswalk Safety at Nonsignalized Crossings During Nighttime: Use of Thermal Video Data and Surrogate Safety Measures.” Transportation Research Record 2586: 90–99.
  • Fujii, Hideki, Hideaki Uchida, and Shinobu Yoshimura. 2017. “Agent-Based Simulation Framework for Mixed Traffic of Cars, Pedestrians and Trams.” Transportation Research Part C: Emerging Technologies 85 (October): 234–248.
  • Gavriilidou, A., W. Daamen, Y. Yuan, and S. P. Hoogendoorn. 2019. “Modelling Cyclist Queue Formation Using a Two-Layer Framework for Operational Cycling Behaviour.” Transportation Research Part C: Emerging Technologies 105: 468–484.
  • Guo, Yanyong, Mohamed Essa, Tarek Sayed, Md Mazharul Haque, and Simon Washington. 2019. “A Comparison Between Simulated and Field-Measured Conflicts for Safety Assessment of Signalized Intersections in Australia.” Transportation Research Part C: Emerging Technologies 101: 96–110.
  • Haworth, N. 2012. “Powered Two Wheelers in a Changing World - Challenges and Opportunities.” Accident Analysis and Prevention 44 (1): 12–18.
  • Huang, Fei, Pan Liu, Hao Yu, and Wei Wang. 2013. “Identifying If VISSIM Simulation Model and SSAM Provide Reasonable Estimates for Field Measured Traffic Conflicts at Signalized Intersections.” Accident Analysis and Prevention 50: 1014–1024.
  • Hussein, Mohamed, and Tarek Sayed. 2017. “A Bi-Directional Agent-Based Pedestrian Microscopic Model.” Transportmetrica A: Transport Science 13 (4): 326–355.
  • Huttenlocher, D. P., W. J. Rucklidge, and G. A. Klanderman. 1993. “Comparing Images Using the Hausdorff Distance Under Translation.” IEEE Transactions on Pattern Analysis and Machine Intelligence 15 (9): 850–863.
  • Huynh, Manh, and Gita Alaghband. 2019. “Trajectory Prediction by Coupling Scene-LSTM with Human Movement LSTM.” In Advances in Visual Computing. ISVC 2019. Lecture Notes in Computer Science, vol 11844, edited by George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Daniela Ushizima, Sek Chai, Shinjiro Sueda, et al., 244–259. Cham: Springer.
  • Huynh, Duc, Manfred Boltze, and Anh Vu. 2013. “Modelling Mixed Traffic Flow at Signalized Intersection Using Social Force Model.” Journal of the Eastern Asia Society for Transportation Studies 10: 1734–1749.
  • Hyndman, Rob J., and Anne B. Koehler. 2006. “Another Look at Measures of Forecast Accuracy.” International Journal of Forecasting 22 (4): 679–688.
  • Iryo-Asano, Miho, and Wael K.M. Alhajyaseen. 2017. “Modeling Pedestrian Crossing Speed Profiles Considering Speed Change Behavior for the Safety Assessment of Signalized Intersections.” Accident Analysis and Prevention 108 (December 2016): 332–342. Elsevier:.
  • Ismail, Karim, Tarek Sayed, and Nicolas Saunier. 2011. “Methodologies for Aggregating Indicators of Traffic Conflict.” Transportation Research Record 2237: 10–19.
  • Ismail, Karim, Tarek Sayed, and Nicolas Saunier. 2013. “A Methodology for Precise Camera Calibration for Data Collection Applications in Urban Traffic Scenes.” Canadian Journal of Civil Engineering 40: 57–67.
  • Ismail, Karim, Tarek Sayed, Nicolas Saunier, and Clark Lim. 2009. “Automated Analysis of Pedestrian-Vehicle Conflicts Using Video Data.” Transportation Research Record 2140: 44–54.
  • Jennings, Nicholas R. 2000. “On Agent-Based Software Engineering.” Artificial Intelligence 117: 277–296.
  • Keneshloo, Yaser, Tian Shi, Naren Ramakrishnan, and Chandan K. Reddy. 2020. “Deep Reinforcement Learning for Sequence-to-Sequence Models.” IEEE Transactions on Neural Networks and Learning Systems 31 (7): 2469–2489.
  • Kiran, Sai, and Ashish Verma. 2016. “Review of Studies on Mixed Traffic Flow: Perspective of Developing Economies.” Transportation in Developing Economies 2 (1): 1–16.
  • Kitajima, Sou, Keisuke Shimono, Jun Tajima, Jacobo Antona-Makoshi, and Nobuyuki Uchida. 2019. “Multi-Agent Traffic Simulations to Estimate the Impact of Automated Technologies on Safety.” Traffic Injury Prevention 20: S58–S64.
  • Kuhn, M., and K. Johnson. 2013. Applied Predictive Modeling. New York: Springer.
  • Lan, Lawrence W., and Chiung Wen Chang. 2005. “Inhomogeneous Cellular Automata Modeling for Mixed Traffic with Cars and Motorcycles.” Journal of Advanced Transportation 39 (3): 323–349.
  • Lee, Kyungjae, Sungjoon Choi, and Songhwai Oh. 2016. “Inverse Reinforcement Learning with Leveraged Gaussian Processes.” IEEE International Conference on Intelligent Robots and Systems, Daejeon, Korea, October 9–14.
  • Lee, Tzu Chang, John W. Polak, and Michael G.H. Bell. 2009. “New Approach to Modeling Mixed Traffic Containing Motorcycles in Urban Areas.” Transportation Research Record 2140: 195–205.
  • Levine, Sergey, and Vladlen Koltun. 2012. “Continuous Inverse Optimal Control with Locally Optimal Examples.” Proceedings of the 29th International Conference on Machine learning.
  • Levine, Sergey, Zoran Popovic, and Vladlen Koltun. 2011. “Nonlinear Inverse Reinforcement Learning with Gaussian Processes.” Advances in Neural Information Processing Systems (NIPS 2011). https://papers.nips.cc/paper/4420-nonlinear-inverse-reinforcement-learning-with-gaussian-processes.
  • Li, Yang, Hao Cheng, Zhe Zeng, Hailong Liu, and Monika Sester. 2021. “Autonomous Vehicles Drive into Shared Spaces: EHMI Design Concept Focusing on Vulnerable Road Users.” ArXiv Preprint ArXiv:2104.06916. http://arxiv.org/abs/2104.06916.
  • Liu, Yan, Rushdi Alsaleh, and Tarek Sayed. 2021a. “Modeling Lateral Interactions Between Motorized Vehicles and Non-Motorized Vehicles in Mixed Traffic Using Accelerated Failure Duration Model.” Transportmetrica A: Transport Science 1: 1–24. https://doi.org/10.1080/23249935.2021.1908443.
  • Liu, Yan, Rushdi Alsaleh, and Tarek Sayed. 2021b. “Modeling the Influence of Mobile Phone Use Distraction on Pedestrian Reaction Times to Green Signals: A Multilevel Mixed-Effects Parametric Survival Model.” Transportation Research Part F: Traffic Psychology and Behaviour 81: 115–129. Elsevier Ltd:.
  • Liu, Dong C., and Jorge Nocedal. 1989. “On the Limited Memory BFGS Method for Large Scale Optimization.” Mathematical Programming 45: 503–528.
  • Liu, Qiyuan, Jian Sun, Ye Tian, and Lu Xiong. 2020. “Modeling and Simulation of Overtaking Events by Heterogeneous Non-Motorized Vehicles on Shared Roadway Segments.” Simulation Modelling Practice and Theory 103: 102072.
  • Ma, Xiaoliang, and Ding Luo. 2016. “Modeling Cyclist Acceleration Process for Bicycle Traffic Simulation Using Naturalistic Data.” Transportation Research Part F: Traffic Psychology and Behaviour 40: 130–144.
  • Mahmud, S. M. Sohel, Luis Ferreira, Md Shamsul Hoque, and Ahmad Tavassoli. 2017. “Application of Proximal Surrogate Indicators for Safety Evaluation: A Review of Recent Developments and Research Needs.” IATSS Research 41 (4): 153–163. International Association of Traffic and Safety Sciences:
  • Makridakis, Spyros. 1993. “Accuracy Measures: Theoretical and Practical Concerns.” International Journal of Forecasting 9 (4): 527–529.
  • Mnih, Volodymyr, Adria Puigdomenech Badia, Lehdi Mirza, Alex Graves, Tim Harley, Timothy P. Lillicrap, David Silver, and Koray Kavukcuoglu. 2016. “Asynchronous Methods for Deep Reinforcement Learning.” 33rd International Conference on Machine Learning, ICML, 4:2850–2869.
  • Nair, Rahul, Hani S. Mahmassani, and Elise Miller-Hooks. 2011. “A Porous Flow Approach to Modeling Heterogeneous Traffic in Disordered Systems.” Transportation Research Part B: Methodological 45 (9): 1331–1345.
  • Nasernejad, Payam, Tarek Sayed, and Rushdi Alsaleh. 2021. “Modeling pedestrian behavior in pedestrian-vehicle near misses: A continuous Gaussian Process Inverse Reinforcement Learning (GP-IRL) approach.” Accident Analysis & Prevention 161: 106355.
  • Ng, Andrew Y., and Stuart Russel. 2000. “Algorithms for Inverse Reinforcement Learning.” International Conference on Machine learning.
  • Nguyen, Long Xuan, and Shinya Hanaoka. 2011. “An Application of Social Force Approach for Motorcycle Dynamics.” In Proceedings of the Eastern Asia Society for Transportation Studies.
  • Nguyen, Long Xuan, Shinya Hanaoka, and Tomoya Kawasaki. 2012. “Describing Non-Lane-Based Motorcycle Movements in Motorcycle-Only Traffic Flow.” Transportation Research Record 2281 (1): 76–82.
  • Nguyen, Long Xuan, Shinya Hanaoka, and Tomoya Kawasaki. 2014. “Traffic Conflict Assessment for Non-Lane-Based Movements of Motorcycles Under Congested Conditions.” IATSS Research 37 (2): 137–147.
  • Ozbay, Kaan, Hong Yang, Bekir Bartin, and Sandeep Mudigonda. 2008. “Derivation and Validation of New Simulation-Based Surrogate Safety Measure.” Transportation Research Record 2083: 105–113.
  • Papadimitriou, Eleonora, George Yannis, and John Golias. 2009. “A Critical Assessment of Pedestrian Behaviour Models.” Transportation Research Part F: Traffic Psychology and Behaviour 12 (3): 242–255. Elsevier Ltd:.
  • Parker, M., and C. Zegeer. 1989. Traffic Conflict Techniques for Safety and Operations: Engineers Guide. Report FHWA-IP-026.
  • Paul, Madhumita, and Indrajit Ghosh. 2020. “Post Encroachment Time Threshold Identification for Right-Turn Related Crashes at Unsignalized Intersections on Intercity Highways Under Mixed Traffic.” International Journal of Injury Control and Safety Promotion 27 (2): 121–135. Taylor & Francis:.
  • Perkins, R. S., and J. I. Harris. 1968. “Traffic Conflict Characteristics - Accident Potential at Intersections.” Highway Research Record No. 224 35-43.
  • Pirdavani, A., T. Brijs, T. Bellemans, and G. Wets. 2010. “Evaluation of Traffic Safety at Un-Signalized Intersections Using Microsimulation: A Utilization of Proximal Safety Indicators.” Advances in Transportation Studies 22: 43–50.
  • Qiao, Qifeng, and Peter A. Beling. 2013. “Inverse Reinforcement Learning with Gaussian Process.” Proceedings of the American Control Conference. ArXiv Preprint ArXiv:1208.2112. Cornell University.
  • Rao, Trinadha, V. and V, and R. Rengaraju. 1998. “Modeling Conflicts of Heterogeneous Traffic at Urban Uncontrolled Intersections.” Journal of Transportation Engineering 124 (1): 23–34.
  • Saccomanno, Frank F., Flávio Cunto, Giuseppe Guido, and Alessandro Vitale. 2008. “Comparing Safety at Signalized Intersections and Roundabouts Using Simulated Rear-End Conflicts.” Transportation Research Record 2078: 90–95.
  • Saunier, Nicolas, and Tarek Sayed. 2006. “A Feature-Based Tracking Algorithm for Vehicles in Intersections.” Third Canadian Conference on computer and Robot vision, CRV 2006.
  • Saunier, Nicolas, Tarek Sayed, and Clark Lim. 2007. “Probabilistic Collision Prediction for Vision-Based Automated Road Safety Analysis.” IEEE Conference on Intelligent Transportation Systems, proceedings, ITSC, 872–878.
  • Savitzky, Abraham, and Marcel J.E. Golay. 1964. “Smoothing and Differentiation of Data by Simplified Least Squares Procedures.” Analytical Chemistry 36 (8): 1627–1639.
  • Sayed, Tarek, and Sany Zein. 1999. “Traffic Conflict Standards for Intersections.” Transportation Planning and Technology 22 (4): 309–323.
  • Shimosaka, Masamichi, Takuhiro Kaneko, and Kentaro Nishi. 2014. “Modeling Risk Anticipation and Defensive Driving on Residential Roads with Inverse Reinforcement Learning.” 17th IEEE International Conference on Intelligent Transportation Systems, ITSC 2014, 1694–1700. IEEE.
  • Shiwakoti, Nirajan, Majid Sarvi, and Geoff Rose. 2008. “Modelling Pedestrian Behaviour under Emergency Conditions - State-of-the-Art and Future Directions.” 31st Australasian Transport Research Forum, ATRF 2008, 457–473.
  • Sokolova, Marina, and Guy Lapalme. 2009. “A Systematic Analysis of Performance Measures for Classification Tasks.” Information Processing and Management 45 (4): 427–437.
  • Styles, Oily, Arun Ross, and Victor Sanchez. 2019. “Forecasting Pedestrian Trajectory with Machine-Annotated Training Data.” In IEEE Intelligent Vehicles Symposium, Proceedings, 716–721. doi:10.1109/IVS.2019.8814207 .
  • Sutton, Richard S., and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
  • Tageldin, Ahmed, and Tarek Sayed. 2016. “Developing Evasive Action-Based Indicators for Identifying Pedestrian Conflicts in Less Organized Traffic Environments.” Journal of Advanced Transportation 50: 1193–1208.
  • Tageldin, Ahmed, and Tarek Sayed. 2019. “Models to Evaluate the Severity of Pedestrian-Vehicle Conflicts in Five Cities.” Transportmetrica A: Transport Science 15 (2): 354–375.
  • Tageldin, Ahmed, Tarek Sayed, and Xuesong Wang. 2015. “Can Time Proximity Measures Be Used as Safety Indicators in All Driving Cultures? Case Study of Motorcycle Safety in China.” Transportation Research Record 2520: 165–174.
  • Tarko, Andrew P., and P. Songchitruksa. 2005. “Estimating the Frequency of Crashes as Extreme Traffic Events.” In 84th Annual Meeting of the Transportation Research Board.
  • Tomasi, Carlo, and Takeo Kanade. 1991. Detection and Tracking of Point Features. Technical Report CMU-CS-91-132.
  • Trinh, Linh Thanh, Kazushi Sano, and Kiichiro Hatoyama. 2020. “Modelling and Simulating Head-on Conflict-Solving Behaviour of Motorcycles Under Heterogeneous Traffic Condition in Developing Countries.” Transportmetrica A: Transport Science 0 (0): Taylor & Francis: 1–25.
  • Vlahogianni, Eleni I., George Yannis, and John C. Golias. 2012. “Overview of Critical Risk Factors in Power-Two-Wheeler Safety.” Accident Analysis and Prevention 49: 12–22.
  • Wang, Xuesong, Yilun Xu, Paul Tremont, and Dongyuan Yang. 2012. “Moped Rider Violation Behavior and Moped Safety at Intersections in China.” Transportation Research Record 2281 (1): 83–91.
  • Wu, Yuhuai, Elman Mansimov, Shun Liao, Roger Grosse, and Jimmy Ba. 2017. “Scalable Trust-Region Method for Deep Reinforcement Learning Using Kronecker-Factored Approximation.” Advances in Neural Information Processing Systems. ArXiv Preprint ArXiv:1708.05144. Cornell University.
  • Yang, Jianguo, Wen Deng, Jinmei Wang, Qingfeng Li, and Zhaoan Wang. 2006. “Modeling Pedestrians’ Road Crossing Behavior in Traffic System Micro-Simulation in China.” Transportation Research Part A: Policy and Practice 40 (3): 280–290.
  • You, Changxi, Jianbo Lu, Dimitar Filev, and Panagiotis Tsiotras. 2019. “Advanced Planning for Autonomous Vehicles Using Reinforcement Learning and Deep Inverse Reinforcement Learning.” Robotics and Autonomous Systems 114: 1–18.
  • Zaki, Mohamed H., and Tarek Sayed. 2013. “A Framework for Automated Road-Users Classification Using Movement Trajectories.” Transportation Research Part C: Emerging Technologies 33: 50–73.
  • Zaki, Mohamed H., Tarek Sayed, and Xuesong Wang. 2016. “Computer Vision Approach for the Classification of Bike Type (Motorized Versus Non-Motorized) During Busy Traffic in the City of Shanghai.” Journal of Advanced Transportation 50: 348–362.
  • Zangenehpour, Sohail, Jillian Strauss, Luis F. Miranda-Moreno, and Nicolas Saunier. 2016. “Are Signalized Intersections with Cycle Tracks Safer? A Case-Control Study Based on Automated Surrogate Safety Analysis Using Video Data.” Accident Analysis and Prevention 86: 161–172.
  • Zaremba, Wojciech, Ilya Sutskever, and Oriol Vinyals. 2014. “Recurrent Neural Network Regularization,” no. 2013: 1–8. http://arxiv.org/abs/1409.2329.
  • Zeng, Weiliang, Hideki Nakamura, and Peng Chen. 2014. “A Modified Social Force Model for Pedestrian Behavior Simulation at Signalized Crosswalks.” Procedia - Social and Behavioral Sciences 138: 521–530. Elsevier B.V.:
  • Zhang, Yihao, Zhaojie Chai, and George Lykotrafitis. 2021. “Deep Reinforcement Learning with a Particle Dynamics Environment Applied to Emergency Evacuation of a Room with Obstacles.” Physica A: Statistical Mechanics and Its Applications 571: 125845.
  • Ziebart, Brian D, Andrew L Maas, J. Andrew Bagnell, and Anind K Dey. 2008. “Maximum Entropy Inverse Reinforcement Learning.” 23rd AAAI Conference on Artificial intelligence. 8:1433–1438.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.