Joint learning of video images and physiological signals for lane-changing behavior prediction

References

• Bachlechner, T., B. P. Majumder, H. H. Mao, G. W. Cottrell, and J. J. McAuley. 2020. “ReZero is All You Need: Fast Convergence at Large Depth.” arXiv:2003.04887.
   Google Scholar
• Cao, Y., J. Xu, S. Lin, F. Wei, and H. Hu. 2019. “GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond.” Proceedings of international conference on computer vision workshops.
   Google Scholar
• Deng, Q., J. Wang, K. Hillebrand, C. R. Benjamin, and D. Soffker. 2020. “Prediction Performance of Lane Changing Behaviors: A Study of Combining Environmental and Eye-Tracking Data in a Driving Simulator.” 21 (8): 3561–3570.
   Google Scholar
• Ding, M., Z. Wang, B. Zhou, J. Shi, Z. Lu, and P. Luo. 2019. “Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow.” arXiv:1911.12739.
   Google Scholar
• Gadde, R., V. Jampani, and P. V. Gehler. 2017. “Semantic Video Cnns Through Representation Warping.” Proceedings of the IEEE International Conference on Computer Vision, 4453–4462.
   Google Scholar
• Gao, J., Y. L. Murphey, J. Yi, and H. H. Zhu. 2020. “A Data-Driven Lane-Changing Behavior Detection System Based on Sequence Learning.” Transportmetrica B: Transport Dynamics. doi: 10.1080/21680566.2020.1782786.
   Google Scholar
• Gao, J., Y. L. Murphey, and H. H. Zhu. 2018. “Multivariate Time Series Prediction of Lane Changing Behavior Using Deep Neural Network.” Applied Intelligence 48 (10): 3523–3537.
   Google Scholar
• Gao, J., Y. L. Murphey, and H. H. Zhu. 2019. “Personalized Detection of Lane Changing Behavior Using Multisensor Data Fusion.” Computing 101 (12): 1837–1860.
   Google Scholar
• Gao, J., J. G. Yi, H. H. Zhu, Y. L. Murphey. 2019. “A Personalized Lane-Changing Model for Advanced Driver Assistance System Based on Deep Learning and Spatial-Temporal Modeling.” SAE International Journal of Transportation Safety 7 (2): 163–175.
   Google Scholar
• Gao, J., H. H. Zhu, and Y. L. Murphey. 2019. “A Personalized Model for Driver Lane-Changing Behavior Prediction Using Deep Neural Network.” 2nd International Conference on Artificial Intelligence and Big Data. IEEE, 90–96.
   Google Scholar
• Gebert, P., A. Roitberg, M. Haurilet, and R. Stiefelhagen. 2019. “End-to-end Prediction of Driver Intention Using 3D Convolutional Neural Networks.” IEEE Intelligent Vehicles Symposium, 969–974.
   Google Scholar
• Ilg, E., N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox. 2017. “Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2462–2470.
   Google Scholar
• Jin, X., X. Li, H. Xiao, S. Shen, Z. Lin, J. Yang, Y. Chen, et al. 2017. “Video Scene Parsing with Predictive Feature Learning.” Proceedings of the IEEE International Conference on Computer Vision, 5580–5588.
   Google Scholar
• Li, J., X. Mei, D. Prokhorov, and Dacheng Tao. 2017. “Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene.” IEEE Transactions on Neural Networks and Learning Systems 28 (3): 690–703.
   Google Scholar
• Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollar. 2017. “Focal Loss for Dense Object Detection.” Proceedings of the IEEE International Conference on Computer Vision, 2980–2988.
   Google Scholar
• Ma, X., Z. Ma, X. Zhu, J. Cao, and F. Yu. 2019. “Driver Behavior Classification under Cut-in Scenarios Using Support Vector Machine Based on Naturalistic Driving Data.” SAE Technical Paper.
   Google Scholar
• Morris, B., A. Doshi, and M. Trivedi. 2011. “Lane Change Intent Prediction for Driver Assistance: On-Road Design and Evaluation.” IEEE Intelligent Vehicles Symposium, 895–901.
   Google Scholar
• Murphey, Y., D S. Kochhar, Y Q. Xie. 2019. “Driver Workload in an Autonomous Vehicle.” SAE Technical Paper.
   Google Scholar
• Nakano, Yukiko I., Elisabeth André, Toyoaki Nishida, Louis-Philippe Morency, Carlos Busso, Catherine Pelachaud, Nanxiang Li, et al. 2016. “Driving Maneuver Prediction Using car Sensor and Driver Physiological Signals.” Proceedings of the 18th ACM International Conference on Multimodal Interaction, 108–112.
   Google Scholar
• Nilsson, D., and C. Sminchisescu. 2018. “Semantic Video Segmentation by Gated Recurrent Flow Propagation.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6819–6828.
   Google Scholar
• Ohn-Bar, E., A. Tawari, S. Martin, and M. M. Trivedi. 2014. “Predicting Driver Maneuvers by Learning Holistic Features.” IEEE Intelligent Vehicles Symposium, 719–724.
   Google Scholar
• Olabiyi, O., E. Martinson, V. Chintalapudi, and R. Guo. 2017. “Driver action prediction using deep (bidirectional) recurrent neural network.” arXiv:1706.02257.
   Google Scholar
• Patraucean, V., A. Handa, and R. Cipolla. 2015. “Spatio-temporal video autoencoder with differentiable memory.” arXiv:1511.06309,
   Google Scholar
• Peng, J., Y. Guo, R. Fu, Wei Yuan, and Chang Wang. 2015. “Multi-parameter Prediction of Drivers’ Lane-Changing Behaviour with Neural Network Model.” Applied Ergonomics 50: 207–217.
   Google Scholar
• Peng, X., R. Liu, Y. L. Murphey, S. Stent, and Y. Li. 2018. “Driving Maneuver Detection via Sequence Learning from Vehicle Signals and Video Images.” 24th IEEE International Conference on Pattern Recognition, 1265–1270.
   Google Scholar
• Ramanishka, V., Y. T. Chen, T. Misu, and K. Saenko. 2018. “Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7699–7707.
   Google Scholar
• Ren, Z., J. C. Yan, B. B. Ni, B. Liu, X. Yang, and H. Zha. 2017. “Unsupervised Deep Learning for Optical Flow Estimation.” AAAI Conference on Artificial Intelligence, 1495–1501.
   Google Scholar
• Sun, D. Q., X. D. Yang, M. Y. Liu, and J. Kautz. 2018. “Pwc-net: Cnns for Optical Flow Using Pyramid, Warping, and Cost Volume.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8934–8943.
   Google Scholar
• Wang, Z., A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. “Image Quality Assessment: From Error Visibility to Structural Similarity.” IEEE Transactions on Image Processing 13 (4): 600–612.
   Google Scholar
• Wang, Y., D. Zhang, Y. Liu, B. Dai, and Loo Hay Lee. 2019. “Enhancing Transportation Systems via Deep Learning: A Survey.” Transportation Research Part C: Emerging Technologies 99: 144–163.
   Google Scholar
• Xie, D. F., Z. Z. Fang, B. Jia, and Zhengbing He. 2019. “A Data-Driven Lane-Changing Model Based on Deep Learning.” Transportation Research Part C: Emerging Technologies 106: 41–60.
   Google Scholar
• Xie, Y. Q., Y. L. Murphey, and D. S. Kochhar. 2019. “Detecting Sequential Human Mental Workload Using U-Net with Continuity-Aware Loss Applied to Streamed Physiological Signals.” IEEE 17th International Conference on Industrial Informatics, 1801–1806.
   Google Scholar
• Xu, H., Y. Gao, F. Yu, and T. Darrell. 2017. “End-to-end Learning of Driving Models from Large-Scale Video Datasets.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2174–2182.
   Google Scholar
• Yao H., X. Tang, H. Wei, and G. Zheng. 2018. “Modeling Spatial-Temporal Dynamics for Traffic Prediction.” arXiv preprint arXiv:1803.01254.
   Google Scholar
• Yin, Z., and J. Shi. 2018. “Geonet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1983–1992.
   Google Scholar
• Yong, H, J. Huang, X. Hua, and L. Zhang. 2020. “Gradient Centralization: A New Optimization Technique for Deep Neural Networks.” arXiv:2004.01461.
   Google Scholar
• Zhang, X., J. Sun, X. Qi, andJian Sun. 2019. “Simultaneous Modeling of Car-following and Lane-changing Behaviors Using Deep Learning.” Transportation Research Part C: Emerging Technologies 104: 287–304.
   Google Scholar
• Zheng, Y., and J. H. L. Hansen. 2017. “Lane-change Detection from Steering Signal Using Spectral Segmentation and Learning-Based Classification.” IEEE Transactions on Intelligent Vehicles 2 (1): 14–24.
   Google Scholar
• Zhou, B., A. Lapedriza, A. Khosla, Aude Oliva, and Antonio Torralba. 2018. “Places: A 10 Million Image Database for Scene Recognition.” IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (6): 1452–1464.
   Google Scholar
• Zyner, A., S. Worrall, and J. Ward. 2017. “Long Short Term Memory for Driver Intent Prediction.” IEEE Intelligent Vehicles Symposium 1484–1489.
   Google Scholar