Near Real-time Map Building with Multi-class Image Set Labeling and Classification of Road Conditions Using Convolutional Neural Networks

References

• Almazan, E. J., Y. Qian, and J. H. Elder. 2016. Road segmentation for classification of road weather conditions. In Computer Vision – ECCV 2016 Workshops, ed. G. Hua and H. Jégou, 96–108. Cham: Springer International Publishing.
   Google Scholar
• Amthor, M., B. Hartmann, and J. Denzler. 2015. Road condition estimation based on spatio-temporal reflection models. In J. Gall, P. Gehler, and B. Leibeed., Pattern Recognition 3–15. Cham: Springer International Publishing.
   Google Scholar
• Arazo, E., D. Ortego, P. Albert, N. E. O’Connor, and K. McGuinness. 2019. Pseudo-labeling and confirmation bias in deep semi-supervised learning, Proceedings of the 36thInternational Conference on MachineLearning, Long Beach, California, PMLR 97, 2019.
   Google Scholar
• Bianco, S., R. Cadène, L. Celona, and P. Napoletano. 2018. Benchmark analysis of representative deep neural network architectures. IEEE Access 6:64270–77. doi:10.1109/ACCESS.2018.2877890.
   Web of Science ®Google Scholar
• Bronte, S., L. M. Bergasa, and P. F. Alcantarilla 2009. Fog detection system based on computer vision techniques. In 2009 12th International IEEE Conference on Intelligent Transportation Systems, 1–6.
   Google Scholar
• Carrillo, J., M. Crowley, G. Pan, and L. Fu 2019. Comparison of deep learning models for determining road surface condition from roadside camera images and weather data. In Transportation Association of Canada and Intelligent Transportation Systems Canada Joint Conference, 1–16.
   Google Scholar
• Chen, P., H. Hang, S. Chan, and J. Lin 2019. Dsnet: An efficient cnn for road scene segmentation. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 424–32.
   Google Scholar
• Chollet, F. 2017. Xception: Deep learning with depthwise separable convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1800–07.
   Google Scholar
• Crevier, L.-P., and Y. Delage. 2001. METRo: A new model for Road-Condition forecasting in Canada. Journal of Applied Meteorology 40 (11):2026–37. doi:10.1175/1520-0450(2001)040<2026:MANMFR>2.0.CO;2.
   Google Scholar
• Drobot, S., A. R. S. Anderson, C. Burghardt, and P. Pisano. 2014. U.s. public preferences for weather and road condition information. Bulletin of the American Meteorological Society 95 (6):849–59. doi:10.1175/BAMS-D-12-00112.1.
   Web of Science ®Google Scholar
• Elhoseiny, M., S. Huang, and A. M. Elgammal 2015. Weather classification with deep convolutional neural networks. In 2015 IEEE International Conference on Image Processing (ICIP), 3349–53.
   Google Scholar
• Gallen, R., A. Cord, N. Hautière, and D. Aubert 2011. Towards night fog detection through use of in- vehicle multipurpose cameras. In Proceedings of the 2011 IEEE Intelligent Vehicles Symposium (IV), 399–404.
   Google Scholar
• Hautiere, N., J.-P. Tarel, J. Lavenant, and D. Aubert. 2006. Automatic fog detection and estimation of visibility distance through use of an onboard camera. Machine Vision and Applications 17 (1):8–20. doi:10.1007/s00138-005-0011-1.
   Web of Science ®Google Scholar
• Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25:1097–1105.
   Google Scholar
• Kurihata, H., T. Takahashi, I. Ide, Y. Mekada, H. Murase, Y. Tamatsu, and T. Miyahara 2005. Rainy weather recognition from in-vehicle camera images for driver assistance. In IEEE Proceedings, Intelligent Vehicles Symposium, 205–10.
   Google Scholar
• Laffont, P.-Y., Z. Ren, X. Tao, C. Qian, and J. Hays. 2014. Transient attributes for high-level understanding and editing of outdoor scenes. ACM Transactions on Graphics [ 149:1–149: 11] 33 (4):. doi:10.1145/2601097.2601101.
   Web of Science ®Google Scholar
• LeCun, Y., Y. Bengio, and G. Hinton. 2015. Deep learning. Nature 521 (7553):436. doi:10.1038/nature14539.
   PubMed Web of Science ®Google Scholar
• Lee, D.-H. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. Workshop on challenges in representation learning, ICML, 3.
   Google Scholar
• Li, Q., Y. Kong, and S. M. Xia 2014. A method of weather recognition based on outdoor images. In Computer Vision Theory and Applications (VISAPP), 2014 International Conference, 510–16.
   Google Scholar
• Li, Z., Y. Jin, Y. Li, Z. Lin, and S. Wang 2018. Imbalanced adversarial learning for weather image generation and classification. In 2018 14th IEEE International Conference on Signal Processing (ICSP), 1093–97.
   Google Scholar
• Lin, D., C. Lu, H. Huang, and J. Jia. 2017. Rscm: Region selection and concurrency model for multi-class weather recognition. IEEE Transactions on Image Processing 26 (9):4154–67. doi:10.1109/TIP.2017.2695883.
   PubMed Web of Science ®Google Scholar
• Lu, C., D. Lin, J. Jia, and C. Tang. 2017. Two-class weather classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (12):2510–24. doi:10.1109/TPAMI.2016.2640295.
   PubMed Web of Science ®Google Scholar
• Lyu, Y., and X. Huang 2018. Road segmentation using CNN with GRU. arXiv preprint.
   Google Scholar
• Narasimhan, S. G., and S. K. Nayar 2003. Shedding light on the weather. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), I–I.
   Google Scholar
• Nolte, M., N. Kister, and M. Maurer 2018. Assessment of deep convolutional neural networks for road surface classification. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC), 381–86.
   Google Scholar
• Omer, R., and L. Fu 2010. An automatic image recognition system for winter road surface condition classification. In 13th International IEEE Conference on Intelligent Transportation Systems, 1375–79.
   Google Scholar
• Opitz, D., and R. Maclin. 1999. Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research 11 (1):169–98. doi:10.1613/jair.614.
   Google Scholar
• Pagani, G. A., J. W. Noteboom, and W. M. F. Wauben 2018. Deep neural network approach for automatic fog detection using traffic camera images. In The 2018 WMO/CIMO Technical Conference on Meteorological and Environmental Instruments and Methods of Observation (CIMO TECO-2018), volume 132 of Instruments and Observing Methods (IOM), 15. World Meteorological Organization.
   Google Scholar
• Pan, G., L. Fu, R. Yu, and M. Muresan 2018. Winter road surface condition recognition using a pre-trained deep convolutional neural network. In Transportation Research Board 97th Annual Meeting, number 18-00838, 17.
   Google Scholar
• Pavlić, M., H. Belzner, G. Rigoll, and S. Ilić 2012. Image based fog detection in vehicles. In 2012 IEEE Intelligent Vehicles Symposium, 1132–37.
   Google Scholar
• Ramanna, S., C. Sengoz, S. Kehler, and D. Pham 2020. Near real-time map building with multi-class image set labelling and classification of road conditions using convolutional neural networks. arXiv preprint.
   Google Scholar
• Roser, M., and F. Moosmann 2008. Classification of weather situations on single color images. In IEEE Proceedings, Intelligent Vehicles Symposium, 798–803.
   Google Scholar
• Russakovsky, O., J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. 2015. ImageNet large scale visual recognition challenge. International Journal of Computer Vision (IJCV) 115 (3):211–52. doi:10.1007/s11263-015-0816-y.
   Web of Science ®Google Scholar
• Sass, B. H. 1997. A numerical forecasting system for the prediction of slippery roads. Journal of Applied Meteorology 36 (6):801–17. doi:10.1175/1520-0450(1997)036<0801:ANFSFT>2.0.CO;2.
   Google Scholar
• Schmidhuber, J. 2015. Deep learning in neural networks: An overview. Neural Networks 61:85–117.
   PubMed Web of Science ®Google Scholar
• Shen, L. 2009. Photometric stereo and weather estimation using internet images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1850–57.
   Google Scholar
• Simonyan, K., and A. Zisserman 2014. Very deep convolutional networks for Large-Scale image recognition. arXiv preprint.
   Google Scholar
• Song, H., Y. Chen, and Y. Gao. 2014. Weather condition recognition based on feature extraction and k-nn. In Foundations and practical applications of cognitive systems and information processing, ed. F. Sun, D. Hu, and H. Liu, 199–210. Berlin, Heidelberg: Springer Berlin Heidelberg.
   Google Scholar
• Szegedy, C., S. Ioffe, V. Vanhoucke, and A. A. Alemi 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, 4278–4284. AAAI Press.
   Google Scholar
• Tan, M., and Q. V. Le 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, 6105–14. PMLR.
   Google Scholar
• Villarreal Guerra, J. C., Z. Khanam, S. Ehsan, R. Stolkin, and K. McDonald-Maier 2018. Weather classification: A new multi-class dataset, data augmentation approach and comprehensive evaluations of convolutional neural networks. In 2018 NASA/ESA Conference on Adaptive Hardware and Systems (AHS), 305–10.
   Google Scholar
• Yan, X., Y. Luo, and X. Zheng 2009. Weather recognition based on images captured by vision system in vehicle. In Proceedings of the 6th International Symposium on Neural Networks: Advances in Neural Networks - Part III, 390–98.
   Google Scholar
• Zhang, Z., and H. Ma 2015. Multi-class weather classification on single images. In 2015 IEEE International Conference on Image Processing (ICIP), 4396–400.
   Google Scholar
• Zhao, B., X. Li, X. Lu, and Z. Wang. 2018. A CNN-RNN architecture for multi-label weather recognition. Neurocomputing 322:47–57. doi:10.1016/j.neucom.2018.09.048.
   Web of Science ®Google Scholar
• Zhu, Z., L. Zhuo, P. Qu, K. Zhou, and J. Zhang 2016. Extreme weather recognition using convolutional neural networks. In 2016 IEEE International Symposium on Multimedia (ISM), 621–25.
   Google Scholar
• Zoph, B., V. Vasudevan, J. Shlens, and Q. V. Le 2018. Learning transferable architectures for scalable image recognition. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8697–710.
   Google Scholar