Multilevel weather detection based on images: a machine learning approach with histogram of oriented gradient and local binary pattern-based features

References

• Ahmed, M. M., Ghasemzadeh, A., Hammit, B., Khan, M. N., Das, A., Ali, E., Young, R. K., & Eldeeb, H. (2018). Driver performance and behavior in adverse weather conditions: An investigation Using the SHRP2 naturalistic driving study data-Phase 2 (Final Report WY-18/05F). U.S Department of Transportation.
   Google Scholar
• Ahmed, M. M., Young, R. K., Ghasemzadeh, A., Hammit, B., Ali, E., Khan, M. N., Das, A., & Eldeeb, H. (2017). Implementation of SHRP2 results within the Wyoming connected vehicle variable speed limit system: Phase 2 early findings report and phase 3 proposal. U.S Department of Transportation.
   Google Scholar
• Ahonen, T., Hadid, A., & Pietikäinen, M. (2006). Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037–2041. https://doi.org/10.1109/TPAMI.2006.244
   PubMed Web of Science ®Google Scholar
• Albousefi, A. A., Ying, H., Filev, D., Syed, F., Prakah, K. O., Tseng, F., & Yang, H. (2017). A two-stage-training support vector machine approach to predicting unintentional vehicle lane departure. Journal of Intelligent Transportation Systems, 21(1), 41–51. https://doi.org/10.1080/15472450.2016.1196141
   Web of Science ®Google Scholar
• Andrey, J., Mills, B., Leahy, M., & Suggett, J. (2003). Weather as a chronic hazard for road transportation in Canadian cities. Natural Hazards, 28, 319–343.
   Web of Science ®Google Scholar
• Babari, R., Hautière, N., Dumont, É., Paparoditis, N., & Misener, J. (2012). Visibility monitoring using conventional roadside cameras – emerging applications. Transportation Research Part C: Emerging Technologies, 22, 17–28. https://doi.org/10.1016/j.trc.2011.11.012
   Web of Science ®Google Scholar
• Barua, S., Haque, N., Das, A., Hadiuzzaman, M., & Hossain, S. (2017). Automatic multiregime fundamental diagram calibration using likelihood estimation. International Journal for Traffic and Transport Engineering, 7(1), 79–92. https://doi.org/10.7708/ijtte.2017.7(1).06
   Google Scholar
• Breiman, L. E. O. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
   Web of Science ®Google Scholar
• Bronte, S., Bergasa, L. M., & Alcantarilla, P. F. (2009). Fog detection system based on computer vision techniques [Paper presentation]. 2009 12th International IEEE Conference on Intelligent Transportation Systems, November. https://doi.org/10.1109/ITSC.2009.5309842
   Google Scholar
• Carrillo, J., Crowley, M., Pan, G., & Fu, L. (2019). Comparison of deep learning models for determining road surface condition from roadside camera images and weather data. Transportation Association of Canada and ITS Canada 2019 Joint Conference and Exhibition, Halifax, NS, 1–17.
   Google Scholar
• Chen, Z., Qin, X., & Shaon, M. R. R. (2018). Modeling lane-change-related crashes with lane-specific real-time traffic and weather data. Journal of Intelligent Transportation Systems, 22(4), 291–300. https://doi.org/10.1080/15472450.2017.1309529
   Web of Science ®Google Scholar
• Chern, N. N. K., Neow, P. A., & Ang, M. H. (2001). Practical issues in pixel-based autofocusing for machine vision. Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation, 3, Seoul, Korea, 2791–2796. https://doi.org/10.1109/robot.2001.933045
   Google Scholar
• Chicco, D. (2017). Ten quick tips for machine learning in computational biology. BioData Mining, 10(1), 1–17. https://doi.org/10.1186/s13040-017-0155-3
   PubMedGoogle Scholar
• Chu, W. T., Zheng, X. Y., & Ding, D. S. (2016). Image2weather: A large-scale image dataset for weather property estimation [Paper presentation]. IEEE 2nd International Conference on Multimedia Big Data, 137–144. https://doi.org/10.1109/BigMM.2016.9
   Google Scholar
• Dalal, N., Triggs, B. (2005). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition San Diego, CA.
   Google Scholar
• Das, A., & Ahmed, M. M. (2021). Exploring the effect of fog on lane-changing characteristics utilizing the SHRP2 naturalistic driving study data. Journal of Transportation Safety & Security, 13(5), 477–502. https://doi.org/10.1080/19439962.2019.1645777
   Web of Science ®Google Scholar
• Das, A., & Ahmed, M. M. (2021). Machine learning approach for predicting lane change maneuvers using the SHRP2 naturalistic driving study data. Transportation Research Record: Journal of the Transportation Research Board, 1–21. https://doi.org/10.1177/03611981211003581
   Google Scholar
• Das, A., Ahmed, M. M., & Ghasemzadeh, A. (2019). Using trajectory-level SHRP2 naturalistic driving data for investigating driver lane-keeping ability in fog: An association rules mining approach. Accident; Analysis and Prevention, 129, 250–262. https://doi.org/10.1016/j.aap.2019.05.024
   PubMed Web of Science ®Google Scholar
• Das, A., Ghasemzadeh, A., & Ahmed, M. M. (2019). Analyzing the effect of fog weather conditions on driver lane-keeping performance using the SHRP2 naturalistic driving study data. Journal of Safety Research, 68, 71–80. https://doi.org/10.1016/j.jsr.2018.12.015
   PubMed Web of Science ®Google Scholar
• Das, A., Khan, M. N., & Ahmed, M. M. (2020a). Nonparametric multivariate adaptive regression splines models for investigating lane-changing gap acceptance behavior utilizing strategic highway research program 2 naturalistic driving data. Transportation Research Record: Journal of the Transportation Research Board, 2674(5), 223–238. https://doi.org/10.1177/0361198120914293
   Web of Science ®Google Scholar
• Das, A., Khan, M. N., & Ahmed, M. M. (2020b). Detecting lane change maneuvers using SHRP2 naturalistic driving data: A comparative study machine learning techniques. Accident; Analysis and Prevention, 142, 105578. https://doi.org/10.1016/j.aap.2020.105578
   PubMed Web of Science ®Google Scholar
• Federal Highway Administration. (2020). How Do Weather Events Impact Roads? – FHWA Road Weather Management. http://www.ops.fhwa.dot.gov/weather/q1_roadimpact.htm
   Google Scholar
• Finch, H. (2005). Comparison of the performance of nonparametric and parametric MANOVA test statistics when assumptions are violated. Methodology, 1(1), 27–38. https://doi.org/10.1027/1614-1881.1.1.27
   Google Scholar
• Friedman, J., Hastie, T., & Tibshirani, R. (2008). The elements of statistical learning (2nd ed.). Springer.
   Google Scholar
• Gallen, R., Cord, A., Hautière, N., & Aubert, D. (2011). Towards night fog detection through use of in-vehicle multipurpose cameras [Paper presentation]. IEEE Intelligent Vehicles Symposium, 399–404. https://doi.org/10.1109/IVS.2011.5940486
   Google Scholar
• Gaweesh, S. M., Khan, M. N., & Ahmed, M. M. (2021). Development of a novel framework for hazardous materials placard recognition system to conduct commodity flow studies using artificial intelligence AlexNet Convolutional Neural Network. Transportation Research Record: Journal of the Transportation Research Board, In-Press, 1–18.
   Google Scholar
• Guerra, J. C. V., Khanam, Z., Ehsan, S., Stolkin, R., & McDonald-Maier, K. (2018). Weather classification: A new multi-class dataset, data augmentation approach and comprehensive evaluations of convolutional neural networks [Paper presentation]. 2018 NASA/ESA Conference on Adaptive Hardware and Systems (AHS), 305–310. https://doi.org/10.1109/AHS.2018.8541482
   Google Scholar
• Hankey, J. M., Miguel A, P., & McClafferty, J. A. (2016). Description of the SHRP2 naturalistic database and the crash, near-crash, and baseline data sets. In Virginia Tech Transportation Institute. https://doi.org/10.1017/CBO9781107415324.004
   Google Scholar
• Hassan, H. M., Abdel-Aty, M. A., Choi, K., & Algadhi, S. A. (2012). Driver behavior and preferences for changeable message signs and variable speed limits in reduced visibility conditions. Journal of Intelligent Transportation Systems, 16(3), 132–146. https://doi.org/10.1080/15472450.2012.691842
   Web of Science ®Google Scholar
• Hautiére, N., Labayrade, R., & Aubert, D. (2006). Real-time disparity contrast combination for onboard estimation of the visibility distance. IEEE Transactions on Intelligent Transportation Systems, 7(2), 201–211. https://doi.org/10.1109/TITS.2006.874682
   Web of Science ®Google Scholar
• Ibrahim, M., Haworth, J., & Cheng, T. (2019). WeatherNet: Recognising weather and visual conditions from street-level images using deep residual learning. ISPRS International Journal of Geo-Information, 8(12), 549. https://doi.org/10.3390/ijgi8120549
   Web of Science ®Google Scholar
• James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. Springer.
   Google Scholar
• Jonsson, P. (2011). Road condition discrimination using weather data and camera images [Paper presentation]. IEEE Conference on Intelligent Transportation Systems, ITSC, 4460318, 1616–1621. https://doi.org/10.1109/ITSC.2011.6082921
   Google Scholar
• Kawai, S., Takeuchi, K., Shibata, K., & Horita, Y. (2014). A smart method to distinguish road surface conditions at night-time using a car-mounted camera. IEEJ Transactions on Electronics, Information and Systems, 134(6), 878–884. https://doi.org/10.1541/ieejeiss.134.878
   Google Scholar
• Khan, M. N., & Ahmed, M. M. (2019). Snow detection using in-vehicle video camera with texture-based image features utilizing K-Nearest neighbor, support vector machine, and random forest. Transportation Research Record: Journal of the Transportation Research Board, 2673(8), 221–232. https://doi.org/10.1177/0361198119842105
   Web of Science ®Google Scholar
• Khan, M. N., & Ahmed, M. M. (2020). Trajectory-level fog detection based on in-vehicle video camera with TensorFlow deep learning utilizing SHRP2 naturalistic driving data. Accident; Analysis and Prevention, 142, 105521 https://doi.org/10.1016/j.aap.2020.105521
   PubMed Web of Science ®Google Scholar
• Khan, M. N., & Ahmed, M. M. (2021). Development of a novel convolutional neural network architecture named Roadweathernet for trajectory-level weather detection using SHRP2 Naturalistic Driving Data. Transportation Research Record: Journal of the Transportation Research, Board, 1–15. https://doi.org/10.1177/03611981211005470
   Google Scholar
• Khan, M. N., Das, A., & Ahmed, M. M. (2020). Non-parametric association rules mining and parametric ordinal logistic regression for an in-depth investigation of driver speed selection behavior in adverse weather using SHRP2 naturalistic driving study data. Transportation Research Record: Journal of the Transportation Research Board, 2674(11), 101–119. https://doi.org/10.1177/0361198120941509
   Web of Science ®Google Scholar
• Khan, M. N., Ghasemzadeh, A., & Ahmed, M. M. (2018). Investigating the impact of fog on freeway speed selection using the SHRP2 naturalistic driving study data. Transportation Research Record: Journal of the Transportation Research Board, 2672(16), 93–104. https://doi.org/10.1177/0361198118774748
   PubMed Web of Science ®Google Scholar
• Lee, J., Hong, B., Jung, S., & Chang, V. (2018). Clustering learning model of CCTV image pattern for producing road hazard meteorological information. Future Generation Computer Systems, 86, 1338–1350. https://doi.org/10.1016/j.future.2018.03.022
   Web of Science ®Google Scholar
• Lee, J., Hong, B., Shin, Y., & Jang, Y. J. (2016). Extraction of weather information on road using CCTV video [Paper presentation]. 2016, International Conference on Big Data and Smart Computing, BigComp 2016, 529–531. https://doi.org/10.1109/BIGCOMP.2016.7425986
   Google Scholar
• MathWorks (2020a). Extract histogram of oriented gradients (HOG) features. https://www.mathworks.com/help/vision/ref/extracthogfeatures.html
   Google Scholar
• MathWorks (2020b). Extract local binary pattern (LBP) features. https://www.mathworks.com/help/vision/ref/extractlbpfeatures.html#responsive_offcanvas
   Google Scholar
• Ojala, T., Pietikäinen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1), 51–59. https://doi.org/10.1016/0031-3203(95)00067-4
   Web of Science ®Google Scholar
• Ojala, T., Pietikäinen, M., & Mäenpää, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987. https://doi.org/10.1109/TPAMI.2002.1017623
   Web of Science ®Google Scholar
• Ozcan, K., Sharma, A., Knickerbocker, S., Merickel, J., Hawkins, N., & Rizzo, M. (2020). Road weather condition estimation using fixed and mobile based cameras. Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, 943, 192–204.
   Google Scholar
• Pan, G., Fu, L., Yu, R., & Muresan, M. (2019). Evaluation of alternative pre-trained convolutional neural networks for winter road surface condition monitoring [Paper presentation]. ICTIS 2019 - 5th International Conference on Transportation Information and Safety, Liverpool, UK, 614–620. https://doi.org/10.1109/ICTIS.2019.8883540
   Google Scholar
• Perry, A. H., & Symons, L. (2003). Highway Meteorology. Taylor & Francis Books, Inc.
   Google Scholar
• Pomerleau, D. (1997). Visibility estimation from a moving vehicle using the RALPH vision system [Paper presentation]. Proceedings of IEEE Conference on Intelligent Transportation Systems, pp. 906–911. https://doi.org/10.1109/ITSC.1997.660594
   Google Scholar
• Qian, Y., Almazan, E. J., & Elder, J. H. (2016). Evaluating features and classifiers for road weather condition analysis [Paper presentation]. International Conference on Image Processing, ICIP, Phoenix, AZ, 4403–4407. https://doi.org/10.1109/ICIP.2016.7533192
   Google Scholar
• Smadi, O. (2015). SHRP2 – Roadway information database. Iowa State University CTRE.
   Google Scholar
• Song, F., Tan, X., Liu, X., & Chen, S. (2014). Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recognition, 47(9), 2825–2838. https://doi.org/10.1016/j.patcog.2014.03.024
   Web of Science ®Google Scholar
• Stocker, M., Silvonen, P., Rönkkö, M., & Kolehmainen, M. (2016). Detection and classification of vehicles by measurement of road-pavement vibration and by means of supervised machine learning. Journal of Intelligent Transportation Systems: Technology, Planning, and Operations, 20(2), 125–137. https://doi.org/10.1080/15472450.2015.1004063
   Google Scholar
• Strang, G. (2009). Introduction to linear algebra (5th ed.). Wellesley-Cambridge Press.
   Google Scholar
• Suard, F., Rakotomamonjy, A., Bensrhair, A., & Broggi, A. (2006). Pedestrian detection using infrared images histograms of oriented gradients [Paper presentation]. Proceedings of the, 2006 IEEE Intelligent Vehicles Symposium,. https://doi.org/10.1109/IVS.2006.1689629
   Google Scholar
• Suthaharan, S. (2016). Support vector machine – machine learning models and algorithms for big data classification (pp. 207–235). Springer.
   Google Scholar
• Tian, S., Bhattacharya, U., Lu, S., Su, B., Wang, Q., Wei, X., Lu, Y., & Tan, C. L. (2016). Multilingual scene character recognition with co-occurrence of histogram of oriented gradients. Pattern Recognition, 51, 125–134. https://doi.org/10.1016/j.patcog.2015.07.009
   Web of Science ®Google Scholar
• Vlahogianni, E. I., Karlaftis, M. G., & Orfanou, F. P. (2012). Modeling the effects of weather and traffic on the risk of secondary incidents. Journal of Intelligent Transportation Systems, 16(3), 109–117. https://doi.org/10.1080/15472450.2012.688384
   Web of Science ®Google Scholar
• Yan, X., Luo, Y., Zheng, X. (2009). Weather recognition based on images captured by vision system in vehicle. In International Symposium on Neural Networks, 390–398. https://doi.org/10.1007/978-3-642-01513-7_42
   Google Scholar