V2X, GNSS, radar, and camera-based intelligent system for adaptive control of heavy mining vehicles during foggy weather

Abstract

One of the most challenging scenarios for drivers of open-pit mines is driving in dense fog. As a result, drivers cannot see the road, which causes significant vehicle colloidal accidents. To overcome the problem, an intelligent vision enhancement system has been developed by real-time processing and integrating data from the vehicle-to-everything (V2X) module, global navigation satellite system (GNSS), radar, and infrared cameras to enhance the visibility range of the operative road to the driver and mitigate the chances of accidents. Final images after image processing and object detection are displayed using a dashboard screen in front of the driver for adaptive control and safe operation. Performance analysis of the visual computing algorithm has been carried out in a mine based on image quality parameters. Field-tested images’ average contrast value, peak signal-to-noise ratio, structural similarity index, visual information fidelity, and universal quality index were 0.87, 19.36, 0.39, 0.82, and 0.56, respectively, and the calculated accurateness of the convolution neural network (CNN)-based object detection model was 90%. These values indicate the output of the image processing algorithm provided better-enhanced images in the foggy condition. With the help of image processing algorithms, the camera view is able to provide a clear vision of more than 30 m in dense fog. Further, using V2X technology, the augmented visibility range has been increased up to 200 m. The 3D augmented reality (AR) view and 2D navigation view provide other surrounding vehicle locations. Untagged objects like rocks or bunds are also detected and displayed using radar.

Graphical Abstract

Highlights

• The system fuses IR cameras, V2X, GNSS, radar, & visual computing for safe driving

• Image processing algorithm fuses CLAHE and DCP methods for clear image output

• CNN-based object detection algorithm integrated with image processing software

• Provides multi-stage safety using camera, 2D navigation, & 3D virtual reality views

• Intelligent system inbuilt with software and advanced analytics for fleet management

Keywords:

• adaptive control
• artificial intelligence
• augmented reality
• intelligent system
• vehicle-to-everything technology

Acknowledgments

The authors are thankful to the Director, CSIR-Central Institute of Mining and Fuel Research, Dhanbad, India, for granting permission to publish the paper. In addition, the authors are grateful to the Ministry of Electronics and Information Technology, the Government of India, and National Mineral Development Corporation Limited, Hyderabad, India, for supporting the project jointly.

Authors’ contributions

Swades Kumar Chaulya: Conceptualized and proposed the technical solution, technical supervision and manuscript writing and editing, Monika Choudhury: Development of algorithm for image processing and drafting of manuscript, Girendra Mohan Prasad: Reviewing the article for intellectual content and verification of analysis results with technical input, Naresh Kumar: Generation of 3D geo-tagged mine map using drone and LiDAR and verification of data, Vikash Kumar: Field deployment at mine site and generation of experimental data, Virendra Kumar: Field studies at mine site and generation of experimental data, Abhishek Chowdhury: Generation of field experimental data and deployment of the system in a mine, Preity Mishra: Laboratory studies, data analysis and validation of data.

Compliance with ethical standards

The authors declare that this paper has never been submitted to other journals for simultaneous review processes and it has not been published before (partially or completely). Moreover, this paper has not been divided into different sections in order to have more submissions. The data has not been fabricated or changed in favor of the conclusions. In addition, no theory, data, or text belonging to other authors and publications was included as if it was our own and proper acknowledgement to other works has been provided.

Research involving human participants and/or animals

The authors declare that no human participant or animal was included in this work. The authors state that this submission does not require any informed consent due to the fact that the results were acquired by computer simulation.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Additional information

Funding

This research was funded by the Ministry of Electronics and Information Technology, Government of India, and National Mineral Development Corporation Limited, Hyderabad, India, jointly with grant no. 16(1)/2018-ESDA.

Notes on contributors

Swades Kumar Chaulya

Swades Kumar Chaulya has been working at CSIR-Central Institute of Mining and Fuel Research (CSIR-CIMFR) in Dhanbad, India since 1993. He holds Ph.D. in Mining Engineering and focuses on designing and developing communication, monitoring, control, and automation systems for mines.

Monika Choudhury

Monika Choudhary is a Project Associate, working on a project that aims to enhance vision in foggy weather at opencast mines. She has a degree in Electronics and Communication Engineering.

Girendra Mohan Prasad

Girendra Mohan Prasad holds a Ph.D. in Applied Physics. His research area includes mine instrumentation, communication, automation, and control.

Naresh Kumar

Naresh Kumar is a Senior Technical Officer. His research focuses on instrumentation, calibration, telecommunication, and safety testing.

Vikash Kumar

Vikash Kumar is a Technical Officer, specializing in artificial intelligence and machine learning. He holds a degree in Computer Science and Engineering.

Virendra Kumar

Virendra Kumar works as an Executive Engineer (Electrical). He has a degree in Electrical Engineering and his research focuses on power distribution management and electrical safety.

Abhishek Chowdhury

Abhishek Chowdhury is a Technical Officer, specializing in Electronics and Instrumentation Engineering. His research includes developing electronic systems, mine instrumentation, automation, and control.

Preity Mishra

Preity Mishra is a Technical Officer with a degree in Electronics and Communication Engineering. Her research focuses on communication, electronics, and instrumentation