Powering AI-driven car damage identification based on VeHIDE dataset

Figures & data

Table 1. Datasets for classification, detection, and segmentation of the vehicle damage.

Dataset	Task	Size	PA	Categories	Damage Categories
de Deijn (2018)	C	1,007	N	4	dent, glass, scratch, and hail
Balci et al. (2019)	C	533	N	2	damaged, and non-damaged
Waqas et al. (2020)	C	600	N	3	medium damaged, huge damage, and no damaged
Patil et al. (2017)	C	1,503	N	7	bumper dent, door dent, glass shatter, scratch, tail lamp broken, scratch, smash, and headlamp broken
Li et al. (2018)	D	1,790	N	3	crack, scratch, and dent
Patel et al. (2020)	D	326	N	N/A	N/A
Dhieb et al. (2019)	D, S	N/A	N	3	minor, moderate, and major
Singh et al. (2019)	S	2,822	N	5	scratch, minor dent, major dent, missing, and cracked
Wang et al. (2022)	C, D, S, SOD	4,000	Y	6	scratch, dent, glass shatter, crack, tire flat, and lamp broken
VehiDE	C, D, S, SOD	13945	Y	8	dents, broken glass, scratch, lost parts, punctured, torn, broken, lights, non-damage

Note: The letters C, D, S, and SOD encompass damage classification, detection, segmentation, and salient object detection, resp.N/A: The authors omitted to include that information.

Figure 1. Architecture overview of the detection system based on YoLo approach.

Figure 2. Mask R-CNN model architecture.

Figure 3. Comparison of VehiDE statistics and other datasets.

Figure 4. Samples of annotated images in the VehiDE dataset. The first row presents the images of damaged vehicle; the second row shows the annotated labels; the last row demonstrates the masks of vehicles.

Figure 6. Visual comparison of diverse Salient Object Detection (SOD) methods.

Figure 5. Mask R-CNN network with parallel semantic segmentation branch.

Table 2. SOD method hyper-parameters. $[i,j]$ denotes the interval between the ith and jth epochs.

	SGL-KRN	PoolNet	U${}^2$ Net	CSNet
Weight Decay	5e−2	5e−2	0	5e−1
Batch Size	6	6	12	16
Maximun Epochs	100	100	600	300
Learning Rate	[1, 50]: 5e−2	[1, 50]: 5e−2	2e−3	[1, 200]: 1e−4
	[51, 100]: 5e−4	[51, 100]: 5e−4		[201, 250]: 1e−5
				[251, 300]: 1e−6

Table 3. Results of CSNet, $U^2$-Net, PoolNet, and SGL-KRN's quantitative salient object detection.

	$F_{\beta }\uparrow$	$F_{\beta }^w\uparrow$	$S_m\uparrow$	$E_m\uparrow$	$\mathrm{MAE}\downarrow$
CSNet (Cheng et al., 2022)	0.634	0.652	0.720	0.770	1.120
U${}^2$-Net (Qin et al., 2020)	0.696	0.701	0.809	0.810	0.730
PoolNet (Liu et al., 2019)	0.823	0.775	0.834	0.885	0.069
SGL-KRN (Xu et al., 2021)	0.832	0.805	0.842	0.938	0.067

Table 4. Results of SGL-KRN's quantitative salient object detection for each class.

	$F_{\beta }\uparrow$	$F_{\beta }^w\uparrow$	$S_m\uparrow$	$E_m\uparrow$	$\mathrm{MAE}\downarrow$
dents	0.689	0.672	0.770	0.879	0.058
broken glass	0.801	0.823	0.834	0.822	0.051
scratch	0.731	0.781	0.758	0.831	0.062
lost parts	0.832	0.855	0.882	0.921	0.032
punctured	0.849	0.782	0.863	0.895	0.043
torn	0.984	0.936	0.992	0.935	0.025
broken lights	0.885	0.925	0.873	0.938	0.035
All classes	0.820	0.761	0.836	0.912	0.046

de Deijn, J (2018). Automatic car damage recognition using convolutional neural networks.

 Google Scholar

Balci, B., Artan, Y, & Elihos, A. (2019). Front-view vehicle damage detection using roadway surveillance camera images. Proceedings of the 5th International Conference on Vehicle Technology and Intelligent Transport Systems – Volume 1: Vehits (pp. 193–198), Crete, Greece, Scitepress.

 Google Scholar

Waqas, U. A., Akram, N., Kim, S. Y., Lee, D., & Jeon, J. Y. (2020). Vehicle damage classification and fraudulent image detection including moiré effect using deep learning. In 2020 . Proceedings of IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 1–5), Edmonton, AB, Canada, IEEE.

 Google Scholar

Patil, K., Kulkarni, M., Sriraman, A., & Karande, S. S. (2017). Deep learning based car damage classification. Proceedings of 16th IEEE International Conference on Machine Learning and Applications (ICMLA 2017) (pp. 50–54), Cancun, Mexico, IEEE.

 Google Scholar

Li, P., Shen, B., & Dong, W. (2018). An anti-fraud system for car insurance claim based on visual evidence. Preprint arXiv:1804.11207.

 Google Scholar

Patel, N., Shinde, S. R., & Poly, F. (2020). Automated damage detection in operational vehicles using mask R-CNN. In Advanced computing technologies and applications (pp. 563–571). Springer.

 Google Scholar

Dhieb, N., Ghazzai, H., Besbes, H, & Massoud, Y. (2019). A very deep transfer learning model for vehicle damage detection and localization. In 2019 31st International Conference on Microelectronics (ICM) (pp. 158–161), Cairo, Egypt, IEEE.

 Google Scholar

Singh, R., Ayyar, M. P., Pavan, T. S., Gosain, S., & Shah, R. R. (2019). Automating car insurance claims using deep learning techniques. Proceedings of 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM) (pp. 199–207), Singapore, IEEE.

 Google Scholar

Wang, X., Li, W., & Wu, Z. (2022). Cardd: A new dataset for vision-based car damage detection. IEEE Transactions on Intelligent Transportation Systems, 24(7), 7202–7214. https://doi.org/10.1109/TITS.2023.3258480

 Web of Science ®Google Scholar

Cheng, M. M., Gao, S. H., Borji, A., Tan, Y. Q., Lin, Z., & Wang, M. (2022). A highly efficient model to study the semantics of salient object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 8006–8021. https://doi.org/10.1109/TPAMI.2021.3107956

 PubMed Web of Science ®Google Scholar

Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O. R., & Jagersand, M. (2020). U2-net: Going deeper with nested U-structure for salient object detection. Pattern Recognition, 106, 107404. https://doi.org/10.1016/j.patcog.2020.107404

 Web of Science ®Google Scholar

Liu, J. J., Hou, Q., Cheng, M. M., Feng, J, & Jiang, J. (2019). A simple pooling-based design for real-time salient object detection. In Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3917–3926), Long Beach, CA, USA, IEEE.

 Google Scholar

Xu, B., Liang, H., Liang, R., & Chen, P. (2021). Locate globally, segment locally: A progressive architecture with knowledge review network for salient object detection. Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35 (pp. 3004–3012), Virtually.

 Google Scholar