A deep learning model to detect foggy images for vision enhancement

Figures & data

Table 1. Considerable research for classifying foggy and clear images.

Authors	Method used	Features	Constraints	Accuracy
Yu et al. [23]	Eigenvalues; SVM	Can handle multiple images at a time	Considers random images	Not provided
Pavlic et al. [24]	Spectral Features; Linear Classifier	Global descriptors are used; Uses a large dataset	Does not perform well in high contrast; Generates horizontal gradients in dark regions	95.35%
Mao et al. [25]	Haze Degree Estimation	No need of entering manual input constraint	Can not be applied to a monochrome light source; Uses a very small dataset	Not provided
Thakur et al. [26]	Visibility Index	Handles both batch and real-time processing	Can not classify noisy images; A small dataset is used	92%
Wan et al. [27]	Gaussian Mixture Model	Can be performed on outdoor images	Picks up random images from the internet	89.8%
Shrivastava et al. [28]	19 Classification Techniques	Fits for both cluster and steady planning structure	Uses a small dataset; Can not deal well with noisy images	89.06%
Hao et al. [29]	Brightness Optimization; Colour Correction	Works with noisy images; Can be used both in the daytime and nighttime scenes	Does not consider haze level in small areas	89%
Chincholkar et al. [30]	CNN	Visibility distance can be computed	Uses a small dataset	82%

Figure 1. The network architecture of the proposed DNN without dropout layer.

Figure 2. The network architecture of the proposed DNN after solving the issue of data overfitting.

Table 2. Environment hyper-parameters.

Hyper-parameter	Value
LR	0.001
Decay of Weights	0.0001
Training Batch Size	64
Number of Epoch	10
Optimizer	Adam

Table 3. The data components used during experimentation of the model proposed.

	Value
Training data shape	(440, 28, 28)
Testing data shape	(110, 28, 28)
Total no. of outputs	2
Output classes	[0 1]

Table 4. Confusion matrices for red, green and blue colour components when dropout is not used.

Red	Confusion Matrix	Output YES	Output NO
	True YES	98	0
	True NO	0	12
Green	Confusion Matrix	Output YES	Output NO
	True YES	98	0
	True NO	1	11
Blue	Confusion Matrix	Output YES	Output NO
	True YES	98	0
	True NO	1	11

Table 5. Classification reports of the colour components when dropout is not used.

Colour Component	Class Label	Precision	Recall	$F_1$ Score
Red	0	1.00	1.00	1.00
	1	1.00	1.00	1.00
Green	0	0.99	1.00	0.99
	1	1.00	0.92	0.96
Blue	0	0.99	1.00	0.99
	1	1.00	0.92	0.96

Table 6. Accuracy reports of the colour components when dropout is not used.

Red	Accuracy			1.00
	Macro Average	1.00	1.00	1.00
	Weighted Average	1.00	1.00	1.00
Green	Accuracy			0.99
	Macro Average	0.99	0.96	0.98
	Weighted Average	0.99	0.99	0.99
Blue	Accuracy			0.99
	MacroAverage	0.99	0.96	0.98
	WeightedAverage	0.99	0.99	0.99

Figure 3. Training and validation loss for the (a) red (b) blue and (c) green colour components when dropout is not used: (a) training and validation loss for the red colour component; (b) training and validation loss for the green colour component and (c) training and validation loss for the blue colour component.

Figure 4. Training and validation accurcy for the (a) red (b) blue and (c) green colour components when dropout is not used: (a) training and validation accuracy for red colour component; (b) training and validation accuracy for the green colour component and (c) training and validation accuracy for the blue colour component.

Table 7. Table for the values of step loss, accuracy, validation loss and validation accuracy for each of the colour components after using dropout.

Metric	Colour component	Epoch
		01	02	03	04	05	06	07	08	09	10
Step Loss	Red	0.4006	0.3013	0.2547	0.2294	0.2356	0.2333	0.2214	0.2199	0.1849	0.2589
	Green	0.3910	0.2639	0.2523	0.2169	0.1830	0.1281	0.1177	0.1266	0.0589	0.0525
	Blue	0.3988	0.2579	0.2580	0.2594	0.2232	0.1717	0.1191	0.1230	0.1442	0.1033
Accuracy	Red	0.9202	0.9202	0.9202	0.9202	0.9202	0.9202	0.9202	0.9259	0.9459	0.9259
	Green	0.8551	0.9148	0.9148	0.9148	0.9148	0.9517	0.9574	0.9716	0.9801	0.9830
	Blue	0.8722	0.9148	0.9148	0.9148	0.9148	0.9148	0.9602	0.9602	0.9545	0.9631
4pc Validation Loss	Red	0.4402	0.4038	0.3273	0.3680	0.3245	0.3448	0.3361	0.3361	0.4948	0.2908
	Green	0.2616	0.2431	0.2388	0.2329	0.1739	0.2301	0.2543	0.2536	0.2363	0.2412
	Blue	0.2572	0.2560	0.2517	0.2534	0.2394	0.2404	0.2686	0.1643	0.1491	0.2033
Validation Accuracy	Red	0.8750	0.8750	0.8750	0.8750	0.8750	0.8977	0.8977	0.9091	0.8977	0.9091
	Green	0.8909	0.8909	0.8909	0.8909	0.9359	0.9218	0.9359	0.9359	0.9359	0.9473
	Blue	0.9091	0.9091	0.9091	0.9091	0.9091	0.9218	0.9091	0.9218	0.9202	0.9561

Table 8. Confusion matrices for red, green and blue components of the final model.

Red	Confusion Matrix	Output YES	Output NO
	True YES	98	0
	True NO	3	9
Green	Confusion Matrix	Output YES	Output NO
	True YES	98	0
	True NO	5	7
Blue	Confusion Matrix	Output YES	Output NO
	True YES	98	0
	True NO	8	4

Table 9. Classification reports of the colour components of the final model.

Colour Component	Class Label	Precision	Recall	F1 Score
Red	0	0.97	0.97	0.97
	1	0.67	0.67	0.67
Green	0	0.95	1.00	0.98
	1	1.00	0.58	0.74
Blue	0	0.92	1.00	0.96
	1	1.00	0.33	0.50

Figure 5. Training and validation loss for the (a) red (b) blue and (c) green colour components of the final model: (a) training and validation loss for the red colour component; (b) training and validation loss for the green colour component and (c) training and validation loss for the blue colour component.

Figure 6. Training and validation accuracy for the (a) red (b) blue and (c) green colour components of the final model: (a) training and validation accuracy for the red colour component; (b) training and validation accuracy for the green colour component and (c) training and validation accuracy for the blue colour component.

Table 10. Accuracy report for each of the colour components of the final model.

Red	Accuracy			0.94
	Macro Average	0.82	0.82	0.82
	Weighted Average	0.94	0.94	0.94
Green	Accuracy			0.95
	Macro Average	0.98	0.79	0.86
	Weighted Average	0.96	0.95	0.95
Blue	Accuracy			0.93
	Macro Average	0.96	0.67	0.73
	Weighted Average	0.93	0.93	0.91

Table 11. Overall accuracy and loss report of the final model.

Component	Validation		Testing
	Accuracy	Loss × 100	Accuracy	Loss × 100
Blue	91.7431	22.8344	93.5780	25.8716
Green	91.7431	23.6588	96.3303	18.8875
Red	92.6605	23.4250	94.4954	26.8286
Average	92.0489	23.3061	94.8012	23.8626

Figure 7. Classification of hazy and clear images on different outputs by different researchers: (a) clear images; (b) foggy images; (c) defogged by the method proposed by Li et al. [3]; (d) defogged by the method proposed by Sebastian et al. [37]; (e) defogged by the method proposed by Ancuti et al. [38]; (f) defogged by the method proposed by Ngo et al. [39]; (g) defogged by the method proposed by Li et al. [40].

Table 12. Comparative analysis of some existing machine learning classifiers.

Classifier	Accuracy	Remarks
Random Forest	93%	High computaional complexity, features are needed to be extracted.
Linear SVM	89%	High computaional complexity, features are needed to be extracted.
Polynomial SVM	88%	High computaional complexity, features are needed to be extracted.
Sigmoid SVM	88%	High computaional complexity, features are needed to be extracted.
Gaussian SVM	89%	High computaional complexity, features are needed to be extracted.
Proposed Model	94%	Low computational complexity, no need to extract features

Table 13. Ablation study of the proposed architecture with some existing deep learning image classification model.

Model	Accuracy
VGG16 [44]	75.59%
ResNet50 [45]	49.28%
MKL-Based Model [46]	71.8%
Alex Net Based Model [47]	90.2%
Proposed Model	94%

Table 14. Comparative analysis of the proposed model with some existing deep learning techniques.

Model	Accuracy
Kang et al. [48]	92%
Wang et al. [49]	91.4%
Xiao et al. [50]	92.6%
Satrasupalli et al. [51]	93%
Proposed Model	94%

Table 15. Cross validation score.

Colour component	Validation type	Accuracy
Red	Repeated K Fold	94.5%
	Time Series	90.2%
	Startified K Fold	94.8%
Green	Repeated K Fold	94.2%
	Time Series	88.4%
	Startified K Fold	91%
Blue	Repeated K Fold	95%
	Time Series	93%
	Startified K Fold	95.6%

Figure 8. Predicting time complexity for the method proposed.

Yu X, Xiao C, Deng M, et al. A classification algorithm to distinguish image as haze or non-haze. In: 2011 Sixth International Conference on Image and Graphics. IEEE; 2011. p. 286–289.

 Google Scholar

Pavlic M, Rigoll G, Ilic S. Classification of images in fog and fog-free scenes for use in vehicles. In: 2013 IEEE Intelligent Vehicles Symposium (IV). IEEE; 2013. p. 481–486.

 Google Scholar

Mao J, Phommasak U, Watanabe S, et al. Detecting foggy images and estimating the haze degree factor. J Comput Sci Syst Biol. 2014;7(6):226–228.

 Google Scholar

Thakur RK, Saravanan C. Classification of color hazy images. In: 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT). IEEE; 2016. p. 2159–2163.

 Google Scholar

Wan J, Qiu Z, Gao H, et al. Classification of fog situations based on gaussian mixture model. In: 2017 36th Chinese Control Conference (CCC). IEEE; 2017. p. 10902–10906.

 Google Scholar

Shrivastava S, Thakur RK, Tokas P. Classification of hazy and non-hazy images. In: 2017 International Conference on Recent Innovations in Signal processing and Embedded Systems (RISE). IEEE; 2017. p. 148–152.

 Google Scholar

Hao S, Wang P, Hu Y. Haze image recognition based on brightness optimization feedback and color correction. Information. 2019;10(2):81.

 Google Scholar

Chincholkar S, Rajapandy M. Fog image classification and visibility detection using cnn. In: International Conference on Intelligent Computing, Information and Control Systems. Springer; 2019. p. 249–257.

 Google Scholar

Li B, Peng X, Wang Z, et al. Aod-net: all-in-one dehazing network. In: Proceedings of the IEEE International Conference on Computer Vision. 2017. p. 4770–4778.

 Google Scholar

Salazar-Colores S, Cruz-Aceves I, Ramos-Arreguin J-M. Single image dehazing using a multilayer perceptron. J Electron Imaging. 2018;27(4):1–11.

 Web of Science ®Google Scholar

Ancuti CO, Ancuti C. Single image dehazing by multi-scale fusion. IEEE Trans Image Process. 2013;22(8):3271–3282.

 PubMed Web of Science ®Google Scholar

Ngo D, Lee S, Nguyen Q-H, et al. Single image haze removal from image enhancement perspective for real-time vision-based systems. Sensors. 2020;20(18):1–21.

 Web of Science ®Google Scholar

Li B.B.S.L.Q.Z.Z.H.Z, Zheng X. Fast region-adaptive defogging and enhancement for outdoor images containing sky. 2020.

 Google Scholar

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.

 Google Scholar

He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. p. 770–778.

 Google Scholar

Zhang Z, Ma H, Fu H, et al. Scene-free multi-class weather classification on single images. Neurocomputing. 2016;207:365–373.

 Web of Science ®Google Scholar

Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.

 Web of Science ®Google Scholar

Kang L-W, Chou K-L, Fu R-H. Deep learning-based weather image recognition. In: 2018 International Symposium on Computer, Consumer and Control (IS3C). IEEE; 2018. p. 384–387.

 Google Scholar

Wang C, Liu P, Jia K, et al. Identification of weather phenomena based on lightweight convolutional neural networks. CMC Comput Mater Continua. 2020;64(3):2043–2055.

 Web of Science ®Google Scholar

Xiao H, Zhang F, Shen Z, et al. Classification of weather phenomenon from images by using deep convolutional neural network. Earth Space Sci. 2021;8(5):e2020EA001604.

 Web of Science ®Google Scholar

Satrasupalli S, Daniel E, Guntur SR, et al. End to end system for hazy image classification and reconstruction based on mean channel prior using deep learning network. IET Image Process. 2020;14(17):4736–4743.

 Web of Science ®Google Scholar

Data availability

The data are available on request from the corresponding author.