Forecasting air quality time series using deep learning

Figures & data

Figure 1. Individual node model.

Table 1. Common activation functions.

Name	Equation	First derivative	Output range
sigmoid
tanh			−1, 1
relu

Figure 2. Different ANN model architectures: (a) simple feed-forward neural network, (b) a recurrent (Elman) neural network, and (c) a deep feed-forward neural network with multiple hidden layers.

Figure 3. Correlogram of O₃ and NOx for 72 hr.

Figure 4. Architecture of an RNN showing layers unfolding in time.

Table 2. Enhanced first-order optimizers used for DL.

Optimizer	Description summary	Source
AdaGrad	Divides the learning rate, α, by the L2 norm	Duchi (2011)
RMSProp	Divides gradient by a running average of its recent magnitude	Tieleman (2012)
Adam	Combines CM with RMSProp	Kingman (2014)
Nadam	Combines NAG with RMSProp	Dozat (2016)

Figure 5. LSTM architecture showing unit time delays (−1), gates, and recurrentactivation functions (σ).

Figure 6. Location of Kuwait and Air Monitoring Station used in the study.

Table 3. Chemical and meteorological parameters captured at the Air Monitoring Station.

Chemical analytes	Meteorological
Nitrous oxide (NO)	Wind direction (WD)
Ammonia (NH3)	Wind speed (WS)
Ozone (O3)	Temperature
Sulfur dioxide (SO2)	Atmospheric pressure (ATP)
Formaldehyde (CH2O)	Solar radiation (SR)
Nitrogen dioxide (NO2)	Relative humidity (RH)
Benzene (C6H6)
Toluene (C7H8)
p-Xylene (C8H10)
m Xylene (C8H10)
1,2,3-trimethylbenzene (C6H3(CH3)3)
o-Xylene (C8H10)
Ethylene glycol tertiary butyl ether (ETB) (C6H14O2)
Styrene (C8H8)
Chlorine (Cl2)
Carbon dioxide (CO2)
Methane (CH4)
Hydrogen sulfide (H2S)
Carbon monoxide (CO)

Figure 7. Station wind rose from 2012 to 2014.

Figure 8. Seasonal bivariate polar plots of 1-hr O_3.

Figure 9. Hourly averages of 1-hr O₃ and NOx.

Figure 10. RMSE of O₃ and NOx from consecutive gaps of data using the first imputation method.

Figure 11. Feature importance from decision tree prediction of 8-hr O₃ exceedances from 1 hr to 12 hr.

Figure 12. Process of converting data input columns into a tensor for training the RNN.

Figure 13. Loss function errors for training and test data sets for different horizons at (a) 24 hr, (b) 36 hr, and (c) 48 hr.

Figure 14. Results of training an RNN with a 24-hr horizon.

Figure 15. Distribution of residual test errors for 24-hr horizon network.

Figure 16. Training error associated with feature reduction on network prediction.

Table 4. Default values for parameter sensitivity analysis.

Parameter	Default value
Input features	26
Prediction horizon (hr)	24
Look back nodes	26
Samples/batch	72
Dropout factor	0.2

Figure 17. Prediction horizons using five features and default parameters.

Figure 18. Impact of (a) batch samples, (b) look-back nodes, and (c) dropout factor parameters on training errors in the model.

Table 5. Comparison of RNN test data results to previously published results.

Source	Prediction horizon	Results (RMSE)	RNN (RMSE)
Luna et al. (2014)	1 hr	6.3–12.3	0.8
Feng et al. (2011)	12 hr	5.5–86.9	1.5
Wang and Lu (2006)	24 hr	7.9–11.2	2.5
Gomez et al. (2003)	24 hr	6.9–9.9

Table 6. Comparison of different forecasting errors over a 24-hr period.

	RNN	FFNN	ARIMA
MAE (ppb)	0.235	15.28	23.574

Figure 19. Comparison of different model forecasts over a 24-hr period.

Duchi, J., E. Hazan, and Y. Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12:2121–2159. http://dl.acm.org/citation.cfm?id=1953048.2021068.

 Web of Science ®Google Scholar

Tieleman, T., and G. Hinton 2012. Lecture 6.5 - RMSprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 4. http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf (accessed February 12, 2018).

 Google Scholar

Kingma, D.P., and J. Ba. 2014. Adam: A method for stochastic optimization. CoRR abs/1412.6980. http://arxiv.org/abs/1412.6980. (accessed February 18, 2018).

 Google Scholar

Dozat, T. (2016). Incorporating Nesterov momentum into ADAM. In 4th International Conference on Learning Representations. https://openreview.net/pdf?id=OM0jvwB8jIp57ZJjtNEZ.(accessed February 12, 2018).

 Google Scholar

Luna, A.S., M.L.L. Paredes, G.C.G. de Oliveira, and S.M. Corra. 2014. Prediction of ozone concentration in tropospheric levels using artificial neural networks and support vector machine at Rio de Janeiro, Brazil. Atmos. Environ. 98:98–104. doi:10.1016/j.atmosenv.2014.08.060.

 Web of Science ®Google Scholar

Feng, Y., W. Zhang, D. Sun, and L. Zhang. 2011. Ozone concentration forecast method based on genetic algorithm optimized back propagation neural networks and support vector machine data classification. Atmos. Environ. 45 (11):1979–1985. doi:10.1016/j.atmosenv.2011.01.022.

 Web of Science ®Google Scholar

Wang, D., and W.-Z. Lu. 2006. Ground-level ozone prediction using multilayer perceptron trained with an innovative hybrid approach. Ecol. Modell. 198 (34):332–340. doi:10.1016/j.ecolmodel.2006.05.031.

 Google Scholar

Gomez, P., Nebot, Angela, Ribeiro, Sabrine, Alquézar, René, Mugica, Francisco,Wotawa and Franz. 2003. Local maximum ozone concentration prediction using soft computing methodologies. Syst. Anal. Modell. Simul. 43 (8):1011–1031. doi:10.1080/0232929031000081244.

 Google Scholar