Simulation-based model comparison methodology with application to road accident models

ABSTRACT

This article assumes the goal of proposing a simulation-based theoretical model comparison methodology with application to two time series road accident models. The model comparison exercise helps to quantify the main differences and similarities between the two models and comprises of three main stages: (1) simulation of time series through a true model with predefined properties; (2) estimation of the alternative model using the simulated data; (3) sensitivity analysis to quantify the effect of changes in the true model parameters on alternative model parameter estimates through analysis of variance, ANOVA. The proposed methodology is applied to two time series road accident models: UCM (unobserved components model) and DRAG (Demand for Road Use, Accidents and their Severity). Assuming that the real data-generating process is the UCM, new datasets approximating the road accident data are generated, and DRAG models are estimated using the simulated data. Since these two methodologies are usually assumed to be equivalent, in a sense that both models accurately capture the true effects of the regressors, we are specifically addressing the modeling of the stochastic trend, through the alternative model. Stochastic trend is the time-varying component and is one of the crucial factors in time series road accident data. Theoretically, it can be easily modeled through UCM, given its modeling properties. However, properly capturing the effect of a non-stationary component such as stochastic trend in a stationary explanatory model such as DRAG is challenging. After obtaining the parameter estimates of the alternative model (DRAG), the estimates of both true and alternative models are compared and the differences are quantified through experimental design and ANOVA techniques. It is observed that the effects of the explanatory variables used in the UCM simulation are only partially captured by the respective DRAG coefficients. This a priori, could be due to multicollinearity but the results of both simulation of UCM data and estimating of DRAG models reveal that there is no significant static correlation among regressors. Moreover, in fact, using ANOVA, it is determined that this regression coefficient estimation bias is caused by the presence of the stochastic trend present in the simulated data. Thus, the results of the methodological development suggest that the stochastic component present in the data should be treated accordingly through a preliminary, exploratory data analysis.

KEYWORDS:

• ANOVA
• DRAG
• Experimental design
• Methodological choice
• Monte Carlo simulation
• Quantitative model comparison
• Road accident models
• Unobserved components model

MATHEMATICS SUBJECT CLASSIFICATION:

• Primary 60, 62
• Secondary 62M10, 62K15

Funding

This work has been carried out in the framework of the research Project FURGOSEG - P24/08 “Development and application of an integrate methodology for the study of van-involved traffic accidents,” of the Spanish National Research Plan 2008–2011, Ministry of Innovation and Science (MICINN).

Notes

1 The DFP algorithm is similar to Newton–Raphson's, but it does not require the analytic calculations of second derivatives and approximates it iteratively.

2 The elasticity estimates of intervention variable are not reported in this table since it is assumed to be a fixed effect and would not be further used.

3 Full results of experimental design for each DRAG parameter are reported in the Appendix, Tables A.1–A.8.