A Monte Carlo filtering application for systematic sensitivity analysis of computable general equilibrium results

ABSTRACT

Parameter uncertainty has fuelled criticisms on the robustness of results from computable general equilibrium models. This has led to the development of alternative sensitivity analysis approaches. Researchers have used Monte Carlo analysis for systematic sensitivity analysis because of its flexibility. But Monte Carlo analysis may yield biased simulation results. Gaussian quadratures have also been widely applied, although they can be difficult to apply in practice. This paper applies an alternative approach to systematic sensitivity analysis, Monte Carlo filtering and examines how its results compare to both Monte Carlo and Gaussian quadrature approaches. It does so via an application to rural development policies in Aberdeenshire, Scotland. We find that Monte Carlo filtering outperforms the conventional Monte Carlo approach and is a viable alternative when a Gaussian quadrature approach cannot be applied or is too complex to implement.

KEYWORDS:

• Monte Carlo filtering
• systematic sensitivity analysis
• computable general equilibrium model
• common agricultural policy
• rural development

Acknowledgements

The views expressed are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission or the University of Aberdeen. The authors would like to thank Maria Espinosa, Sergio Gomez y Paloma, three reviewers and the editor for valuable comments, and Javier Alba (and the IPTS-IT department) for technical assistance.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Pillar 1 is generally aimed at supporting producers through direct support and market measures.

2 Because it is unlikely that linear approximations to $\mathbf{\text{h}}\left(\mathit{\delta }\right)$ are appropriate in a CGE model, $E\left[\mathbf{\text{h}}\left(\mathit{\delta }\right)\right]$ is not a good approximation to the integral in Equation (2) and therefore employing mean values for exogenous variables will traditionally lead to significant approximation error in the estimate of the mean of results.

3 The convergence has to be discussed with respect to particular indicators. For instance, in our application, the traditional Monte Carlo approach requires more than 3,000 simulations to converge towards the mean agricultural GDP effect (we do not report these results), but needs 5,000 simulations to be within acceptable error margin (i.e. 0.01). Meanwhile, other variables of interest may still have higher error margins. More importantly, the sign for the total GDP effect has yet to converge towards the mean (negative) effect even after 5,000 MC simulations.

4 Since weights $w_k$ are equal and must sum to 1, then $w_k=1/2n$.

5 The KS test is conventionally used to assess the hypothesis that two samples were drawn from different populations (Neuhauser, Welz and Ruxton, 2017). Unlike the parametric t-test or the Wilcoxon–Mann–Whitney (or Mann–Whitney U) test, which test for differences in the location of two samples (differences in means or differences in average ranks respectively), the KS test is also sensitive to differences in the general shapes of the distributions in the two samples (i.e., to differences in dispersion, skewness, etc.). The Epps–Singleton test does not compare distributions directly but compare the empirical characteristic functions. This test has arguably similar power than the KS test though this has been subject to recent debate (Neuhauser, Welz and Ruxton, 2017). Finally, the KS test is easy to interpret and implement. We follow Saltelli et al. (2004; 2008) in the use of the KS test.

6 While this is an arbitrary number of simulations, we choose 550 simulations because it is roughly the double of the number needed to run the Stroud simulations (i.e., 272 × 2 = 544), as further explained in Section 5.2.

7 550 simulations are run for the MC and MCF approaches.