Quasi-Monte Carlo application in CGE systematic sensitivity analysis

ABSTRACT

The uncertainty and robustness of Computable General Equilibrium (CGE) models can be assessed by conducting a Systematic Sensitivity Analysis (SSA). Different methods have been used in the literature for SSA of CGE models such as Gaussian Quadrature and Monte Carlo methods. This article explores the use of Quasi-random Monte Carlo methods based on the Halton and Sobol’ sequences as means to improve the efficiency over regular Monte Carlo SSA, thus reducing the computational requirements of the SSA. The findings suggest that by using low-discrepancy sequences, the number of simulations required by the regular MC SSA methods can be notably reduced, hence lowering the computational time required for SSA of CGE models.

KEYWORDS:

• Quasi-Monte Carlo
• CGE
• systematic sensitivity analysis
• GTAP
• Gaussian quadrature

JEL CLASSIFICATION:

• C63
• D58

I. Introduction

The use of Systematic Sensitivity Analysis (SSA) in the context of Computable General Equilibrium (CGE) models is gaining a new momentum within the CGE literature. SSA sought to address the input and calibration-data uncertainty of the models in order to capture the uncertainties surrounding CGE models and their results.

The applicability of SSA in CGE models becomes more prominent in cases where the models assess the macroeconomic implications of climate related changes. Modelling the economic effects of climate change is subject to substantial uncertainty resulting from the uncertainty in the environmental input data used. As discussed by Chatzivasileiadis et al. (2018), for example, in the IPCC RCP8.5 scenario, the expected mean sea level rise (SLR) in cm, by the year 2100, is 74 cm with a range of [52, 98] relative to the mean over 1986–2005 [Church et al. (2013)]. The uncertainty regarding the estimates of SLR is reflected in this range of almost half a meter. Thus, economic models that assess the effects of climate change need to reflect this uncertainty in their input data. Otherwise, the economic assessments connected to uncertain physical changes will suffer from a much higher level of uncertainty that gets compounded in the different modelling stages [Chatzivasileiadis et al. (2018)]. SSA can be used to estimate the statistical moments of the model outputs that are driven by the underlying uncertainties of the model inputs, such as stochastic shocks and parameters [Villoria and Preckel (2017)]. Nonetheless, the literature assessing the total economic effects of climate change does not reflect the combined effects of input uncertainty in a systematic way. Of course, this can be a result of the computational requirements of the SSA, on-top of the already computationally expensive economic models.

This article discusses the use of quasi-random Monte Carlo methods in SSA of CGE models. The main purpose is to economise the SSA process without losing any important information, while incorporating the uncertainty of climate input data in the economic findings.

We begin by defining the general form of a CGE model as:

(1) $G(x,\beta )=0$(1)

The most common approach of SSA in CGE models is based on Gaussian Quadrature (GQ) design. That choice is based on the fact that GQ requires only a few data points to approximate the central moments of stochastic variables, making this method very computational inexpensive. An alternative to the GQ is the regular Monte Carlo (rMC) SSA that requires a large amount of realisations such that: the mean ($\bar{x}$) in the univariate case as defined by Equation (2), is an unbiased estimator of $K(\beta )$,

(2) $I=E[K(\beta )]=\underset{a}{\overset{b}{\int }}K(\beta )p(\beta )d\beta =\bar{x}$(2)

(3) $I_N=\sum _{n=1}^N{w}_{n}K({\beta }_n)$(3)

where $w_n=\frac{1}{N}$ for each realizations n, x (Equation. (1)) represents a vector of results or endogenous variables (such as prices, welfare etc.), $\beta$ (Equation. (1)) is a vector of exogenous variables, $x^{\star }(\beta )\equiv K(\beta )$ is a vector of results for each given parameter $\beta$ and $p$ is the non-zero probability density function (pdf).

The difference between QG and rMC is in the way $w_n$ is defined. While in rMC all realizations have equal weight 1/N, in QG we choose the most appropriate points within the interval [a, b] and associated weights $w_n$, such that, the crude moments of the approximating distribution equals the moments of the true distribution from zero to some specified order Chatzivasileiadis et al. (2018). Thus, the GQ method is able to economise the computational requirements of the SSA (i.e. small number of simulations are required).

Economising the SSA through GQ is not without its drawbacks. For example, a significant amount of information is lost regarding the shape of the distribution, its higher-order moments, and its range. This information might be important when the shocks applied to the CGE model are expected to have asymmetric impacts.¹

By using rMC SSA instead, we sample from the total distribution of inputs, thus avoiding the GQ drawbacks at the cost of computation time. Moreover, the number of points required by the GQ is based on the shocks evaluated and the degree of the quadratures. For example in the case of Chatzivasileiadis et al. (2018), 4096 simulations were required by the Liu quadrature based on the GEMPACK software. Thus, in that case, the advantage of GQ that economises the SSA is questioned. ‘[…]An advantage of the rMC method, compared to GQ, lies on the estimation of the error. In the rMC method, the error is estimated from the generated data, whereas in the QG more global measures of error estimation are required such as the Chebyshev’s inequality for the confidence bounds. The Chebyshev’s inequality, will produce confidence bounds that are extremely conservative compared to the Central Limit Theorem which provides narrower confidence intervals if the available number of data points is sufficiently large[…]’ (Chatzivasileiadis et al., 2018; Villoria and Preckel 2017).

From the above, the conundrum choosing between; economising the SSA with GQ and gaining the advantages of rMC SSA is apparent. This article discusses a way to keep the advantages of the rMC SSA methodology while minimising the number of simulations required. We contribute to the existing literature by applying a Quasi-Monte Carlo (QMC) SSA on a static CGE model in order to economise the SSA without losing important information as in the GQ SSA, in a computationally inexpensive way, in contrast to the rMC SSA.

II. Methods and data

rMC and QMC

Based on the model of Chatzivasileiadis et al. (2016) and the input SSA discussed in Chatzivasileiadis et al. (2018), we explore the use of quasi-random realisations in the MC SSA of CGE models. We begin with a brief description of the differences between rMC and QMC SSA.

Going back to Equation (2), according to the Strong Law of Large Numbers for the rMC method, the approximation is convergent with probability one, i.e.

(4) $\underset{N\to \mathrm{\infty }}{lim}{I}_N\to I$(4)

Then the error of the MC integration is

(5) ${\epsilon }_N=I-I_N\approx \sigma N^{-\frac{1}{2}}x$(5)

if N is sufficiently large based on the Central Limit Theorem, where $\sigma$ is

(6) $\sqrt{Var(x)}=[\underset{a}{\overset{b}{\int }}(K(\beta )-E[K(\beta )]{)}^{2}p(\beta )\mathrm{d}\beta {]}^{\frac{1}{2}}$(6)

Based on the above, the convergence rate of the rMC is $O(N^{-\frac{1}{2}})$ which is independent of dimensions.

In the case of rMC, we randomly select points and proximate Equation (2) by the empirical average in Equation (3). In QMC the points are selected semi-deterministically, such that the chosen points provide the best possible spread. The chosen points are highly equidistributed thus providing greater uniformity compared to the pseudo-random numbers. In QMC, the resulting convergence rate is $O((logN{)}^k{N}^{-1})$, which is dependent on the k dimensions.² As a result, for the same number of evaluations, the QMC method, using low-discrepancy sequences, achieves higher accuracy thus faster convergence. This characteristic of the QMC method is very appealing in our case. We are interested in faster convergence, thus smaller number of simulations required to conduct a SSA of our CGE model.

One issue with QMC is that since the convergence is dependent on the dimensions of the problem, improved accuracy is lost in problems of high dimension. Another problem is due to existing correlations between the points of the quasi-random sequence used. The latter problem can be resolved by skip, leap over, or scramble the values in the sequence.

SSA input generation

In our analysis, we have used two different low-discrepancy sequences: the Halton and Sobol’ sequences Jank (2005). The Halton sequence is generated by using different prime bases to generate the sequence, where for the Sobol’ a base of 2 is used with a reordering of the coordinates in each dimension. In the Halton sequence we skip 1,000 and leap 100 using RR2 scrambler. The Sobol’ sequence is generated using leap 10,000 and leap 100 using the MatousekAffineOwen scrambler.

Based on Chatzivasileiadis et al. (2018), we have 13 regions and a total of 39 shocks applied to the CGE model. The process initially used for the rMC is the following:

1. Generate 39 pseudo-random samples (10,000 realisations) based on the uniform distribution;

2. Transform to the triangular distribution given the minimum, median and maximum for each regional changes of Land, Capital and Productivity;

3. Run 10,000 simulations based on these inputs.

In the QMC method the only difference is in step 1. Instead of the 39 pseudo-random samples we generate 39 quasi-random sequences based on the Halton and Sobol’ sequences. Steps 2 and 3 are followed as described above.

III. Results

For simplicity, we focus on one result of the CGE model; the Hicksian Equivalent Variation (HEV) for two of the regions only: Central-Asia and North-America. The choice of the regional results is arbitrary since all regions show similar differences between the rMC and the two QMC SSA methods.

First we discuss the differences between the shocks’ distributions. Here we only present the Halton sequence-based inputs, since we saw no substantial difference in the histogram with the Sobol’ generated inputs. Looking at both Figures 1 and 2 for Land, Capital and Productivity shocks, we see the expected; the QMC shocks are evenly distributed without any clustering (i.e. spikes in the histogram as around −20 for Central Asia’s productivity in Figure 1) compared to the rMC shocks’ histograms.

Figure 1. Shocks histogram based on the triangular distribution using pseudo-random sampling and Halton sequence for Central Asia.

Figure 2. Shocks histogram based on the triangular distribution using pseudo-random sampling and Halton sequence for North America.

In Central Asia (Figure 3) the mean is stable after approximately 4,000 simulations which are already 96 less than the QG method required as discussed above. In the Halton-based SSA results, we find that the mean converges to the rMC 10,000 runs mean (black line) after only 500 simulations and the confidence interval (CI) is similar at 2,000 simulations with the CI of 4,000 simulations in the rMC. Looking at the Sobol’ QMC results, even though the mean converges at around 800 simulations, the Sobol’-based QMC is outperformed by the Halton QMC method, but is preferred from the rMC SSA. The same conclusion can be derived by looking at Figure 4 of the results’ standard deviation (SD). The SD is more stable in the Halton QMC method after 500 simulations compared to the 1,500 required in the Sobol’ QMC. Clearly though, both quasi-random methods produce significantly lower SD estimates compared to the rMC results.

Figure 3. DIVA-C HEV mean convergence for the triangular distribution using: random sampling, Halton and Sobol’ sequences for Central Asia.

Figure 4. DIVA-C HEV Standard deviation changes for the triangular distribution using: random sampling, Halton and Sobol’ sequences for Central Asia.

In North America (Figure 5), the superiority of the Halton-based method is more prominent. Clearly, the Halton-based results converge after 500 simulations, but the Sobol’-based results have not converged yet, before 2,000 simulations. Nevertheless, these results are superior to the 4,000 simulations required by the rMC method. The North American SD results (Figure 6) indicate that the SD reaches a plateau after 500 simulations in the Halton and 1,000 in the Sobol’-based results and that plateau is smaller than the rMC one.

Figure 5. DIVA-C HEV mean convergence for the triangular distribution using: random sampling, Halton and Sobol’ sequences for North America.

Figure 6. DIVA-C HEV Standard deviation changes for the triangular distribution using: random sampling, Halton and Sobol’ sequences for North America.

IV. Conclusion

In this article, we discuss the use of QMC methods based on the Halton and Sobol’ sequences for SSA of CGE model. Our main interest is to provide evidence of efficiency gains with using QMC in SSA compared to rMC.

Based on our empirical results, both Halton and Sobol’ based QMC methods are more efficient than the rMC method, but the Halton-based results are preferred. This is based mainly on the convergence of the results. Our results indicate that there is a clear computational-time advantage in the use of low-discrepancy sequence compared to pseudo-random numbers and this method should be considered when conducting a SSA of a CGE model. In this article we have only used two quasi-random sequences, but it would be interesting in the future to compare these results with other quasi-random sequences such as the Faure sequence, the Hammersley set and the Niederreiter sequence.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

¹ Additional information about the differences between GQ and rMC can be found in Chatzivasileiadis (2018) and Villoria and Preckel (2017).

² See Caflisch (1998) for more information.