Backtesting extreme value theory models of expected shortfall

Abstract

We use stock market data to analyze the quality of alternative models and procedures for forecasting expected shortfall (ES) at different significance levels. We compute ES forecasts from conditional models applied to the full distribution of returns as well as from models that focus on tail events using extreme value theory (EVT). We also apply the semiparametric filtered historical simulation (FHS) approach to ES forecasting to obtain 10-day ES forecasts. At the 10-day horizon we combine FHS with EVT. The performance of the different models is assessed using six different ES backtests recently proposed in the literature. Our results suggest that conditional EVT-based models produce more accurate 1-day and 10-day ES forecasts than do non-EVT based models. Under either approach, asymmetric probability distributions for return innovations tend to produce better forecasts. Incorporating EVT in parametric or semiparametric approaches also improves ES forecasting performance. These qualitative results are also valid for the recent crisis period, even though all models then underestimate the level of risk. FHS narrows the range of numerical forecasts obtained from alternative models, thereby reducing model risk. Combining EVT and FHS seems to be best approach for obtaining accurate ES forecasts.

Keywords:

• Extreme value theory
• Skewed distributions
• Expected shortfall
• Backtesting
• Filtered historical simulation

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplemental data

Supplemental data for this article can be accessed at https://doi.org/10.1080/14697688.2018.1535182.

ORCID

Alfonso Novales http://orcid.org/0000-0002-2137-1789

Laura Garcia-Jorcano http://orcid.org/0000-0002-3656-8787

Notes

† In other studies VaR is the primary measure of interest, with ES left as a secondary consideration. Examples are Zhou (2012), Degiannakis et al. (2013), and Tolikas (2014), where not much focus is placed on ES forecasting patterns.

† An overview of ES backtesting procedures used in recent literature can be seen in Table A1 in the online appendix.

‡ Artzner et al. (1999) state four axioms which any risk measure used for effective risk regulation and management should satisfy: (i) homogeneity, (ii) subadditivity, (iii) monotonicity, and (iv) translation invariance. Such risk measures are said to be coherent.

§ Specifically, they show that VaR is subadditive in the relevant tail region when asset returns exhibit multivariate regular variation, for both independent and cross sectionally dependent returns, provided the mean is finite.

† For example, Kerkhof and Melenberg (2004) found methods that performed better than comparable VaR backtests.

‡ We provide a description of asymmetric probability distributions in Appendix A.1–A.3. All computations were performed with the R software (version 3.1.1) package rugarch (version 1.3-4) designed for the estimation and forecast of various univariate ARCH-type models.

§ An alternative leptokurtic and asymmetric distribution that has been considered in this context is the skewed generalized-t (SGT) distribution proposed by Theodossiou (1998). The SGT distribution has the attractive feature of encompassing most of the distributions that are usually assumed for standardized returns, such as the Gaussian, generalized error distribution (GED), Student-t and skewed Student-t distributions. Recently, Ergen (2015) has considered the skewed-t distribution proposed by Azzalini and Capitanio (2003) and Aas and Haff (2006) propose the use of the generalized hyperbolic skew Student-t distribution for unconditional and conditional VaR forecasting.

† Note that we focus on the lower tail of the data, and we have adapted all the formulations accordingly.

‡ The implied assumption is that the tail of the underlying distribution begins at the threshold u. From our sample of T data a random number of observations, $T_u$, will exceed this threshold. If we assume that the $T_u$ excesses over the threshold are i.i.d. with an exact GPD distribution, Smith (1987) has shown that the maximum likelihood estimates $\hat{\xi }={\hat{\xi }}_N$ and $\hat{\beta }={\hat{\beta }}_N$ of the GPD parameters ξ and β are consistent and asymptotically normal as $T_u\to \mathrm{\infty }$, provided $\xi >-1/2$. Under the weaker assumption that the excesses are i.i.d. from a $F_u\left(y\right)$ which is only approximately GPD he also obtains asymptotic normality results for ξ and β.

† Confidence bands are constructed applying the delta method, assuming that the sample mean follows a normal distribution.

† Except for IBM under fat-tailed distributions.

‡ Although the Johnson $S_U$ skew parameter is not significant at $5\mathrm{\%}$ for IBM and at $10\mathrm{\%}$ for BP.

§ In the estimation of EVT models, we use the R packages ismev (version 1.41) and evir (version 1.7-3).

¶ Figures 1 and 2 show the right tail, considering losses as positive numbers.

† Continuity and strict monotonicity allow for expected shortfall to be expressed as the expected value of returns below the value at risk.

‡ We have adapted the test statistics to apply to negative values of ${\mathrm{VaR}}_{\alpha }$ and ${\mathrm{ES}}_{\alpha }$. Acerbi and Szekely (2014) define them for positive ES values.

† Acerbi and Szekely (2014) show that the $Z_2$ test is more powerful than the $Z_1$ test when the null and alternative hypothesis differ in volatility, while $Z_1$ is more powerful than $Z_2$ in the case of different tail indices.

‡ Rosenblatt (1952), Crnkovic and Drachman (1996), Diebold et al. (1998), and Berkowitz (2001) are often credited with introducing PIT into the financial risk management backtesting literature. Graham and Pál apply a further transformation to the exponential context because it allows to solve for the saddle point analytically. This solution is, moreover, well defined over the complete interval of interest for tail losses.

† Another necessary condition for the series to be i.i.d. is that the $P\mathrm{\&}L$ time horizons do not overlap; otherwise, serial interdependencies may occur within the data.

‡ For more details, Lugannani and Rice (1980), Daniels (1987), and Wong (2010).

§ That amounts to return violations, in probability terms, following a uniform (0,1) distribution.

† $H_t\left(\alpha \right)=\left(1/\alpha \right){\int }_0^{\alpha }{\mathbb{1}}_{(u_t\le u)}\mathrm{d}u=\left(1/\alpha \right){\mathbb{1}}_{(u_t\le \alpha )}{\int }_0^{\alpha }{\mathbb{1}}_{(u_t\le u)}\mathrm{d}u=\left(1/\alpha \right){\mathbb{1}}_{(u_t\le \alpha )}{\int }_{u_t}^{\alpha }1\mathrm{d}u=\left(1/\alpha \right){\mathbb{1}}_{(u_t\le \alpha )}(\alpha -u_t)$.

‡ Note that we use the conditional mean restriction in the definition of autocorrelations. As a result, tests based on ${\gamma }_{Tj}$ are expected to have power against deviations from $H_{0c}$, where $H_t\left(\alpha \right)$ are uncorrelated but have mean different from $\alpha /2$.

† Figures 3 and 4 show only the negative returns so as to maintain a clear perspective on the different VaR and ES estimates.

25 Lambert and Laurent (2001) and Giot and Laurent (2003a) have shown that for various financial daily returns, it is realistic to assume that the standardized innovations ${\hat{z}}_t$ follow a skewed Student-t distribution.

26 The skewness parameter $\xi >0$ is defined so that the ratio of probability masses above and below the mean is $\frac{\mathrm{Prob}(z\ge 0|\xi )}{\mathrm{Prob}(z<0|\xi )}={\xi }^2.$

27 This parametrization is used by R package rugarch, which we use for estimating the parameters of our models.

Additional information

Funding

The authors gratefully acknowledge financial support from the grants Ministerio de Economia y Competitividad ECO2015-67305-P, Generalitat Valenciana PrometeoII /2013/015, Programa de Ayudas a la Investigación en Macroeconomía, Economía Monetaria, Financiera y Bancaria e Historia Económica 2015-2016 from Banco de España.