Powerful Backtests for Historical Simulation Expected Shortfall Models

Abstract

Since 2016, the Basel Committee on Banking Supervision has regulated banks to switch from a Value-at-Risk (VaR) to an Expected Shortfall (ES) approach to measuring the market risk and calculating the capital requirement. In the transition from VaR to ES, the major challenge faced by financial institutions is the lack of simple but powerful tools for evaluating ES forecasts (i.e., backtesting ES). This article first shows that the unconditional backtest is inconsistent in evaluating the most popular Historical Simulation (HS) and Filtered Historical Simulation (FHS) $ES$ models, with power even less than the nominal level in large samples. To overcome this problem, we propose a new class of conditional backtests for $ES$ that are powerful against a large class of alternatives. We establish the asymptotic properties of the tests, and investigate their finite sample performance through some Monte Carlo simulations. An empirical application to stock indices data highlights the merits of our method.

KEYWORDS:

• Backtest
• Expected shortfall
• Historical simulation
• Filtered historical simulation
• Risk management

Acknowledgments

We would like to thank two anonymous referees, the associate editor and editor for their constructive comments that have significantly improved the article. We would also like to thank Juan Carlos Escanciano, Cheng Liu, Xi Qu, and the participants of 7th Annual Meeting of Young Econometricians in Asia-Pacific for their helpful comments. All errors are our own.

Disclosure Statement

The authors report there are no competing interests to declare.

Notes

1 Some papers propose backtesting ES by backtesting multiple VaRs, see for example, Emmer, Tasche, and Kratz (2015), Kratz, Lok, and McNeil (2018), Couperier and Leymarie (2020) as well as the related papers by Hurlin and Tokpavi (2006), Pérignon and Smith (2008) and Colletaz, Hurlin, and Perignon (2013).

2 HS and FHS methods use unconditional quantiles of raw data or standardized innovations, respectively, to forecast conditional quantiles and further calculate ES of asset returns.

3 Our argument is straightforward, as we show that unconditional backtests for HS and FHS ES are like evaluating the empirical cdf at the 5% empirical quantile, which equals 5% under both the null and alternative hypotheses.

4 The cumulative violation process is the integral of the violations over the coverage level in the left tail and accumulates all violations in the left tail just like the ES accumulating the VaRs in the left tail. Recall that violations are indicators for whether portfolio losses exceed the VaR.

5 Barone-Adesi, Giannopoulos, and Vosper (1999) use bootstrapped standardized residuals to approximate the distribution of ${\epsilon }_t,$ while another method is using the empirical cdf of standardized residuals, see for example, Gao and Song (2008) and Escanciano and Pei (2012). We follow the second method here, although we expect both methods to perform similarly as sample size goes to infinity.

6 A specification of the conditional distribution $F_t(·,{\Omega }_{t-1},{\theta }_0)$ is assumed for DE’s tests, which are originally introduced for parametric distributions, but, as pointed out in Footnote 6 of DE, they are readily extended to semiparametric specifications like those considered here, especially for cases with negligible estimation effects (see Assumption A4). We can actually show this using a similar argument as our proof for Theorem 3 with $g_1^{\ast }$ replaced by $H_{t-j,\alpha }({\theta }_0)-\alpha /2,$ $j\ge 1.$

7 Su et al. (2021) develop an empirical likelihood unconditional backtest for ES, which requires less finite moments than existing backtests and allows for robustness to heavier tails. However, their test requires the standardized innovation to follow a parametric distribution with Lipschitz continuous density function, which is not satisfied for the HS and FHS models considered here.

8 They differ in how the parameter θ₀ of the HS or FHS model is estimated. In the recursive scheme, the estimator ${\widehat{\theta }}_t$ is computed with all the sample available up to time t. In the rolling scheme, only the last R values of the series are used to estimate ${\widehat{\theta }}_t,$ that is, ${\widehat{\theta }}_t$ is constructed from the sample at periods $s=t-R+1,\dots ,t.$ Finally, in the fixed scheme, the parameter is not updated when new observations become available.

9 They consider a Box-Pierce type test given by $P{\sum }_{j=1}^m{\rho }_{H_{t-j,\alpha }({\theta }_0)-\alpha /2}^2$.

10 Actually, our method can also be extended to the choices of $g({\Omega }_{t-1})=g_2^{\ast }(Y_{t-1},Y_{t-2}):=E\left[H_{t,\alpha }({\theta }_0)|Y_{t-1},Y_{t-2}\right]$ and $g({\Omega }_{t-1})=g_3^{\ast }(Y_{t-1},Y_{t-2},Y_{t-3}):=E\left[H_{t,\alpha }({\theta }_0)|Y_{t-1},Y_{t-2},Y_{t-3}\right],$ and then our test will be powerful against alternatives under which $E\left[H_{t,\alpha }({\theta }_0)-\alpha /2|Y_{t-1},Y_{t-2}\right]\ne 0,$ and $E\left[H_{t,\alpha }({\theta }_0)-\alpha /2|Y_{t-1},Y_{t-2},Y_{t-3}\right]\ne 0,$ respectively. However, the resulting tests will be a little bit complicated then.

11 The Legendre polynomials $b_M(u)$ are defined as $b_M(u)=\frac{1}{M!}\frac{d^M}{d{u}^M}\left[{(u^2-u)}^M\right],$ where M is a nonnegative integer and $u\in [0,1]$. The first several Legendre polynomials are $b_0(u)=1,$ $b_1(u)=2u-1,$ $b_2(u)=6u^2-6u+1,$ and $b_3(u)=20u^3-30u^2+12u-1.$

12 We thank one of the referees for suggesting studying the interesting alternatives with structural breaks.

13 For FHS ES models, Novales and Garcia-Jorcano (2019) report low power of the unconditional backtests for some alternatives in their simulations, although they don’t raise this issue up or justify it theoretically. For HS or FHS VaR models, Pérignon and Smith (2008) and Escanciano and Pei (2012) report low power of the unconditional backtests for some alternatives in their simulations.

14 See for example, Van der Vaart (1998).

Additional information

Funding

Zaichao Du’s work was supported by National Natural Science Foundation of China, 72173029, Innovative Research Groups Project of the National Natural Science Foundation of China, 72121002, and the Innovation Program of Shanghai Municipal Education Commission, 2023SKZD01.