Bias, exploitation and proxies in scenario-based risk minimization

Abstract

When minimizing a risk measure over a set of scenarios, solutions are often optimistic in the sense that the in-sample, or perceived, risk is much less than the out-of-sample risk. Optimism, which can be attributed to the bias of the risk estimator and the exploitation of the scenarios' idiosyncracies by the optimization, increases with the amount of sampling error inherent in the scenarios and the flexibility afforded by the problem's formulation. Minimizing a proxy, namely an estimator of a risk measure different from the one that is actually of interest, can reduce optimism and improve out-of-sample performance. The effectiveness of a proxy depends on the sizes of its perceived risk, bias and exploitation in relation to those of the estimator for the risk measure of interest.

Keywords:

• optimism
• bias
• exploitation
• proxy
• optimization
• scenario
• risk

Acknowledgements

Oleksandr Romanko's research was partially supported by Mitacs. We are grateful to Taehan Bae for valuable suggestions and discussions.

Notes

1. The second inequality holds by definition, since π( w *(π)) ≤ π( w ) for all w . However, the perceived risk p _N ( w *(p _N , S ^o ), S ^o ) may exceed both the true optimal risk π( w *(π)) and the actual risk π( w *(p _N , S ^o )), e.g. if p _N is positively biased or S ^o is an uncharacteristically ‘bad’ sample. Mak et al. 17 prove that when π is the mean.

2. We use this convention so that losses are positive rather than negative.

3. The terms ‘CVaR’, ‘expected shortfall’ and ‘conditional tail expectation’ are often used inter-changeably, although subtle differences may exist in their respective definitions. While such differences are inconsequential for continuous distributions, they can become apparent in the discrete case (see 1,26 for details).

4. Since the χ²(2) approximation for the distribution of the Jarque-Bera statistic is known to be imprecise for small samples, we used the critical values tabulated by Wuertz and Katzgraber 29.

5. We chose this range to avoid numerical difficulties when generating scenarios of joint asset returns. Specifically, the procedure used sometimes gave large simulation errors for assets with Jarque-Bera statistics above 822.

6. The source code was downloaded from http://www.iot.ntnu.no/users/kaut/

7. All optimizations are performed using CPLEX 12.2 on a server with 8 Quad-Core AMD Opteron Processors 8356 (32 cores in total) and 256 Gb of RAM. Four threads are used for solving all problems and CPLEX parameters are left at their default values except when minimizing variance, where cross-over is explicitly selected.

8. The estimators can be ordered based on the increasing extremeness of their associated risk measures as follows: , h _0.90,N , h _0.95,N , h _0.99,N , h _0.999,N .

9. The optimism for is slightly negative when n = 5000, i.e. the actual risk is less than the perceived risk. Since this does not persist for larger sample sizes, we attribute it to an uncharacteristically high average perceived risk for the 25 trials considered. In this case the sample variance remains an effective proxy even though the optimism is negative.