Multiple comparisons problem: Recent advances applied to energy and emissions

Abstract

Within the field of empirical finance, the econometric analysis of markets commonly suffers from the well-established problem of data-snooping bias, whereby there is a likelihood that statistically significant results may be identified by pure random chance alone. This is the multiple comparisons problem resulting from Multiple Hypothesis Testing (MHT). Recent advances in MHT techniques to control the multiple comparisons problem are uniquely showcased within a VAR and Granger causality testing of energy and emissions market interactions. Four generalized p-value-based MHT techniques show no evidence of interactions between European Union Allowance (EUA) prices and a range of energy prices – spanning key oil, gas, coal and electricity markets – over the first half or so (2008–2010) of Phase II of the EU Emissions Trading Scheme. The generalized familywise error rate procedures show evidence of regional and cross-regional interactions within European electricity markets. However, in contrast, the more conservative false discovery proportion procedures identify much fewer statistically significant relationship and, indeed, show little evidence of such cross-regional electricity market interactions.

Keywords:

• multiple comparisons problem
• multiple hypothesis testing
• generalized familywise error rate
• false discovery proportion
• energy and emissions markets

JEL Classification:

• C12
• C32
• C52