On the efficiency of German growth forecasts: an empirical analysis using quantile random forests and density forecasts

ABSTRACT

We use quantile random forests (QRF) to study the efficiency of the growth forecasts published by three leading German economic research institutes for the sample period from 1970 to 2017. To this end, we use a large array of predictors, including topics extracted by means of computational-linguistics tools from the business-cycle reports of the institutes, to model the information set of the institutes. We use this array of predictors to estimate the quantiles of the conditional distribution of the forecast errors made by the institutes, and then fit a skewed t-distribution to the estimated quantiles. We use the resulting density forecasts to compute the log probability score of the predicted forecast errors. Based on an extensive in-sample and out-of-sample analysis, we find evidence, particularly in the case of longer-term forecasts, against the null hypothesis of strongly efficient forecasts. We cannot reject weak efficiency of forecasts.

JEL CLASSIFICATION:

• C53
• E32
• E37

KEYWORDS:

• Growth forecasts
• forecast efficiency
• quantile-random forests
• density forecasts

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

For obtaining the data, please send an e-mail to the corresponding author.

Notes

¹ Other aspects of macroeconomic forecasts for Germany have been studied in recent research by, for example, Heilemann and Stekler (2013), who focus on the time-varying accuracy of forecasts, Kirchgässner and Müller (2006), who analyse costly forecast revisions, and Döpke and Fritsche (2006), who use panel-data methods to show that macroeconomics forecasts are unbiased and weakly efficient.

² Our description of regression trees is relatively compact. For a more detailed description and numerical examples, see Behrens, Pierdzioch, and Risse (2018a).

³ The research institutes are: Deutsches Institut für Wirtschaftsforschung, Ifo Institut, and Institut für Weltwirtschaft.

⁴ In order to account for data revisions, we use a backward-looking moving-average of order 12 to smooth out the effects of retrospective data revisions (CPI, M1, real effective exchange rate, industrial production, and orders).

⁵ We also tested the weak efficiency of the forecasts by comparing the log probability score of a quantile-regression model that features only a constant with the log probability score of a quantile-regression model that features the lagged forecast error as a predictor. The findings from this comparison (not reported, but available upon request) corroborate the results reported in Table 2

⁶ Recent empirical findings reported by (Döpke, Fritsche, and Müller 2019) indicate that forecaster’s behaviour has changed following the financial crisis.

⁷ Deleting the years of the Great financial crisis from the analysis leaves the results of the AG tests qualitatively unaffected. Results are not reported, but are available upon request.

⁸ The overlapping nature of the research institutes growth forecasts implies that, in our empirical analysis, the PIT is not independently distributed. As a result, application of the Ljung-Box test statistic yields significant results (results are not reported but available upon request).

⁹ Rossi and Sekhposyan (2014) describe in detail tests useful for evaluating predictive densities.

Additional information

Funding

This research was supported by the Deutsche Forschungsgemeinscahft (Project: Exploring the experience-expectation nexus in macroeconomic forecasting using computational text analysis and machine learning; Project number: 275693836)