On the Accuracy of the Private Sector Macroeconomic Forecasts of the South African Economy

Abstract

This paper evaluates the forecasting accuracy of private sector forecasters who participate in the annual “Media24 Economist of the Year” forecasting competition in South Africa. Our primary aim is to examine whether the accuracy of private sector forecasters improved over time, particularly their ability to predict the 2008/2009 recession and whether there was a distinct change in forecasting accuracy along this turning point of the business cycle. Our estimates from a forecasting error measure known as Theil's U-Statistic show that, on the average, the Root Weighted Mean Squared Error (RWMSE) of the growth forecast for the current period was 0.62 of an adaptive-naïve forecast, whereas the inflation forecast was 0.71 of an adaptive-naïve forecast. In order to determine whether there was an improvement in forecasting accuracy after the recession, we segregate the sample period along this break date and compare the size of the forecast errors between the two periods. To this end, we find that with respect to the growth predictions, there was a marginal reduction in the magnitude of the forecasting errors. However, in the case of inflation forecasts (both current and year-ahead), there was a marked reduction in the size of both the RWMSE and Theil's U-Statistic, implying that the post recessionary period was characterized by an improvement in accuracy of inflation forecasts made by the private sector. Furthermore, with respect to year-ahead forecasts, the results from both the nonparametric and the more formal parametric test allows us to reject the hypothesis of equality between the mean squared errors of the competing forecasts. Although the private sector forecasters were unable to accurately predict the recession, they were at the least able to produce forecasts that were more accurate than the adaptive-naïve model.

Notes

1 A series of competitions organized by teams led by forecasting researcher Spyros Makridakis intended to assess and compare the accuracy of different forecasting methods. The M-4 Competition which is currently being run is based on forecasted data for over 3000 time series for the Unites States. These include series from a number of domains including: micro, finance etc. and for various frequencies i.e. yearly, quarterly and monthly. For more details see Makridakis and Hibon (2000).

2 This summary statistic developed by Eon Smit, an adjudicator of the competition, encompasses all forecasts made during the year and attaches weights to each forecast. The initial forecast for the year is assigned the largest weight and the size of the weights attached to subsequent forecasts for that year, systematically descends until the end of the horizon.

3 Naturally a low rate of unemployment is in itself an important goal, and sustained GDP-growth is associated with a decline in unemployment.

4 This measure should not be confused with a previous error measure namely U_r, proposed in (Theil, 1958) which had serious defects and was subsequently discarded as a historic oddity.

5 See (Diebold & Mariano, 2002), for a brief discussion on these of alternative approaches.

6 See also Batchelor (2001: 230) in which a similar question is posed.

7 The test is a variant of the procedure suggested by (Morgan, 1940) and is discussed in more detail by (Stekler, 1991).