An evaluation of New Zealand macroeconomic survey forecasts

Abstract

In evaluating the one-year-ahead survey forecasts of key indicators for New Zealand, this study tests rationality under flexible loss to allow for the possibility of an asymmetric loss associated with forecast errors. For 1988–2011, the consensus forecasts of inflation, output growth, and interest rate are rational (efficient) under symmetric loss, out-perform the naïve benchmark, and are directionally accurate. In contrast, the consensus forecasts of unemployment and exchange rates are not of value since, while efficient, they are not directionally accurate and fail to out-perform the naïve benchmark. Utilizing actual and survey data, we have formulated a model to forecast the unemployment rate. Comparable unemployment forecasts from this model are superior to the survey forecasts for 2000–2011. As the final step, we examine the accuracy of individual forecasters. Our findings reveal both differences and similarities between the consensus and individual forecasts in terms of rationality and other accuracy measures.

Keywords:

• Survey of Expectations
• forecast evaluation
• rationality
• asymmetric/symmetric loss
• directional accuracy

Notes

 1. See, among others, Capistrán (2008) and Elliott et al. (2005, 2008). The studies by Diebold, Gunther, & Tay (1998), and Granger and Pesaran (2000) focus on density forecasts but illustrate the importance of the loss function for forecasting.

 2. The complete set of survey questions is available on the RBNZ website. For some of these series, the survey forecasts for quarters t + 1 and t + 7 are available for the period since 1993. We have chosen to study the one-year-ahead forecasts because they are available for all the series since 1987. This provides us with a reasonable number of observations for our analysis.

 3. Consistent with the first two survey questions, the inflation rate in quarter t + 3 is measured from quarter t–1 through to t + 3; the same applies to the GDP growth rate.

 4. The actual data on real GDP, CPI inflation, and unemployment rate are obtained from New Zealand's national statistical office, Statistics New Zealand. The actual data on interest rate and exchange rates are obtained from the RBNZ. Except for real GDP, these series are not subject to revisions. In this study, we measure the actual series on output (GDP) growth using the currently available final data; similar conclusions are obtained when we use real-time data to measure the actual series.

 5. In quarter t when the forecasts are made, the forecast errors for quarters t, t + 1, t + 2, and t + 3 are not yet known. As a result, the forecast error follows a fourth-order moving-average process under the null hypothesis of rationality. With the forecast errors generally heteroskedastic, we utilize the Newey–West (1987) procedure to correct for both the inherent fourth-order serial correlation and heteroskedasticity.

 6. See Elliott and Timmermann (2008) for good background reading.

 7. With d > 1, the J test statistic (χ ² _d–1) is applicable for Inst 2-5. However, this statistic is not applicable for Inst 1 since d = 1.

 8. In testing directional accuracy, we reach the same conclusion when using Fisher's exact test. This test is based on a two-by-two contingency table whose elements are the numbers of correct and incorrect sign forecasts. For more information, see Sinclair, Steklar, and Kitzinger (2010).

 9. For instance, Baghestani (2008) shows that the Federal Reserve and private (survey) forecasts of the US unemployment rate beat the naïve forecasts and are also directionally accurate.

10. Not all forecasters provide forecasts in every quarter. As such, the number of observations ranges from 48 to 85 for the individual forecast series examined here. Following Elliott et al. (2008, pp. 124–125), we should note that for some of these series, the power of rationality tests under flexible loss may be weak due to small sample sizes.