Directional forecasts of GDP and inflation: a joint evaluation with an application to Federal Reserve predictions

Abstract

Many studies have undertaken separate analyses of the Fed's forecasts of real Gross Domestic Product (GDP) growth and inflation. This article presents a method for jointly evaluating the direction of change predictions of these variables. We conclude that some of the inflation forecasts, examined separately, were not valuable. However, the joint pattern of GDP and inflation projections was generally in accord with the economy's movements.

‘… directional forecasting … is now an increasingly popular metric for forecasting performance….’ (Pesaran and Timmermann, 2004, 414)

Acknowledgements

The authors wish to thank Roy Bachelor, David Hendry, Fred Joutz and participants in the 26th International Symposium on Forecasting and the GWU Forecasting Brownbag for helpful comments and suggestions. We also thank an anonymous referee for beneficial comments. All remaining errors are our own.

Notes

¹ On this point, see Leitch and Tanner (1991).

² We use the term GDP to represent output, measured as either GDP or Gross National Product (GNP) depending on the time period as indicated in the data section below. Although this article focuses on these variables in the context of monetary policy, our procedure is equally applicable to microeconomics problems, e.g. the direction of change of sales or prices.

³ The Greenbook is the data and analytic summary notebook that is distributed before each meeting of the Federal Open Market Committee (FOMC) that determines monetary policy.

⁴ No change is classified with the negative changes. Note that we are focusing on the direction of change in the inflation rate, which is equivalent to measuring accelerations and decelerations of the price level.

⁵ The contingency table methodology is used to test whether the sign of the predicted change is probabilistically independent of the sign of the actual change. This is also a test of the hypothesis that the forecasts are more accurate than those of a naïve random walk model in predicting the direction of change (Stekler, 1994, p. 497).

⁶ Mishkin's popular money and banking textbook (Mishkin, 2006) says: ‘Caring about inflation and output fluctuations is consistent with many statements by Federal Reserve officials that controlling inflation and stabilizing real output are important concerns of the Fed’ (p. 429).

⁷ Naik and Leuthold (1986) also used a 4 × 4 contingency table in their qualitative analysis of forecasting performance. Their study focused on a different topic the ability to predict turning points (also see Kaylen and Brandt, 1988).

⁸ Merton (1981) and Henriksson and Merton (1981) had used a test based on the hypergeometric distribution. This is identical to Fisher's exact test. Their test assumes known row and column frequencies, which is not assumed for the Pesaran–Timmermann test.

⁹ With larger tables such as the 4 × 4 tables, the negative effect of applying the discrete categories to a continuous distribution is minimal because the number of categories (16) allows for a better approximation to the continuous distribution. It is, therefore, not necessary to use the correction and the standard chi-square statistic can be used.

¹⁰ See the tables in the Appendix for the 4 × 4 cell counts.

¹¹ These forecasts were made in the last month of the quarter and refer to the economic conditions prevailing in that quarter (i.e. horizon zero). Some, but not all, of the data referring to these conditions were available in the last month. Therefore, these ‘forecasts’ are a mix of estimates and true predictions.

¹² Our sample spans several terminology changes at the BEA. For the beginning of our sample the ‘final’ data were released 45 days after the end of the quarter with no further revisions until more comprehensive revisions were made that might include definitional or classificational changes. Starting in 1974, these 45-day numbers were also referred to as 1st revision numbers because a second revision was introduced approximately 75 days after the end of the quarter. Since 1988 the data we use have been referred to by the BEA as ‘preliminary’ data and are released approximately 2 months after the end of the quarter (with ‘final’ data being released approximately 3 months after the end of the quarter). The only real-time data consistently available throughout our sample are the 45 to 60 day numbers, so we focus on these for analysis.

¹³ Note that in the 2 × 2 case, the Pesaran–Timmermann test statistic is equivalently a test of independence and of predictive ability.

¹⁴ One assumption of the Pesaran–Timmermann test statistic is that the probability of changes in direction must be time-invariant. Due to the longer expansions since the mid-1980s, this may not be the case, in particular for GDP growth. To determine the effect of the longer GDP expansions, we split the sample pre- and post-1984. The results were similar to the full sample except for leads 4 and 5 months. For lead 4 months there were only 29 observations (and only one recession) in the post-1984 subset which may explain the lack of significance for the Pesaran–Timmermann test statistic for lead 4 months for this subset. For lead 5 months, the test statistic is not significant at the 5% level for either subset, even though for the full sample it is significant. This occurs because the off-diagonal elements are clustered differently in the two subsamples. These results are available from the authors upon request.