On evaluating the performance of different forecasters

ABSTRACT

In this article, we have evaluated the performance of different forecasters and tested association between their performances for different pairs of variables. We have used three data sets of track records of professional U.S. economic forecasters participating in the Blue Chip consensus forecasting service (the data sets contain the root mean square errors (RMSE) of different forecasters for different years). To evaluate the performance of forecasters we have covered three well-known tests, namely the usual F test (cf. Fisher (1923)), Kruskal Wallis test (cf. Kruskal and Wallis (1952)), and Extension of Median test (cf. Daniel (1990)). To test the association between the forecaster's performances for different pairs of variables, we have considered Gini mean correlation coefficient r_g1 (cf. Yitzhaki, S., and Olkin, I. (1991) and Yitzhaki (2003)), Modified rank correlation coefficient (cf. Zimmerman (1994)) and three modifications of Spearman rank correlation coefficient. We have observed that different forecasters do not necessarily offer same average performance. Moreover, an evidence of association between two criteria does not always lead us reaching at the same decision. The outcomes of the study may help the practitioners in selecting the best forecaster(s) for policymaking purposes.

KEYWORDS:

• Association parameters
• Forecasters performance
• Location Parameters
• Parametric and Nonparametric

MATHEMATICS SUBJECT CLASSIFICATION:

• 62G99

Acknowledgments

The authors are thankful to the anonymous reviewers for their constructive comments that helped in improving the last version of the paper. The author Muhammad Riaz is also indebted to King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia for providing excellent research facilities and a dynamic research environment.