Fiscal deficit forecasts in Europe: evidence for a double standard?

ABSTRACT

Fiscal forecasts produced by international institutions came under strong criticism after the Eurozone sovereign debt crisis due to excessive optimism. Presently, international organizations are also accused of applying a double standard. Their opponents claim they depict negative picture for populist governments. This paper evaluates forecasts provided by the IMF, European Commission and the OECD based on a panel of EU economies and selected large countries. Five years after the Sovereign debt crisis, we still find excessively optimistic forecasts for Portugal and Spain. Moreover, the EC and OECD are being indulgent to countries under the excessive deficit procedure. There is also a strong autocorrelation of forecast errors and cyclical biases – European Commission overestimates governments’ propensity to tighten fiscal policy during expansion and forecasts an overly pessimistic picture during a slowdown. However, we find no evidence suggesting that fiscal forecasts stigmatize the governments accused of populism.

KEYWORDS:

• Deficit forecasts
• Excessive deficit procedure
• Fiscal supervision
• Public finance

JEL classification codes:

• H62
• H68

1. Introduction

The problem of fiscal forecast accuracy plays a relevant role in the European Union (EU) countries. According to the Stability and Growth Pact, member states are obliged to keep their government fiscal deficits under 3% of gross domestic product (GDP) and public debt below 60% of GDP. Furthermore, each country is obliged to achieve the so-called medium-term objective (MTO) – a desired level of structural balance dependent on the nominal GDP growth rate, interest rates, etc. In case when countries do not comply European Commission (EC) invokes so-called Excessive deficit procedure (EDP). Under its corrective arm government are obliged to reduce imbalance, otherwise EC imposes financial sanctions, e.g. freeze of EU funds or financial penalties up to 0.5% of GDP. Public finances are also monitored by the International Monetary Fund (IMF) and the Organization for Economic Cooperation and Development (OECD), but their recommendations are less binding.

The competences of international supervision of public finances are systematically increasing, not without controversy. The process of forecasting the financial situation cannot be fully separated from the political agenda. International institutions sometimes have contradictory goals comparing to the national governments what may rise conflicts. Our aim is to determine whether, in such cases, international institutions systematically tend to apply different standards in the assessment of public finances for two certain groups of governments. The first group can be described as populist governments. It includes Polish PiS, Hungarian Fidesz and Romanian PSD (Italian M5S-LN government rule was too short to correctly measure its effects). We refer here to the definition of populism which is currently applied by a mainstream European press (see e.g. Guardian (2018) and BBC (2019)). This concept surpasses the traditional definition assuming a more expansive fiscal policy, which is applicable for Romanian PSD government only. In the EU, new populism is rather related to Euroscepticism and controversial institutional policy, i.e. both the Polish PiS and Hungarian Fidesz are accused of violating the rule of law by the European Parliament. The second group describes indebted Mediterranean economies, i.e. Spain, Italy and Portugal. We apply a panel study to verify this hypothesis, where government-specific errors will be represented by dummy variables.

Our research shows no strong evidence suggesting that fiscal forecasts stigmatize governments accused of populism or violating the rule of law. On the other hand, international institutions do tend to be more lenient to indebted countries such as Portugal, Spain and Italy. In the case of the EC and the OECD, the bias was also greater for economies under the corrective arm of the EC EDP. In addition, there are cyclical problems related to long-term forecasts for emerging European economies: the EC tends to overestimate the propensity to consolidate public finances during expansion and presents an overly pessimistic picture during a slowdown. Finally, we also found a strong serial correlation between forecast errors.

This paper is structured as follows. The next section discusses political clashes related to EU financial framework. Section 3 provides an in-depth literature on the fiscal forecast accuracy. Section 4 describes the forecasting procedures of international institutions and the content of our datasets. Section 5 presents the methodology of our research. Section 6 discusses our empirical results. Finally, section 7 concludes the paper.

2. Literature Review – political clashes over the EU fiscal framework

This section provided a quick overview of the political clashes regarding reforms of financial framework in the EU and the international supervision.

First of all, there is no economic consensus on the outcomes. While some authors claim that EDP recommendations help to consolidate fiscal policy in the indebted countries (De Jong & Gilbert, 2020), others highlight the lack of instrument in the toolbox of international institutions to improve public finances in the countries which do not cooperate (Reuter, 2015; Schuknecht et al., 2011). Finally, some authors directly undermine the credibility of European fiscal rules and the international supervision of the IMF and OECD (Belke, 2017) after the European sovereign debt crisis using the following arguments: first, the fiscal forecast overestimated the governments’ ability to consolidate public finances in indebted Portugal, Italy and Spain. Second, the EC selectively released France, Portugal and Spain from the sanctions, what motivates other countries to a non-compliance with the fiscal rules.

The subsequent reforms of the fiscal framework provided mixed results. European fiscal board (EFB, 2020) highlighted that majority of the EU countries reduced deficit, but did not lower public debt and failed to build enough fiscal reserves for a downturn. Also, the response to the financial reform was not homogenous across the EU. On the one hand, countries with healthy public finances, such as Germany or the Netherlands are showing strong support to the fiscal discipline. On the other, countries which perform worse like France, Italy or Spain are prone to wage ‘war of attrition’ – national political groups are resisting fiscal adjustments until it is clear that some other group will pay its cost (Doray-Demers & Foucault, 2017).

The drawback of this strategy is the increasing criticism of financial framework in the countries with weaker fiscal discipline, which especially intensified in the recent years. The political fragmentation in the EU (see, e.g. Gidron & Hall (2017)) has led also to strong accusations of applying a double standard toward favored and unfavoured national governments. What was probably the most vocal clash occurred between Italy and France (Reuters, 2018). The EC forced the Italian government to lower its expected deficit for 2019 from 2.4% of GDP to 2.04%. At the same time, France was allowed to temporarily exceed the 3% threshold to fulfill promises pledged to the Yellow Jackets movement (fr. Gilets Jaunes). The deputy prime minister of Italy, Matteo Salvini, publicly denounced the approach of French commissioner, Pierre Moscovicci.

The reverberation of this conflict was also visible in the deficit forecast of the IMF. Despite Lega Nord-Five Star Movement declarations that they would lower the deficit after 2019, the IMF analysts predict constant deterioration. This assessment of IMF might be correct (finally governing coalition collapsed), especially given the costly pledges of, e.g. universal income, lower VAT and lower retirement age. However, during the previous Renzi government in Italy, the IMF frequently and consistently provided overly optimistic forecasts, despite the worsening realities of the government budgets. The evolution of the IMF forecasts for Italy in the last five years is presented in Figure 1. The errors in the forecasts in the previous years could undermine the credibility of the institution.

Figure 1. IMF deficit forecast for Italy – World Economic Outlook (WEO) October Editions. Each line represents forecasts available in different years. The 2018 edition (black solid line) provides up to date estimate of 2017 government deficit and forecast from 2018 to 2023. Source: IMF WEO database.

Another interesting example is Poland. Since 2016, the PiS (Law and Justice) government has been in open conflict with the EC regarding the rule of law, and the EC has, for two consecutive years, reduced the expected deficit in their forecast (Figure 2). The problem is complex. The ruling party introduced generous social programs and, at the same time, successfully improved the collection of tax revenues (Poniatowski et al., 2018). This increase in revenues was hardly predictable. Furthermore, if the tax collectors had been unsuccessful, more negative scenarios would have been likely. The literature on this subject highlights the negative link between damage to institutional quality and attachment to disciplinary fiscal rules (Wyplosz, 2012). Therefore, more cautious forecasts could be proof of responsibility. If the EC absolutely trusted the governmental forecasts and PiS failed to improve collection, the institution would expose the bondholders to the greater credit risk.

Figure 2. EC deficit forecast for Poland – Winter Editions. Source: EC forecast – statistical annex.

Mentioned examples showed only few dilemmas faced by the international institutions. Macroeconomic projections are often compared between the countries. Therefore, forecasting frameworks need to maintain some consistency of the errors. Permanent biases are likely to undermine credibility of the international supervision. The next section will show these problems were very urgent in the past, especially in case of sovereign debt crisis. Our aim is to verify whether situation improved since that time.

3. Literature Review – fiscal forecasting

This section provides an insight into the subject of fiscal situation forecasting performance and its institutional implications. Debate on the topic effectively started at the beginning of the new millennium. Short-term estimates of current year deficits (as a percentage of GDP) provided by international financial institutions such as the OECD or the IMF came under severe criticism, as they were frequently less accurate than the consensus of professional forecasters (Batchelor, 2001; Pons, 2000). Furthermore, some authors highlighted the lack of statistical efficiency and inconsistency of forecasts for G8 countries (Artis & Marcellino, 2001); there were examples of countries where a significant positive (Japan, Italy) or negative (Canada) forecast bias occurred. This phenomenon was explained by an ‘asymmetrical loss function’ in countries where deficit forecasts were politically sensitive, in other words, due to implementation of new fiscal policies, international institutions tended to provide cautious estimates.

The accuracy of forecasts prepared by public institutions likely improved within the next five years or greater interest was related to the long-term forecasts. As a result, the topic of the difference between the forecasts of international agencies and government institutions began to dominate in the literature. According to the majority of authors, deficit forecasts prepared by national governments suffered from political motivations. Their performance was less accurate, despite the use of superior information which is not available outside the ministries of finance or other budgeting entities (Brück & Stephan, 2006; Jonung & Larch, 2006; Leal et al., 2008; Merola & Pérez, 2013).

Positive bias in governments forecasts has been rationalized. In contrast to international agencies, government entities can use detailed information on tax collection or planned expenditures. At the same time, politicians have strong motivations to present overly optimistic macroeconomic forecasts (in comparison to future realizations) and to depict success stories of their current policies, for instance, by presenting remarkably strong GDP growth or high wage dynamics, and neglecting to include high unemployment rates (Brück & Stephan, 2006). Moreover, such forecasts are prone to the political cycle—governing parties have the temptation to increase spending and boost consumption prior to elections to influence the voting outcome. In order to prevent misleading of stakeholders (e.g. bond holders, societies), academics highlight the need for international supervision of fiscal policies (Jonung & Larch, 2006).

Unfortunately, research on fiscal supervision and forecast accuracy suggests that external forecasts are also prone to the previously mentioned problems. Some authors (Leal et al., 2008; Merola & Pérez, 2013) have confirmed the supremacy of the EC/IMF over national agencies in accurately predicting outcomes of economic policies in the G8 space. But their analyses also confirmed the existence of the same problems typically seen in the governments’ projections including systematic positive bias and the existence of the influence of political cycles in the forecasts’ errors. In addition, the authors highlighted another problem: the forecast errors of national governments and international institutions tend to be correlated with each other. These findings likely imply overconfidence of the external forecasters in the information provided by national authorities.

Another strong critique of independent agencies’ forecasts came after the European sovereign debt crisis. The example of Greece prior to the introduction of ECB–EC–IMF economic adjustment programs provides a situation where both the EC and the IMF consistently maintained forecasts suggesting prompt deficit reduction, despite that country having missed selected targets, year after year, even prior to the crisis. Furthermore, Greece was able to mislead both economists from the EC and statisticians from Eurostat about its real economic performance.

Beetsma et al. (2013) and Frankel & Schreger (2013) pointed out that the Greek case was not the only one where the forecast failed to describe reality. The fiscal projections of EC tended to present overly rapid fiscal consolidation in EU countries under corrective arm of the EDP. Furthermore, the problem of over-optimistic forecasts has not been evident in other developed economies with even larger deficits (i.e. the U.S.A. and Japan). Thus, the problem of bias was not related, for example, to government investment activity, but rather to overconfidence in the corrective action of EDP procedure (Pina & Venes, 2011).

4. International deficit forecasts

This section describes the dataset used in this research. Government deficit/surplus forecasts are published semiannually by international institutions. Our first data source is the IMF World Economic Outlook report (WEO), published in April and October. There are also two interim rounds of forecast updates (in January and July), where only new GDP growth estimates are published.

The IMF’s database published simultaneously with the report contains information for 194 countries. We selected 36 of these to construct our sample. The selected countries include EU and OECD members, as well as other large non-OECD economies (e.g. China, Russia).

The data used is from 2008 to October 2018 (last release at the moment of writing). Prior to this period, the IMF did not report deficit forecasts for some of the emerging European economies of interest (e.g. the Czech Republic, Poland). The database contains information about government net lending or borrowing expressed as a percentage of GDP. Amongst macroeconomic variables, we use information about the annual dynamics of GDP growth, its deflator, and the Consumer Price Index (CPI). We also use time series of the current account balance as a percentage of GDP to account for the twin deficit theorem¹ (Corsetti & Müller, 2006; Piersanti, 2000).

Each forecast covers a five-year horizon. We analyzed estimates for the current year (i.e. when the report was published) called ‘nowcasts’ and for the next three years. For example, from the report published in April 2015, we collected the nowcast estimate for 2015 and the estimates for 2016, 2017, and 2018 only.

The next source of data is the EC forecasts. Similar to the IMF, the EC also revises its forecasts twice per annum and provides two interim rounds where GDP forecasts are updated. The deficit forecasts are published during the spring and autumn (usually in April–May and November). The database contains information for 27-member states, the United Kingdom, the U.S.A., and Japan. We decided not to use information regarding candidate countries (e.g. Turkey) due to a short history, inconsistent reporting, and missing forecasts.

The number of indicators provided by the EC is greater than the number in the IMF reports. Statistical annexes also include information regarding the detailed structure of GDP, including public and private consumption expenditure, or gross fixed capital formation. Furthermore, labour market indicators include compensation of employees.

The EC forecasts also have a much shorter horizon of two years. In the spring round, the institution provides estimates for the current and next years. In the autumn round, estimates with a two-year horizon are also available. For example, the report published in November 2015 contained forecasts for 2015, 2016, and 2017. The horizon did not change in the next spring release, which provided information for 2016 and 2017.

Finally, we also use OECD economic outlook forecasts. The procedure of the update is semiannual and similar to that of the EC, but forecasts are published later compared to the IMF and EC (i.e. in May and December). The number of macroeconomic variables provided by the OECD is the same as in the report produced by the EC. The database contains information about: GDP, private and public consumption forecasts, CPI inflation and various price deflators, current account balances, and compensation of employees. The OECD produces forecasts for a similar horizon to the EC and shorter than the horizon of the IMF.

We excluded economies with episodes of adjustment bailout programs (i.e. Greece, Cyprus) as international institutions have capability to shape fiscal policy there. Secondly, less accurate forecasts for Greece may result from the lack of reliable times series, describing public finances up to 2016 as Greek authorities intentionally falsified statistics prior the Eurozone Sovereign Debt crisis. There were also periods when some international institutions did not publish forecasts for Greece and this fact created breaks in the analyzed time series. We also excluded small open economies with severe banking crises (i.e. Ireland, Iceland, and Slovenia). Such episodes create obvious outliers, which do not provide reliable information about performance of the major economies in the EU. The summary of investigated countries is presented in Table 1.

Table 1. Countries used in the panel by the international institution from which the data were sourced.

International Monetary Fund (IMF)	Organization for Economic Cooperation and Development (OECD)	European Commission (EC)
Australia	Australia	Belgium
Austria	Austria	Germany
Brazil	Belgium	Estonia
Bulgaria	Canada	Spain
China	Czech Republic	France
Croatia	Denmark	Italy
Czech Republic	Estonia	Latvia
Denmark	Finland	Lithuania
Estonia	France	Luxembourg
Finland	Germany	Malta
France	Hungary	Netherlands
Germany	Israel	Austria
Hungary	Italy	Portugal
India	Japan	Slovakia
Indonesia	Korea	Finland
Italy	Latvia	Bulgaria
Japan	Luxembourg	Czech Republic
Korea	Netherlands	Denmark
Latvia	New Zealand	Croatia
Lithuania	Norway	Hungary
Netherlands	Poland	Poland
New Zealand	Portugal	Romania
Norway	Spain	Sweden
Poland	Sweden	U.S.A.
Portugal	Switzerland	Japan
Romania	U.S.A.
Russia
Slovak Republic
South Africa
Spain
Sweden
Switzerland
United Kingdom
U.S.A.

5. Methodology

The aim of our analysis is to verify whether international financial institutions provide biased forecasts in relation to the different EU countries. We repeat the calculations previously used in the literature (Artis & Marcellino, 2001; Brück & Stephan, 2006; Pina & Venes, 2011). The starting point for our analysis is the following model: (1) $\mathrm{d}\mathrm{e}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}{\mathrm{t}}_t=a_0+a_1\ast \mathrm{f}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{s}{\mathrm{t}}_{t,h}+e_t$(1) where $\mathrm{d}\mathrm{e}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}{\mathrm{t}}_t$ is a final realization of the deficit in year t, $\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{s}{\mathrm{t}}_{t,h}$ describes a forecast for year t prepared in the horizon h. For example, when h = 2, this indicates a forecast prepared two years prior to data realization.

If the forecasts are unbiased, the parameter $a_0$ should be statistically insignificant and $a_1$ equal to one. Therefore, all forecast errors should be well described by random disturbances $\left(e_t\right)$. We assume there is no multiplicative error in the forecasts and that a₁ = 1. Therefore, Equation (1) is transformed into: (2) $\mathrm{d}\mathrm{e}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}{\mathrm{t}}_t-\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{s}{\mathrm{t}}_{t,h}=a_0+e_t$(2) Next, we attempt to identify the factors, which may explain forecast errors. First, fiscal forecasts related to revenue collection depend on the realization of macroeconomic assumptions regarding GDP growth and its structure (e.g. private consumption and gross fixed capital formation), inflation, and wages. Second, the data on both fiscal revenues and expenditures are prone to revisions, such as in the process of consolidation of public sector finances. As a result, forecasters are sometimes preparing their estimates based on incomplete information about the current state of public finances. This phenomenon should result in systematic and different errors between the countries, as data collection and revision procedures may vary. To account for these problems, we expand Equation (2) to: (3) $e_{\mathrm{f}\mathrm{i}\mathrm{n},t,h}=a_0+a_X\ast m{a}_X+a_{2X}\ast m{a}_X^3+a_3\ast \mathrm{r}\mathrm{e}{\mathrm{v}}_{t-1,h}+u_t$(3) where $e_{\mathrm{f}\mathrm{i}\mathrm{n},t,h}$ stands for the error of financial institutions’ forecast $(\mathrm{d}\mathrm{e}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}{\mathrm{t}}_t-\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{s}{\mathrm{t}}_{t,h})$, $m{a}_X$ describes the vector of macroeconomic assumptions, $\mathrm{r}\mathrm{e}{\mathrm{v}}_{t-1,h}$ measures the magnitude of previous year deficit revisions between the moment when the forecasts were formulated and their final values. Also, $a_0$, $a_X$, $a_{2X}$, and $a_3$ are estimated parameters, and $u_t$ is the equation residual. We introduced the third power of macroeconomic assumptions error to reflect stronger deficit increases during more severe downturns. We rejected to use second power in order to distinguish between positive and negative surprises. In the case of unbiased forecasts, parameter $a_0$ should be equal to 0 (statistically insignificant). Parameters describing macroeconomic variables $a_X$ and $a_{2X}$ are expected to be positive, for example, better activity, labour market conditions, or higher inflation should result in lower deficits.

As the next step, we introduce control variables describing both political and institutional factors. We use a panel structure to derive both cross-country and period fixed effects. Then we add variables describing whether the EC opened excessive deficit procedure against the country $\left(\mathrm{E}\mathrm{D}{\mathrm{P}}_t\right)$, whether World Bank governance indicators describe the government as dedicated to preserving the rule of law $(\mathrm{r}\mathrm{u}\mathrm{l}\mathrm{e}\mathrm{\_}\mathrm{l}\mathrm{a}{\mathrm{w}}_t)$, and whether the government is described as populist (or Eurosceptic) by the mainstream European press (e.g. Guardian, 2018; BBC, 2019 - $\mathrm{G}\mathrm{o}{\mathrm{v}}_t$). Five separate dummies take positive value when they describe one of governments being in power: Polish PiS, Romanian PSD, Italian M5N and Lega Nord coalition and Hungarian Orban’s Fidesz and negative otherwise.

Due to the positive result of the test for autocorrelation, we also include the deficit forecast error for the horizon h related to previous reports $\left(e_{\mathrm{f}\mathrm{i}\mathrm{n},t-1,h}\right)$. In Autumn reports $e_{\mathrm{f}\mathrm{i}\mathrm{n},t,h}$ and $e_{\mathrm{f}\mathrm{i}\mathrm{n},t-1,h}$ describe the same year. For example, for a report published in 2018 $h=1$ refers to 2019 in case of both reports. In case of Spring reports the approach is slightly different – we compare errors for the previous year. For example, horizon $h=1$ denotes 2019 for a report published in Spring 2018. We compare it to the Autumn report published in 2017 where $h=0$ denotes 2017 and $h=1$ 2018.

The final equation has following form: (4) $e_{\mathrm{f}\mathrm{i}\mathrm{n},t,h}=a_0+a_X\ast m{a}_X+a_2\ast \mathrm{r}\mathrm{e}{\mathrm{v}}_{t-1,h}+a_{X2}\ast m{a}_X^3+a_3\ast \mathrm{E}\mathrm{D}{\mathrm{P}}_t+a_4\ast \mathrm{G}\mathrm{o}{\mathrm{v}}_t+a_5\ast e_{\mathrm{f}\mathrm{i}\mathrm{n},t-1,h}+a_D\ast \mathrm{E}\mathrm{v}\mathrm{e}\mathrm{n}{\mathrm{t}}_t+\mu +{\phi }_t+u_t$(4) where $\mu$ is a cross-country effect and ${\phi }_t$ is a period effect. Our hypothesis states that values of $\mu$ are skewed regionally. Furthermore, we expect the parameters corresponding to political variables to be statistically significant.

Secondly, for a robustness check we will also directly test the significance of the two dummy groups: populist and Mediterranean economies, based on the pooled data without the corresponding fixed effects. (5) $e_{\mathrm{f}\mathrm{i}\mathrm{n},t,h}=a_0+a_X\ast m{a}_X+a_2\ast \mathrm{r}\mathrm{e}{\mathrm{v}}_{t-1,h}+a_{X2}\ast m{a}_X^3+a_3\ast \mathrm{E}\mathrm{D}{\mathrm{P}}_t+a_4\ast \mathrm{p}\mathrm{o}\mathrm{p}\mathrm{u}\mathrm{l}\mathrm{i}\mathrm{s}{\mathrm{t}}_t+a_5\ast \mathrm{M}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{n}{}_t+a_6\ast e_{\mathrm{f}\mathrm{i}\mathrm{n},t-1,h}+a_D\ast \mathrm{E}\mathrm{v}\mathrm{e}\mathrm{n}{\mathrm{t}}_t+u_t$(5) In both cases, we are using White’s diagonal method to achieve standard errors, that are robust to observation-specific heteroscedasticity in the disturbances, but not to correlation between residuals for different observations. The equations presented in Tables 2–5 are the result of this estimation. We also repeated the computation with White’s period method, which assumes that the errors for a cross-section are heteroskedastic and serially correlated (cross-section clustered). The modification of estimation techniques does not alter the final conclusions.

Table 2. International Monetary Fund (IMF) forecast errors – estimated models.

	T0	T1	T2	T3
Fiscal forecast error (previous report)	0.47*** (0.06,7.83)	0.51*** (0.04,12.51)	0.54*** (0.05,11.77)	0.49*** (0.05,9.72)
GDP – forecast error	0.27*** (0.05,4.92)	0.25*** (0.06,4.24)	0.15** (0.06,2.3)	0.21*** (0.08,2.66)
GDP – forecast error (^3)			0.01** (0.00,2.5)	0.00* (0,1.77)
Constant	0.1** (0.04,2.54)	0.17*** (0.04,3.83)	0.07 (0.05,1.51)	−0.01 (0.06, −0.16)
Hungarian reform of retirement system - dummy	−8.93*** (0.44, −20.42)
Basic diagnostics
Cross-sections/Periods	34/22	34/20	34/15	34/13
Observations	640	572	416	350
R^2/Adjusted R^2	0.42/0.36	0.62/0.59	0.57/0.52	0.68/0.64
Redundant fixed effects tests - Statistic (p-value)
Cross-section F	1.57 (0.02)	1.35 (0.10)	1.91 (0.00)	2.68 (0.00)
Cross-section Chi-square	54.7 (0.01)	47.39 (0.05)	66.48 (0.00)	90.55 (0.00)
Period F	1.34 (0.15)	6.32 (0.00)	0.9 (0.55)	2.23 (0.01)
Period Chi-square	28.75 (0.09)	113.89 (0.00)	13.21 (0.43)	27.51 (0.00)
Cross-Section/Period F	1.52 (0.01)	3.34 (0.00)	1.63 (0.01)	2.62 (0.00)
Cross-Section/Period Chi-square	83.17 (0.01)	163.14 (0.00)	77.8 (0.00)	113.89 (0.00)

Note: The estimated model indicates no significance of control variables related to institutional quality or certain governments. However, IMF deficit nowcasts and one year ahead forecasts are systematically biased – the constant is statistically significant and greater than zero. The forecast errors are not randomly distributed as well – a parameter corresponding to the previous iteration error is significant and positive. Therefore, autocorrelation of errors occurs.

The relationship between macroeconomic surprises and deficits are positive, in line with the intuition. However, the cross-section fixed effects tend to be statistically significant in the case of nearly every time horizon. Detailed discussion about their values is presented in Table 5.

*** denotes significance at α = 0.01, ** denotes significance at α = 0.05, * denotes significance at α = 0.1.

Table 3. European Commission (EC) forecast errors – estimated models.

	T0	T1	T2
Fiscal forecast error (previous report)	0.32*** (0.07,4.34)	0.47*** (0.05,8.78)	0.22*** (0.07,3.05)
Private Consumption – forecast error	0.14** (0.06,2.50)	0.15*** (0.05,2.81)	0.46*** (0.15,3.03)
Private Consumption – forecast error^3	0.00*** (0.00, −2.90)	0.00** (0.00, −2.24)	−0.02** (0.01, −2.6)
Gross Fixed Capital Formation – forecast error	0.02* (0.01,1.90)
GFCF – forecast error^3	0.00*** (0.00, −2.75)
Public Consumption – forecast error^3	0.00*** (0.00, −3.53)
Wages – forecast error		−0.08** (0.03, −2.59)
GDP deflator forecast error		0.18*** (0.05,3.66)
Constant	0.27*** (0.07,3.89)	0.48*** (0.05,8.86)	0.47** (0.2,2.32)
Polish reform of retirement system – dummy	−9.64*** (0.34, −28.31)
Polish reform of retirement system – dummy 2		−8.89*** (0.31, −29.06)
Hungarian reform of retirement system – dummy	−6.24*** (0.36, −17.56)
Hungarian reform of retirement system – dummy 2	−6.53*** (0.64, −10.22)
EDP	−0.29* (0.15, −1.86)		0.73* (0.43,1.72)
Basic diagnostics
Cross-sections/Periods	26/18	26/18	26/8
Observations	468	468	208
R^2/Adjusted R^2	0.62/0.57	0.59/0.55	0.59/0.55
Redundant fixed effects test-statistics (p-value)
Cross-section F	1.95 (0.00)	6.34 (0.00)	1.34 (0.14)
Cross-section Chi-square	51.91 (0.00)	150.6 (0.00)	37.4 (0.05)
Period F	2.1 (0.01)	13.51 (0.00)	4.52 (0.00)
Period Chi-square	38.7 (0.00)	215.33 (0.00)	35.47 (0.00)
Cross-Section/Period F	1.94 (0.00)	9.34 (0.00)	2.67 (0.00)
Cross-Section/Period Chi-square	83.95 (0.00)	317.68 (0.00)	84.72 (0.00)

Note: There is no evidence that the EC tends to stigmatize populist governments or those with lower institutional quality. However, weak evidence of indulgence for countries under EDP is visible (negative beta in the case of nowcast).

Similar to the case of the IMF, both constants and parameters corresponding to the deficit forecast error in the previous report are positive and statistically significant. The latter confirms autocorrelation of forecast errors. The cross-country fixed effects are non-redundant as well.

Additionally, parameters corresponding to wages in the one year ahead equation and to the third power of consumption in the two-year horizon are negative. As a result, in case of stronger positive macroeconomic surprises, we do not observe an adequate deficit reduction. The problem is likely a result of overestimating governments’ propensity to reduce deficits during an economic expansion.

*** denotes significance at α = 0.01, ** denotes significance at α = 0.05, * denotes significance at α = 0.1.

Table 4. Organization for Economic Cooperation and Development (OECD) forecast errors – estimated models.

	T0	T1	T2
Fiscal forecast error (previous report)	0.29*** (0.08,3.52)	0.47*** (0.07,6.52)	0.22*** (0.08,2.86)
GDP – forecast error			0.28*** (0.1,2.86)
Private Consumption – forecast error		0.14** (0.07,2.16)
Private Consumption – forecast error^3	0.05** (0.02,2.17)
Gross Fixed Capital Formation – forecast error	0.04*** (0.01,3.01)
GFCF – forecast error^3	0.00*** (0.00, −3.11)
Constant	0.32*** (0.06,5)	0.16*** (0.06,2.92)	0.56*** (0.12,4.73)
Polish reform of retirement system – dummy	−9.87*** (0.32, −30.71)
Polish reform of retirement system – dummy 2		−8.4*** (0.4, −20.95)
Hungarian reform of retirement system – dummy	−7.96*** (0.31, −25.88)
Hungarian reform of retirement system – dummy 2	−7.44*** (0.82, −9.09)
EDP	−0.51*** (0.16, −3.23)		−0.88** (0.38, −2.31)
Basic diagnostics
Cross-sections/Periods	28/17	28/15	28/6
Observations	448	392	155
R^2/Adjusted R^2	0.69/0.65	0.67/0.63	0.67/0.63
Redundant fixed effects tests - Statistic (Prob)
Cross-section F	2.46 (0.00)	2.39 (0.00)	1.47 (0.08)
Cross-section Chi-square	69.45 (0.00)	66.94 (0.00)	44.68 (0.02)
Period F	1.66 (0.05)	2.42 (0.00)	0.5 (0.78)
Period Chi-square	29.08 (0.02)	36.61 (0.00)	3.2 (0.67)
Cross-Section/Period F	2.22 (0.00)	2.56 (0.00)	1.31 (0.15)
Cross-Section/Period Chi-square	96.75 (0.00)	103.84 (0.00)	46.87 (0.04)

Note: There is no evidence that the OECD tends to stigmatize populist governments or those with lower institutional quality. However, there are similar problems with OECD forecasts, like those of the other institutions; i.e. we found systematic bias, autocorrelation of forecasts, and existence of statistically significant country effects.

Additionally, this institution put too much confidence in the corrective arm of EDP. Countries under EC supervision are projected to consolidate their budget deficits at a rate which is later not achieved.

*** denotes significance at α = 0.01, ** denotes significance at α = 0.05, * denotes significance at α = 0.1.

Table 5. Cross-country fixed effects – a summary.

	European Commission			Organization for Economic Cooperation and Development			International Monetary Fund
Forecast horizon	T0	T1	T2	T0	T1	T2	T0	T1	T2	T3
Australia				−0.12	−0.13	−0.42	−0.33	−0.59	−0.59	−1.67
Austria	−0.02	−0.07	−0.01	−0.04	0.00	−0.24	−0.01	0.12	0.08	0.57
Belgium	−0.11	−0.06	−0.06	−0.18	−0.24	−0.29
Brazil							−0.30	−0.45	−0.41	−3.22
Bulgaria	−0.15	0.05	−0.23				0.11	0.05	−0.13	0.47
Canada				0.51	0.47	0.51
China							0.06	−0.32	−0.79	−1.57
Croatia	−0.16	−0.09	−0.06				0.05	0.70	0.96	1.58
Czech Republic	0.27	0.42	0.95	0.10	0.29	0.98	0.30	0.65	0.89	2.78
Denmark	0.69	0.72	1.01	0.55	0.72	1.25	0.56	0.60	0.81	1.69
Estonia	0.21	0.11	0.09	−0.09	−0.12	−0.47	−0.13	−0.01	−0.08	0.48
Finland	−0.21	−0.39	−0.35	−0.20	−0.43	−0.49	0.09	0.01	−0.18	−0.57
France	0.11	−0.04	−0.31	0.07	−0.21	0.15	−0.05	−0.07	−0.19	−1.24
Germany	0.02	0.11	0.47	−0.11	0.08	−0.08	0.08	0.29	0.44	1.18
Hungary	−0.06	−0.44	−0.67	0.03	−0.05	−0.69	−0.09	0.31	0.16	1.08
India							−0.09	0.03	−0.03	0.58
Indonesia							0.02	0.00	−0.02	−0.18
Israel				0.36	0.22	0.53
Italy	−0.17	−0.33	−0.74	−0.25	−0.39	−0.75	0.00	−0.07	−0.20	−0.39
Japan	0.01	0.14	0.42	0.71	0.86	1.13	0.36	0.11	0.23	1.21
Korea				0.02	0.19	0.28	0.04	0.16	−0.09	−0.16
Latvia	0.07	0.44	1.15	0.08	−0.54	−1.01	−0.06	−0.16	0.03	0.12
Lithuania	−0.08	0.14	0.30				−0.19	0.66	0.84	2.66
Luxembourg	0.56	0.55	1.33	0.16	0.37	0.58
Malta	0.53	0.22	0.50
Netherlands	0.05	0.05	0.15	0.14	0.02	0.48	−0.02	0.17	0.65	1.82
New Zealand				0.29	0.17	−0.33	0.16	−0.02	0.02	0.27
Norway				−0.25	−0.27	1.14	−0.10	−0.37	−1.07	−2.71
Poland	0.23	0.30	−0.08	0.35	0.33	−0.03	−0.01	−0.01	−0.07	−0.14
Portugal	−1.07	−0.97	−1.60	−0.73	−0.72	−0.55	−0.88	−0.53	−0.14	−1.46
Romania	−0.07	−0.08	−0.05				−0.22	−0.23	−0.51	−0.71
Russia							0.13	0.39	0.39	1.23
Slovakia	−0.05	0.12	0.19	−0.07	0.06	−0.43	0.02	0.18	0.39	1.30
South Africa							−0.08	−0.01	−0.01	−0.44
Spain	−0.56	−0.57	−1.80	−0.48	−0.89	−0.42	−0.51	−0.48	−0.25	−1.28
Sweden	0.11	−0.12	−0.07	−0.15	−0.07	−0.07	0.30	0.23	0.03	−0.29
Switzerland				−0.32	−0.05	−0.52	0.04	−0.05	−0.14	0.06
United Kingdom	0.16	0.13	0.01	−0.10	0.02	0.09	0.22	0.22	−0.23	0.01
U.S.A.	−0.30	−0.35	−0.56	−0.57	−0.94	−1.12	0.38	0.07	0.11	0.86

Note: The values of the fixed effects for Portugal and Spain are lower compared with those for the other countries. The discrepancy between their values and cross-country means frequently exceeds two standard deviations for different institutions and different horizons. Blank cells indicate that the country was not part of a data panel.

6. Estimation results

This chapter presents a summary of the outcomes of our research. The full detailed estimates of the fixed effects panel models are available in Tables 2–5. The results of pooled regressions are present in Tables 6–8.

Table 6. Pooled regressions – forecast errors for the IMF.

Forecast Horizon	T0	T1	T2	T3
Fiscal forecast error (previous report)	0.53*** (0.05, 0.00)	0.50*** (0.04, 0.00)	0.71*** (0.03, 0.00)	0.75*** (0.03, 0.00)
Populist governments	−0.03 (0.20, 0.90)	−0.12 (0.26, 0.63)	−0.33** (0.14, 0.01)	−0.45* (0.24, 0.06)
Mediterranean economies	−0.35*** (0.12, 0.00)	−0.34** (0.15, 0.03)	−0.15 (0.15, 0.32)	−0.30* (0.17, 0.07)
GDP – forecast error	0.26*** (0.05, 0.00)	0.39*** (0.05, 0.00)	0.26*** (0.05, 0.00)	0.26*** (0.06, 0.00)
Constant	0.14*** (0.04, 0.00)	0.24*** (0.05, 0.00)	0.10** (0.05, 0.04)	0.08 (0.06, 0.20)
Basic diagnostics
R^2	0.52	0.61	0.69	0.72
Observations	640	572	416	350

***denotes significance at α = 0.01, ** denotes significance at α = 0.05, * denotes significance at α = 0.1.

Dummies corresponding to Hungarian and Polish retirement system reform were present in the model and intentionally removed from the output. The model confirms conclusions presented in Table 2.

Table 7. Pooled regressions – forecast errors for the European Commission.

Forecast Horizon	T0	T1	T2	T3
Fiscal forecast error (previous report)	0.39*** (0.07, 0.00)	0.36*** (0.05, 0.00)	0.13** (0.06, 0.04)
EDP	−0.44*** (0.12, 0.00)	0.18 (0.13, 0.17)	0.73*** (0.27, 0.01)
Populist governments	−0.10 (0.16, 0.54)	−0.45*** (0.15, 0.00)	−0.51 (0.43, 0.23)
Mediterranean economies	−0.37** (0.14, 0.01)	−0.78*** (0.16, 0.00)	−1.36*** (0.33, 0.00)
Private Consumption – forecast error	0.16*** (0.05, 0.00)	0.18*** (0.05, 0.00)	0.36*** (0.12, 0.00)
Gross Fixed Capital Formation – forecast error	0.02* (0.01, 0.05)	0.00*** (0.00, 0.00)	0.04** (0.02, 0.03)
GDP deflator forecast error		0.14** (0.06, 0.01)
Constant	0.37*** (0.07, 0.00)	0.58*** (0.08, 0.00)	0.77*** (0.15, 0.00)
Basic diagnostics
R^2	0.55	0.44	0.21
Observations	468	468	208

*** denotes significance at α = 0.01, ** denotes significance at α = 0.05, * denotes significance at α = 0.1.

Dummies corresponding to Hungarian and Polish retirement system reform were present in the model and intentionally removed from the output. The model confirms conclusions presented in Table 3.

Table 8. Pooled regressions – forecast errors for the OECD.

Forecast Horizon	T0	T1	T2	T3
Fiscal forecast error (previous report)	0.40*** (0.09, 0.00)	0.57*** (0.06, 0.00)	0.24** (0.12, 0.04)
EDP	−0.42*** (0.12, 0.00)	−0.09 (0.15, 0.54)	−0.70*** (0.26, 0.01)
Populist governments	−0.03 (0.15, 0.86)	−0.06 (0.18, 0.75)	−0.47 (0.36, 0.20)
Mediterranean economies	−0.35** (0.15, 0.02)	−0.40** (0.17, 0.02)	−0.44* (0.23, 0.05)
GDP – forecast error			0.45** (0.21, 0.03)
Private Consumption – forecast error	0.18** (0.07, 0.01)	0.24*** (0.06, 0.00)
Constant	0.34*** (0.07, 0.00)	0.19*** (0.06, 0.00)	0.55*** (0.14, 0.00)
Basic diagnostics
R^2	0.62	0.57	0.29
Observations	448	392	155

*** denotes significance at α = 0.01, ** denotes significance at α = 0.05, * denotes significance at α = 0.1.

Dummies corresponding to Hungarian and Polish retirement system reform were present in the model and intentionally removed from the output. The model confirms conclusions presented in Table 4.

The estimated models indicate three negative phenomena visible in the evaluated forecasts. First and least troublesome – each of the international institution forecasts has a systematic negative bias (forecasts are more pessimistic comparing to further realizations): the parameter $a_0$ is non-zero and is statistically significant in the present and next year horizons. The largest bias is present in the EC case and amounts, respectively, to 0.28 and 0.47pp (percentage points). The OECD systematically provides overly optimistic forecasts for the current year (T0) by 0.32pp, but bias is lower in the one-year horizon (0.16pp). Finally, the bias presented in forecasts provided by the IMF is the lowest and equals, respectively, 0.10pp and 0.17pp. Our findings are contrarian to the previous literature on this subject. Researchers for G8 countries reported positive bias, e.g. Beetsma et al. (2013) and Frankel & Schreger (2013). There are two plausible reasons behind this phenomenon. First of all, this Eurozone sovereign debt crisis resulted in the increased scope of the fiscal supervision – authors (Beetsma et al., 2019) show that introductions of fiscal councils help to limit optimistic bias. Secondly previous research was conducted mainly for advanced economies, what might distort the overall picture.

Second, the forecast errors of government deficits are not randomly distributed; estimates are prone to autocorrelation problems. Therefore, we inserted an autoregressive component in each equation. In each case, these parameters were statistically significant. The values of this beta parameter were highest in the IMF case – close to 0.5 for each forecast horizon. Beta estimates for the OECD and EC cases were lower and more dispersed (equal to 0.22–0.47).

Third, the cross-section fixed effects were statistically significant in nearly every equation. In the EU, overly optimistic forecasts are particularly visible for Portugal and Spain, and, in the case of long-term estimates, for Italy. On the other hand, the improvement of fiscal balance after the global financial crisis was underestimated for Denmark and the Czech Republic. For countries outside the EU, the European-based institutions (EC and OECD) tend to present overly optimistic estimates regarding the U.S.A.

Amongst the macroeconomic variables, we identified a strong relationship between errors of deficit forecast and GDP growth assumption or its components, such as private consumption or gross fixed capital formation. Our analysis provides negative and statistically significant parameters for the third power in the EC forecasts. In the majority of horizons, the effect is not strong. For example, the 3pp positive surprise in the consumption dynamics (i.e. dynamics of indicator is 3 percentage point higher comparing to the forecast) results in a higher deficit forecast error by 0.1pp for nowcasts and one-year forecasts made by the EC. A more severe problem occurs in cases of horizons longer than two years, where the response is equal to 0.6pp. This observation shows that the EC tends to overestimate the propensity of national governments to consolidate public finances. Similarly, during a slowdown, forecasts are likely to provide an overly pessimistic picture.

Surprisingly, we see no coincidence between current account balances and government deficit forecast errors, even for long-term forecasts. The institutions forecast does not reflect propensity to simultaneous increase of external and fiscal deficit, despite strong macroeconomic foundations.

In line with the findings of Pina & Venes (2011), we found evidence that the OECD and EC are overconfident in the positive effect of EDP corrective arm. The current year forecasts are optimistically biased by, respectively, 0.5pp and 0.3pp. In the case of the EC, this bias can lead to premature decisions regarding closing of the procedure.

The pooled regressions confirm conclusions from the fixed effects panel models. The forecast for Mediterranean economies were on average too optimistic by approximately 0.3pp in case of the IMF, 0.4-1.3pp by the EC and the 0.4pp by the OECD. The forecasts for populistic governments were free of potential political biases or even too optimistic in some horizons. These models also confirm statistical artifacts related to the EDP procedure – international organizations produced too optimistic short-term forecasts for countries under corrective arm.

7. Conclusions

Our study does not confirm the hypothesis that international institutions tend to stigmatize ‘populist’ governments. However, we identify different and equally important problems associated with deficit forecasting.

First, all of the agencies threat leniently heavily indebted countries. Some indulgence is visible for economies under corrective arms of EDP in case of EC and OECD as well. The motives for such a forecasting approach can only be guessed. For example, the institutions may not be willing to trigger negative confidence shocks. The problem could also be a result of excessive trust in government predictions. Nonetheless, the results of underestimating the scale of government deficits have consequences. Although Spain has managed to reduce its sovereign debt in relation to its GDP, Italy, France, and Portugal are still heavily indebted.

In the case of the EC, another problem relates to forecasting deficits in emerging economies. We found cyclical biases in the EC forecasts, which could be a result of insufficient resources being dedicated to forecasting. Other problems may be related to a lack of analysis of documents in a country’s language or to acquiring information from a limited number of media outlets providing news coverage only in English.

Finally, our research confirmed the existence of problems described in the academic literature, specifically, the selective positive bias for some countries (e.g. Portugal, Spain) and strong autocorrelations of forecast errors. Both problems undermine the credibility of international institution forecasting methods.

Acknowledgements

The paper was prepared within the framework of research project No. 2018/29/N/HS4/00334. Analysis of international financial institutions fiscal forecasts consistency for European Union economies, financed by the National Science Center, Poland.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Narodowe Centrum Nauki: [Grant Number 2018/29/N/HS4/00334].

Notes on contributors

Jakub Rybacki

Jakub Rybacki is a PhD student at the SGH Warsaw School of Economics, Economic Advisor at the Polish Economic Institute.

Notes

1 The theorem assumes a common and positive relationship between fiscal and current account deficits in the long term; e.g. an increase of government expenditures financed by debt increase should widen the external imbalance.