Sustainability assessment of pension systems of new EU member states using data envelopment analysis with sensitivity and cross-efficiency analysis

Abstract

This paper assesses the sustainability of pension systems of New Member States of the European Union that have undergone the transition to a market economy and the establishment of a multi-pillar pension model. Investigating pension sustainability from a slightly different perspective, we apply the Mercer CFA Institute Global Pension Index methodology, whose sustainability sub-index measures indicators that have a significant influence on the likelihood that the current pension system will be able to provide benefits into the future. However, this methodology predefines the weights for each indicator, which can be limiting for countries that have large oscillations between indicators. To obtain a sustainability analysis without predefined weights, we apply data envelopment analysis (DEA) to calculate efficiency scores using sustainability indicators as inputs. Furthermore, since DEA generally allows each country to load on their strong indicators as much as possible, allowing thus for self-appraisal, we perform the analysis of the perturbations in the data and examine how these changes affect the overall relative positions of the countries in the set. Also, we explore the cross-efficiency approach to define peer-appraisal as a more objective efficiency score. As a final step, a comparison of all EU Member States provides an overall perspective.

Keywords:

• Data envelopment analysis
• Mercer CFA Institute Global Pension Index
• pension sustainability

JEL CLASSIFICATIONS:

• C67
• H55
• J14

1. Introduction

Population aging poses a major challenge for pension systems sustainability in European countries. Reduced fertility rates and an increase in life expectancy lead to a deterioration in the ratio of the elderly and the working population, which burdens pension systems. Aside from unfavorable demographic trends, pension systems of European countries, particularly former socialist economies, are characterized by poor design features including high shares of early retirement, high system dependency ratios, and fiscal imbalances (Krpan et al., 2020). A common strategy used to study European pension systems’ sustainability is by analyzing the indicators offered by the Pensions strand of the Open Method of Coordination (OMC) (Eurostat, 2020a). However, Monash Centre for Financial Studies developed a pension index called the Mercer CFA Institute Global Pension Index, which is based on three sub-indices: adequacy, sustainability, and integrity. The overall index value for each system represents the weighted average of the sub-indices. The weightings used are 40 percent for the adequacy sub-index, 35 percent for the sustainability sub-index, and 25 percent for the integrity sub-index which have remained unchanged since the first Index in 2009 (Mercer, 2019).

By focusing solely on the sustainability sub-index of the Mercer CFA Institute Global Pension Index our paper uses indicators representing the long-term sustainability of current pension systems on a set of New Member States of the European Union to measure sustainability by employing the new approach in this area, Data Envelopment Analysis (DEA). Measurement and the analysis of the sustainability of the pension systems of New Member States of the European Union (NMS), which is the focus of this paper, is lacking in the Mercer Global Pension Index assessment¹. The main reason for omitting NMS by the Monash Centre for Financial Studies is the unavailability of data, especially for non-OECD countries, which is why most research on pension systems includes solely the most developed economies. We, therefore, analyzed these countries and investigated their level of sustainability.

The sustainability sub-index, being a part of the Mercer CFA Institute Global Pension Index, includes several indicators that are believed to influence the likelihood that the current system will be able to provide benefits into the future. It encompasses factors concerning the economic importance of the private pension system, its funding level, the length of expected retirement both now and in the future, the labor force participation rate of the older population, the current level of government debt, and the level of real economic growth. The first part of the indicators measures the financing of private pension systems and their economic importance. These include mandatory contributions towards funded public benefits, expressed as a percentage of wages, the proportion of the working-age population that are members of private pension plans, and total assets in funded and private pension arrangements as a percentage of GDP. It is here assumed that a higher share of funding implies a greater level of sustainability since it reduces future dependence on the public pension while also adjusting the expectations of many workers. This is in line with World Bank’s (1994) initial objectives for the introduction of mandatory private pension schemes, which is supported by many international institutions (e.g. OECD).

The next two indicators are general government consolidated gross debt as a share of GDP and real economic growth rate averaged over seven years. They assess the financial capacity of the future government concerning coverage and levels of future pension benefits, among others. The pressure of aging on the long-term sustainability of pension systems and the government’s labor market and pension system’s policies focused on encouraging people to work longer present the next perspective contained in the sustainability sub-index. It is represented by the following indicators: the gap between life expectancy at birth and the state pension age, both now and projected for 2040, then projected old-age dependency ratio in 2040, estimated total fertility rate for the 2015–2020 period, and labor force participation rates for the 55-64 age group, as well as for the population over 65. Lastly, the sustainability sub-index evaluates the possibility of older workers to access part of their retirement savings or pension and continue working, which would enable these individuals to continue contributing to the pension system.

The use of the DEA method on a set of indicators, that were chosen by the Monash Center to be the representatives of the long-term sustainability of current pension systems, for relatively homogeneous countries allows for a comparative comparison of several perspectives of the pension systems of the chosen economies. Additionally, it gives insight into the comparative strengths and weaknesses of an individual country concerning these indicators. Furthermore, the efficiency scores from DEA analysis allow for the evaluation of the performances of the pension systems of the selected economies to their future long-term sustainability. This is especially important in conditions where countries are undergoing reforms to the pension system in an aging environment and financial imbalances. As a next step, sensitivity and cross-efficiency analyses allow us to compare the resilience of the DEA efficiency results with the results that are obtained when an individual indicator is omitted from the analysis or when weights from other economies from DEA are used. Finally, a comparison of all EU Member States is conducted to provide an overall perspective.

The paper is structured as follows: the first section discusses the sustainability sub-index and previous empirical research. The second section is concerned with the Data Envelopment Analysis methodology. The third section presents the data and delivers our results. The fourth section is devoted to sensitivity and cross-efficiency analysis. The fifth section brings the research conclusions.

2. Literature review

A large number of existing studies have examined the sustainability of pension systems. In a broader sense, these can be divided into sustainability analyses within the Overlapping Generations (OLG) framework or other complex forecasting models, which are often single-country studies; and comparative analyses that aim to classify or rank countries using a set of sustainability indicators or a synthetic indicator. Several OLG studies of pension systems and pension reforms are important to mention. Seminal contributions have been made by İmrohoroglu et al. (1995), Razin and Sadka (1999), Fougere and Merette (1999), Börsch-Supan et al. (2006), Verbič et al. (2006), Boldrin and Montes (2009), Fanti and Gori (2010), Georges et al. (2016), Pavković (2021) and many others. A different method employed by De La Fuente and Doménech (2013) is a preliminary estimate of the impact of the reform of the Spanish public pension system constructed using a decomposition of public pension expenses as a percentage of GDP, whereas the other half is related to the evolution of the ‘generosity’ ratio.

In the same manner, European Commission (2009), Moscarola (2009), Pallares-Miralles et al. (2012), Marcinkiewicz and Chybalski (2014), Chybalski (2014), Krpan et al. (2019), and Škuflić et al. (2020) approach the analysis of sustainability by studying public pension expenditure as a percentage of GDP that Eurostat (2020a) considers as the main indicator of sustainability. In 2018, with 6.48 percent of GDP, Estonia recorded the lowest share of pension expenses among NMS, while Croatia and Poland had the largest pension expenditure, amounting to more than 10 percent (Eurostat, 2020b). Eurostat (2020a) lists the employment rate of older workers and the duration of working life, as two other primary indicators of sustainability, but these will be included in this study. World Bank’s experts such as Kasek et al. (2008), Pallares-Miralles et al. (2012) present more than twenty pension sustainability indicators, including demographic factors, labor market indicators, replacement rates, the structure of pension expenditure, and many more. Holzmann et al. (2004) argue that it is important to consider unfunded public pension liabilities as part of an assessment of the overall fiscal situation of pension systems and show that almost all of the NMS rank high in levels of implicit pension debt (Slovenia, Romania, and Poland are in the first five among low and middle-income countries).

However, most authors recognize the need for developing different techniques and indicators to approach the pension sustainability issue. In recent years, synthetic indicators have become one of the most commonly used tools of analytical measurement in practice in many fields of social reality (Alonso-Fernandez et al., 2018), and also in pension economics. The two most important synthetic indicators of pension systems include The Allianz Pension Sustainability Index (PSI) and Mercer CFA Institute Global Pension Index (Mercer, 2019, 2021), which is used in this research. The latest version of the PS index available is from 2016 and analyzes 54 countries according to the following indicators: old-age dependency ratio, level of pension benefit from the first pillar and coverage of workforce, legal/effective retirement age, the strength of the funded pillar and reserve fund (as a percent of GDP), pension payments over GDP, public indebtedness over GDP, and need for welfare support. Hence, the PS index encompasses private pension schemes as a determinant of pension sustainability, as does the Mercer CFA index, but unlike the Mercer CFA index includes Central and Eastern European (CEE) economies. Their estimates show that Latvia, Estonia, and Lithuania top the list for CEE countries, while Estonia managed to even enter into the top ten in the overall ranking. Conversely, Croatian and Slovenian pension systems remain at the bottom of the CEE list.

Several authors have developed their pension indicators, such as Chybalski (2016) and Alonso-Fernandez et al. (2018). Chybalski (2016) points out that the multidimensional nature of his indicators enables comparisons of many different empirical pension systems. Since the calculation of the indicators does not require prior data standardization, this method is more resistant to the relativeness of the measurement and comparisons in cross-section studies. However, the relativeness enables the analysis since the efficiency-inefficiency border is not determined. Therefore, the proposed approach is efficient only when a few pension systems are compared, and then it is possible to rank pension systems. Chybalski (2016) shows that the fastest increase in efficiency is reported by Baltic states: Lithuania, Latvia, and Estonia, once again confirming a relatively higher level of pension sustainability in these economies. Different from Chybalski (2016) that uses the Open Method of Coordination (OMC) definitions and methodology, this study offers a test of the Mercer CFA Institute Global Pension Index methodology for the pension sustainability issue. Additionally, in assessing the sustainability of New Member States of the European Union, using indicators proposed by the Monash Centre for Financial Studies, this paper uses the new DEA methodology in this area, while also emphasizing private pension schemes, a path rarely taken in the pension sustainability analyses. A closer look at the DEA methodology and relevant literature is subject of the next section.

3. Data envelopment analysis

In economics, efficiency is a concept defined through the ratio of outputs and inputs. In this paper, we use the DEA methodology and the slack based measure (SBM) of efficiency to evaluate the sustainability of pension systems of eleven New Member States of the European Union. DEA models assume that the decision-making units (DMUs) use a certain number of inputs to produce a certain number of outputs. Following the logic of a production process, a DMU aims to produce more output with the given level of inputs or to use as small inputs as possible to achieve a certain level of output. The basic idea is to detect a DMU which, relative to other DMUs, uses its inputs in a more efficient way to produce its outputs, or achieves higher outputs with its inputs. In this sense, DEA can be used to estimate relative efficiency not only of production units but also of other homogenous and comparable units that can be described in terms of their inputs and outputs. However, in the efficiency analysis concerned with the actual production process, the inputs and the outputs are distinctively defined. In other cases, efficiency is a relative feature that is defined by a set of criteria stemming from the nature of the study. Following the defined set of criteria, variables in DEA are divided into the outputs, whose values we prefer to be as great as possible, and inputs, whose values we prefer to be as small as possible.

This study employs variables that we consider as potential determinants of the sustainability of a pension system, treating some of them as inputs and the other ones as outputs. This allows us to define the relative efficiency of pension systems in EU countries in the area of sustainability and to observe their positions relative to others. The relative efficiency of sustainability in a particular pension system defines its sustainability prospects relative to others, which is why we treat this analysis as the sustainability analysis. The greatest advantage of DEA methodology is that it enables the efficiency analysis based on multiple criteria. Furthermore, pre-defined weights for particular variables are not required. The method generally finds weights for each DMU that puts this DMU in the best possible position regarding others. This allows excluding any subjectivity from the study and allows detecting the comparative strengths or weaknesses of a particular DMU. Also, the comparative advantages of DMU can be used to mitigate deficiencies in a certain amount. This approach enables a new perspective on the sustainability analysis compared to the existing ones, especially in the pension systems' sustainability assessment literature.

Although similar to the traditional DEA models, such as CCR (Rhodes et al., 1978) or BCC (Banker et al., 1984), SBM represents the extension of traditional models and allows dealing with problems where assumptions of traditional models are too limited. It was proposed by Tone (2001). The first step is to introduce the assumptions of the model. Let there be N DMUs that are all characterized by the same set of inputs $x_i,i\in \left\{1,\dots ,m\right\}$ that ‘produce’ the same set of outputs $y_r,r\in \left\{1,\dots ,s\right\}.$ Then $\mathit{X}=\left[x_{ij}\right]\in {\mathbb{R}}^{\mathit{mxN}}$ and $\mathit{Y}=\left[y_{rj}\right]\in {\mathbb{R}}^{\mathit{sxN}}$ Y are input and output matrices, respectively, for the observed set of DMUs. The production possibilities set is defined as: (1.1) $$P=\left\{\left(\mathbf{x},\mathbf{y}\right)|\mathbf{x}\ge X\mathbf{\lambda },\mathbf{y}\le Y\mathbf{\lambda },\mathbf{\lambda }\ge \mathbf{0}\right\}$$(1.1)

The SBM model is non-radial, meaning that it takes into account the case in which the observed changes of inputs cause disproportionate changes of outputs. The original SMB measure is unit invariant concerning units of data and is monotone decreasing in each slack in input and output. The SBM of efficiency can take values between zero and one, but in contrast to traditional DEA models, the unity (indicating efficiency) is obtained if and only if DMU is on the efficiency frontier of the production possibilities set with no input or output slacks (strongly efficient).

While traditional models ignore possible slacks, SMB deals with slacks directly by punishing undesirable excesses or shortfalls. However, when the number of DMUs is too small compared to the number of employed criteria, the measure suffers from insufficient discriminatory power, as do all DEA models. This problem can be solved by the use of the super-efficiency models. The super-efficiency model for SBM (super-SBM) is proposed by Tone (2002). The production possibilities set now excludes the evaluated DMU₀ and in this case turns to: (1.2) $$P\left({\mathbf{x}}_0,{\mathbf{y}}_0\right)=\left\{\left(\overline{\mathbf{x}},\overline{\mathbf{y}}\right)|\overline{\mathbf{x}}\ge \sum _{j=1,\ne 0}^N{\lambda }_j{\mathbf{x}}_j,\overline{\mathbf{y}}\le \sum _{j=1,\ne 0}^N{\lambda }_j{\mathbf{y}}_j,\overline{\mathbf{x}}\ge \mathbf{0},\overline{\mathbf{y}}\ge \mathbf{0},\mathbf{\lambda }\ge \mathbf{0}\right\}$$(1.2)

The formulation of the super-SBM model is as follows. In the input-oriented model, the measure of SBM efficiency with variable returns to scale is obtained by solving the following program for each DMU₀=DMU_j (Tone, 2002): (1.3) $$\begin{array}{c}\min \delta =\frac{1}{m}{\sum }_{i=1}^m{\overline{x}}_i/x_{i0}\\ \mathrm{s}.\mathrm{t}.\mathrm{}\overline{\mathbf{x}}\ge \sum _{j=1,\ne 0}^N{\lambda }_j{\mathbf{x}}_j,\\ \mathrm{}\overline{\mathbf{y}}\le \sum _{j=1,\ne 0}^N{\lambda }_j\overline{\mathbf{y}},\\ \mathrm{}\sum _{j=1,\ne 0}^N{\lambda }_j=1,\\ \mathrm{}\overline{\mathbf{x}}\ge {\mathbf{x}}_0,\mathrm{}\mathrm{}\overline{\mathbf{y}}={\mathbf{y}}_0,\mathbf{\lambda }\ge \mathbf{0},\end{array}$$(1.3)

In (1.3) ${\mathbf{x}}_0$ and ${\mathbf{y}}_0$ are defined as: (1.4) $${\mathbf{x}}_0=X\mathbf{\lambda }+{\mathbf{s}}^{-},$$(1.4) (1.5) $${\mathbf{y}}_0=Y\mathbf{\lambda }-{\mathbf{s}}^{+},$$(1.5) and derived from the projection of DMU₀ on the production possibility frontier where ${\mathbf{s}}^{-}\ge \mathbf{0}$ is a vector of input excess and ${\mathbf{s}}^{+}\ge \mathbf{0}$ a vector of output shortfall. The rationality for this measure is to minimize a sort of weighted l₁ distance from an observed DMU₀ to the production possibility set excluding the DMU₀. The procedure for solving this program can be found originally in Tone (2002).

In case of the whole EU reference set, some data was missing, so it was replaced with a significantly large value (for inputs) or a small value (close to zero) (for output), as proposed by Kuosmanen (2009). In this way, we were able to keep a country in the sample, but these 'penalty' values are not recognized as strengths that contribute to the efficiency of a considered DMU and therefore are not included in the objective function (zero weight is assigned). Furthermore, since the model does not work with negative data, and Greece had a negative value for variable S8 (real economic growth rate averaged over seven years), Greece was excluded from the analysis.

Software DEA Solver Pro v.12 was used for the analysis. The obtained efficiency scores indicate how efficient are the observed DMUs. The higher the efficiency score, the more efficient is DMU compared to others. Thus, the model allows the ranking of DMUs by their efficiency score. Slacks can indicate which variable needs the most attention, in terms of possible increases or decreases to reach the production possibilities frontier.

In classical DEA models, as well as in the model (1.3), it is assumed that the observed set of DMUs has at least one input and at least one output. However, in many cases, especially in those where DMUs are not actual production units, DMUs can be described either only with inputs or only with outputs. In those situations, Kostrzewa et al. (2011) propose a modified approach in DEA, in which it is enough to assume that all DMUs have only one input (or output) which has the same value for all DMUs (for example, that all DMUs have one input equal to one). In such models, the obtained result is a modified synthetic indicator of efficiency given only values of input or outputs.

The last part of our study includes cross-efficiency and sensitivity analysis. As previously mentioned, DEA models allow each DMU to put itself in the best possible position regarding others by loading on its best qualities. The cross-efficiency model is based on the idea of how well a DMU would position if assigned weights of other DMUs. This approach was developed by Sexton et al. (1986) and further developed by Doyle and Green (1994). The cross-efficiency method allows peer evaluation rather than self-evaluation (Cook & Zhu, 2015). The output of the cross-efficiency analysis is a cross-efficiency matrix, presented in Table 1. Each E_ij in the cross-efficiency matrix is an efficiency score of a DMU_i using the optimal weights of a DMU_j. The row and column arithmetic averages have their special interpretations. The column averages are interpreted as peer appraisals, where row averages are the average appraisal of peers.

Table 1. Cross-efficiency matrix.

	Rated DMU
Rating DMU	1	2	3	4	5	6	Averaged appraisal of peers
1	E11	E12	E13	E14	E15	E16
2	E21	E22	E23	E24	E25	E26
3	E31	E32	E33	E34	E35	E36
4	E41	E42	E43	E44	E45	E46
5	E51	E52	E53	E54	E55	E56
6	E61	E62	E63	E64	E65	E66
	Ē1	Ē2	Ē3	Ē4	Ē5	Ē6
	Averaged appraisal by peers (peer appraisal)

Source: Doyle and Green (1994).

Finally, we perform a sensitivity analysis of the efficiency scores by excluding one by one variable. By doing this, we aim to find out how heavy each DMU is loading on a specific variable and conclude whether its position is resilient enough to changes in the variable set.

4. Data and findings

Our sample contains data describing the pension systems of 11 New Member States (NMS) of the European Union (Bulgaria, Croatia, Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Slovakia, and Slovenia). The data includes the values of 12 indicators that are chosen to recreate the structure of the Mercer CFA Institute Global Pension Index sustainability sub-index, and a thorough explanation of the methodology can be found in Mercer (2021). All values are taken for the year 2018 and represent common indicators available in statistical databases:

• Proportion of the working-age population that are members of private pension plans (S1),

• Total assets in funded and private pension arrangements as a percentage of GDP (S2),

• Gap between life expectancy at birth and the state pension age (2020) (S3a),

• Projected gap between life expectancy at birth and the state pension age in 2040 (S3b),

• Projected old-age dependency ratio in 2040 (S3c),

• Estimated Total Fertility Rate (TFR) for 2015-2020 (S3d),

• Mandatory contributions set aside for retirement benefits as a percentage of wages (S4),

• Labor force participation rate (55-64) (S5a),

• Labor force participation rate (65+) (S5b),

• General government consolidated gross debt as a percentage of GDP (S6),

• Older workers (65+) yes/no (S7),

• and Real economic growth rate averaged over seven years (four previous years, three estimated) (S8).

4.1. Descriptive statistics and main results

Data on private pension schemes (indicators S1 and S2) were taken from the OECD database - Pension Markets in Focus (2019) that provide information on funded and private pension systems across the world. Furthermore, European Commission (2018) published the so-called Country fiches on pension projections that were prepared together with national pension ministries or offices that give information on projected life expectancy and state retirement age (indicators S3a, S3b), as well as on share of mandatory contributions for private schemes (indicator S4). Most of the data were taken from the Eurostat database (2020 b) (indicators S3c, S5a, S5b, S6, and S7). Finally, estimates of total fertility rates (indicator S3d) were taken from the United Nations database (2020), whereas rates of real economic growth (indicator S8) were calculated using the International Monetary Fund’ (2020) data and projections.

In our analysis, all variables are indicators that are believed to affect the long-term sustainability of the pension system. Therefore, we treat all of them as inputs. Since we only have inputs in our analysis, we assume that all DMUs have a unit output. The indicators whose higher values are believed to be positively linked with the concept of the long-term sustainability of the pension system are used in the analysis in their inverse form. Therefore, variables S1, S2, S3d, S4, S5a, S5b, S7, and S8 are used in the analysis as the following inputs: 1/S1, 1/S2, 1/S3d, 1/S4, 1/S5a, 1/S5b, 1/S7, and 1/S8. The descriptive statistics of the data are presented in Table 2, to give an aggregate overview of the analyzed indicators and get an initial insight into the situation in the economies in the year 2018. A more thorough analysis of the time dimension of the data is available in Škuflić et al. (2020), where cross-sectional dimension is also studied within a panel setting.

Table 2. Descriptive statistics.

	S1 (%)	S2 (%)	S3d (%)	S4 (%)	S5a (%)	S5b (%)	S8 (%)	S3a (yrs)	S3b (yrs)	S3c (%)	S6 (%)
Min	18.7	5.2	1.4	0.0	44.8	3.0	2.8	11.7	14.5	39.4	8.4
Max	100.0	27.3	1.7	7.3	73.9	14.0	4.3	17.0	19.4	52.0	74.8
Mean	63.8	11.3	1.6	4.5	59.3	6.8	3.4	14.0	16.3	44.6	43.9
SD	22.6	6.2	0.1	2.4	10.2	3.2	0.4	1.7	1.4	3.9	20.2

Source: OECD (2019).

Note: Variable S7 is omitted from Table 4 because its values do not vary between the countries.

Although pension reforms are aimed at increasing the state pension age and consequently reducing time spent in retirement and expenditures on retirement benefits, the average gap between life expectancy at birth and the state pension age is expected to increase until 2040 from 14 to 16.31 years (according to the results from Table 2 for indicators S3a and S3b). The highest increase is expected for Bulgaria, Hungary, and Latvia, while Croatia is the only analyzed economy where the reduction in the time spent in retirement is expected. The projected share of the population over 65 in the working-age population for 2040 (an indicator S3c) ranges from 39,4 percent in Slovenia to 52 percent in Lithuania. Data on the indicators of the labor force participation of older workers (S5a and S5b), especially those over 65 (S5b), show relatively high variability, ranging from only 2.95 percent in Croatia to 13.98 percent in Estonia. Below average are also Hungary, Slovakia, Slovenia, Poland, and Bulgaria. The same indicator for the population 55-64 ranges from 44.75 percent in Croatia to 73.85 percent in Lithuania. The other two Baltic economies join Lithuania in recording the highest labor force participation rate of the elderly. The average proportion of the working-age population that are members of private pension arrangements is 63.8 percent with the relatively highest variability among the NMS. It ranges from 18.7 percent in Hungary to 100 percent in Latvia.

The ability of future governments to pay pension benefits, represented by the indicator S6, is reduced for Croatia, Slovenia, and Hungary, recording general government consolidated gross debt of above 70 percent of GDP. The above-average is also Slovakia and Poland. Estonia, on the other hand, recorded 8.4 percent. Another indicator that is expected to affect the government's financial position is the indicator S8. The real economic growth rate, averaged over seven years, ranges from 2.8 percent in Croatia to 4.33 percent in Romania. Finally, the average share of total assets in funded and private pension arrangements was 11.3 percent, ranging from 5.23 percent in Romania to 27.35 percent in Croatia. Since there are 12 variables/indices in our model, and we also have 11 DMUs, standard SBM is not discriminatory enough. Therefore, we use the super-SBM model. Super efficiency models also allow efficiency ranking or DMUs (Chen & Du, 2015). The results obtained by solving the model (1.3) are presented in Table 3. Scores greater or equal to 1 indicate pension systems which are relatively more successful than the others according to the given criteria, indicating that they have greater sustainability prospects. The efficiency scores exist on an ordinal scale which is used for ranking. For example, score 1.3 is better than 1, but the distances between them are unknown, meaning that 1.3 is not necessarily 30% better than 1.

Table 3. Super-SMB scores and ranks.

No.	DMU	Score	Rank	Alt. rank
1	Bulgaria	1.006381	8	2
2	Croatia	1.06526	2	9
3	Czech Republic	1.011386	6	7
4	Estonia	1.323985	1	1
5	Hungary	1.008692	7	11
6	Latvia	1.04935	3	5
7	Lithuania	1.003278	9	6
8	Poland	1.034523	4	3
9	Romania	1.022784	5	4
10	Slovakia	0.760621	11	10
11	Slovenia	1.00307	10	8

Source: Own.

Having the greatest l₁ distance from the production possibilities set (1.2), Estonia is recognized as the economy with the highest efficiency score or overall best long-term sustainability perspective of the pension system in comparison to the other countries in the set. Estonia is followed by Croatia and Latvia. Slovenia and Slovakia are at the bottom of the list. Other countries are in between. The reason for such scores and rankings could be found by analyzing equations (1.4)(1.4) $${\mathbf{x}}_0=X\mathbf{\lambda }+{\mathbf{s}}^{-},$$(1.4) and (1.5)(1.5) $${\mathbf{y}}_0=Y\mathbf{\lambda }-{\mathbf{s}}^{+},$$(1.5) which represent each DMU in terms of its projection on the nearest point on the production possibilities frontier (whereas that particular DMU is excluded from the set), corrected for slacks. In this way, slacks in the value of a certain indicator, expressed in percentages and displayed in Table 4, indicate deviations from the projection of a particular DMU on the production possibilities frontier for the analyzed indicator. Therefore, slacks allow us to detect the weaknesses and strengths of the individual DMU in the particular aspect of the long-term sustainability of the pension system. This alternative rank is based on the sum of the negative percentage deviations.

Table 4. Deviations (in percent) from projections on the production possibilities frontier.

DMU	1/S1	1/S2	1/S3d	1/S4	1/S5a	1/S5b	1/S8	S3a	S3b	S3c	S6
BG	−3	0	−5	−17	−6	−48	0	7	−1	−5	0
HR	−15	61	−9	−17	−39	−79	−18	−5	17	−11	−89
CZ	−21	−34	6	4	0	−31	−6	−17	−8	4	−47
EE	4	35	2	20	14	139	−2	−14	−4	8	165
HU	−75	−41	−4	−98	0	−53	0	3	−5	7	−40
LV	17	−19	8	0	−3	−31	−3	22	12	−8	−77
LT	−23	−49	−2	−50	4	−8	−11	−3	0	−12	−4
PL	−4	0	−11	22	−15	−51	2	0	12	6	−59
RO	−34	−58	2	−5	−26	0	26	−2	−7	0	−8
SK	−57	−24	−9	−29	−20	−65	−3	−16	0	−8	−55
SI	0	−12	0	−74	−22	−17	−4	−5	−10	4	−40

Source: Own.

Note: Variable S7 is omitted from Table 4 because its values do not vary between the countries.

To understand this alternative ranking, Table 4 presents positive and negative values of slacks. Positive values of slacks in Table 4 indicate that the analyzed economy has positive percentage deviations in the value of the observed indicator from projections on the frontier of the production possibilities set. They, therefore, imply the analyzed economy’s strengths in this particular aspect of the long-term sustainability of the pension system. On the other hand, negative values of slacks indicate that the analyzed country should strive to reduce the value of that indicator since it negatively deviates from its projection to the frontier of the production possibilities set for this particular indicator. In this way, negative deviations indicate possible improvements in certain areas that affect the long-term sustainability of the pension system of the analyzed economy.

The results presented in Table 4 can be interpreted for the individual country, but also each indicator. They show that Estonia has comparative strengths in almost all indicators, except in the S3a (Gap between life expectancy at birth and the state pension age (2020)) and S8 (Real economic growth rate averaged over seven years (four previous years, three next years estimated). Its comparative strength is the highest in the value of the indicator S6 (variable General government consolidated gross debt as a percentage of GDP). Specifically, its projection on the production possibilities frontier has the value of this indicator 165 percent higher than the actual registered value for Estonia, implying that the general gross debt of Estonia is significantly lower than in the rest of NMS. Similarly, Estonia’s projection on the production possibilities frontier has the value of the input 1/S5b (S5b - Labor force participation rate (65+)) 139 percent higher than the actual registered value for Estonia. Note that the first eight inputs in Table 4 are inverse values of the used variables. Therefore, the interpretation should be adjusted accordingly.

The same principle is applied in the interpretation of other indicators. Table 4 reveals why Croatia is ranked as the second-best concerning the efficiency score presented in Table 3 and Latvia as the third best. Precisely, Croatia has a relatively high positive slack in input 1/S2 indicating a relatively high value of variable S2 (Total assets in funded and private pension arrangements as a percentage of GDP), which is thought to be positively related to the sustainability of the pension system. Here it is important to note that, among the observed countries, Croatia is the only one that is not drifting away from the funded pensions, by reducing its relative importance. Latvia’s comparative strengths are variables S3a (Gap between life expectancy at birth and the state pension age in 2020), and variable 1/S1 (S1 - Proportion of the working-age population that are members of private pension plans). Note that Romania has a positive slack in variable 1/S8 (Real economic growth rate averaged over seven years (4 years, 3 estimated), but the model recognized that Romania had no positive deviations in other inputs. Other countries also did not have such distinctive preferred differences relative to others.

The vast majority of the analyzed economies (except for Estonia) show relatively poorer results concerning the two indicators: S6 (General government consolidated gross debt as a percentage of GDP) and S5b (Labor force participation rate (65+)). This conclusion is carried out based on negative deviations from their projections to the production possibilities frontier in the values of these two indicators. Estonia set a high standard for other economies, registering the lowest general government consolidated gross debt as a percentage of GDP of 8.4 percent, with the second in line being Bulgaria with 22.3 percent.

Furthermore, Estonia also registered the highest labor force participation rate of the population over 65, with 13.98 percent. The other two Baltic economies, Latvia and Lithuania followed with 9,78 percent and 9,35 percent, respectively. Moreover, we observe that Croatia and Hungary have very large negative slacks in many variables. Croatia shows the highest negative deviations concerning the general government consolidated gross debt as a share of GDP. Also, the Croatian Government should encourage higher labor force participation of the elderly, especially those over 65. Slovakia, Hungary, Poland, and Bulgaria should follow the same strategy. When it comes to the economic growth rate, Romania shows its comparative strength. But, its weakness lies in the lowest share of total assets in funded and private pension arrangements in GDP among NMS.

Since Croatia is ranked very high, we conclude that efficiency scores of the model (1.3) are obtained by putting a lot of weight on the comparative strengths of DMUs. If we sum all positive percentage deviations for each DMU and rank countries accordingly, from highest to lowest values, we would obtain the ranking presented in the third column of Table 3. However, if we were to rank DMUs based on the sum of the negative percentage deviations, we would get a different ranking. This alternative ranking is presented in the last column of Table 3, called Alternative rank (Alt. rank). Estonia is still at the top of the list, but the position of Croatia and Hungary has changed significantly. On the other hand, the position of Bulgaria and Lithuania is significantly improved. However, when the best and the worst country were excluded from the sample, Latvia’s pension system is proven as the most efficient and Croatia’s as the least efficient.

The model assumes that each projection on the production possibility frontier is possible and can be expressed as a combination of other DMUs, called peers. The peers form a reference set of a DMU. The relative strength of a DMU within the observed set can also be evaluated in terms of frequency of appearance in the reference set of other DMUs (Friedman et al., 2002). Table 5 shows reference sets for all DMUs.

Table 5. Reference sets and peers.

	BG	HR	CZ	EE	HU	LV	LT	PL	RO	SK	SI
BG				0.50		0.38			0.12
HR				1.00
CZ				0.78				0.22
EE	1.00
HU						0.37		0.48	0.14
LV				1.00
LT				0.14		0.86
PL				0.56					0.44
RO				0.27		0.47		0.26
SK				0.51		0.49
SI			0.75		0.25			0.01
count	1	0	1	8	1	5	0	4	3	0	0

Source: Own.

A DMU in the first column is projected as a combination of other using the weights shown in the table (empty cells indicate that a specific DMU is not in the reference set of an observed DMU) (Gardijan et al., 2021). The last row of Table 5 shows the number of occurrences of a specific DMU in the reference sets of other DMUs. It is obvious that the most efficient DMU, which is Estonia, can be ranked first in the light of this criterion as well. It is followed by Lithuania. Although Croatia is ranked as a second given its efficiency score, it now shows some difficulties and the lack of relative strength in terms of the long-term sustainability perspectives of the pension system.

Before running sensitivity and cross-efficiency analyses, the efficiency of pension systems of eleven New EU Member States was compared to the rest of the EU member states. Table 6 presents the newest rankings.

Table 6. Comparison of the Efficiency of Pension Systems of EU Member States.

Rank	Country	Rank	Country
1	Estonia	15	Bulgaria
2	Ireland	16	Austria
3	Netherlands	17	Lithuania
4	Denmark	18	Portugal
5	Finland	19	Czech Republic
6	Latvia	20	Belgium
7	Sweden	21	Croatia
8	Luxembourg	22	Germany
9	Poland	23	Slovakia
10	France	23	Slovenia
11	Malta	23	Spain
12	Romania	23	Italy
13	Hungary
14	Cyprus

Source: Own.

The leading countries among the New EU Member States, Latvia and Estonia, are now placed high in the rankings of all EU member states (excluding Greece), while others are spread across the board. Comparing the average rankings older Member States do have a better average ranking (13.29) than the new members (15). However, the date of the accession is not a relevant determinant of the efficiency of the pension systems (given the used criteria).

4.2. Sensitivity and Cross-Efficiency analysis

The results of the efficiency of the pension systems of selected countries presented so far in respect of the indicators that the Monash Centre considers as indicators of the long-term sustainability of current pension systems are based on the use of 12 indicators. Below we conduct the sensitivity analysis of the results obtained and present the results of 12 analyses that have been obtained so that in each next step one indicator is excluded from the analysis. In this way, the existing efficiency assessment relies on the use of 11 indicators. This approach allows us to detect how the order of economies is changing and how important a particular indicator is for the final rating of the country.

Table 7 shows that regardless of which indicator is excluded from the analysis, Estonia remains first. Croatia, which was ranked second shows great dependence on the second indicator because when it is taken from the analysis, Croatia takes ninth place. In this context, Latvia takes second place. In all other cases, except where the analysis omits the ninth indicator, Latvia takes third place. In addition to Estonia and Latvia, high resilience also shows Poland. On the other hand, Slovakia takes the last place, regardless of which indicator is omitted from the analysis. Slovenia also shows weaker sustainability results, which do not depend on the chosen indicator. On the other hand, Romania and Hungary, like Croatia, show dependence on the indicators in which they have a good relative position.

Table 7. Rankings when one indicator is dropped from the analysis (12 columns for each new ranking).

	-S1	-S2	-S3	-S4	-S5	-S6	-S7	-S8	-S9	-S10	-S11	-S12
BG	8	7	8	7	8	8	8	7	10	8	6	10
HR	2	9	2	2	2	2	2	2	2	2	2	2
CZ	6	5	7	8	6	6	6	5	6	6	7	6
EE	1	1	1	1	1	1	1	1	1	1	1	1
HU	7	6	6	6	7	7	7	6	9	7	9	7
LV	3	2	3	3	3	3	3	3	4	3	3	3
LT	9	8	9	9	10	9	9	8	7	10	8	8
PL	4	3	4	5	4	4	4	4	3	4	4	4
RO	5	4	5	4	5	5	5	10	5	5	5	5
SK	11	11	11	11	10	11	11	10	10	11	9	11
SI	10	10	10	10	9	10	10	9	8	9	9	9

Source: Own.

The final step is to present the results of the cross-efficiency analysis that evaluates the efficiency of a country in such a way that it uses the optimal weights of other countries in the efficiency assessment.

Table 8 presents the efficiency scores obtained by a cross efficiency model, from which similar conclusions are drawn as from the previous part of the analysis. The most resilient from the top are Estonia, Latvia, and Poland, while the bottom is still held by Lithuania, Slovakia, and Slovenia. The biggest change in ranking concerning our results shows Croatia.

Table 8. The results of the cross-efficiency analysis.

	BG	HR	CZ	EE	HU	LV	LT	PL	RO	SK	SI
BG	1	0.851	0.921	1	1	1	0.913	0.999	1	0.866	0.907
HR	0.960	1	0.833	1	0.742	1	0.887	0.901	0.773	0.904	0.694
CZ	0.966	0.885	1	1	1	1	0.925	0.972	0.994	0.898	1.000
EE	0.980	0.838	0.896	1	0.875	0.941	0.791	1	1	0.906	0.838
HU	0.969	0.897	1	1	1	1	0.916	1	0.995	0.901	0.981
LV	0.976	0.872	0.963	1	1	1	0.912	1	0.957	0.874	0.954
LT	1	1	1	1	1	1	1	1	1	1	1
PL	0.991	0.912	0.930	1	0.916	1	0.876	1	1	0.944	0.880
RO	0.925	0.829	0.932	1	0.887	1	0.937	1	1	0.862	0.857
SK	1	1	1	1	1	1	1	1	1	1	1
SI	0.968	0.896	1	1	0.962	1	0.913	0.997	1	0.890	1
average	0.976	0.907	0.952	1	0.944	0.995	0.915	0.988	0.974	0.913	0.919
rank	4	11	6	1	7	2	9	3	5	10	8
SD	0.021	0.062	0.053	0.000	0.079	0.017	0.054	0.029	0.065	0.046	0.092

Source: Own.

5. Conclusion

This paper employs Data Envelopment Analysis (DEA) to obtain the sustainability assessment of pension systems of eleven New Member States of the European Union: Bulgaria, Croatia, the Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Slovakia, and Slovenia. The conducted analysis is based on several indicators that compose the sustainability sub-index of the Mercer CFA Institute Global Pension Index, developed by the Australian Monash Center for Financial Studies. The chosen indicators are believed to influence the long-term sustainability of current pension systems of NMS, whereby several aspects are emphasized: the economic importance of the private pension system, its funding level, the length of expected retirement both now and in the future, the labor force participation rate of the older population, the current level of government debt and the level of economic growth. In particular, it is believed that the chosen indicators affect the likelihood that the current pension system will be able to provide benefits into the future. Accordingly, the DEA methodology and the super slack based measure (SBM) of efficiency is used in obtaining efficiency scores for pension systems of analyzed economies. All indices are treated as inputs, in their original or inverse form, whereby it is assumed that all DMUs (analyzed economies) have a unit output.

Estonia is recognized as the economy with the highest efficiency score or overall best long-term sustainability perspective of the pension system in comparison to the other countries in the set. Results show that Estonia has comparative strengths in almost all indicators, except in the Gap between life expectancy at birth and the state pension age for 2020, and the real economic growth rate averaged over seven years. Estonia also stands out in other empirical studies. Price and Rudolph (2013) and Altiparmakov and Nedeljković (2018) acknowledge Estonia as one of the rare countries that successfully implemented the austerity measures that are mandatory to finance pension privatization. Krpan et al. (2019) show that Estonia ranks first among NMS in the labor force participation rate of the elderly and very high in the fiscal balance of the pension system. When comparative strengths are given the most weight, Estonia is followed by Croatia and Latvia. Slovenia and Slovakia are at the bottom of the list. However, if we were to rank DMUs based on their weaknesses, the position of Croatia and Hungary would deteriorate significantly, and those of Bulgaria and Lithuania would improve. The vast majority of the analyzed economies (except for Estonia) show relatively poorer results concerning the two indicators: General government consolidated gross debt as a percentage of GDP and Labor force participation rate of the population over 65. Therefore, governments of New Member States should generally encourage higher labor force participation of the elderly, especially those over 65 to improve the sustainability of their pension systems. Finally, the sensitivity and cross-efficiency analyses are conducted to detect how the order of economies is changing when a particular indicator is dropped from the analysis and how important a particular indicator is for the final rating of the country. Additionally, cross-efficiency analysis uses the weights of the other countries in obtaining the efficiency results. Both results show that the most resilient economies from the top are Estonia, Latvia, and Poland, while the bottom is still held by Lithuania, Slovakia, and Slovenia. The biggest change in ranking concerning our results shows Croatia.

There are several limitations to this approach to measuring pension sustainability. A limitation already presented is the notion of the role of private pension schemes in the sustainability of pension systems. They are included in this study under the assumption that they reduce the burden of aging in the first pension pillar, although their importance also depends on the way the transition is carried out. Indicator of part-time employment and the possibility of phased retirement is used according to the methodology of Mercer (2019), but the index gives a score between 0 and 2 depending on the circumstances and practice, meaning it is not an objectively measured factor, so this could be enhanced in future studies. Also, given the extremely large number of sustainability indicators and related factors, it is possible to expand the set of indicators in future research. Future research could focus on combining a greater number of indicators of both public and private pension schemes, with different sets of economies.

In summary, this paper shows how different techniques and indicators should be used to approach the concept of pension sustainability. The present findings confirm the rankings of New Member States from different empirical studies, emphasizing the favorable situation in Baltic countries. Aside from the common classification and clustering of economies, the results provide evidence to comparative strengths and weaknesses of analyzed pension systems, forming a basis for suggestions to improve each retirement income system.

Disclosure statement

There are no relevant financial or non-financial competing interests to report.

Notes

1 Poland is the only New Member State currently included in the Mercer CFA Institute Global Pension Index.