Constructing Knowledge Economy Composite Indicators using an MCA-DEA approach.

Abstract

Composite indicators are a remarkably useful tool in policy analysis and public communication for assessing phenomena, such as Knowledge-Based Economy (KBE), that cannot be expressed by means of a simple indicator. The objective of this study is to propose and compare three MCA-DEA models from a “Benefit of Doubt” (BoD) approach in order to build KBE Composite Indicators. To show the effectiveness of the models, this paper proposes a case study of 36 European countries to assess the degree of development of KBE. The results revealed differences with respect to the optimal weights assigned to the sub-indicators, the discriminating power, the operability, and the participatory nature of the models. Model 1 yielded high scores for every country and low discriminating power. Model 2 favored the most efficient countries in terms of KBE and allows for the incorporation of expert knowledge, thereby giving flexibility to the process. Model 3 made it possible to construct composite indicators from an optimal balance approach and yielded low results overall. These results demonstrate the necessity to analyze the different choices for measuring KBE in order to determine which indicator is more suitable for each context.

Keywords:

• Knowledge economy
• management
• composite indicators
• innovation
• GP
• BoD.

JEL CODES:

• F63
• O33
• O57
• O38

1. Introduction

Economic theory has exposed the importance of technology and knowledge in economic development and human welfare (Tchamyou, 2017; Dodgson & Gann, 2018). A historical analysis of the processes of transformation of the productive system allows us to identify the growing importance of the concept of “knowledge” as a driving force in development processes (Karahan, 2012). The concept of “Knowledge-Based Economy” (KBE) was initially introduced by Machlup (1962) and made popular by Drucker (1969). This concept is currently used to refer to productive systems whose basis is, precisely, knowledge (Chen & Dahlman, 2005). The Organization for Economic Cooperation and Development (OECD) described KBE as “the economies which are directly based on the production, distribution, and use of knowledge and information” (OECD, 1996:3). Indeed, knowledge has always been a driver of economic growth, but nowadays it has grown significantly. This idea is encapsulated in Foray and Lundwall (1996: 27) as follows “Economy is more strongly and more directly rooted in the production, distribution and using of knowledge than ever before”. If we focus on companies, we can say that those whose activity is knowledge-intensive have a value-added multiplier and greater productivity, compared to those whose activity is less knowledge-intensive (Lee & Gibson, 2002).

Regarding the growing importance of knowledge in production systems, research on KBE has grown significantly in recent decades, including studies at the company and/or national level. Economists have revised long-term growth theories to accommodate the role of knowledge (Romer, 1994). However, it is complicated to quantify the concept of knowledge and, therefore, the description and measurement of KBE are conditioned by the construction of reliable indicators (Karahan, 2012), the data available, and their capacity to reflect a complex reality. This is not a minor issue, as the possibility of proposing objective approaches to the concept of knowledge, as well as to measurements of KBE, is essential to understand its dynamics and design more effective policies.

The first step towards measuring and monitoring the achievements of the KBE is to identify the conceptual framework and the aspects included in the measurement systems of KBE. The second step would be to define methodologies for the construction of composite indicators (CI) that are capable of adapting to different contexts and reflecting the many aspects of the phenomenon in question. Thus, creating CIs is a very useful solution for monitoring a complex problem, as they synthesize in a single piece of data the information contained in a number of variables about different aspects of a given issue (Guaita et al., 2020; Salinas et al., 2020). Therefore, their use is advisable when the complexity of the phenomenon makes it impossible to work with a single variable, as would be the case with KBE. There are several proposals designed to measure and monitor countries’ achievements as regards KBE. However, despite the importance of measuring these, the number of methodological proposals is very limited, and there is yet no analysis of how each of them can be adjusted to different contexts or concrete needs. This work focuses on this research gap. The aims of this project are, on the one hand, to propose new methodologies for the creation of KBE composite indicators and, on the other hand, to compare the differences between them in a specific context. The starting point is the idea that a single methodology for creating descriptive indicators of KBE cannot respond to the needs of different societies, or describe the particular aspects of a knowledge-based economy. In order to be able to offer a proposal, this paper provides a review of the main KBE frameworks and a description of the main measurement indicators. This will allow us to come up with new methodologies for creating indicators that complement the current ones, but which also conform to the generally accepted conceptual frameworks.

The proposals put forward in this study are based on Multi-Criteria Analysis (MCA) techniques, which are greatly suitable when working from a multidimensional approach. To achieve the objectives of this study, we propose and compare three MCA-DEA models that were developed following a “Benefit of Doubt” (BoD) approach, which will allow us to build KBE composite indicators. Model 1 is a basic BoD model. Model 2 is a BoD model with proportional sub-indicator share restrictions of a participatory nature. Model 3 is a common weighted DEA that uses a balanced analysis approach to aggregate simple indicators by means of a MINMAX Goal Programming method. To demonstrate the usefulness of the models, a case study including 36 European countries was carried out, taking as a reference the system of indicators proposed by the European Commission. This framework considers 27 indicators associated with innovation, grouped into ten sub-indexes: Human resources, Attractive research systems, Innovation-friendly environment, Finance and support, Firm investments, Innovators, Linkages, Intellectual assets, Employment impacts, and Sales impacts. The robustness of the models was confirmed by using a Spearman’s Rank correlation coefficient and compared with the Composite Innovation Indicators of the European Innovation Scoreboard.

2. Theoretical framework and main indicators proposed

When talking about KBE, reference is often made to a process of structural transformation at the national level, and so the focus of international organizations has been on developing indicators capable of describing the weight that knowledge has in their economies (Karahan, 2012). Carter (1996) pointed out that measuring knowledge is not only a problem at the business level, but that the challenge extends to country-level measurement too. There is currently no widely accepted framework for measuring KBE, but there are several proposals from international organizations such as the Asia Pacific Economic Cooperation (APEC), the Organization for Economic Cooperation and Development (OECD), the World Bank (WB), and the European Union (EU). One of the main disadvantages of the above proposals and other alternatives is that, as Shapira et al. (2006) puts it, they are conditioned by the fact that the indicators selected are those whose availability is ensured in all the countries involved in the analysis. Moreover, these indicators are conditioned by the quality of the statistical sources, something that in less developed countries can be a problem (Devarajan, 2013).

The World Bank’s (WB) proposal, known as Knowledge Assessment Methodology (KAM), born in 1999, states that KBE is based on four pillars: Education and Human Resources, Economic Incentive and Institutional Regime, Communication Technology, Innovation System and Information (World Bank Institute, 2007). The framework proposed by this organization takes into consideration 83 different variables that serve as proxies for the four pillars described above. These variables are grouped as follows: Economic Incentive and Institutional Regime Index (19), ICT Index (12), Overall Economic Performance (9), Innovation System Index (24), Education and Human Resources Index (19). Concerning the methodology for aggregating information, this institution has used two modes, the Knowledge-based Economy Index (KEI) and the Basic Scorecard, both of them included within the KAM. The former is calculated on the basis of 12 indicators. The result of this indicator provides a picture of the degree of development of a country or region in terms of KBE. The methodology on which it is based relies on the simple average of normalized values of the 12 indicators of the basic scorecard. Whereas the Basic Scorecard offers an overview of the performance of the country as concerns KBE by using 14 variables (Table 1). The KAM Basic Scorecard is a disaggregated representation of the KEI.

Table 1. Indicators included in the measurement of KBE by several international organizations.

World Bank Knowledge Economy Indicators (Basic Scorecards)	OECD Knowledge Economy Indicators	European Union Knowledge Economy Indicators (European Innovation Scoreboard)	APEC Knowledge Economy Indicators
1. Performance  1.1 Average annual GDP growth (%)  1.2 Human Development Index 2. Economic Incentive and Institutional Regime  2.1 Tariff and non-tariff barriers 2.2 Regulatory Quality  2.3 Rule of Law 3. Education and Human Resources 3.1 Adult Literacy rate (%age 15 and above) 3.2 Secondary Enrolment 3.3 Tertiary Enrolment 4. Innovation System 4.1 Researchers in R-D, per million populations 4.2 Patent Applications granted by the USPTO, per million populations 4.3 Scientific and technical journal articles, per million populations 5. Information Infrastructure 5.1 Telephones per 1000 persons, (telephone mainlines + mobile phones) 5.2 Computers per 1000 persons 5.3 Internet Users per 10000 persons	1. Knowledge-Based Economy 1.1 Knowledge Investment (education, R&D and software) as % of GDP 1.2 Education of the adult population as % of the population aged 25-64 1.3 R&D expenditure as a percentage of GDP 1.4 Basic research expenditure as a percentage of GDP 1.5 Expenditure of Business R&D in domestic product of industry 1.6 Expenditure of Business R&D in manufacturing 1.7 Share of services in R&D expenditure 1.8 Expenditure on innovation as a share of total sales 1.9 Investment in venture capital as a percentage of GDP 2. Information and Communication Technology 2.1 ICT spending as % of GDP 2.2 PC penetration in households 2.3 Number of internet host per 1000 inhabitants 2.4 Percentage share of ICT industries in GDP 2.5 Share of ICT in patents granted by USPTO 3. Science and Technology Policies 3.1 Publicly funded R&D as % of GDP 3.2 Government R&D expenditure on health-defense-environment 3.3 Government R&D expenditure in total R&D expenditure 3.4 Business R&D expenditure in total R&D expenditure 3.5 Share of Government-Business R&D expenditure financed together 3.6 Tax subsidies rate for R&D 4. Globalization 4.1 Share of foreign affiliates in R&D 4.2 Share of foreign and domestic ownership in total inventions 4.3 Number of international technological alliances 4.4 Percentage of scientific publications with a foreign co-author 4.5 Percentage of patents with a foreign co-investor 5. Output and Impact 5.1 Scientific publications per 100 000 population 5.2 Share of countries in total EPO patent application 5.3 Share of firm creating any innovative output 5.4 GDP per employed person 5.5 Share of knowledge-based industries in total value added 5.6 Share medium-high technology industries in manufacturing export 5.7 Technology balance of payments as a percentage of GDP	1. Innovation Drivers 1.1 New S&E graduates per 1000 population aged 20-29 1.2 Population with tertiary education per 100 population aged 25-64 1.3 Number of broadband lines per 100 population 1.4 Participation in life-long learning per 100 population aged 25-64 1.5 Percentage population age 20-24 completed secondary education 2. Knowledge Creation 2.1 Public R&D expenditures (% of GDP) 2.2 Business R&D expenditures (% of GDP) 2.3 Share of medium high-tech and high-tech R&D 2.4 Share of enterprises receiving public funding for innovation 2.5 Share of University R&D expenditures financed by business sector 3. Innovation and Entrepreneurship 3.1 SMEs innovating in-house (% of SME) 3.2 Innovative SMEs co-operating with others (% of SMEs) 3.3 Innovative expenditures (% of turnover) 3.4 Early-stage venture capital (% of GDP) 3.5 ICT expenditure (% of GDP) 3.6 SMEs using non-technological change (% of SMEs) 4. Application 4.1 Employment in high-tech services (% of total workforce) 4.2 Exports of high technology products as share of total exports 4.3 Sales of new-to-market products (% of turnover) 4.4 Sales of new-to-firm not new-to-market products (% of turnover) 4.5 Employment in medium-high tech manufacturing (% of total) 5. Intellectual Property 5.1 New European Patent Office patents per million 5.2 New United States Patent and Trademark Office per million 5.3 New Triad patents per million population 5.4 New community trademarks per million population 5.5 New community industrial designs per million population	1. Business Environment 1.1 Knowledge based Industries as % of GDP 1.2 Services Exports as of GDP 1.3 High-Tech Exports as of GDP 1.4 Foreign Direct Investment inward flow as % of GDP 1.5 Government transparency rating by World Competitiveness Yearbook 1.6 Financial transparency rating by World Competitiveness Yearbook 1.7 Competition policy rating by World Competitiveness Yearbook 1.8 Openness rating by World Competitiveness Yearbook 2. ICT Infrastructure 2.1 Number of mobile telephones in use per 1000 inhabitants 2.2 Number of telephone mainlines in use per 1000 inhabitants 2.3 Number of computers per 1000 inhabitants 2.4 Number of internet users as % of population 2.5 Internet hosts per 10000 2.6 Expected e-commerce Revenues, M$US 3. Innovation System 3.1 Scientists Engineers in R&D per million of the population 3.2 Full-time researchers per million of the population 3.3 Gross Expenditure on R&D (% of GDP) 3.4 Business Expenditure on R&D (% of GDP) 3.5 US Patents per annum 3.6 The number of technological cooperation among companies 3.7 The number of technological cooperation between company-university 4. Human Resource Development 4.1 Secondary enrolment (% of age group) 4.2 Natural Sciences Graduates per annum 4.3 Knowledge Workers (% of labor force) 4.4 Newspaper (per 1000 inhabitants) 4.5 Human Development Index

Source: APEC (2000); European Commission (2010), OECD (1999), World Bank Database (n.d.).

The OECD issued another methodology for the analysis of KBE. This institution defined KBE in 1996 as “economies which are directly based on the production, distribution, and use of knowledge and information” (OECD, 1996: 20). Although the OECD proposed a direct measurement of KBE, there are several obstacles. For instance, it is difficult to set up intellectual capital accounts in a way that is comparable to accounts of conventional fixed capital. In short, knowledge is difficult to measure and value. In this sense, the proposals for indicators have been developed little by little, and have at least tried to reflect the rates of return in education and training (Leung, 2004). The first attempt at measurement proposed by the OECD emerged in 1999 (Table 1), from a body of 32 indicators grouped into 5 dimensions (OECD, 1999). This proposal grants special relevance to information and communication technologies, to science, and to technological activities, all of which are conditioned by the way in which these activities are globalized.

The European Commission has also developed a measurement methodology based on the European Innovation Scoreboard (EIS). The proposal is not too different from previous ones, including a set of selected indicators to measure the performance of KBE in Europe. This methodology focuses on innovation systems to identify indicators of KBE. Its scoreboard has been designed on the basis of the main drivers of KBE, and also includes measures related to the innovation outputs. As shown in Table 1, the proposed indicators are distributed in 5 categories that consider input and output measures. Parallel to this proposal, the European Commission publishes the Global Innovation Scoreboard, which offers a reduced version of the EIS based on 12 indicators.

The Asia Pacific Economic Cooperation (APEC) has also made a proposal for measuring KBE. The rationale behind this system was to create a framework that could serve as a guide for the effective use of knowledge among its member countries. This proposal includes 26 indicators, grouped into 4 dimensions (Table 1) (APEC, 2001). It should be noted that in order to create the four dimensions of KBE, the APEC carried out an analysis of its main pillars. This analysis highlighted the importance of innovation and technological change, the improvement of human resources, the need for efficient communication infrastructures, and a business network that fosters innovation and the creation of companies.

The above methodologies represent groups of indicators that seek to measure or describe a complex reality such as KBE. The major challenge lies in how to aggregate these simple indicators to build composite indicators capable of measuring the same concept and analyzing how they interact with each other (Karahan, 2012), for if we merely build groups of indicators, we will only provide a very basic description (Leung, 2004).

3. The challenge of building a meaningful KEB indicator

The concept of composite indicators (CI) was introduced in a practical way in the 1990s as a procedure to synthesize information about a complex and multidimensional phenomenon (Sumner & Tezanos Vázquez, 2014). Many institutions have since created CIs to measure a wide range of phenomena. In fact, 160 composite indicators have been established and employed around the world (Foa & Tanner, 2012). The use of CIs takes on special relevance in the description of complex realities in which many variables are involved (Martín et al., 2017; Rodríguez et al., 2016). These indicators provide an overview of a group of partial indicators, thus avoiding comparison based on multiple individual data. The worth of these indicators lies in their capacity to synthesize information and provide a useful tool for decision-making (Cherchye et al., 2011). Aggregating information into a single variable allows for an easier interpretation of the phenomenon in question and, thus, makes it possible to capture the performance of regions or countries (Hudrliková, 2013). Such an approach is very useful for policymakers, academics, experts, journalists, etc., given that they can analyze a phenomenon with only one piece of data (Saltelli et al., 2006). The OECD (n.d.) Glossary defines CIs as: “A composite indicator is formed when individual indicators are compiled into a single index, on the basis of an underlying model of the multi-dimensional concept that is being measured”. However, these indicators raise a number of issues that should not be overlooked (Nardo et al., 2005). To some extent, their construction is conditioned by a degree of subjectivity, which may come to undermine their credibility (Booysen, 2002). This would also include the definition of the phenomenon at issue, which of the partial indicators should be included, how these should be aggregated, how inaccurate or poor quality data should be treated (Cherchye et al., 2011), how to avoid redundant information or how to establish the weights of partial indicators, or even their misinterpretation or simplicity (Zermeño et al., 2019).

On the more technical side, CIs are mathematical combinations of a set of multidimensional indicators (Nardo et al., 2005, Saisana et al., 2005), and normal measures that aggregate the information generated by a complex phenomenon. A basic condition of CIs is transparency in their construction; thus, allowing for discussion and replication on the part of stakeholders (Saisana et al., 2005). Partial indicators can be aggregated without being scaled by means of percentages or ratios, if all the variables are measured with the same unit (Talukder et al., 2017). In other cases, the indicators are expressed in different units (Salzman, 2003) or measurement techniques. When this occurs, it is necessary to apply a process of normalization, which consists of making partial indicators expressed in different units comparable on a common basis (Sajeva et al., 2005). During the process of normalization, measurement errors can be made and the indicator may lose quality (Hudrlíková & Kramulová, 2013). Therefore, this process must be carefully chosen, as different normalization techniques can yield different values (OECE, 2008) and therefore, lead to significant differences in the results of the CI (Cherchye et al., 2007; Tate, 2012).

The methodology for estimating the relative weights assigned to the partial indicators is one of the most delicate stages in the process of creating CIs (Hudrliková, 2013). One of the most widely used methods to construct composite indicators when the information about the weights is unknown is the Data Envelopment Analysis (DEA), developed using the Benefit of Doubt Approach (BoD). DEA is often applied to analyze the best practices or the efficiency level in the production or delivery of a service (Phucharoen & Sangkaew, 2020). The main disadvantage of this method is that without setting constraints the weight is given by an indicator in which the analysis unit is the best (Cherchye et al., 2009); therefore, cases displaying a good performance in a single indicator obtain a better position in the ranking of the global indicator. This methodology can be found explained in detail in Cherchye et al. (2008). One of the most widespread methodologies is Linear Aggregation (LIN) since its calculation is very simple. However, this form of aggregation implies full compensability, and the poorer performance of one indicator can be balanced with higher values in others. In addition to the above, the OECD (2008) has suggested additional ones such as geometric aggregation and the non-compensatory multi-criteria approach. The former benefits the higher-performing countries the most, so countries with poorer performance tend to prefer linear aggregation. Multi-Criteria Analysis (MCA) represents the non-compensatory approach. This methodology can lead to problems when there is a large number of criteria being analyzed (Munda and Nardo (2009), although some solutions to this weakness have been proposed (Munda, 2010). An additional proposal is the Distance Method (DP2), which offers another form of aggregation of partial indicators (Pena, 1977). This methodology has some advantages related to key aspects in the process of building CIs. It assigns weights to partial indicators objectively, avoids duplicate information, and makes it possible to aggregate variables expressed in different units (Rodríguez et al., 2018; Rodríguez et al., 2018; Rodríguez et al., 2017; Martín et al., 2019; Guaita et al., 2019).

As discussed in the previous section, several alternative indicators have been proposed to measure the main dimensions, criteria, and sub-criteria representative of KBE. One of the key aspects, as mentioned above, lies in the assignment of weights to each of the partial indicators used. The criterion used for this purpose should take into account the context in which the indicator is applied, or even the existence of a specific goal that might condition the measurement of KBE (Freudenberg, 2003). Therefore, bearing in mind the primary objective of creating a representative CI of KBE, there are different proposals for constructing indicators, each of which is better suited to a particular context or objective. Under this premise, we offer below a proposal of alternative indicators that differ in the methodology with which they are created, but that share a common body of partial indicators. These indicators are those proposed by the European Commission.

4. Material and methods

4.1. Case study

The case study in this paper is based on 10 sub-indicators, made up of 27 indicators, as identified by the European Commission to measure KBE under the framework of the European Innovation Scoreboard (EIS) (EU, 2019). The EIS focuses mainly on innovation processes so as to define KBE indicators and has been designed to capture the main drivers of a knowledge-based economy plus some measures of innovation outputs (Karahan, 2012). The measurement framework of the EIS includes 4 dimensions, 10 sub-indicators, and 27 indicators, as shown in Table 2.

Table 2. Dimensions, sub-indicators, and indicators considered to the Knowledge Economy Composite Indicators.

1. Framework conditions	1.1. Human resources (HR)	1.1.1. New doctorates and graduates
		1.1.2. Population aged 25-34 with tertiary education
		1.1.3. Lifelong learning
	1.2. Attractive research systems (ARS)	1.2.1. International scientific co-publications
		1.2.2. Top 10% most cited publications
		1.2.3. Foreign doctorate students
	1.3. Innovation-friendly environment (IFE)	1.3.1. Broadband penetration
		1.3.2. Opportunity-driven entrepreneurship
2. Investments	2.1. Finance and support (FS)	2.1.1.R&D expenditure in the business sector
		2.1.2. Venture capital expenditures
	2.2. Firm investment (FI)	2.2.1. R&D expenditure in the business sector
		2.2.2. Non-R&D innovation expenditures
		2.2.3. Enterprises providing trainning to develop or upgrade ICT skills of their personnel
3. Innovation activities	3.1. Innovators (I)	3.1.1. SMEs with products or process innovations
		3.1.2. SMEs with marketing or organisational innovations
		3.1.3. SMEs innovating in-house
	3.2. Linkages (L)	3.2.1. Innovative SMEs collaborating with others
		3.2.2. Public-private co-publications
		3.2.3. Private co-funding of public R&D expenditures
	3.3. Intellectual assets (IA)	3.3.1.PCT patent applications
		3.3.2. Trademark applications
		3.3.3. Design applications
4. Impacts	4.1. Employment impacts (EI)	4.1.1. Employment in knowledge-intensive activities
		4.1.2. Employment fast-growing enterprises of innovative sectors
	4.2. Sales impacts (SI)	4.2.1. Medium and high-tech product exports
		4.2.3. Knowledge-intensive services exports
		4.2.4. Sales of new-to-market and new-to-firm product innovations

Source: own elaboration based on European Union (2019).

Framework conditions compiles information on the main drivers of innovation performance external to the firm, related to human resources, research, and innovation opportunities. Investments includes investments made in both the private and public sectors. Innovation activities encompasses different issues related to the promotion of innovation in the business sector, to the collaboration and dissemination of this type of activity, and to the generation of intellectual assets. Finally, Impacts covers the effect on employment and the sales of firms’ innovation activities.

The EIS calculates the composite indicators (CIs) for each country using a Min-Max normalization method and an aggregation method based on unweighted averages. Besides, there is a reference year, which is usually the first one in the series. The MinMax method calculates re-scaled scores by first subtracting the Minimum score and then dividing by the difference between the Maximum and the Minimum score. The composite indicator is calculated as the unweighted average of the re-scaled scores of all indicators, in which all indicators have the same weight, i.e. 1/27. The sub-indicators used in this study correspond to the period 2011-2018, taking as a reference the value of the EU in 2011. For more details on methodological approaches to the calculation of sub-indicators, see EU (2019) and Hollanders and Janz (2013).

Basically, the spirit of the EIS lies in the cross-country comparison of innovation indicators to identify the strengths in terms of KBE of member countries. In this regard, it is particularly important to pay attention to how sub-indicators are aggregated to calculate the final composite indicator, since, depending on the purpose of the indicator, one method may be more appropriate than another. There is not just one way of designing composite indicators; however, choosing one or another method may yield different results (Greco et al., 2019). In this study, we compare the performance of three MCA-DEA models so as to develop composite indicators on KBE. In order to do so, we took as a reference the 10 sub-indicators proposed by the EIS for the 36 European countries in 2018 (relative to EU 2011). The data concerning the original sub-indicators used in the case study are presented in Table 3.

Table 3. Original data of sub-indicators in 2018 relative to EU 2011, Composite Indicators for each country regarding model 1, model 2, model 3, and Unweighted average.

Countries	HR	ARS	IFE	FS	FI	I	L	IA	EI	SI	CIs Model 1	CIs Model 2	CIs Model 3	CIs Unweighted average
Belgium	129.7	170.1	168.2	118.4	142.6	135.2	163.8	87.2	79.8	103.1	0.9767	0.7178	0.7813	0.6127
Bulgaria	64.5	23.1	85.1	17.0	49.4	24.5	31.3	78.8	113.5	39.0	0.6428	0.2137	0.4047	0.2441
Czechia	91.7	73.6	118.6	51.1	112.6	88.0	87.3	62.1	123.6	95.8	0.7755	0.4783	0.7634	0.4335
Denmark	220.6	207.0	288.1	116.7	124.6	86.9	144.6	159.3	105.1	77.6	1.0000	0.8338	0.6904	0.6857
Germany	108.5	96.8	156.2	109.6	170.2	123.5	138.0	144.6	102.1	123.2	1.0000	0.7774	0.9330	0.6084
Estonia	134.2	106.3	138.9	96.8	108.0	97.7	125.9	124.3	69.3	67.6	0.7584	0.5990	0.5601	0.4978
Ireland	160.7	147.3	154.5	78.7	101.5	119.4	82.1	50.3	173.6	131.5	1.0000	0.6010	1.0000	0.5741
Greece	95.8	66.6	64.6	48.5	78.7	132.4	115.9	35.0	87.9	68.2	0.8650	0.4552	0.5476	0.3874
Spain	141.7	86.5	169.3	82.3	76.3	40.9	60.5	69.3	97.5	87.5	0.7495	0.4850	0.6570	0.4197
France	156.0	128.9	138.2	139.8	97.8	114.9	96.0	83.4	92.4	91.2	0.9472	0.6339	0.6954	0.5406
Croatia	61.0	38.0	65.3	33.1	111.6	86.6	65.3	29.2	67.5	36.3	0.5949	0.2862	0.3982	0.2849
Italy	64.1	101.5	106.9	57.8	84.8	118.5	49.6	97.9	76.5	85.0	0.8054	0.4463	0.6144	0.4062
Cyprus	118.3	123.4	113.7	27.0	84.7	74.8	50.8	102.0	75.0	104.3	0.8364	0.4273	0.7117	0.4094
Latvia	77.1	46.1	143.7	106.5	55.3	36.1	49.8	52.0	98.6	55.5	0.7699	0.3472	0.4730	0.3375
Lithuania	115.6	42.0	191.3	56.2	91.3	100.3	111.0	49.9	44.4	56.6	0.6878	0.4032	0.4518	0.3925
Luxembourg	155.8	217.0	212.7	127.7	77.9	127.5	70.5	153.3	140.5	83.6	1.0000	0.6965	0.7044	0.6340
Hungary	53.7	55.9	144.7	46.2	98.0	30.9	57.1	40.1	124.2	84.1	0.7571	0.3374	0.6858	0.3462
Malta	78.2	60.2	217.0	5.2	100.0	53.8	16.6	169.7	157.7	77.2	1.0000	0.1838	0.7056	0.4218
Netherlands	173.7	191.4	263.4	129.4	84.9	114.1	149.1	120.8	118.8	95.5	0.9977	0.7653	0.7426	0.6614
Austria	142.6	147.5	124.0	92.3	138.6	136.2	172.1	141.8	67.9	85.6	1.0000	0.6870	0.6796	0.5874
Poland	70.4	34.6	197.9	39.1	87.3	15.0	32.4	67.4	96.5	56.1	0.7127	0.2635	0.5066	0.3113
Portugal	98.1	116.2	205.1	82.9	97.0	155.6	57.7	72.7	81.7	56.2	0.9684	0.5218	0.5051	0.4773
Romania	16.7	27.2	121.6	29.4	10.9	0.0	40.8	22.3	48.4	63.5	0.5305	0.1498	0.3832	0.1754
Slovenia	126.0	88.7	140.3	31.2	126.4	62.1	104.5	79.2	85.3	68.9	0.6516	0.4612	0.6049	0.4182
Slovakia	86.1	46.7	90.9	26.1	79.7	37.9	60.1	38.7	113.3	114.5	0.8707	0.3168	0.7993	0.3356
Finland	192.0	152.5	288.1	124.2	154.7	152.7	157.9	147.6	83.7	88.0	1.0000	0.8333	0.7397	0.7051
Sweden	213.9	187.2	272.4	119.5	148.1	104.8	153.0	151.9	140.5	90.6	1.0000	0.8647	0.8244	0.7201
United Kingdom	180.8	177.3	155.5	112.4	116.0	94.3	132.3	79.4	153.2	117.4	1.0000	0.7188	0.9259	0.6213
Iceland	154.7	174.5	288.1	114.6	119.9	116.1	172.4	60.1	146.2	32.5	1.0000	0.6180	0.5160	0.6199
Israel	158.7	114.4	119.9	78.1	217.1	76.7	129.3	101.8	184.9	101.6	1.0000	0.6693	1.0000	0.6034
North Macedonia	40.6	60.2	55.6	15.3	74.0	48.1	26.0	15.8	6.5	51.6	0.4198	0.1498	0.3431	0.1881
Norway	174.8	157.2	227.4	127.0	137.0	163.2	163.6	56.4	82.5	53.2	1.0000	0.7284	1.0000	0.6193
Serbia	66.8	35.2	63.0	40.1	95.0	87.4	65.6	23.8	39.8	69.3	0.6056	0.2802	0.4999	0.2876
Switzerland	239.0	234.2	232.4	147.6	208.7	142.8	164.7	168.6	117.2	119.2	1.0000	1.0000	1.0000	0.8207
Turkey	43.7	30.5	123.6	45.7	110.6	136.2	43.2	8.3	10.8	56.9	0.8556	0.1716	0.4163	0.2959
Ukraine	100.8	15.0	6.0	7.6	52.9	15.6	3.0	13.1	77.4	34.7	0.5070	0.0617	0.3431	0.1531

Source: European Commission (2019) and the authors.

4.2. Methodology

The construction of CIs requires a multidimensional approach, which allows for the addition of different criteria in a structured manner. In this regard, Multi-Criteria Analysis (MCA) techniques are particularly appropriate (El Gibari et al., 2019). In this paper, we present three models that have been developed using a “Benefit of the Doubt” approach from Multi-Criteria Analysis to build composite indicators on Knowledge Economy. Model 1 is a basic BoD model. Model 2 is a BoD model with proportional sub-indicator share restrictions. Model 3 is a common weights DEA model using a Goal Programming (GP) technique.

The BoD approach has its origin in the Data Envelopment Analysis (DEA) in an environment of inaccurate information. The DEA methodology is a linear programming technique that evaluates a set of homogeneous productive units using both input and output variables in an uncertain environment, in which the weights associated with these variables are not known, nor is the form of the function that relates these variables. The very name “benefit of the doubt” captures the essence of this type of model and emphasizes its ability to obtain the most appropriate weighting scheme for each decision unit performance benchmarking from the data available for each unit of analysis (Cherchye et al., 2007). A great number of works have already successfully used this approach for the construction of composite indicators, such as those of Cherchye et al. (2011), Mizobuchi (2014), Karagiannis (2017), or Verbunt and Rogge (2018). The only difference with respect to traditional DEA models is that only the output variables are fixed, considering just one dummy input variable whose value equals 1 for each unit of analysis. The BoD model compares the performances of one country with the others, assigning higher weights to the first indicators and lower to the seconds. In doing so, the model chooses the most favorable set of weights for each unit of analysis.

Model 1 is a basic BoD model and its objective function is formulated as shown in equation (1)(1) $$\mathrm{CIc}=\underset{w_{c,i}}{\max }\sum _{i=1}^m{w}_{c,i}{I}_{c,i}$$(1) : (1) $$\mathrm{CIc}=\underset{w_{c,i}}{\max }\sum _{i=1}^m{w}_{c,i}{I}_{c,i}$$(1) s.t. (1a) $$\sum _{i=1}^m{w}_{c,i}{I}_{j,i}\le 1\mathrm{ }$$(1a)

(n constraints, one for each country j) (1b) $$w_{c,i}\ge 0$$(1b)

(m constraints, one for each indicator i)

Where $j=1,2,\dots .,n$ and i = 1,2,…….,m, w_c,i represents the weight of country c in relation to sub-indicator i. I_c,i represents el sub-indicator i associated with country c. And I_j,i represents sub-indicator i for any country j.

Model 2 is a variation of model 1 but which includes proportional sub-indicator share restrictions. Such restrictions were introduced by Wong and Beasley (1990) to make it easier for experts to quantify their opinion in terms of percentage values.

The objective function of model 2 is as follows: (2) $$\mathrm{CIc}=\underset{w_{c,i}}{\max }\sum _{i=1}^m{w}_{c,i}{I}_{c,i}$$(2) s.t. (2a) $$\sum _{i=1}^m{w}_{c,i}{I}_{j,i}\le 1\mathrm{ }$$(2a) (2b) $${\propto }_i\le \frac{w_i{I}_{j,i}}{\sum _{i=1}^m{w}_i{I}_{j,i}}\le {\beta }_i\mathrm{ }$$(2b) (2c) $$w_{c,i}\ge 0$$(2c)

Where $j=1,2,\dots .,n$ and i = 1,2,…….,m, w_c,j represent the weight of country c in relation to sub-indicator i. I_c,i represents sub-indicator i associated with country c. I_j,i represents sub-indicator i for each country j, and ${\propto }_i$ and ${\beta }_i$ represent the respective lower and upper limits of the contribution of sub-indicator i for each country j with respect to the total of sub-indicators.

Model 3 uses a different approach to determine the CIs for each country, the MinMax Goal Programming (GP) approach. GP is based on a satisfying or non-optimizing rationale, where the decision-making authority tries to bring a number of relevant goals as close as possible to pre-set levels of aspiration (Jones & Romero, 2019).

This model seeks to minimize the maximum deviation, in terms of efficiency, among the possible deviations of all the analysis units.

This model is based on Hatefi and Torabi (2010) and is formulated according to equation (3): (3) $$\text{Min D}$$(3) s.t. (3a) $$\mathrm{D}-\mathrm{dj}\ge 0\mathrm{ }$$(3a) (3b) $$\sum _{i=1}^m{w}_i{I}_{ji}+d_j=1,\mathrm{ }$$(3b) (3c) $$w_i\ge 0,\mathrm{ }$$(3c) (3d) $$d_j\ge 0,$$(3d)

Where $j=1,2,\dots .,n$ and i = 1,2,…….,m, w_i represents the common weight assigned to sub-indicator i for each country. I_ji represents sub-indicator i associated with country j. d_j represents deviation for country j, and D represents the largest possible deviation.

The composite indicator calculated for each country is determined by the difference 1-$d_j.$

5. Results and discussion

The results of the CIs obtained from the application of each of the three models are shown in Table 3. In addition, we provide the results of the CIs using the unweighted average as an aggregation method, which is the aggregation method proposed by the EIS. The main differences between the three models are determined by 1) the weights assigned to the sub-indicators, 2) the simplicity of use, 3) the discriminatory power and 4) the participatory nature.

Figure 1 shows a comparison between the values of the CIs obtained from the application of the three models to each country. As can be seen, model 1 yields higher values for all countries than the rest of the models since it assigns weights to each of the sub-indicators, which provide an optimal overall score in the case of each country. This is a fairly adaptable model when selecting optimal weights, as it only has to meet two restrictions: restriction (1a), which standardizes the weights, and restriction (1 b), which indicates that the CI is an increasing function of the sub-indicators, so an improvement in one of the sub-indicators will always result in an improvement in the final CI score. Applying model 1 is appropriate in those cases where there is no agreement on the weights assigned to each indicator or sub-indicator. This model assigns the best scores (CI = 1,0000) to Denmark, Germany, Ireland, Luxembourg, Malta, Netherlands, Austria, Finland, Sweden, United Kingdom, Israel, Norway, and Switzerland. The main advantage of this method is that it is generally accepted by the parties being assessed (Bas Cerdá, 2014).

Figure 1. Comparative Composite Indicators for each country regarding model 1, model 2 and model 3. Source: The authors.

When constructing the CI, model 2 provides higher values to countries that excel at everything. The main advantage of this model is that it takes into account some constraints on the relative contributions of the dimensions analyzed, according to expert opinions. Notice how some countries that had reached the highest score according to model 1, have obtained a much lower score according to model 2. For example, Malta obtained a CI = 1,0000 according to model 1, but according to model 2, it obtained a CI = 0,1838. This huge difference in the overall value of the indicator is due to the fact that, although there are some sub-indicators with high values, such as IFE (217.0) or IA (169.7), there are other sub-indicators with very low values, such as FS (5.2). These “bad” sub-indicators greatly penalize the value of the CI. Something similar happens with Ireland, Luxembourg, Austria, or Bulgaria.

Model 3 yields a more balanced result, although it is overall lower. This model gives priority to obtaining the “least bad” result and, therefore, the overall value is much more balanced for all the countries, i.e. the score is split between all of them. Thus, it can be seen that some countries with low sub-indicators have obtained a higher relative score according to this model, even above model 1 and, of course, model 2. Clear examples of this are Czechia, Slovenia, or Slovakia.

With regard to simplicity of use, models 1 and 2 required replication of the model for each decision unit, in this case, for each country. As a result, each model had to be replicated 36 times. One of the main limitations of these two models is that they allow for null values to be assigned to the weights associated with some sub-indicators in the calculation of the CI. This may be an undesirable circumstance for the analyst when, for example, one wishes to include every sub-indicator in the CI. On the other hand, model 2 can sometimes provide inconsistent results, derived from a lack of compliance with restriction (2 b). When this occurs, it is necessary to use some kind of approximation. In this case study, the limits used to delimit the contribution of each indicator with respect to the CI were ${\propto }_i=$0.1 and ${\beta }_i=$0.7. When dimensions such as these are imposed, inconsistencies can sometimes arise. For example, it may well be that a sub-indicator has a value of zero or a significantly lower value than the rest of the indicators for a country. In such a case, it is necessary to incorporate an additional restriction that assigns the value zero to the weight associated with that specific indicator. In our case study, this approximation had to be applied to the ARS indicators in Germany, IA in Greece, I in Romania, and SI in Norway. Although the range of acceptance is quite broad, the results of the CIs have shown significant differences. Although model 3 uses a linear programming technique like the other two models, it only requires a replica, as it uses common weights for all countries, so its application is much simpler. This model has provided similar results in terms of performance as other models based on the DEA approach and in other contexts (Hatefi & Torabi, 2010).

With respect to discriminating power, the worst model was model 1, which obtained 13 ties. As such, it may be difficult to prioritize countries on the basis of these criteria and, therefore, to develop rankings. Model 3 obtained 4 ties for the best positions. Model 2 has been the most discriminating and has not resulted in any tie. It is also to be expected that the narrower the interval between ${\propto }_i\text{ y }{\beta }_i,\mathrm{ }$ the greater the discriminating power of the model.

With regard to the possibility of incorporating experts’ opinions or defining, usually minimal, thresholds for the different sub-indicators, the only valid model is model 2. This model is the most appropriate for constructing CIs with a minimum degree of quality, which can clearly point at the countries that excel at everything. Additionally, it is also advisable when experts’ opinions must be taken into consideration.

The unweighted average aggregation method is the easiest to apply, as it only uses a sum weighted by 1/$n_i,$ where $n_i$ is the number of countries considered in the analysis for obtaining the CI. However, it is the least accurate method, as it does not identify optimal sub-indicators, but rather distributes the values of all sub-indicators equally regardless of their efficiency. With respect to discriminating power, this aggregation method did not obtain any tie, so it is adequate if a ranking is desired, regardless of the efficiency of the sub-indicators with respect to the CI. Although this system of aggregation per se does not consider participation, merely by transforming it into a system of weighted averages, it would already allow for the incorporation of weights specifically defined by decision-makers.

In order to choose the most appropriate model, the analyst must take into account all these scenarios and must consider that different aggregation methods will yield different results (De Castro-Pardo et al., 2020).

An analysis of the ranking of countries with respect to the results of the CIs (Table 4) shows that, although the best position (highlighted in bold) and the worst positions (highlighted in italics) are the same for all models, the middle positions of the ranking vary. The most important deviations have been found in the CIs provided by model 3. This is so because the aim of this model is not to find an optimal one, but to find solutions that are as good as possible. For this reason, the results are more balanced and, therefore, the positions in the ranking vary to a larger extent. Denmark, Luxembourg, Austria or Norway are examples of countries that have obtained very good positions with model 1, while with model 3, their position in the ranking has significantly dropped. At the same time, model 2 has placed the countries with the best scoring sub-indicators in the top positions, i.e. the most efficient in everything. This is the case of Switzerland, Sweden, and Denmark. The ranking put forward by model 1 is the closest to that given by the unweighted average aggregation method. Both the average and standard deviation of the distances between the two rankings are lower than in the other two cases. On the other hand, model 2 shows the most significant differences with respect to the ranking provided by the unweighted average. This is due to the fact that model 2 gives more importance to the countries that scored higher on all sub-indicators, rather than distributing the weights equally.

Table 4. Ranking of countries regarding model 1, model 2, model 3 and unweighted average.

Countries	Rank Model 1	Rank Model 2	Rank Model 3	Rank Unweighted average
Belgium	15	9	8	10
Bulgaria	31	31	32	33
Czechia	23	18	9	18
Denmark	1	3	16	4
Germany	1	5	4	11
Estonia	25	16	22	16
Ireland	1	15	1	14
Greece	19	21	24	25
Spain	27	19	19	20
France	17	13	15	15
Croatia	33	28	33	32
Italy	22	22	20	23
Cyprus	21	23	12	22
Latvia	24	25	29	27
Lithuania	29	24	30	24
Luxembourg	1	10	14	6
Hungary	26	26	17	26
Malta	1	32	13	19
Netherlands	14	6	10	5
Austria	1	11	18	13
Poland	28	30	26	29
Portugal	16	17	27	17
Romania	34	35	34	35
Slovenia	30	20	21	21
Slovakia	18	27	7	28
Finland	1	4	11	3
Sweden	1	2	6	2
United Kingdom	1	8	5	7
Iceland	1	14	25	8
Israel	1	12	1	12
North Macedonia	36	34	35	34
Norway	1	7	23	9
Serbia	32	29	28	31
Switzerland	1	1	1	1
Turkey	20	33	31	30
Ukraine	35	36	36	36

Source: The authors.

The Spearman’s Rank correlation test was carried out to validate the proposed models. At the same time, the CIs provided by the three models were compared with the model used by the EIS. The test indicates whether or not there is a positive correlation between the two sets of indicators calculated by using each one of the three models proposed by the EIS’ CIs. So the null and alternative hypotheses were put forward as $H_0:\mathrm{ }{r}_s=0$ and $H_1:\mathrm{ }{r}_s>0,$ so, for model 1:

H₀: There is no correlation between the CIs obtained by model 1 and the CIs obtained by EIS.

H₁: There is a positive correlation between the CIs obtained by model 1 and the CIs obtained by EIS.

Equally, we put forward both the null and the alternative hypotheses to validate models 2 and 3 by comparing them with the EIS’ CIs.

The Spearman’s Rank correlation coefficient scored 0.869 for model 1, 0.957 for model 2, and 0.733 for model 3. In addition, the p-value of the test was 0 for all three models, so H₀ is rejected with a significance of 0.01. On the other hand, there is a high positive correlation between the CIs related to the three models proposed here and the EIS’ CIs. These results show the robustness of the models and their appropriateness for building Knowledge Economy Composite Indicators.

6. Conclusions

Composite indicators are quite useful for evaluating phenomena, as is Knowledge-Based Economy, that cannot be encapsulated in a single simple indicator. However, the way in which indicators and sub-indicators are aggregated to calculate a CI must be considered in terms of the analyst’s objectives, the purpose of the CI, and the context and resources available. The BoD approach can be very useful for building CIs, as it allows for an adaptive and optimal weighting of indicators in a multidimensional environment.

The three models proposed are robust and allow for the construction of valid CIs; however, they can bring about changes in the intermediate positions of the ranking. In order to select the most appropriate model for each situation, the analyst must decide which characteristics he or she prefers to prioritize for the calculation of the global assessment. The basic BoD model is generally accepted by all those being evaluated. For this reason, it may be recommended in contexts of weak consensus or conflict, since the scores of all the countries are usually higher and the best positions of the ranking are more equally distributed. The BoD model with proportional sub-indicator share restrictions is recommended for more demanding evaluations, which require better-defined rankings, in which it is possible to pinpoint the countries that excel at everything. In addition, it facilitates a participatory approach, which allows for the incorporation of stakeholders’ opinions.

The GP model provides balanced results and may be recommended when indicators favoring weaker countries are desired. In addition, its application is very simple, which makes it suitable when resources are limited. In the case study presented, all models provided the best position in terms of innovation to Switzerland. Whereas Romania, North Macedonia, and Ukraine held the worst positions in all cases. It would be interesting to apply these methodologies to different groups of countries, in order to see how they perform in different areas. Such would be the case of comparing countries with very different levels of development or, on the contrary, comparing countries with very homogeneous characteristics. This task, in addition to the above, could be taken as a basis for developing research aimed at monitoring KBE when applying different public policies or business undertakings. In the authors’ opinion, this would be the natural evolution of the research proposed here. As a public policy recommendation, we recommend that alternative systems for measuring KBE, such as those proposed, be taken into consideration. In so doing, it will be possible to compare the results with more traditional ones, allowing for a wider range of information to support decision-making.

Disclosure statement

No potential conflict of interest was reported by the author(s).