The Productivity of Economics Research in Slovakia

Abstract

The research productivity of economics departments and tenure-track staff (associate and full professors) in Slovakia is analyzed based on journal publications and citations extracted from the SSCI database. Results suggest that the current university system in Slovakia is ineffective in incentivizing associate and full professors to publish in peer-reviewed journals. The majority of tenure-track staff has done little or no research over the past twenty-six years. The research criteria imposed by the Accreditation Commission for awarding professors’ titles are not fully enforced. Economics faculties have low-quality standards in hiring tenure-track staff. Finally, results suggest that the majority of tenure-track staff (93%) do not have sufficient research skills to supervise Ph.D. students.

Keywords:

• associate/full professors
• citations
• economics departments
• journal publications
• rankings
• research productivity

JEL Classification:

• A11
• P2

This article analyzes the productivity of economics departments and associate and full professors in Slovakia on the basis of published research. A great deal of literature has been published on the ranking of economics departments and faculty staff. However, most articles focus on Western countries (Conroy and Dusansky 1995; Davis and Papanek 1984; Dolado, Garcia-Romero, and Zamarro 2003; Dusansky and Vernon 1998; Fusfeld 1956; Laband 1986; Lubrano et al. 2003; Mixon and Upadhyaya 2001; Scott and Mitias 1996; Sinha and Macri 2002). Studies investigating research productivity in economics in transition countries are relatively scarce (Ciaian, Pokrivcak, and Drabik 2008; Ciaian, Pokrivcak, and Rajcaniova 2005; Machacek and Kolcunova 2005; Turnovec 2005). This article aims to contribute to this relatively scarce literature by addressing the situation of economics research in a transition country¹.

Analyzing the research productivity of a department or its staff members is not a straightforward task. Staff members of any economics department or any university in general are engaged in multiple activities that may or may not lead to tangible output, such as teaching, student supervision, publication in various outlets (e.g., books, journals), journal article refereeing, journal editorial work, and department management (Harris 1988, 1990). The journal publications and their citations, however, are the most visible and most commonly used variables to measure the performance of researchers and economics departments. The first reason is that only peer-reviewed journal publications undergo strict quality control and are therefore the most valued research outputs. The second reason is that the quality and quantity of journal publications are relatively easily measurable because of the availability of databases and indicators (e.g., EconLit, Social Science Citation Index [SSCI]) that provide detailed information on published articles, citations, and journal quality. The measurement of other outputs, such as journal article refereeing, department management, and nonjournal publications (e.g., books, conference proceedings, monographs), suffers from the unavailability of adequate and consistent indicators comparable across individuals and departments (Lubrano et al. 2003; Neary, Mirrlees, and Tirole 2003). In Slovakia, the analysis of journal publications and their citations is relevant because the criteria imposed by the Accreditation Commission for awarding associate and full professor positions require publication in peer-reviewed journals (AK 2013a; Law No. 131 2002). Thus, the analysis in this article provides some evidence on the effectiveness of enforcement of these criteria and their strictness. Furthermore, the economics departments covered in this article have Ph.D. programs in economics; hence, this analysis indirectly evaluates the quality of doctoral programs in Slovakia as well.

Considerable heterogeneity exists in methodologies applied in the literature to quantify research productivity. However, it is widely accepted that the key indicators that determine research productivity in economics are journal publications and citations (Lubrano et al. 2003; So 1998). Less agreement or strong heterogeneity among studies is found on the specific calculation of productivity (e.g., how to account for the number of coauthors or the length of the article), which journals to count, and how to evaluate them. For example, to produce rankings of economics departments and economists, some studies consider only a restricted set of core economics journals. Dusansky and Vernon (1998), Graves, Marchand, and Thompson (1982), Kalaitzidakis, Mamuneas, and Stengos (2003), McPherson (2012), and Scott and Mitias (1996) take into account the top eight, twenty-four, thirty, thirty-six, and fifty economics journals, respectively. In contrast, Lubrano et al. (2003) include all economics journals in their analysis.

The most common indicator used in the literature to measure journal quality is the impact factor (IF) from Thomson Reuters Journal Citation Reports. Recently, alternative metrics for measuring journal quality were proposed, such as an eigenfactor score and article influence score (AIS). Other journal quality indicators include the self-citation threshold approval rating (STAR), h-index, and immediacy impact factor. However, most of the journal quality metrics use citations as a basis for measuring journal quality.² The journal quality metrics differ in terms of the factors they control or that are generally thought to distort citation impact, such as journal length (e.g., number of published articles), the quality of the citing journal, and self-citations (Bergstrom 2007; Bergstrom, West, and Wiseman 2008; Chang, Maasoumi, and McAleer 2016; Engemann and Howard 2009; Kalaitzidakis, Mamuneas, and Stengos 2003; Kodrzycki and Yu 2012; Liebowitz and Palmer 1984; Marchant 2009; Münich and Skoda 2016; Palacios-Huerta and Volij 2004).

Research productivity can be measured by using either a stock or a flow approach (Hogan 1984). In the stock approach, the quality of a department depends on the productivity of the current staff. This approach also takes into consideration publications by the staff at previous employers (Baumann, Werden, and Williams 1987; Ciaian, Pokrivcak, and Rajcaniova 2005; Conroy and Dusansky 1995; Mixon and Upadhyaya 2001; Sinha and Macri 2002). The flow approach counts publications of staff members affiliated with the evaluated institution at the time of publication, irrespective of their current affiliation. It also takes into account publications by staff who are not current employees of the department but were employed in the department at the time of publication (Dolado, Garcia-Romero, and Zamarro 2003; Kalaitzidakis, Mamuneas, and Stengos 2003).

In this article, the research productivity of economics departments and tenure-track staff (associate and full professors) in Slovakia is evaluated based on their journal publications and citations in the SSCI database for the period 1990–2016. In order to provide a comprehensive analysis of research productivity, all articles and their citations published in economics journals that have an impact factor (i.e., economics journals in the SSCI Journal Citation Reports) were evaluated. This allows us to weigh the distribution of the publications of departments and tenure-track staff by the quality of publications. An article is used as the unit for measuring the research productivity of a department or a tenure-track staff.³ Three indicators for measuring journal quality—IF, AIS, and the normalized eigenfactor (NE)—were applied to account for factors identified in the literature as relevant to journal quality (e.g., quality of citing journal, journal self-citation) and to check the robustness of our analysis. Finally, the stock approach was employed to quantify the research output of departments and staff members, because it allows analysis of the distribution of productivity among both departments and current tenure-track staff, including those with zero publication output.

The rest of the article is organized as follows. The following section presents the methodology employed in the article. The third section compares the development of economics publications in Slovakia and neighboring transition countries. The fourth section describes the publication productivity of economics departments and tenure-track staff in Slovakia. Finally, conclusions are offered.

METHODOLOGY

Publication productivity in economics was evaluated at the faculty (institute) level,⁴ the department level, and for tenure-track staff. In all, we consider fourteen economics faculties and two economics research institutes the Slovak Academy of Sciences (SAV).⁵ The fourteen faculties are divided into seventy-nine departments (see Appendix, Table 1A). The number of tenure-track staff totals 409. The main reason for including the SAV institutes in the analysis is that, according to earlier studies, they were dominant economics research institutions in Slovakia in the past, in particular during the communist period (Ciaian, Pokrivcak, and Drabik 2008; Ciaian, Pokrivcak, and Rajcaniova 2005). The inclusion of SAV in the rankings thus allows examining whether its dominance in research persists in the present. Note that economics departments also exist within noneconomics faculties or universities. For example, the Technical University of Košice (TUKE) has a Faculty of Economics. Only economics faculty departments will be taken into account. All faculties considered in this article are doctorate-granting institutions.

Following Ciaian, Pokrivcak, and Rajcaniova (2005), Dusansky and Vernon (1998), and Garcia-Castrillo, Montanes, and Sanz-Gracia (2002), the stock approach was applied to quantify the publication output of departments and staff members. This approach has the advantage of facilitating analysis of the productivity of both departments and the current staff employed, the distribution of productivity across departments and staff, and the prevalence of unproductive departments and staff. The flow approach usually considers only staff who publish; those who have not published are not observed, because no records in databases are tracking them. Thus, the share of unproductive staff and departments cannot be fully measured.⁶

The stock approach quantification of the publication output in this article is based on the publication and citation count of the tenure-track staff employed in the economics departments and research institutions in Slovakia in the academic year 2015/2016.

Staff employed part time are not taken into account because no reliable data are available. In addition, we consider only the staff in tenure-track positions: associate professors and full professors. Junior staff members (e.g., Ph.D. students) are not considered, because by the design of the Slovak university system, tenure-track staff represent the key human capital of the departments, guaranteeing the sustainability and quality of teaching and research. Associate professors and full professors are responsible for designing the content of teaching courses, for supervision of Ph.D. students, guaranteeing the quality of study (bachelor’s, master’s, and Ph.D.), and research programs, and are members of the scientific board at institutes (AK 2013b; Law No. 131 2002). Thus, high publication activity among tenure-track staff is expected to exhibit a synergic effect on teaching potential and the potential to guide and supervise junior staff (e.g., Ph.D. students) in producing high-quality research (García-Gallego et al. 2015).

The source of information on publication activity used in this article is the SSCI database, covering publication activity for all tenure-track staff for the period 1990–2016.⁷ Following the literature, the only articles considered are published in economics IF journals, which implies that they were subject to the usual peer-review process. Conference articles, working papers, books, book chapters, letters to the editor, symposia, book reviews, and similar types of articles are excluded.⁸ Conference articles and working papers are omitted, because they are most often submitted to a journal for publication (Neri and Rodgers 2006). Books and book chapters are excluded, because they usually undergo little peer review or include a collection of articles previously published in journals (Hartley, Monks, and Robinson 2001). As in other studies, noneconomics journals are not counted, because the goal is to evaluate the productivity of economics departments and tenure-track staff (Graves, Marchand, and Thompson 1982; Kalaitzidakis, Mamuneas, and Stengos 2003; McPherson 2012).⁹ However, the statistics on publication activity of tenure-track staff in the results section also report those for noneconomics journals.

Journal Quality Indicators

Three indicators were utilized for measuring journal quality—IF, AIS, and NE—available from the Thomson Reuters Journal Citation Report (JCR). All three indicators are average values for the three-year period 2013–2015.¹⁰

Impact factor is the most popular indicator used to measure journal quality. It is calculated as a ratio of total citations of articles published in the journal in the two previous years and the total number of articles published in the journal in the previous two years. Some of the advantages of IF are that it is based on objective citation data; it is a well-established metric for journals’ scientific prestige; and its calculation is easily understood (Nisonger 2004). This indicator is relevant to consider in our analysis because under the current Slovak university system, it is used as a quality indicator of the publication productivity of candidates for tenure-track positions (AK 2013a; Law No. 131 2002).

The IF is criticized on several grounds. It does not control for journal and author self-citations, which are commonly viewed as less valuable than citations from other journals and other authors. It does not take into account the quality of citing journals; and it gives the same weight to all-citing journals, including low-quality journals. Different scientific fields, including subfields within economics, have different citations patterns; hence, its value is not fully comparable among fields. The IF is susceptible to cronyism and ceremonial citations, when friends/colleagues cite one another to increase their mutual citation count and when authors cite authority in the field to demonstrate knowledge of the research, respectively. The two-year period used for IF calculation is arbitrary and relatively short, which may lead to significant year-to-year fluctuation of its value (Lubrano et al. 2003; Nisonger 2004)

Recently, the AIS and the NE have been emphasized in the literature as more accurate measures of journal quality, as they address some of the weaknesses of IF (Chang, Maasoumi, and McAleer 2016; Münich and Skoda 2016). Chang, Maasoumi, and McAleer (2016) show that the IF of economics journals can lead to a distorted evaluation of journal quality compared to indicators that take into consideration the frequency of citations and the quality of citing journals, such as NE or AIS. Further, Münich and Skoda (2016) show that this is particularly problematic for Czech economics journals, which are important publication outlets for Slovak researchers. According to Münich and Skoda (2016), some of the Czech economics journals have relatively high IF but extremely low AIS, indicating that they either have a high self-citation rate or are predominantly cited in low-quality journals.

For these reasons, AIS and NE are also used as journal quality metrics in our quantification of publication productivity for economics departments and tenure-track staff in Slovakia. The NE is calculated from the eigenfactor score. The eigenfactor score measures a journal’s influence/importance to the scientific community. The eigenfactor score is calculated based on the number of journal citations in the past five years, taking into consideration the quality of citing journals. Citations in more prestigious journals (i.e., highly cited journals) receive greater weight than those in low-quality journals (i.e., seldom cited journals). In contrast to IF, the eigenfactor does not take into consideration journal self-citations (Bergstrom 2007; Bergstrom, West, and Wiseman 2008). The NE rescales the total number of journals in the JCR such that the average journal has a score of 1. For example, a journal with an NE value of 3 is three times as influential as an average journal (“Eigenfactor Metrics” 2012; “ InCites” 2015).

The eigenfactor score considers total citations of the journals. For example, journal A, which publishes 100 articles a year, has an eigenfactor score twice as high as journal B, which publishes fifty articles a year, provided that each article in both journals has the same number of citations. The AIS adjusts the eigenfactor score by the total number of published articles in the journal. To obtain the AIS, a journal’s eigenfactor score is divided by the number of articles in the journal, normalized so that the average article in JCR has an AIS of 1. An AIS value greater than 1 indicates that each article in the journal has above-average influence, while a value less than 1 indicates that each article in the journal has below-average influence (“Eigenfactor Metrics” 2012; “InCites” 2015).

Productivity Calculation for Tenure-Track Staff and Departments

Quantifying the publication productivity of tenure-track staff and departments was based on two equally weighted indicators: (1) the count of articles published in economics IF journals and (2) the count of citations of articles published in economics IF journals. Citations were also considered because the approach that takes into consideration only the number of published articles is often criticized on the grounds that low-quality articles can be found even in high-quality journals (Coupé 2003).

Publication measures are weighted by the journal quality (i.e., IF, NE, or AIS) and the number of authors per article (coauthorships). Journal quality and coauthorship are commonly used in rankings in the literature as proxies to control for article quality and to allocate the merit of the publication among the authors. It is also important to account for the number of authors in order to avoid double counting of publications if coauthors are affiliated with the same department (e.g., Coupé 2003; McPherson 2012).

The publication output of tenure-track staff is calculated as the sum of articles published in IF journals and their citations adjusted by journal quality (IF, NE, or AIS) and the number of coauthors.

In line with the literature, total and per capita measures were used to calculate the department publication output (Combes and Linnemer 2003; Dusansky and Vernon 1998; Neary, Mirrlees, and Tirole 2003; Pomfret and Wang 2003). According to Pomfret and Wang (2003), the total research output captures overall strength or the reputation, whereas the per capita research output measures productivity of a given department. The advantage of using a combined indicator is that it weights both aspects of the research output measurement. As a result, for each department we consider the total number of IF journal articles and their citations produced by all tenure-track staff employed in the department and the average number of IF journal articles and citations per tenure-track staff member. Analogous to the publication output of the tenure-track staff, each IF journal article and citation is adjusted by journal quality and the number of coauthors to obtain the departments’ publication score. The total publication score of a department is then calculated as the arithmetic mean of the total and per capita published number of IF journal articles and the citations.¹¹

Overall, a higher number and quality of published articles and citations lead to a larger publication output of a staff member and places her/him in a higher position in the staff ranking. The departments, which employ staff who are active in publishing in good journals and have many cited articles attain higher output and obtain higher placement in the departmental ranking.

Six rankings for departments and tenure-track staff were generated. Three rankings were calculated for each of the three journal quality indicators (i.e., for IF, NE, and AIS). This allows comparing the robustness of the rankings of economics departments and staff when different journal quality metrics are applied. For each of the three journal quality indicators, publication productivity was calculated with all publications and their citations included and with excluded articles and their citations published in Czech and Slovak journals.¹² The purpose of calculating the second publication productivity with excluded articles published in Czech and Slovak journals is to analyze the international competitiveness of economics departments and tenure-track staff in Slovakia as well as to control in part for location bias, such as knowledge of the Slovak or Czech language or Slovak academics’ membership on editorial boards in Slovak and Czech journals. Publication in Slovak and Czech journals may indicate the strong competitiveness of a department or a staff member in research on domestic and regional issues. Pomfret and Wang (2003) argue that the competitiveness effect cannot be separated from the location bias and propose excluding domestic journals (i.e., Australian journals in the case of Pomfret and Wang 2003) from the analysis.

THE DEVELOPMENT OF ECONOMICS PUBLICATIONS: AN INTERNATIONAL COMPARISON

The economics articles published in IF journals showed significant growth in Slovakia over the past two decades. However, they started from a relatively low level, as observed in the early transition period (early 1990s). The total number of published economics articles increased by more than threefold in Slovakia over the period 1993–2016, from twenty articles in 1993 to ninety-one articles in 2016. Compared to neighboring transition countries, Slovakia shows the smallest growth. A significant increase is observed in Poland followed by the Czech Republic and Hungary. For example, in Poland the number of published articles increased by more than thirteen-fold (to 324 articles) over the same period, from around the same level in 1993 as in Slovakia (i.e., twenty-two articles). In contrast, in Hungary the number of published economics articles increased from a lower level in 1993 (fifteen articles) to around the same number of articles in 2016 as in Slovakia (ninety-seven articles). The development of the number of published economics articles in the Czech Republic is between the number in Hungary and Poland (Figure 1).

FIGURE 1 Development of the Number of Published Journal Papers in Economics in Slovakia and Neighboring Transition Countries in the Period 1993–2016.

Source: Web of Science 2017.

Economics researchers in Slovakia publish lower-quality articles (i.e., in journals with low IF) than researchers in neighboring transition countries. The average IF of published articles was 0.62 in Slovakia in the period 1993–1995. The neighboring transition countries had an average IF higher than 0.87 in the same period. The average IF of published articles increased by 41% in Slovakia in the period 2014–2016 compared to the period 1993–1995. This increase in the average IF is significantly greater than in other selected countries. However, the average IF in Slovakia (0.86) remains below the average IF in neighboring transition countries (0.95 or higher) in the period 2014–2016 (Table 1).

TABLE 1 Average Impact Factor of Published Economics Papers in Slovakia and Neighboring Transition Countries

	1993–1995	2014–2016	Index 2014–2016 relative to 1993–1995 (1993–1995 = 100)
Slovakia	0.62	0.86	141
Czech Republic	0.88	1.06	121
Poland	0.87	0.95	109
Hungary	0.91	1.07	117

Source: Web of Science 2017.

As many as 89% of articles were published in low-quality journals (with an IF lower than 40% of the IF of the American Economics Review [AER]) in Slovakia in the period 2014–2016. In the period 1993–1995, 100% of the articles were published in low-quality journals. The share of low-quality journals is higher in Slovakia than in neighboring transition countries. The share of articles published in higher-quality journals (with an IF higher than 80% of the IF of the AER) is relatively small (1.2%) in Slovakia in the period 2014–2016. In the 1993–1995 period, none of the Slovak economics researchers published in higher-quality journals. Again, this share is smaller than that of other selected countries, where more than 2.5% of articles were published in higher-quality journals in the period 2014–2016. For example, 6%, 4%, and 2.5% of economics articles were published in higher-quality journals in Hungary, Poland, and the Czech Republic, respectively, in the period 2014–2016 (Figure 2).

FIGURE 2 The Distribution of the Quality of Economics Papers Published in Slovakia and Neighboring Transition Countries (Impact Factor of AER = 1).

Source: Web of Science 2017. Note: The quality of each publication is measured in relative terms by dividing its Impact Factor (IF) by the IF of the American Economics Review (AER).

In summary, an increase in economics publication activity in Slovakia has occurred over time. However, most economics articles are published in lower-quality journals, and Slovakia lags behind neighboring transition countries in the average quality of published economics articles.

PUBLICATION PRODUCTIVITY OF ECONOMICS DEPARTMENTS AND TENURE-TRACK STAFF IN SLOVAKIA

Statistics on Published Articles

The tenure-track staff employed in economics departments in Slovakia during the academic year 2015/2016 published 1,275 articles in IF journals over the period 1990–2016. Around 70% of these articles were published in economics journals, and the rest (30%) were published in noneconomics journals. Some noneconomics articles by associate and full professors in economics departments were published in fields that are complementary to economics research (e.g., mathematics, statistics). Publications in other noneconomics fields (e.g., Biologia, Studia Psychologica, Mliekarstvo, Biodiversity and Conservation) may indicate that tenure-track staff cooperate with researchers in other fields in interdisciplinary research.

Tenure-track staff publish mainly in Slovak and Czech journals—65% of the articles published in 1990–2016. If we consider only economics journals, 85% of the economics articles appear in Slovak and Czech journals. This publication pattern is similar for both associate professors and full professors. With respect to noneconomics publications, the share of Slovak and Czech journals is significantly smaller: 20% for associate professors and 17% for full professors (Figure 3). These results are comparable to the findings of Ciaian, Pokrivcak, and Drabik (2008), who report that 86% of all journal publications published by staff in economics departments were published in Slovak peer-reviewed journals in Slovakia in the period 1990–2004.

FIGURE 3 Distribution of Published Papers by Type of Journal (%).

Note: Associate professor = doc; full professor = prof.

The most common publication outlet is Ekonomicky Casopis, which accounted for 40% of the published articles by economic tenure-track staff in Slovakia in the period 1990–2016. This journal is produced at the SAV (Institute of Economic Research, Bratislava, Slovakia), and it is the only Slovak IF journal in economics. The other journals each account for less than 7% of the articles published. Of the top eight economics journals in which tenure-track staff published in 1990–2016, five are Slovak or Czech. Full professors publish a larger share of articles (45%) in Ekonomicky Casopis than associate professors do (35%). Among the rest of the top eight economics journals, with exception of Ekonomie a Management (E+M), the difference between full professors and associate professors is relatively small (Figure 4).

FIGURE 4 The Top Eight Most Published Economics Journals (%).

Note: Associate professor = doc; full professor = prof.

Tenure-track staff at the SAV and the University of Economics in Bratislava (EUBA) published as many as 71% and 56%, respectively, of their journal articles in Ekonomicky Casopis. Other universities published between 9% and 32% of their articles in this journal. These rates are lower than those reported by Ciaian, Pokrivcak, and Drabik (2008) for 1990–2004, when SAV and EUBA staff published around 90% of their articles in Ekonomicky Casopis.

The publication of higher-quality articles may require closer cooperation between scientists. When scientists combine skills, they gain from complementarity effects, leading to the production of higher-quality articles, which ultimately may increase the chance of publishing in higher-quality international journals. This seems to hold true here. On average, articles published in international journals have 3.5 authors, compared to 2.1 authors of articles published in Slovak and Czech journals. The average number of coauthors is lower in articles published in economics journals (2.2) than in articles published in noneconomics journals (3.5). However, the absolute difference in the number of coauthors between international and Slovak and Czech journals is larger for economics journals—3.1 versus 2.0 authors, respectively—than for noneconomics journals—3.7 versus 2.9 authors, respectively.

The average value of IF, NE, and AIS for journals in which associate and full professors from Slovak economics departments published in 1990–2016 is 0.72, 0.27, and 0.18, respectively. The low values of NE and AIS indicate that associate and full professors in Slovak economics departments publish in journals that have low influence and below-average quality. The average value of these three indicators for economics publications is significantly lower than that for noneconomics publications. The difference between economics and noneconomics publications is particularly high for NE and AIS (more than 70%) compared with IF (45%) (Figure 5). The quality of economics journal publications between the associate professors and the full professors shows no significant difference if we consider IF. However, NE and AIS indicate that full professors publish in better-quality journals than do associate professors.

FIGURE 5 Average Impact Factor, Article Influence Score and Normalized Eigenfactor of Published Papers by Tenure-Track Staff.

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS.

A significant difference is observed between Slovak and Czech publications and international publications in all three journal quality indicators. Articles published in international journals have greater average values of IF, NE, and AIS than articles published in Slovak and Czech journals. For economics publications, the average IF, NE, and AIS of international journal publications are 0.93, 0.39, and 0.33, compared to 0.53, 0.03, and 0.06 for Slovak and Czech publications, respectively. For noneconomics publications, the gap in the average value of the three indicators is also significant (Figure 6).

FIGURE 6 Comparison of the Average Impact Factor, Article Influence Score, and Normalized Eigenfactor Between Slovak and Czech Publications and International Publications.

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS.

Statistics on Citations of Published Articles

Each article published in 1990–2016 by tenure-track staff employed at economics departments in Slovakia in the academic year 2015/2016 was cited on average around three times (i.e., 2.9). Articles published in noneconomics journals were cited more often (4.0 times) than those published in economics journals (2.5 times). Publications in international journals had significantly more citations than those published in Slovak and Czech journals. The average number of citations of articles in international economics (noneconomics) journals is 5.6 (4.5), compared to 1.9 (1.6) citations for articles published in Slovak and Czech journals.

Articles by associate professors tended to be cited slightly more than articles by full professors. The average number of citations of articles by associate professors in economics (noneconomics) journal is 2.7 (4.6), compared to 2.3 (3.1) citations for articles by full professors.

The citation pattern is heterogeneous across articles. As many as 43% of the articles, most of which are in Slovak and Czech journals, have zero citations. Associate and full professors produced only a few articles (7%, or ninety-five articles) that were cited relatively widely (with ten or more citations). The rest of the articles (50%) were cited between one and nine times.

The correlation analysis estimated by using ordinary least squares (OLS) regression indicates that the following factors affect the citation intensity of articles by tenure-track staff in Slovakia: journal quality, the language of the article, the number of coauthors, the number of pages, the age of the article, and the type of tenure-track position (associate professors versus full professors) (Table 2). As expected, the number of citations per article increases with the IF and AIS value of the journal (see the estimated model 1 [M1] and model 3 [M3] in Table 2). However, NE has a statistically insignificant impact on the number of citations (M2). This result could be explained by the fact the NE counts total citations of journals, not the intensity of citations per published article. Articles in English generate between 0.5 and 0.8 more citations per article than those published in Slovak or Czech, probably because articles in English are accessible to more readers. With respect to the number of coauthors, the number of citations increases by around 0.4 per each additional author. As argued above, this could be because having more coauthors likely leads to higher-quality articles on account of the skill complementarity effect. As expected, older articles are cited more than newer articles. On average, an additional year adds between 0.05 and 0.09 citations per article. Articles published in Slovak and Czech journals have around 1.85 fewer citations than articles published in international journals when IF and NE variables are included in the regression (M1, M2). When AIS is used as the journal quality metric, the Slovak and Czech journal dummy is statistically insignificant (M3). This could be due to the fact that the AIS metric corrects for the weaker citation potential of articles published in Slovak and Czech journals, for which IF is distorted, as argued by Münich and Skoda (2016). Finally, articles by full professors are cited between 0.62 and 0.75 times less than those by associate professors. This result is surprising, given that the awarding of a full professorship (including the requirement for a certain number of citations) requires the satisfaction of more criteria than is the case for an associate professorship; moreover, full professors are expected to be more experienced researchers and thus to produce higher-quality articles with more citations.¹³

TABLE 2 Regression Results Estimating the Determinants of Journal Article Citations (Dependent Variable: Number of Citations per Article)

	M1	M2	M3
Slovak/Czech journal dummy	−1.842***	−1.876***	−0.513
Number of authors	0.357**	0.469***	0.372**
Economics journal dummy	1.411	1.847	1.815*
Article age	0.0894***	0.0660***	0.0534**
Number of pages	0.00456	0.00448	0.00338
English-language dummy	0.849***	0.670**	0.509*
Full professor dummy	−0.687*	−0.755**	−0.624*
IF	2.084***
NE		2.048
AIS			8.777***
Constant	−0.415	0.339	−1.128
Observations	1,273	1,273	1,273
R^2	0.087	0.104	0.121

Notes: OLS with robust standard errors; ***, **, and * statistical significance at the 1%, 5%, and 10% levels, respectively. Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS.

Productivity of Departments and Faculties

Table 3 reports the ranking of faculties for the three journal quality indicators following our methodology, which takes into consideration published economics articles and citations. The ranking is relatively robust across the three indicators for the same set of journals: rankings with all publications versus rankings without counting publications in Slovak and Czech journals. With the exception of the Faculty of Economics and Management (SPU), the maximum change in rank is by two places. The correlation coefficient between rankings for the three journal quality indicators, for the same set of considered journals, is more than 94% (Table 4).

TABLE 3 Ranking of Economics Faculties and Research Institutes in Slovakia by Journal Quality Indicator

			Ranking with all economics journals (No.)			Ranking without Slovak/Czech economics journals (No.)
Faculty/Institute	University	Share of Slovak/Czech publications in total publications	IF	NE	AIS	IF	NE	AIS
Institute for Forecasting	SAV	53	1	2	1	1	2	1
Faculty of National Economy	EUBA	72	2	1	2	2	1	2
Faculty of Economics	TUKE	67	3	4	3	3	5	4
Institute of Economic Research	SAV	97	4	5	5	10	12	12
Faculty of Commerce	EUBA	98	5	6	7	14	14	14
Faculty of Economics	UMB	70	6	7	6	4	4	5
Faculty of Economic Informatics	EUBA	85	7	8	8	6	7	6
Faculty of Economics and Management	SPU	75	8	3	4	5	3	3
Faculty of Business Management	EUBA	92	9	9	9	11	10	10
Faculty of Economics and Entrepreneurship	PEVŠ	56	10	10	10	15	15	15
Faculty of Management	UK	52	11	11	11	9	8	8
Faculty of Management	PU	43	12	13	13	15	15	15
Faculty of Business Economics in Košice	EUBA	35	13	12	12	7	6	7
Faculty of Operation and Economics of Transport and Communications	ŽU	20	14	15	14	13	13	11
College of International Business	ISM	42	15	14	15	12	11	13
Faculty of Economics	UJS	10	16	16	16	8	9	9

Notes: College of International Business (ISM); Comenius University in Bratislava (UK); J. Selye University (UJS); Matej Bel University in Banská Bystrica (UMB); Paneuropean University (PEVŠ); Slovak Academy of Sciences (SAV); Slovak University of Agriculture in Nitra (SPU); Technical University of Košice (TUKE); University of Economics in Bratislava (EUBA); University of Prešov (PU); University of Žilina (ŽU); impact factor (IF); normalized eigenfactor (NE); article influence score (AIS).

TABLE 4 Correlations Between the Rankings of Faculty by Journal Quality Indicator

		Ranking with all economics journals			Ranking without Slovak/Czech journals
		IF	NE	AIS	IF	NE	AIS
Ranking with all publications	IF	100	95	96	58	49	53
	NE		100	99	64	61	62
	AIS			100	67	62	65
Ranking without Slovak/Czech journals	IF				100	97	97
	NE					100	98
	AIS						100

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS.

When all economics publications are counted, the most productive research institute in economics in Slovakia is the Institute for Forecasting at SAV, followed by the Faculty of National Economy (EUBA); both institutions are ranked by at least two indicators in first and second place, respectively. Last in the rankings are the Faculty of Operation and Economics of Transport and Communications (ZU), the College of International Business (ISM), and the Faculty of Economics (UJS) irrespective of which journal quality metrics are used (Table 3).

A significant difference is seen between rankings with all economics publications considered and ranking of only international economics journals (i.e., excluding Slovak and Czech journals). The correlation between the two types of rankings is between 50% and 67% (Table 4). This indicates that some faculties are relatively competitive in publishing in Slovak and Czech economics journals, but their rank drops significantly when only international economics publications are counted. The rank of the top two institutions is not affected when only international economics journals are considered. A significant drop in productivity relative to the top institution and thus in ranking is observed in particular for the Institute of Economic Research (SAV) and the Faculty of Commerce (EUBA). These two institutions drop to the bottom of the ranking because Slovak and Czech publications are responsible for a significant share, more than 95%, of their published articles. In contrast, the Faculty of Economics (UJS) experiences a significant improvement in its ranking, rising at least seven places, and the Faculty of Business Economics in Košice (EUBA), climbing at least five places (Table 3).

As in the faculty ranking, no significant difference is seen in the departments’ rankings across the three journal quality indicators when the same set of journals is considered. The correlation between rankings for the three journal quality indicators is more than 94% (Table 5). Again a relatively high difference, although smaller than for the faculty rankings, is found between the rankings with all economics publications included and the rankings when only international economics journals are considered with the correlation coefficient dropping to between 62% and 70% (Table 5). This indicates a relatively large discrepancy between some departments in their competitiveness to publish in Slovak and Czech journals rather than in international ones.

TABLE 5 Correlations Between the Rankings of Departments by Journal Quality Indicator

		Ranking with all economics journals			Ranking without Slovak/Czech journals
		IF	NE	AIS	IF	NE	AIS
Ranking with all publications	IF	100	97	96	62	63	62
	NE		100	98	64	68	66
	AIS			100	70	69	71
Ranking without Slovak/Czech journals	IF				100	90	97
	NE					100	94
	AIS						100

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS.

Figure 7 shows the distribution of economics publication output for Slovak economics faculties. The figure reveals that economics publication activity is concentrated at a few faculties, while many produce little or zero research output. The inequality in the publication output among faculties is the largest for the NE metric, followed by AIS and IF. In terms of IF, when all publications are counted, the top four faculties (25% of the sixteen faculties) produce 64% of the publication output by tenure-track staff in Slovakia in 1990–2016. In terms of NE and AIS, the top four faculties produce 86% and 71% of total output, respectively. Given that NE and AIS are more appropriate than IF for distinguishing journal quality, our results imply that the publication of higher-quality articles (i.e., publication in journals with a higher NE and AIS) shows a higher concentration at a few faculties rather than a concentration of lower-quality publications.

FIGURE 7 The Distribution of Economics Publication Output (Lorenz curve) Across Faculties (%).

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS.

Many faculties have extremely low productivity. Half of them (eight) produce less than 12% of the total economics research output when IF is used as the journal quality metric and less than 10% when NE and AIS are used. The output concentration is even higher when only international economics publications are counted. The top four faculties produce almost all international economics publications in Slovakia (more than 88%), irrespective of which journal quality metric is used. The half of faculties account for less than 3% of the international publication output. These results suggest that most faculties in Slovakia are internationally uncompetitive in economics, given that they produce little or no publications in international economics journals (Figure 7).

The economics research concentration is even more apparent at the department level.¹⁴ A small group of departments generates most economics research in Slovakia. Again, the inequality of publication output among faculties is the largest for the NE metric, followed by AIS and IF. In terms of IF, the top ten departments (or 13% of the seventy-nine departments) produce 53% of the publication output in Slovakia when all economics publications are considered. In terms of NE and AIS, the top ten departments produce 82% and 61% of the economics publication output, respectively, again indicating that higher-quality economics research is more concentrated than lower-quality research. The top thirty (38%) departments account for almost all the research (more than 85%) in Slovakia irrespective of which journal quality metric is used. The rest of the departments (forty-nine) make an insignificant contribution—less than 15% of total output—to economics research in Slovakia. Eighteen departments have zero publication output when all economics publications are considered (Figure 8).

FIGURE 8 The Distribution of Economics Publication Output (Lorenz curve) Across Departments.

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS

The results worsen significantly when only international economics publications are counted, i.e. by excluding Slovak and Czech journals. In this case, the top ten departments are even more prominent, given that they generate more than 82% of the international economics publications in Slovakia. The top twenty-nine departments produce 100% of the international economics publication output. Around two-thirds of the departments (fifty) have zero international publications, which implies that they are completely disconnected from and are uncompetitive in the international economics research market (Figure 8).

Research concentration is also high across tenure-track staff within faculties. Usually, research activity is not evenly spread across tenure-track staff at different faculties: a few individuals generate most of publications. Table 6 reports the concentration ratio for the top three tenure-track staff (CR3) and the top five tenure-track staff (CR5) at the faculty level measured as their publication output as a share of the total faculty economics publication output. When all economics publication output is counted, the average CR3 concentration ratio is between 69% and 73%, depending on the journal quality metrics, while the average CR5 concentration ratio is between 79% and 83%. Several faculties have a ratio greater than 90%: at least three faculties for CR3 and at least five faculties for CR5. For international publications (excluding Slovak and Czech journals), the average CR3 concentration ratio increases to around 95% and is almost the same as the CR5 concentration ratio, 97%. Note that the concentration ratios are negatively correlated with faculty size when all economics publication output is considered, varying between −44% and −61%. However, for international publications, the concentration ratios appear to be independent of faculty size, with a correlation between the faculty size and the concentration ratio between −13% and 5%. These results suggest that research productivity for each economics faculty in Slovakia is heavily dependent on few individuals. Productive individuals are usually in the minority at all economics faculties. This is particularly the case for international publications, in which a maximum of the top three tenure-track staff do almost all the research at a given faculty. The implication is that one staff member can significantly affect a faculty’s ranking.

TABLE 6 Concentration Ratio of Research Within Economics Faculties in Slovakia (Share of Economics Publications of the Top Three and the Top Five Tenure-Track Staff in the Total Faculty Publication Output)

			All publications (share of total faculty production)						International publications (share of total faculty production)
			Top 3 TT staff			Top 5 TT staff			Top 3 TT staff			Top 5 TT staff
Faculty/Institute	University	Number of staff*	IF	NE	AIS	IF	NE	AIS	IF	NE	AIS	IF	NE	AIS
Faculty of Economic Informatics	EUBA	39	40	38	40	51	51	55	100	100	100	100	100	100
Faculty of Business Management	EUBA	30	41	44	48	59	60	62	100	100	100	100	100	100
Faculty of National Economy	EUBA	44	63	92	73	70	94	79	90	98	89	95	99	96
Faculty of Commerce	EUBA	27	55	52	54	67	65	67	100	100	100	100	100	100
Faculty of Business Economics in Košice	EUBA	16	85	89	87	98	99	99	100	100	100	100	100	100
College of International Business	ISM	11	100	100	100	100	100	100	100	100	100	100	100	100
Faculty of Economics and Entrepreneurship	PEVŠ	6	100	100	100	100	100	100	na.	na.	na.	na.	na.	na.
Faculty of Management	PU	23	64	56	57	85	83	82	na.	na.	na.	na.	na.	na.
SAV**	SAV	9	81	92	88	94	97	96	100	100	100	100	100	100
Faculty of Economics and Management	SPU	36	55	69	67	68	78	77	99	100	99	100	100	100
Faculty of Economics	TUKE	20	49	50	51	65	66	65	59	60	61	74	74	76
Faculty of Economics	UJS	10	100	100	100	100	100	100	100	100	100	100	100	100
Faculty of Management	UK	34	74	76	75	88	91	89	100	100	100	100	100	100
Faculty of Economics	UMB	53	36	58	52	47	67	62	79	82	82	93	94	93
Faculty of Operation and Economics of Transport and Communications	ŽU	49	91	86	87	100	100	100	100	100	100	100	100	100
Average		27	69	73	72	79	83	82	94	95	95	97	97	97
Correlation between the number of staff and the concentration ratio			−60	−44	−54	−61	−42	−52	−13	−5	−12	1	5	−1

Notes: *Note that the sum of staff over all faculties does not add up to the total 409, because two staff members form the EUBA are not affiliated with a specific faculty. They are included only when universities are evaluated. **We report results for SAV overall and not by the institute, because of the low number of staff with associate and full professor titles. Abbreviations: zero value of international publication output (na.); tenure track (TT); College of International Business (ISM); Comenius University in Bratislava (UK); J. Selye University (UJS); Matej Bel University in Banská Bystrica (UMB); Paneuropean University (PEVŠ); Slovak Academy of Sciences (SAV); Slovak University of Agriculture in Nitra (SPU); Technical University of Košice (TUKE); University of Economics in Bratislava (EUBA); University of Prešov (PU); University of Žilina (ŽU); impact factor (IF); normalized eigenfactor (NE); article influence score (AIS).

The concentration of research is not unique to Slovak economics departments. Other studies find research concentration within and across countries. For example, Scott and Mitias (1996) rank eighty U.S. universities on the basis of articles published by staff members in the top thirty-six journals over the 1984–1993 period, and find large differences across departments and large rank changes when departments are judged on the basis of research concentration, rather than aggregate research output. Further, Scott and Mitias (1996) find that “major league” (i.e., the top nineteen) departments dominate the list of the top fifty individual publishers. Their dominance is more pronounced when the quality of publications is taken into account (i.e., publication in the top five economics journals). Similarly, Combes and Linnemer (2003) report the concentration of research output among research centers in Europe.

Studies also show that research concentration tends to decrease over time. Kim, Morse, and Zingales (2006) show that over the period 1970–2001 elite universities in the United States lost their competitive edge over other universities in terms of their ability to boost staff members’ research productivity. They found that in the 1970s, residence at an elite university had a large impact on individual productivity, but in the 1990s this effect disappeared. Guimaraes (2002) shows that Portuguese researchers consistently increased their publication activity over the period 1986–2000, driven by more publications in international journals. In his evaluation of worldwide research productivity—citations and publications—of academic staff and economics departments, Coupé (2003) finds that the United States held a dominant position in the production of economics research, but this dominance has decreased over time. Similarly, based on a literature review, Neary, Mirrlees, and Tirole (2003) conclude that the gap between Europe and the United States in economics research is narrowing, but it remains significant.

Our results suggest that SAV retains significant dominance in economics research in Slovakia, and no significant change is observed over time. The two research institutes at SAV produce between 31% and 42% of the economics publications, depending on the journal quality metrics, or on average between 15% and 21% per institute. The remaining 58% to 69% of publication output is produced by fourteen faculties, or on average less than 5% per faculty. When our results are compared with those from more than one decade ago by Ciaian, Pokrivcak, and Rajcaniova (2005), the dominance of SAV has not changed significantly. According to Ciaian, Pokrivcak, and Rajcaniova (2005), who calculated IF weighted research production in the period 1990–2004, SAV produced 43% of all research publications when all economics journals are considered and 23% without counting Slovak and Czech journals. Our estimates for the IF-based indicator are 31% and 41%, respectively, indicating that the dominance of SAV has not reduced significantly.

Productivity of Tenure-Track Staff

The tenure-track staff employed at economics departments in Slovakia in the academic year 2015/2016 published around 3.1 articles on average per member over the period 1990–2016, of which 2.2 were published in economics journals and 0.9 in noneconomics journals. As expected, on average, full professors published more articles than associate professors did. The average number of published articles in economics (noneconomics) journals is 3.3 (1.3) for full professors, compared to 1.7 (0.8) of articles for associate professors. Full professors also appear to be internationally more competitive. On average, full professors published 0.5 (1.1) articles in international economics (noneconomics) journals, compared to 0.3 (0.6) articles by associate professors (Figure 9). The higher number of published articles by full professors than by associate professors could be explained by the differences in the career length and the higher criteria (it takes longer and the criteria are stricter) required to attain full professorship than associate professorship. Hence, full professors are expected to produce on average more publications (AK 2013a; Law No. 131 2002).

FIGURE 9 Average Number of Published Papers by Type of Journal and Tenure-Track Position.

Note: Associate professor = doc; full professor = prof.

Our results indicate that a sizable number of tenure-track staff at economics faculties and research institutes in Slovakia are unproductive. As shown in Figure 10, 31% (or 130 of the total 409) of the tenure-track staff members have zero publications in an IF journal, whereas 42% (or 173) of the tenure-track staff members have zero publications in an economics IF journal. For comparison, Ciaian, Pokrivcak, and Drabik (2008) report that around 80% of the staff at Slovak economics departments in 2003/2004 did not have a single peer-reviewed journal article published. However, this result is not fully comparable, given that they include also junior staff in their analysis. In a similar study on the Czech Republic, Machacek and Kolcunova (2005) report that more than 70% of the associate and full professors who received the title in the period 1999–2005 did not have a single publication in a peer-reviewed journal (Machacek and Kolcunova 2005).

FIGURE 10 The Distribution of Published Papers by Type of Journal and Tenure-Track Staff (%).

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS; associate professor = doc; full professor = prof.

An assessment of international competitiveness indicates that the proportion of those working in economics departments who are unproductive is exceptionally high. As many as 76% (312) of the tenure-track staff have not published in an international journal, and 85% (349) have published zero articles in an international economics journal. Many staff members are competitive only locally, publishing only in Slovak or Czech journals. Around 43% (1.5%) of the tenure-track staff published at least one article only in Slovak and Czech economics (noneconomics) journals. As expected, a larger share of full professors has more international (economics and noneconomics) publications, compared to associate professors. However, a larger proportion of full professors have published articles only in Slovak and Czech economics journals (49%) than is the case among associate professors (41%) (Figure 10).

A significant share of unproductive staff is also reported in studies on other countries, including developed ones. However, most of them either use a more restricted set of journals or they report a smaller share of unproductive staff. For example, Pomfret and Wang (2003) show that 60% of academic staff at economics departments in Australia did not publish a single article in the eighty-eight journals examined for the period 1990–2001, whereas the top 4.7% of researchers published around 40% of all journal articles. Based on a comprehensive evaluation of economics research centers in eighteen European countries for the period 1971–2000, Combes and Linnemer (2003) report that at least 55% of European researchers have not published an article in a journal referenced in EconLit; at least 40% have more than one publication; 25% have more than two publications; and 10% of the researchers have more than eight. Only 1% of the most productive European economists have more than thirty-three published articles. Mata (1995) shows that 73% of Portuguese researchers in economics published only a single article in the period 1980–1994. This number drops to 46% in Guimaraes (2002), for the period 1986–2000, indicating that the publication concentration among active researchers in Portugal contracted over time.¹⁵

On average, each tenure-track staff member has nine citations for articles published in the period 1990–2016, of which 5.4 citations are for economics articles and 3.6 are for noneconomics articles. If we consider only the staff who have published, the average number of citations is 13.1 per staff member, of which 7.9 citations are for economics articles and 5.3 are for noneconomics articles. As expected, full professors have more citations than associate professors: 11.3 citations versus 7.9 citations on average over all tenure-track staff or 14.5 citations versus 12.3 citations on average over the staff who published. As many as 25% of the articles by staff who have published have zero citations. This proportion is 27% for associate professors and 21% for full professors.

Figure 11 shows the distribution of economics publication output across tenure-track staff, which takes into consideration both published articles and citations adjusted by the journal quality. As with the output distribution for departments, a small number of associate and full professors in Slovakia conducts economics research, while the vast majority of the tenure-track staff does little or no research. The top ten tenure-track staff (or 2.5% of the staff) generated the majority (more than 65%) of the internationally competitive economics publication output (excluding Slovak and Czech journals) in Slovakia in the period 1990–2016. The distribution of economics international output is more unequal for the NE and AIS metrics than for IF, which suggests that higher-quality international research is skewed more toward fewer individuals. The share of internationally unproductive personnel is as high as 90%; that is, 90% of all associate and full professors generated less than 4% of the international economics publication output, irrespective of which journal quality metrics is used.

FIGURE 11 The Distribution of Economics Publication Output (Lorenz curve) Across Tenure-Track Staff (%).

Note: Impact factor = IF; normalized eigenfactor = NE; article influence score = AIS; associate.

When all economics publications are counted, including articles published in Slovak and Czech journals, the distribution improves, but a large proportion of associate and full professors is still unproductive in economics research. The top ten tenure-track staff (or 2.5% of the staff) generated between 31% and 71% of the publication output, depending on the journal quality metric, in Slovakia in 1990–2016, while between 90% and 96% of the output was produced by 34% of tenure-track staff. This implies that 66% of the staff produced less than 10% of the economics research output (Figure 11).

These results suggest that the minimum criteria required by the Accreditation Commission for awarding titles as associate/full professor (AK 2013a; Law No. 131 2002) are not rigorously enforced by economics faculties in Slovakia. The minimum Accreditation Commission criteria explicitly require producing research output and publishing in IF journals. More precisely, a researcher needs to have at least three (six) articles published in IF journals to be eligible to become an associate professor (full professor). The high proportion of associate and full professors without a single journal publication shows that the minimum Accreditation Commission’s criteria are not being fully implemented, and many faculty members receive titles without producing a single peer-reviewed article. Further, these results suggest that economics faculties in Slovakia have relatively low standards for hiring tenure-track staff in terms of publication requirements. The Slovak economics faculties employed many associate and full professors in academic year 2015/2016 who have little or no peer-reviewed research output.

The low research productivity of tenure-track staff may affect the quality of economics Ph.D. programs in Slovakia. Lubrano et al. (2003) defines a minimum level, below which a person is thought to be unable to supervise a Ph.D. student. According to Lubrano et al. (2003), this threshold is one article published in a top economics journal with one coauthor, or its equivalent, over a ten-year period. If we consider a more relaxed threshold of at least two articles published in an international economics IF journal in twenty-six years, only twenty-nine tenure-track staff (or 7% of the staff considered) fulfill this minimum requirement. Only around 5% of the associate professors (thirteen) and 12% of the full professors (sixteen) reach this threshold. These results suggest that the majority of associate and full professors (93%) employed at economics faculties in Slovakia do not have sufficient intellectual capacity and skills to supervise Ph.D. students, which may dampen the quality of the doctorate in economics awarded in Slovakia.

CONCLUSION

This article analyzes the productivity of economics research in Slovakia based on journal publications and citations extracted from the SSCI database for tenure-track staff (associate and full professors) over the period 1990–2016. By using three journal quality indicators—IF, AIS, and NE—to account for the quality of publications and citations, the robustness of the results was validated.

Overall, economics departments and associate and full professors still show low publication productivity twenty-six years after the fall of communism. The results are consistent across the three journal quality indicators used to measure publication productivity. The low publication productivity suggests that the university system in Slovakia is ineffective in motivating associate and full professors to conduct research and to publish in peer-reviewed journals. Although the minimum criteria imposed by the Accreditation Commission explicitly require publication in peer-reviewed journals, economics departments in Slovakia awarded associate professorships and full professorships to researchers who had not published a single peer-reviewed journal article. As many as 31% of all associate and full professors at Slovak economics departments in academic year 2015/2016 do not have a single peer-reviewed journal publication. Further, following the approach of Lubrano et al. (2003), the results suggest that only around 7% of all associate and full professors have the appropriate skills to be supervisors of Ph.D. students.

The results of this paper provide some evidence that full professors perform better than associate professors in research, but this is not confirmed consistently across all indicators. As a result, we cannot fully confirm that the full professor title is qualitatively superior to the associate professor title in producing more and better-quality journal publications. Full professors show higher publication productivity according to some indicators (e.g., the number of published articles per person, international competitiveness, publication in higher-quality journals), but are weaker or no better than associate professors according to others (e.g., citations per article, the share of published articles in interactional journals). Higher performance by full professors is expected, given that the criteria adopted by the Accreditation Commission that they must meet in order to be awarded this title are stricter than those for becoming an associate professor, including more publications and citations. At the same time, because full professors in general have longer careers behind them than do associate professors, they are expected to be more experienced researchers and to produce more and better articles.

Another important finding is that publication productivity is concentrated in a few departments and is conducted by a small number of associate and full professors. More than half the economics faculties and departments generate little or no economics publication output. The concentration of research is particularly great for higher-quality publications. For example, most economics departments and associate and full professors are uncompetitive with regard to publishing articles in international economics journals. The top ten economics departments (13%) produce more than 82% of the international economics publications; the top twenty-nine departments produce 100% of the total international economics publications, while around two-thirds of the departments have zero international publications. The top ten associate and full professors (or 2.5% of all tenure-track staff) generate the majority (more than 65%) of the internationally competitive economics publications in Slovakia. The share of unproductive associate and full professors in terms of international publication is 90%. As many as 76% of the associate and full professors do not have a single publication in an international journal. A high proportion (45%) of associate and full professors publishes only locally in Slovak and Czech journals, where publication is perceived to be less competitive.

The results seem to indicate that productive individuals are usually in the minority at all economics faculties in Slovakia. A maximum of the top three tenure-track staff do almost all internationally competitive economics research at a given economics faculty in Slovakia, which implies that if a productive person departs, the entire research program at the faculty might be jeopardized.

The findings of this paper have important policy implications. The current university system in Slovakia is ineffective at motivating economics research, in particular articles published in internationally competitive journals. The minimum criteria by the Accreditation Commission for awarding (associate and full) professorships are not fully enforced, and economics faculties have low quality standards in hiring researchers for these positions. The current university system needs to be significantly overhauled in order to change the current level of research productivity at economics faculties. New policy measures need to be introduced to motivate the production of high-quality research. At the same time, enforcement of the minimum criteria for awarding associate and full professorships needs to be improved in order to guarantee that researchers who receive these titles (or are hired for these positions) publish in peer-reviewed journals. Finally, unproductive faculties in research should lose their accreditation for awarding the Ph.D. as part of improving the quality of doctoral programs in Slovakia.

Additional information

Funding

This work was supported by the Slovak Research and Development Agency [APVV-16-0321].

Notes on contributors

Pavel Ciaian

Pavel Ciaian is a Team Leader in the Economics of Agriculture Unit at the Joint Research Centre of the European Commission. Drahoslav Lancaric is a Senior Researcher on the Faculty of Economics and Management at the Slovak University of Agriculture in Nitra. Jan Pokrivcak is Professor and Head of the Department of Economic Policy on the Faculty of Economics and Management at the Slovak University of Agriculture in Nitra.

Notes

1. This work was supported by the Slovak Research and Development Agency under Grant No. APVV-16-0321. The authors are solely responsible for the content of the article. The views expressed are purely those of the authors and may not under any circumstances be regarded as stating an official position of the European Commission.

2. An alternative method for valuation of journal quality is based on expert judgment. The disadvantage of this approach is that it can be subjective and represent the personal bias of the experts (Peters et al. 2014; So 1998).

3. According to Kodrzycki and Yu (2012) and Palacios-Huerta and Volij (2004), the article rather than the paper-page is more meaningful, because the former is the natural unit for measuring research output, whereas paper length is heavily influenced by journal editorial policies.

4. Under the Slovak university system, a faculty is a division within a university, which consists of a number of departments and conducts teaching and research in one field, or a number of related fields. Only economics faculties are considered in this article.

5. The fourteen economics faculties considered in the article are at the following ten universities: College of International Business (ISM), Comenius University in Bratislava (UK), J. Selye University (UJS), Matej Bel University in Banská Bystrica (UMB), Paneuropean University (PEVŠ), Slovak University of Agriculture in Nitra (SPU), Technical University of Košice (TUKE), University of Economics in Bratislava (EUBA), University of Prešov (PU), and University of Žilina (ŽU) (see Appendix Table 1A).

6. One could recover this information with the flow approach if the list of all employed staff at economics departments is collected over the whole study period, which, however, is unavailable.

7. Note that the data were collected during October and December 2016, which implies that only those publications were considered for 2016 which were registered in the database at the time of collection.

8. Note that this approach is a less restrictive approach than the one used in other articles, in which economics departments’ publication productivity is often quantified based only on selected economics journals. For example, Graves, Marchand, and Thompson (1982), Kalaitzidakis, Mamuneas, and Stengos (2003), and McPherson (2012) consider only the top twenty-four, thirty, and fifty economics journals, in ranking of economics departments in the United States, worldwide, and United States, respectively.

9. Journals in the following fields are considered here as economics journals: economics; business; finance and business; operations research and management science; and agricultural economics, policy, and management.

10. If IF is not available for the period 2013–2015, the last three available years were used. Note that some journals may not have an IF since the start date of the sample, but their articles might have been registered in the SSCI database. For example, Finance a Úvěr received an IF in 2000, but its articles were registered in the SSCI database prior to this date. Articles published in Ekonomicky Casopis and Politická Ekonomie were registered in the SSCI database prior to the start date of the sample. Unfortunately, consistent data were lacking to account for the date when the journals analyzed received an IF. For this reason, this factor cannot be considered. All articles registered in the SSCI database published in a journal with an IF in 2013–2015 (or the last three available years for discontinued journals) were included in this article.

11. Note that both total and per capita indicators are calculated as relative values compared to the department with the highest value of the respective indicator in order to ensure the same unit of the two indicators is used when the department’s total publication score is calculated.

12. A similar approach is used by Bauwens (1998) and Pomfret and Wang (2003), who also exclude domestic journals to construct a ranking of economics universities and economists in Belgium and Australia, respectively. Also excluded are Czech journals, because of the common past history of the two countries and the existence of strong relations between Slovak and Czech universities in many areas, including journal editorial boards.

13. According to the minimum criteria set by the Accreditation Commission, among others, a researcher needs to have at least fifteen (thirty) citations to be eligible to receive an associate (full) professorship, of which one-third need to be listed in the Web of Science or Scopus databases (AK 2013a; Law No. 131 2002).

14. Note that SAV does not have departments; hence, it is not included in this figure.

15. Note that Guimaraes (2002) and Mata (1995) use the flow approach; hence, they evaluate the performance of active researchers. Researchers with zero publications are not accounted for in their data; hence, one cannot provide an overall distribution of publication performance based on their analysis.

APPENDIX

TABLE 1A The Economics Departments Evaluated

University	Abbreviation	Faculty/Institute	Abbreviation	Department
Technical University of Košice	TUKE	Faculty of Economics	EF	Department of Finance
				Department of Economic Theories
				Department of Regional Science and Management
				Department of Banking and Investment
				Department of Applied Mathematics and Business Informatics
Slovak University of Agriculture in Nitra	SPU	Faculty of Economics and Management	FEM	Department of Economics
				Department of Finance
				Department of Economic Policy
				Department of Informatics
				Department of Management
				Department of Marketing and Trade
				Department of Mathematics
				Department of Social Science
				Department of Statistics and Operations Research
				Department of Accountancy
Matej Bel University in Banská Bystrica	UMB	Faculty of Economics	EF	Department of Tourism and Hospitality
				Department of Economics
				Department of Corporate Economics and Management
				Department of Finance and Accounting
				Department of Quantitative Methods and Information Systems
				Department of Language Communication in Business
				Department of Public Economics and Regional Development
				Institute of Managerial Systems in Poprad
University of Economics in Bratislava	EUBA	Faculty of Business Economics in Košice	PHF	Department of Economics
				Department of Commercial Entrepreneurship
				Department of Corporate Financial Management
				Department of Quantitative Methods
				Department of Management
				Department of Information and Language Communication
		Faculty of National Economy	NHF	Department of Banking and International Finance
				Department of Economics
				Department of Finance
				Department of Economic Policy
				Department of Pedagogy
				Department of Insurance
				Department of Social Development and Labor
				Department of Public Administration and Regional Development
		Faculty of Business Management	FPM	Department of Business Economy
				Department of Business Finance
				Department of Management
				Department of Production Management and Logistics
				Department of Information Management
		Faculty of Economic Informatics	FHI	Department of Applied Informatics
				Department of Mathematics and Actuarial Science
				Department of Operations Research and Econometrics
				Department of Statistics
				Department of Accounting and Auditing
		Faculty of Commerce	OF	Department of Marketing
				Department of International Trade
				Department of Services and Tourism
				Department of Goods Quality Management
				Department of Business Law
University of Žilina	ŽU	Faculty of Operation and Economics of Transport and Communications	FPEDAS	Department of Road and Urban Transport
				Department of Quantitative Methods and Economic Informatics
				Department of Air Transport
				Department of Communications
				Department of Water Transport
				Department of Railway Transport
				Department of Economics
Comenius University in Bratislava	UK	Faculty of Management	FM	Department of Management
				Department of Economics and Finance
				Department of Marketing
				Department of Information Systems
				Department of Strategy and Entrepreneurship
Pan-European University	PEVŠ	Faculty of Economics and Entrepreneurship	FEP	Institute of Economics
				Institute of Management and Marketing
				Institute of International Business
University of Prešov	PU	Faculty of Management	FM	Department of Economics and Economy
				Department of Accounting and Controlling
				Department of Finance
				Department of Management
				Department of Marketing and International Trade
				Department of Environmental Management
				Department of Tourism and Hotel Management
				Department of Mathematical Methods and Managerial Informatics
				Department of Managerial Psychology
				Department of International Communication
College of International Business	ISM			Department of Economics, Management and Marketing
				Department of Social Sciences
J. Selye University	UJS	Faculty of Economics	EF	Department of Management
				Department of Mathematics and Informatics
				Department of Management
Slovak Academy of Sciences	SAV	Institute of Economic Research	EU
		Institute for Forecasting	PU