![Sample our Economics, Finance,Business & Industry journals, sign in here to start your access, latest two full volumes FREE to you for 14 days]({"type":"image" , "src" : "/sda/5931/TOC-Subject-Sample-2021-Economics-Finance-Business-Industry.jpg"})
Abstract
Although some authors highlight the benefits of journal rankings, previous research is often highly critical of them, insinuating that they can lead to desingularization of academic journals (i.e. their impoverishment and standardization) and dequalification of researchers (i.e. a weakening of researchers’ ability to evaluate academic research). However, as very few authors have empirically assessed these presumptions, we aim to address this gap in the literature. Based on Lucien Karpik’s notions of singularities, judgment devices, forms of involvement, and emulation and rivalry, we assess whether the processes surrounding the production and use of journal rankings might lead to desingularization and dequalification. Our findings support previous research by highlighting that processes where passivity and heteronomy (i.e. lack of autonomy) prevail are conducive to desingularization, rivalry and dequalification. Our findings, however, introduce some nuances into the debate by underscoring instances where emulation logic is employed instead of mere rivalry logic, and where substantial judgment devices and active involvement are mobilized in the production and use of rankings, thereby somewhat alleviating the spread of desingularization and dequalification. Ultimately, our study raises questions that point to a need for serious collective reflection within the academic community on the processes by which published research is evaluated.
Acknowledgments
We sincerely thank all those interviewed for the time they gave to participate in this research project. We are also grateful for the constructive comments received from Associate Editor Carlos Larrinaga, two reviewers, Marion Brivot, Henri Guénin, as well as participants at the 2016 Alternative Accounts Conference (Ottawa) and the 2016 École de Comptabilité Conference (Québec City). Finally, we gratefully acknowledge the financial support of the Social Sciences and Humanities Research Council of Canada.
Notes
1 The impact factor is produced through the Clarivate Analytics’ SSCI database. The two-year impact factor corresponds to the average number of citations received, in year X, by the articles published in a given journal in years X-2 and X-1. It is worth noting that other tools to measure the impact of academic journals have been developed over the years, such as Elsevier’s CiteScore. In this paper, we focus on the impact factor of Clarivate Analytics.
2 While Karpik stresses the notion of consumer commitment, we prefer the notion of consumer involvement which, in our opinion, more adequately reflects the substance of our empirical data.
3 The ABS ranking is now known as Academic Journal Guide (AJG). The change occurred after our data collection had been completed.
4 While anonymity is an important concern for us, it would have been difficult not to mention the names of the ABS and ABDC rankings because, as national rankings, the mere mention of the participants’ country would have easily allowed readers to identify the rankings in question. The measures taken to protect the confidentiality of the collected data are discussed below.
5 In total, 25 people who had collaborated in developing an influential ranking were contacted: 11 accepted our invitation to be interviewed; four declined citing a lack of time; and ten people gave no response to our interview request.
6 In October 2016, Thomson Reuters sold its scientific division to Onex and Baring Asia. The new independent company managing the Web of Science is known as Clarivate Analytics (Clarivate Analytics, Citation2016a).
7 We contacted 26 people via email who participate in tenure committees and/or research funding adjudication committees. Overall, 15 people granted us an interview, six refused alleging either a lack of time or confidentiality concerns, and five did not respond to our interview requests.
8 Eleven interviewees sent a modified version of their transcription. Only minor changes were made to clarify and add information regarding the evaluation process. We used the modified transcriptions in our data analysis.
9 It is worth noting that deans may rely on different judgment devices in developing the recommendations they send to the FT organization.
10 We use the term ‘expert’ to mean someone chosen for her/his specific knowledge of a subject.
11 While we cannot rule out the possibility that these interviewees emphasized a reassuring facade in order to secure their organization’s legitimacy in a context of growing criticisms (e.g., Adler & Harzing, Citation2009; Willmott, Citation2011), by and large the participants were articulate and provided plausible examples and anecdotes in order to support their views.
12 The previous guide serves as a starting point in the evaluation process. Consequently, panel members are tasked with the revision of previously given grades, and the assignment of grades to journals not previously included in the guide.
13 We recognize that our analysis focuses on accounting academia but we believe similar evaluation processes are likely used in the other fields of business academia.
14 Most of the interview evidence mobilized in this subsection was gathered with Canadian interviewees.
15 It is important to note that the people working on adjudication committees also occupy other positions in academia, such as journal editors, members of editorial boards, and active researchers. In this subsection, we are interested in what they told us as adjudication committee members. We acknowledge they may think and behave differently in other areas of activity.
16 Given the objective of our study, we focus on the evaluation of the researcher’s publication record. The assessment of the research project is important but falls outside the scope of our study.