The cock, the Academy, and the best scientific journal in the world

ABSTRACT

The reader is invited to travel to Ancient Greece, contemporary Brazil, and other places in a fantasy search for the best scientific journal. This whimsical search does not rely on the impact factor, the most popular tool used in real life for finding good journals. Instead, it takes advantage of the so-called authority factor, a recently proposed alternative to the impact factor. The authority factor of a particular journal is the mean h-index (Hirsch's index) of the most suitable group of this journal's editors. Having no connection to any major function of scientific journals, and also being arbitrary (which group of editors to select?), this factor is poorly suited for any technical analysis, but it seems to work well for “small-talk” editorials and self-promotion by complacent editors. Interestingly, the highest authority factor we could find belongs to the journal Temperature. This claim, however, should not be taken too seriously.

Keywords:

• Brazil
• Impact factor
• journal quality
• journal Temperature
• scientific publishing

Abbreviations

CAPES	=	Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
CNPq	=	Conselho Nacional de Desenvolvimento Científico e Tecnológico
FAPESP	=	Fundação de Amparo à Pesquisa do Estado de São Paulo

In Ancient Greece

An interesting case happened in Plato's Academy ∼24 centuries ago. In one version, it was Socrates who came up with a definition of a human being as a “featherless biped.” Plato presented this definition to Academy members and was much praised. Then Diogenes, known for his lack of respect to authorities and mischievous behaviors, plucked a cock and brought it into Plato's Academy. “Behold!” he said, “Here is Plato's human being!”

In prehistoric Africa

A new species of extinct hominin that satisfies the Socrates-Plato definition, Homo naledi, has been just reported by Lee Berger and colleagues in two breathtaking articles published in the journal eLife.^1,2 This bizarre animal with human-like feet and a tiny brain lived at one point (the fossils remain undated) in the area known today as the Cradle of Humankind, South Africa.

In modern Brazil

This editorial was written for the special issue of the journal Temperature entitled “Temperature sciences in Brazil.” This issue is dedicated to the tremendous progress achieved by Brazilian scientists over recent years in their research on body temperature regulation. This special issue will be summarized by Cândido Coimbra (Guest Editor) and Christiano Machado-Moreira and Samuel Wanner (Associate Guest Editors) in their editorial; watch for the next issue of Temperature. In addition to reporting research, this special issue talks about the history of thermophysiology in Brazil³ and the recent collaboration between Brazilian and Hungarian thermoregulation researchers within the Science without Borders program.⁴ It also touches on the 2014 World Cup and 2016 Olympic games – events that are very important for Brazil.⁵ The Brazilians are huge soccer fans, and being the inventors of samba and celebrators of the Carnival, they are also known for their temperament and passion – even as scientists. And, arguably, Brazilian scientists are obsessed with the impact factor.⁶ According to the opinion piece recently published in the journal Physiologist by Martin Frank, its Editor, “In Brazil, there is a lot of pressure to publish in journals with a high journal impact factor. To be considered good, investigators need to publish at least one out of 3 papers in a journal with a journal impact factor of 5.2 or higher.”⁶ Dr. Frank then refers to a different measure, the journal authority factor, and suggests that it should be used instead of the impact factor. “Next time, don't consider the journal's impact factor. Next time consider the journal authority factor.”⁶ What is the authority factor? Should the Brazilians (and everyone else) start using it?

In a world governed by the authority factor

The authority factor was proposed by Mark Johnston, the Editor-in-Chief of the journal Genetics, in his recent editorial.⁷ It is defined as the mean h-index (Hirsch's number of highly cited papers⁸) of journal's editors. Dr. Johnston argues that Nature and other journals run by professional and often young editors (who have no or low h-indices) are not as good as some society journals that have decorated science patriarchs and matriarchs on their boards (high h-indices), and that the authority factor would shed light on this difference in journal quality. I think that the authority factor would be a reasonable measure of journal quality, if the following two assumptions were true: (i) when authors submit their paper to a journal, they do it primarily in order to receive scientific advice from the journal's editors, and (ii) the quality of the advice received correlates with the editors' overall research output, as measured by h-index. These assumptions, however, have nothing to do with reality.

In the world of research labs

In the real world of research labs, when authors submit a paper, they want fast processing, high-quality production, rapid and wide dissemination, and – if possible – active promotion. They want their published product to be well-packaged and readily available to colleagues, so that their colleagues start using it (= citing). In other words, when authors submit to a journal, they are looking for the typical services provided by a publisher – not for scientific advice! In the real world of research labs, the impact factor is a logical concept; please see ref. 9 for a review by Eugene Garfield, the founder of the Institute of Scientific Information (currently, part of Thomson Reuters), who devised the impact factor. As for the subjective authority factor that has no connection to any major function of a journal, it cannot be taken seriously. But it may be fun to toy with!

Back to the world of authority factor

To determine who is who in scientific publishing, Johnston⁷ calculated the authority index for a number of journals; some of his results are shown in Table 1. Dr. Johnston carefully selected the groups of editors to include in his calculations. For the journal Nature, he calculated the average h-index of staff editors (Biology). Did he then calculate the average h-index of the staff members for the journal Genetics, which he runs as the Editor-in-Chief? No, for Genetics, he calculated the average h-index of members of the Editorial Board, and not all members, but only the most senior ones, Senior Editors. These calculations put the journal Genetics on top of Cell and Nature.

Table 1. Impact factor and authority factor for selected journals.

#	Journal	Impact factor	Authority factor
1	Temperature	Not awarded	96
2	PNAS	9.8	60
3	Genetics	4.9	45
4	Nature Genetics	29.6	9
5	Nature Cell Biology	20.1	7
6	Nature	42.4	7
7	Cell	33.1	6
8	Molecular Cell	14.5	5

The authority factor for Temperature was calculated by the author on October 2, 2015, as explained in the text, using the h-indices reported by Google Scholar (for two Advisors, 35 and 120) or Web of Science (for the other two Advisors, 67 and 163). All other numbers are taken from Johnston⁷ and rounded to the nearest tenth (impact factor) or to the nearest whole number (authority factor).

Now, are there journals that are even better than Genetics? Forgive me this heretic thought, but could Temperature, the journal that I run as the Editor-in-Chief, be one of them? For a journal of a perinatal age (Temperature is in its second year), it is well-read. The journal's website has ∼3,500 unique visitors per month, and the top 5 most read articles^10-14 have received > 1,700 views per article by unique readers (mean for 5 articles; as of December 3, 2015). This is despite the fact that Temperature changed its publisher and was moved to a different website in the middle of its first year; all earlier statistics have been lost and are not accounted for. Even more importantly, Temperature has gathered a truly impressive constellation of board members and authors. In this journal, you can read papers written by^15,16 and about^17,18 members of the National Academy of Sciences of the USA, as well as by members of several European academies.^19,20 This is what I call an “Academic” journal! Temperature even has published a paper by a co-winner of a Nobel Prize for Peace²¹ and an interview with a scientist who has been recently predicted by Thomson Reuters to win a Nobel Prize in Physiology or Medicine.¹⁷ Short of publishing Socrates and Plato, how more “authoritative” can a journal be?

So I decided to calculate Temperature's authority factor. Whereas Johnston⁷ made his calculations for Genetics using Senior Editors, I used the list of Scientific Advisors for Temperature. If authors submit papers to receive scientific advice, then this is clearly the most relevant group of editors. Let's check what Temperature's Scientific Advisors are made of! Arguably, the best way to find a researcher's h-index is Google Scholar (see my recent editorial²²), which tracks a wide range of publications. Google Scholar profiles are supposed to be edited by their owners (and most are) to eliminate publications authored by other researchers with the same name and to include all publications by the owner, even if the owner's name was misspelled or written in a different alphabet, or even if a different name (e.g., maiden name) of the owner was used. For these reasons, the h-indices for two Temperature's Advisors who had their Google Scholar profiles at the time of my calculations were taken from those profiles. Web of Science was used to determine the h-indices of the other two. And here is the result: with the authority index of 96, Temperature beats handily not only Cell and Nature, but even PNAS and Genetics (Table 1)!

If, based on the measure proposed by Johnston⁷ and promoted by Frank,⁶ one defines the best journal in the world as “the journal with the highest authority index,” then I am proud to present this best journal to our readers: behold, it is Temperature! Ta-da!

Back to Brazil

The importance of the impact factor in Brazil was elevated to its present level after one of the largest funding agencies in the country – Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, often referred to as “Capes”) – started to heavily rely on the impact factor while making funding decisions for graduate programs. The statement by Dr. Frank about the requirement for Brazilian students to publish in journals with an impact factor of 5.2 or higher,² as cited earlier in this editorial, in not entirely correct. The 5.2 value is applicable only to the field of “Biological Sciences II,” as defined by CAPES, which includes morphology, physiology, biochemistry, biophysics, and pharmacology. Different values of the impact factor apply to other disciplines. For example, some neuroscience programs belong to the “Interdisciplinary” area of the CAPES classification. In this area, a paper receives the highest evaluation if it is published in a journal with an impact factor of 2.5 or higher. But the main principle – judging a publication based largely on the impact factor of the journal in which it is published – is described correctly. Other major funding agencies in Brazil – Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) – are also believed to use the impact factor for evaluating the productivity of their applicants and grantees, even though they do not have a formal, CAPES-like, impact factor-based system. While many Brazilian scientists and journal editors protested against the heavy reliance of foundations on the impact factor, some editors allegedly adapted to the new trend by trying to inflate their journals' impact factors;²³ you can follow this story by reading the series of editorials by Milton Ruiz, the former Editor-in-Chief of the journal Revista Brasileira de Hematologia e Hemoterapia.^24,25

Not all Brazilian scientists share the same attitude toward the impact factor – some are relatively relaxed about it. The current issue of Temperature is the best proof of this: more than 50 researchers with current and former Brazilian addresses have published their work in this issue. This is despite the fact that Temperature is too young to have an impact factor. Furthermore, there was competition for this special issue's pages: 29% of the submitted original-research papers and reviews were rejected. Among the published papers, some received very high marks from Temperature's expert reviewers; I am sure that the editorial by Cândido Coimbra and his team in the next issue will talk about these papers.

Those Brazilians who are critical about the impact factor are certainly not alone (see, e.g., refs. 6, 7). The journal eLife, cited at the beginning of this article, also takes a strong anti-impact-factor position²⁶ and, moreover, has a separate section on its website entitled “eLife will not promote the Impact Factor” (capitalization by eLife). But even eLife's Editor-in-Chief Randy Schekman and Executive Director Mark Patterson have to admit the obvious: “it remains sadly true that at many institutions in countries where the internal resources may be inadequate to give proper consideration to expert letters and thoroughly review a candidate's published work, the impact factor remains a convenient crutch on which to base an imperfect evaluation of merit.”²⁶ The way this admission is worded reminds me how some US media reported about Forbes' naming Russian President Vladimir Putin the most powerful person in the world – no shortage of qualifiers, expressions of regret, or diminishing adjectives!

My bet, however, is that Brazilian foundations and scientists – together with the rest of us – will continue using the impact factor to evaluate the quality of scientific journals. The impact factor is not the only measure of journal quality. But it is certainly the most powerful one we have today. It is quantitative; it is function-based; it is objective; and it simply makes sense. A journal that publishes papers that are cited a lot (= used heavily by scientists) is a good journal – even if it has young professional editors, or even if it is not affiliated with a major scientific society. Furthermore, while desired as a measure of journal quality, the impact factor (of a journal) is also a good measure of scientific merit of an individual paper published in this journal (please see ref. 27 for discussion). This is not totally unexpected: authors tend to send their best papers to the best journals. By the way, do you remember which journal is the best in the world – at least according to one definition? He-he.

Back to Ancient Greece and prehistoric Africa

The legend goes further that, after Diogenes' demonstration of the featherless biped, the words “with broad flat nails” were added to the Socrates-Plato definition of a human being. Perhaps testing this definition, the team of Lee Berger paid special attention to the proximal phalanges of their recently discovered hominin, Homo naledi, and found that they were markedly curved!¹ Would the definition of the best journal we jokingly analyzed in this editorial improve if we added something about fingernails to it?

Disclosure of potential conflicts of interest

The author serves as Temperature Editor-in-Chief. The following Brazilian foundations mentioned in this article supported the research of several postdoctoral fellows and students in the author's laboratory: CAPES (M. Camila Almeida), FAPESP (Tatiane B. Nucci and Renato N. Soriano), and CNPq (Samuel P. Wanner).