The h-index, the citation rating, impact factors and the aspiring researcher

Innumerable scientific experiments have clearly demonstrated that all living creatures, from the amoeba upwards, will, if offered a choice between rewards and penalties, choose the path which brings rewards. Laboratory rats, when they learn that pressing a pedal produces a food pellet, happily go on pressing, and bring up their offspring to do the same. Though their outward behaviour may sometimes bely it, university academics are even brighter than Wistar white rats, and therefore adapt themselves even more promptly to any system that brings them rewards.

Clearly, some impartial method has to be used for selecting academics for appointment, for promotion, and, most especially, for the distribution of research funding. Without one there is a very strong risk of nepotism and favouritism, leading to unsuitable appointments. This was found, for example, in the early Victorian era. The civil service had to be expanded considerably in order to find suitable posts for all the Fitzroy family – the illegitimate children of the previous monarch. By the time of the Crimean war, the resulting chaos had become so obvious that following up earlier suggestions by Bentham (1822, reprinted 1993), Northcote and Trevelyan (1853) set up a system of competitive examination, based on the classical Chinese model. As there was no obvious method of proving that an expert in, say, botany was “better” than an expert in history, the examination was based on a knowledge of classical literature. This had the disadvantage, as Parkinson (1958) pointed out, that there was a risk that someone might use an impersonator at the exam, and when subsequently sent out to, say, reorganise the economy of Central India, would prove unable to write Greek verse when the occasion arose. The old system still lingered on in various quarters – Humphrey Lyttleton's description of his selection as an officer in the Grenadier Guards provides a convincing explanation of why the British army tends to lose all its battles in the first few years of any war (Lyttleton, 1954).

Various other selection systems have been devised, though, with Dean Swift (Swift, 1704, reprinted 1909), it is sometimes unclear “whether they dealt in pearls or meal, and consequently whether we are more to value that which passed through or that which staid behind”. The current method for separating the wheat from the scientific chaff is the h-index. Scientists have developed a habit of producing a citation to support any statement. Where a lay person will remark that the sun rose this morning, a scientist will automatically add “(Hemmingway, 1926)”. A researcher's h-index is the number of papers they have published with at least that number of citations, so a person with an h-index of 30 has written 30 papers that have received at least 30 citations. This, of course, encourages prolixity. George Huntington, for example, could only have achieved an h-index of 2 as he only ever published two medical papers, even though one, produced at the age of 22 describing his eponymous chorea (Huntingdon, 1872) was subsequently eulogised by Sir William Osler (Osler, 1950) who said there are “few instances in which a disease has been more accurately, more graphically or more briefly described.” After this, Huntington confined himself to assiduously treating his patients, playing the flute, fishing and being a prominent member of the Philadelphian society (Lourie, 1982). What an example to us all.

In the far-off days before computers (b.c.) citations had an important function. The journal literature was tiny, but searching it was difficult. Valiant attempts were made to list all scientific or medical papers – one of us at least harbours day-dreams in which the entire Internet will permanently seize up and he will be the only living person who knows how to use the Surgeon-General's Index-Catalogue, but normally an author would have been expected to scour all the relevant journals and make a list of papers that would enable subsequent readers to track his line of thinking and provide a guide for anyone wanting to read further on the topic. Nowadays, anyone with access to a computer can produce lists of relevant papers, and, on almost any topic, there will be far more papers published than anyone could possibly read. The list of citations has therefore lost much of the bibliographical part of its original function and has become more of a showcase for bumping up the author's academic reputation, and, incidentally, the reputation of the author's parent institution: the present King's College London, for example, is the result of a series of mergers, and at one stage fell back in the citations' ratings due to authors mentioning “Institute of Psychiatry” or “Guys & St Thomas' Medical School” as their institutional affiliation. Thus, Gaynor and Brown (2013) will not benefit King's College in its campaign to improve its research rating, while Patel (2013) will. The current British Research Excellence Framework includes explicit provision for assessing the impact of research on other academics, so citing, say (Brooks et al., 2013) or (Byford, 2013) might give a minute boost to King's College, while citing (Woods, 2013) would possibly benefit Durham instead.

The level of your h factor depends on the scope of the database in terms of the journals covered, how those journals are mapped onto topics and whether other forms of publication are included, so one of us has an h factor of 34 in Scopus, 37 in Web of Science (WoS) and 65 in Google Scholar. However, another of us has an h factor of 0 in the Web of Science, 1 in Scopus and 3 in Google Scholar. In this case, the low scores reflect the typical output of the author who has written web reviews for this journal for many years (Lipczynska, 2012; Lipczynska, 2009), but whose only publication listed in the Web of Science was an editorial (Lipczynska, 2011). She fares slightly better in Scopus and the Web of Science, but it is worth noting that certain kinds of output (like web reviews) may well not count towards your overall h factor score [the other author of this piece doesn't care]. Similarly, (Ennis et al., 2012) had no citations in WoS as at May 2013 but three in Google Scholar, probably because Google Scholar reports information more promptly. Differences appear because of time scale – Scopus is limited to 1996 onwards and so loses citations from before this time, whereas the others go back much further. Another reason for higher figures from Google Scholar is that it includes books, so authors like Darwin or Freud would have far higher ratings in Google, if they were in a position to care.

How do you increase your h factor?

Make friends

h factors increase with the number of co-authors, presumably because they will all cite papers in their own output, so it is advisable to make friends early and to add co-authors who contribute enough, but not too much. Age also makes a difference as time allows more citations to be added to your existing portfolio even if you publish no more papers.

A formal citation pact would probably count as fraud if discovered, but there is no harm in giving friends a leg up on the understanding that they will return the compliment. Thus, squeezing in a mention of disclosure, citing (Korsbeck, 2013) would improve her h rating, and we could informally suggest ways in which she could recompense us in her next paper.

Selective referencing

The next important tip is to look at the papers on either side of the h factor threshold. For instance, if we cite (Knight et al., 2006; David et al., 2002; Whittington & Wykes, 1994), then the one of us who cares will increase her h factor from 37 to 38.

Awareness of seasonal fluctuation

There is very little that authors can do about seasonal differences. Journals may claim an average or a maximum number of days between submission and a decision to publish, but so much depends on the rate at which papers trickle through the refereeing process. The time of year in which papers are published then becomes largely unpredictable although evidence from physics suggests that July is a good month to submit (Schreiber, 2012). A measure that is considered useful is the centile rating for an individual paper. These can be general, as in WoS where there are citation counts for areas of research [papers in neuroscience are more frequently cited than those in psychology or psychiatry (Guha, 2010a). These citation rates have been more refined so journals are mapped to areas and the individual paper given a centile rating [obviously being first is best] from a comparison with other papers appearing in that journal for a particular area. The problem of these measures is the seasonal effect. (Mueser, 2012) or (Tacchi et al., 2012), publishing in December, will stand much less chance of being cited in the same year than (Mond, 2013) or (Hardy et al., 2013) in the following issue of the same journal. Papers published early in the year do even better than those published in August so (Grano et al., 2013) might well worry that they may already be falling behind. Book publishers cottoned onto this some time ago. We now expect books with a “2014” imprint to start appearing on sale by September 2013, to the confusion of library cataloguers and of anybody else who believes what they see in print.

Bizarrely papers can be cited even earlier than they are "published" in print. The ability for journals to provide the electronic copy of a paper allows it to be discovered by search engines and therefore gives it much needed coverage. So choosing a journal with Online First, such as this one, will aid citations and can overcome some of the difficulties produced by the seasonal effect. The citations are then counted when the printed issue appears. So, for instance, one paper published electronically in January 2013 will appear in this edition or the next (Trujols et al., 2013) and so will have benefited from a year's exposure to citations.

Titles

Choosing the right title for a paper is increasingly important. Literature searches through the printed volumes of Psychological Abstracts used to bring up both the title of the paper and its abstract simultaneously. Few academics searching the on-line equivalent bother to look at abstracts, so getting as many key terms as possible into the title is important. One of us once published a paper on cognitive predictors of outcome in community care (Wykes, 1994) with a title which omitted the word “cognitive” and so has had significantly less attention than other papers on the topic. A fully informative title like (Gaynor & Brown, 2013) "Self-referrers to community workshops: Who are they and why do some participants not consult with their GP about their mental health difficulties?" is probably more efficient than an eye-catching title like "Good friends are hard to find?" or "Hell on Earth" (Forrester-Jones et al., 2012; Andersen & Larsen, 2012) in this respect. It may not get you noticed but it is more likely to get you cited.

Thus, there are various ways in which authors can adapt their writing patterns to maximise the impact and, therefore, the financial rewards for their research. The most recent book on doing so that we have noticed is Scientific Writing for Impact Factor Journals (Lichtfouse, 2013), but there are innumerable other guides to playing the system. What all of them actually teach, of course, is “how to write badly” (Billig, 2013). Having spent a school career, learning how to pass the exam rather than learning, aspiring researchers have to learn how to improve their h factor rather than learning how to do and write up research: one more hoop to jump through on the way to becoming a successful academic. h-index pathology may even be bad for you (Horne et al., 2009). It is certainly questionable as to whether it assists contributors to the Journal of Mental Health in doing anything towards improving mental health.

Given that the list of references has lost a lot of its original bibliographic function, and that some method is needed for assessing the relative standing of researchers, it is difficult to suggest alternative approaches. One suggestion might be for journal editors, in addition to requiring all the listed authors of a paper to confirm not only whether they actually contributed to the research but also whether they had actually read each cited publication or not. If WoS and Google Scholar only counted citations that had actually been read, a clearer picture of academic standing might emerge.

But perhaps even more bizarrely in the academic world of metrics (although in this field it is hard to judge) is that you do not even have to exist to be world renowned on Google Scholar. Take the case of Ike Antkare who was outed as a fake in 2010 in a paper purporting to be written by himself (Labbé, 2010). At that time, he had 102 publications and an h-index of 94 which at that time was less than Freud, (h-index of 183), but better than Einstein in the 36th position with a h-index of 84. Using one of the other Google metrics, the h_m-index, Ike Antkare was in the sixth position outclassing all scientists in his field (computer science). This was carried out using a computer programme that generated fake papers, which each referenced all the other papers generated. All that had to happen to be included on Google Scholar was to refer in the paper to a paper already referenced in Google Scholar. Once on Google Scholar, of course, there were then references to Ike in all the other metrics generating systems.

The journal impact

Whether you are cited or not getting your paper into a journal with a high impact factor also provides a boost to academic credibility. The first author published her first paper in Nature (Wykes & Johnsonlaird, 1977) assuming it was a magazine with little scientific relevance. We do not want our readers to be so naive at the beginning of their careers. Impact is measured by the average number of citations of papers in a specific journal over a 3-year period. Some journals will display their impact factor on their websites; the majority will be found using a tool such as the Journal Citation Reports from Thomson Reuters. Even if a paper does not attract many citations, like the previous one in Nature, the effect of the journal title is to give the paper a halo of respectability (or not). Getting a paper accepted by a high impact journal is assumed to be more difficult although we have not tested it. There has been some debate about the credibility of journal impact factors as a scientific measurement – DORA, the Declaration on Research Assessment which originated from the American Society for Cell Biology (2012) notes that the journal impact factor as a tool was developed to help librarians choose titles for purchase and is therefore not fit for purpose in terms of evaluating true academic output (American Society for Cell Biology, 2012). If the journal publishes reviews they are often cited, as this is one way that academics do not have to read a lot of papers. This is also true in our journal where for many years the most downloaded article was (Hayward et al., 2002), a review on stigma. Nowadays, the most read using the downloaded papers as a metric is different to its citation rate. In our journal, some papers seem to be more cited than read (Berk et al., 2010) but at other times more often read than cited (Fernandez-Aranda et al., 2012) with 200+ downloads.

Open access publishing

Paying for publication may seem like cheating but it is fast becoming respectable. This allows authors to making their paper Open Access (in electronic form viewable by anyone on a computer). This can greatly increase the likelihood of citations. Many grant funders now insist on this form of publication and have an agreement with some publishers but more often there is a charge, which falls on the authors. This certainly increases readership; therefore, in this journal the most frequently downloaded articles are (Ben-Zeev et al., 2010) and (Wykes & Callard, 2010), from an issue of the journal that was made freely available. It was notable that the download figures for that issue “very broadly tapered off in accordance with their position” (Guha, 2010a) – the opening editorial by (Wykes & Callard, 2010) receiving most attention and the book review which closed off the issue (Guha, 2010b) receiving least. This was most likely due to journalists' fatigue: position within an issue of a journal is probably far less important now than it was when print was the only form of access.

Our holiday message

Dear readers (we hope no-one under the age of 9 years), the existence of Father Christmas has been somewhat disputed but we may be able to discover that he has an h-index. Making many friends early in your career, paying for publication or getting into an issue which is going to be made freely available will all help to increase your h-index and aid an academic career, although increasing age does just as well. As editors, we would be failing in our duty if we did not advise on boring but realistic titles that may get you cited. But a salutary tale is Dan Shechtman who won the Nobel prize for chemistry in 2011. He has an h-index of 11 on Scopus rather low, given his clear impact on the field of chemistry. Changes to tighten up metrics may happen but whatever method is chosen it is obvious that academics will learn to manipulate it as effectually as any laboratory rat would. Our only hope is that, in the midst of all this, something will continue to be done in our primary task of improving mental health.

Acknowledgement

Professor Til Wykes acknowledge support from the NIHR Biomedical Research Centre at the South London and Maudsley NHS Foundation Trust and the Institute of Psychiatry, King's College London in addition to her NIHR Senior Investigator award.

Declaration of Interest: The authors report no conflict of interest.