Citations, impact factors and the Australian Journal of Psychology in the future

This editorial aims to situate the Australian Journal of Psychology (AJP) in the context of the processes by which scientific journals are judged as significant and worthy of consideration for publication, subscribership and readership.

AJP remains the premier journal of the Australian Psychological Society, publishing articles of scientific rigour available to, and speaking the same language as, the international community of psychologists. The Journal has a responsibility to publish research that is Australian in content and context, which might find difficulty in getting published in the journals of other countries and societies because it is concerned with matters unique to Australia and Australian society. While, for example, the work of Schweitzer, Perkoulidis, Krome, Ludlow, and Ryan (2005) on prejudice in Australia, has international relevance, there is a specific need for that work to be published in an Australian publication, for it to be noticed in its home context. And while the work of Norman Feather on “tall poppies” and reactions to their fall has international significance (Feather, 1994), Australia has a distinct reputation for having specific attitudes to the tall poppy and work on that topic should be published nationally as well as internationally (Feather & Nairn, 2005). The Journal, however, must also seek to attract authors from international sites who can see it as an outlet for their work, which has prestige and profile.

Journal impact

In the contemporary scientific world, the number of journal articles published by an author is no longer sufficient to register significance. Where the articles are published and the reputation of the journal becomes critical. The development of the citation index has led to an eruption of literature on not only the reputation of the individual scientist, but also the impact of the journal (Garfield, 1972). Impact measures the frequency with which the average cited article in a particular journal is cited, subsequent to publication. The measure of the number of citations an average article achieves becomes another candidate for a league table of repute. To attract eminent authors, the impact factor (IF) must be high.

In psychology, the journals with the highest impact factors are journals of review, such as the Annual Review of Psychology and the Psychological Bulletin, which, in 2005, have impact factors, published by Thomson ISI, of 9.78 and 9.75, respectively. In judging the IF of a journal such as the AJP, we should set aside review journals, which have a built-in bias for citation, because they are used as convenient guides to the past literature, and we should examine the IF of journals that report empirical data, the keystone of the scientific endeavour. No psychological journal approaches the impact of general scientific journals such as Nature or Science. But journals such as the Journal of Abnormal Psychology can boast a factor of 4.38 and the Personality and Social Psychology Bulletin, a factor of 2.75.

Impact of the Australian Journal of Psychology

If we examine the impact of the AJP, on first examination the picture is not good, but at the same time not bleak. The impact factor for the AJP over the last few years is shown in Table I. This gives a picture of a decline followed by a moderately large increase, with the IF always less than unity. This absolute level of impact appears low, but in relative terms it is not so. Compare in Table II the impact factors for a range of broad-based psychology journals and the AJP fares pretty well. The questions for the AJP are, how do we maintain this increase and can it be further improved? An examination of the citation of papers in the AJP gives some clues.

Table I. Changes in impact factor of Australian Journal of Psychology over time

Year	2001	2002	2003	2004	2005
Impact	0.404	0.426	0.318	0.304	0.638
Note. Impact = citation to recent articles divided by no. recent articles.

Table II. Impact factor of selected generalist psychology journals in 2004 and 2005

Journal	Impact factor
Annual Review of Psychology	9.78
Psychological Bulletin	9.75
Psychological Review	7.99
American Psychologist	6.46
Psychological Science	4.50
Journal of Abnormal Psychology	4.38
Current Directions in Psychological Science	2.13
British Journal of Psychology	1.8
American Journal of Psychology	1.06
European Psychologist	0.85
Canadian Journal of Psychology	0.65
International Journal of Psychology	0.65
Australian Journal of Psychology	0.64
New Zealand Journal of Psychology	0.58
Anxiety, Stress and Coping	0.52
Journal of Mind and Behavior	0.44

Biases in the measurement of impact

There are a number of biases in the assessment of IF that need to be considered and which, frequently, in the drive to establish league tables of repute, are ignored. The first of these is the degree to which articles in a journal are self-citation, citations of a paper in the journal by the same authors in another journal. Publishing authors are likely to cite their own work because they know that research better than anyone else. The question is, to what extent is it necessary to take out self-citations in assessing the overall impact of a journal? If an author is publishing several articles, it seems unfair to deny the journal the benefit of having the work published and cited, whoever may be the citing author.

In the listing of IF figures for 2005 by Thomson ISI fully 47% of the citation to the AJP are self-citation. This figure can be increased by counting citations by authors in the same laboratory or institution, a factor that is difficult to factor out of any statistical assessment of impact. Adair and Vohra (2003) have recently shown that an increasing share of citations in psychology journals comes from such in-house citations. And if one extends the notion of “invisible colleagues”, then cross-citations by close colleagues is a major factor in the development of a discipline (cf. Innes, 1980a).

A second factor that needs to be considered is the relative rate of citations. If one counts only those papers in the AJP in 2005 that are not the target of self-citation, the number of citations is rarely greater than unity. However, fully 50% of articles published in the scientific literature never get cited (Innes, 1980b, Redner, 1998, cited in Buchanan, 2000). So, even one citation puts an article in the sixth decile of the distribution. If every article in an issue received only one citation, then the IF would be 1, placing that journal into a much higher league, akin to the American Journal of Psychology (Table II). Impact facors less than unity are not unusual.

A third factor to be considered is the size of the field in which the research is conducted. If one publishes an article on the interpretation of cranial size for the development of social cohension (e.g., Festinger, 1983), then one will likely have a significantly lower likelihood of citation than a paper that is, for example, more fashionably in the area of imaging functional magnetic resonance associated with some social act (e.g., Decety & Jackson, 2006).

Small citation rates do not, in themselves, index insignificant research. It is a little known fact that the distribution of citation follows a scale-invariant power law (Redner, 1998). Articles of importance that are published in small fields have, relative to other papers in that area, the same relative increase in citations as do significant articles in large fields.

A fourth factor that has to be acknowledged are the demonstrated differences in citation patterns in different fields in psychology. For example, in perception studies the literature cited is significantly older than in studies on learning (Innes, 1973), so a paper in the former field can have a higher possibility of citation over a longer period. More recently, it has been shown that journals such as the Journal of Personality and Social Psychology cite a larger proportion of old articles than does the Journal of Experimental Psychology (Adair & Vohra, 2003).

Finally, a fifth factor is the timing of the measurement of impact. Fields of study differ in the time when papers have their highest citations (Glanzel & Moed, 2002). In some disciplines the peak of citations is reached at 2 years (the time interval used in the Thomson ISI calculation), while in other fields the peak is reached after 5 years, whereby the IF may be significantly underestimated. There seem to be no data on the lag period of subdisciplines within a field.

All of these findings have implications for assessing the IFs of journals. The measurement of IF as presently used by Thomson ISI emphasises the citation of recently published articles (and this is reflected even more strongly in another measure, the “immediacy index”) so that possibly faddish, fashionable articles define a journal's impact, rather than articles of a longer lineage. The controversy attending the publication of highly cited psychology authors on the internet by Thomson ISI demonstrates the lacunae that emerge with the use of numerical data (Hebert, 2004), where classically important authors fail to show impact. Although no journal would wish to be identified as the outlet for the sleeping beauties of a discipline (Lange, 2005, van Raan, 2004), where there is a very significant delay in the recognition of a piece of research that appeared ahead of its time, journals that have longer term durability have a respectable role to play in the development of a field of endeavour. Impact and immediacy do not define such longevity of impact. Journals such as the AJP, where there is a mix of disciplines, may be more negatively affected by such factors than those more identified with a particular subfield.

Increasing the impact of the Australian Journal of Psychology

Citation and impact, therefore, have to be read carefully and placed into context. What does this mean for the present and the future of the AJP? If impact is going to be a feature of the lives of authors and editors, and in this age of league tables of scientific reputation (cf. the numerous tables assessing importance of universities from the Times Higher Education Supplement to the Shanghai Jiao Tong University), there seems no reason that they will go away, what is to be done to optimise impact?

First, future editors can maximise impact by seeking and encouraging submissions from already established authors and give those authors priority in acceptance. Frequently published authors, by definition, are in press more often, they cite themselves more often and hence enhance their citations and the IF of the journals in which they publish.

Second, there is a strong case for the publication of more special issues, in which groups of authors focus on a particular topic. Networks of authors working on similar themata are likely to, and can be encouraged to, cite authors in related articles in the same issue. Although such blatant self-citation may be detected and downplayed, there will be later benefit for the Journal because these authors will cite these articles in other journals in later years.

Third, the AJP should consider the benefit of articles of review or commentary as a general principle, in addition to, or to replace, the publication of empirical studies. Examination of ranks of IF clearly show the high relative impact of reviews of empirical literature or theoretical analysis, because these are used as single, prominent sources in brief literature reviews.

Fourth, editors can simultaneously enhance author prominance and review impact by the publication of the presentations by distinguished scientific awardees. Papers by Coltheart (2005) and Day (2006), can significantly boost impact because their work is salient, draws attention to the Journal and gives access easily to a broad range of work.

Fifth, very serious consideration should be given to the publication of methodolgical commentary. The huge impact of such commentaries is attested by citation of major papers in the Psychological Bulletin (Sternberg, 1992) and by the subsequent comments by authors (Cronbach, 1992; Fiske & Campbell, 1992).

Conclusion

The citation of research and the assessment of the relative impact of journals are central to the scientific endeavour and, in the light of the research quality framework (RQF) in Australia, they will continue to be so. The Australian Journal of Psychology owes the profession in Australia, and the contributing authors, a responsibility to increase its impact factor. There are means, not cynical in intention, that can be used to enhance impact. Previous editors, such as Bill Noble and Bryan Byrne (2004) have demonstrated that impact can be improved. Future editors should take note. Scientific reputation and the significance of a journal can be manipulated.