An opportunity for editors of I.S. journals to relate their experiences and offer advice. The editorial view of David Avison Guy Fitzgerald and Philip Powell, Editors: of the Information Systems Journal: Second in a series

As editors of the Information Systems Journal (ISJ), we are pleased to be invited to contribute to the Editor's View series in the European Journal of Information Systems. We agree with much of Ray Paul's (2005) original article in the series, so rather than reiterate these points or provide a parochial piece on where we differ, we think it is more interesting and relevant to raise and discuss one important issue: we choose journal ratings and rankings as ‘our issue’.

However, in order to provide context, we provide a brief motivation for the ISJ. We established the journal around 17 years ago because we felt that the existing journals did not provide suitable outlets for research adopting our view of the world. We recognised the role and importance of the computer, but we took the view that the information systems (IS) perspective was somewhat different. In our view, IS people stood with their backs to the technology and looked outward towards the world at large, to impacts of IT on organizations and society. The impact of technology is not just concerned with efficiency, but more significantly, it is an important change agent with an ‘enabling’ and potentially transforming quality for people and organizations. This wider perspective meant that IS had to embrace a number of associated disciplines and their research approaches, some of which were very different to the generally quantitative and positivistic methods found in the journals of the time, even IS ones, such as the MIS Quarterly. There was a need for a different kind of journal that addressed this broader IS perspective, perhaps inter-disciplinary in nature, and that accepted a more qualitative, interpretive philosophy. Encouraged by the publishers, Blackwell Publishing, and an exciting and international board of editors, the ISJ was launched at the International Conference in Information Systems (ICIS) in December 1990. It was quickly recognised as a major international journal with particular strengths in qualitative research, and also publishing papers that were both interesting and varied.

But the purpose of this article is not to discuss the ISJ in particular, but to discuss the issue of journal rankings and ratings in the field of IS as a whole. In particular, we are concerned about the very limited number and scope of IS journals that are generally considered to be part of an ‘A’ list used by Deans to assess the quality of IS research output. Take almost any ranking and MIS Quarterly appears top with one or two more, usually Information Systems Research and/or Communications of the ACM and/or Management Science, in either second or third position. For example, the ISWorld ‘ranking of rankings’ is a compendium of nine articles ranking IS journals where the MIS Quarterly is given an average weighting of 1.11 and there is a jump to Information Systems Research and Communications of the ACM, which have an average weighting of 2.67 and 2.75, respectively, and a further jump to Management Science at 4.14 (ISWorld, living).

We are not suggesting that these are not worthy of their ranking nor are we commenting on the fact that these happen to be US journals. Indeed, the UK Academy for Information Systems (UKAIS) in its January 2006 Newsletter itself suggests that there are only four top-ranking journals in IS and these are listed as MIS Quarterly, Information Systems Research, Communications of the ACM and Management Science. The UKAIS is suggesting that only articles published in these four journals can be considered ‘top rank’. Some Deans, particularly those in the US, are even suggesting that their probationary colleagues should target only MIS Quarterly and Information Systems Research. All other journal targets are deemed inappropriate outlets for good research.

Conferences as well as journals are suffering from this phenomenon. Even top conferences such as the International Conference in Information Systems (ICIS) and the European Conference in Information Systems (ECIS) are not seeing the best papers (even in their early form – ICIS 2005, for example, did not accept any submissions deemed as ‘research in progress’). If our top conferences are not propagating our top quality research then the discipline will suffer. Furthermore, the present obsession with publishing in the top journals only has reduced the value of academics publishing books and research monographs with the consequent reduction in top quality academic books. Again, we think this is a loss to the discipline as a whole. Junior academics are no longer inspired by the finest research at conferences and our undergraduates have a poor choice of course reading material. There is also the ‘crowding out effect’, as more and more people seek to publish in a smaller set of ‘top’ journals then fewer will actually manage to and more academics will be deemed ‘second rate’.

The issue is not a minor one of marginal interest caused by envy, but one of serious concern to the discipline of information systems. It means that very few journals are seen as ‘excellent’ when used as the basis for evaluating individuals in respect of probation, promotion, tenure and evaluation. If papers are not accepted in these top journals, then the papers (and the academics who authored them) have much reduced status as worthy researchers in IS.

Further, it is not just about ranking individuals. In the UK, for example, as a result of the Government's Research Assessment Exercise (RAE) in which university departments are ranked according to their research, these lists are used to help in the ranking process. Universities also use rankings to evaluate academic departments. This may impact on decisions about funding levels to departments, whether they become ‘research active’ or merely teaching departments and in some cases on their prospects of survival. Thus, these lists have become of critical importance not just for individual academics but for departments and institutions as well, and yet there are usually only three or four journals, all from the US, that appear to count.

It might be argued that such a system is working well as it allows the cream to rise to the top. However, our alternative interpretation is that it is a system that is strangling the academic discipline of information systems. There has been much debate about the discipline of information systems, its nature, and its future (see Iivari et al., 2004 and King and Lyytinen, 2006, for example), but clearly the discipline needs to renew itself and keep a lively and innovative set of people with new ideas coming through to replace a largely aging upper echelon. In our view this use of such a restricted ‘A’ list of qualifying journals is more likely to destroy the profession rather than invigorate it. We are a profession that seems to be devouring our young.

The result is that too many researchers are submitting their work to the same few journals and because of the numbers of submissions overall, their chance of acceptance in these journals is very low. The editors of these journals are concerned about the numbers of submissions and the pressures on their referees, whilst the authors complain about the slow processing of papers and the common experience of rejection. Such high levels of rejection are depressing for the authors, especially new and younger members of faculty possibly seeking their first publication that reflects their main PhD contribution. Rejection will have serious consequences on careers and even good researchers may feel a failure and leave academia as a result.

Although of course a few papers are published in these top-ranked journals – papers that are probably very good – the large majority of papers are rejected. At one level this can be seen as wasted time and effort on behalf of authors, reviewers and editors, but many very good papers are rejected because the numbers are so restricted in these top-ranked journals.

This phenomenon also leads to conservatism and conformity in the discipline. Authors and reviewers both attempt to ensure that a paper fits the particular pattern, style, and content of the journal. The equivalent of the film Death Wish 3 follows Death Wish II to draw on the success of Michael Winner's Death Wish (the title of the film series chosen is not random). There is little room for innovation, creativity or risk. Very few risks are taken and readers complain of the formulaic nature of papers published and that they are basically uninteresting. Thus, relatively few papers, generally of a similar type, make it through to publication.

Yet it is also fairly evident that some authors understand only too well the formula, with some names appearing rather frequently as authors in these journals. Although this work passes the quality standards of the journal, editors of these journals need to seek out and encourage alternative research and research writing. The information systems discipline is frequently described as interdisciplinary and yet there is little evidence of this in these top journals. The discipline of information systems needs to keep, encourage and develop new researchers, new directions, and new approaches (Mingers, 2003). Ours is a rapidly changing discipline and we cannot afford to be so conservative, restrictive and complacent. Although editors change, as with other recruitment, people tend to appoint new colleagues who have similar views and culture.

Some of the top journals also pride themselves in their approach to reviewing by adopting a developmental approach. Essentially this means that reviewers and associate editors not only review a paper but also help to develop it over a protracted series of iterations of critique and revision if they feel that it might have a good chance of being published eventually. While this has many merits and can improve a paper, it also tends to drive a paper to conformity so that its interesting ‘edge’ is lost in the process (a kind of lowest common denominator rather than highest common factor).

Galliers and Meadows (2003) looked at four ‘leading IS journals’ (Information Systems Research, MIS Quarterly, Journal of Strategic Information Systems (JSIS) and the Information Systems Journal) in their 2003 study. Their research suggested that the ISJ and JSIS were much more international in terms of their editorial boards, authors, and journals cited, when compared to the two US-based journals. These authors also argued that there exists clear ‘parochialism’ in, for example, the literature that MISQ and ISR draw on to inform their research efforts, as well as their publication patterns.

We find it particularly concerning (as well as ‘odd’) that European academics apparently also accept this state of affairs. For example, by regarding only the four journals MIS Quarterly, Information Systems Research, Communications of the ACM and Management Science in their ‘A’ list, the UKAIS, the UK's premier society for IS academics, under-values European-based journals. According to Galliers and Meadows (2003), European-based journals are more ‘international’ than the UKAIS-nominated ‘A’ list journals. Further, according to their study, European authors represented only 2 and 5%, respectively, of all papers published in ISR and MISQ during the period 1994–2000. This sets particularly difficult standards for UK-based academics in information systems hoping to achieve a high ranking in the next RAE in 2008.

What can be done? We make four related suggestions.

1. The first is that we (not they) can change this. We are the people that determine what are the top journals. We must say to our colleagues that the information systems discipline – like many others – has, say, 10 top journals representing a variety of topics, research approaches, and sources. If we do not say it to our colleagues in IS (both in our own institutions and countries as well as at international events), to those in other disciplines and to our Deans, no one else will. In the UK, we need to influence the thinking of our peers who undertake the RAE.

2. The second is that all top journals should be international and be more open to other ways of thinking. We could follow the example of ICIS, which in 2005 made an important and significant step in choosing two people as joint chairs (one from North America and another from elsewhere) for the overall program and each of its eight themes. Too often the editorial boards of journals stem from one region of the world. It is not surprising that these journals are seen as narrow.

3. Third, the Association of Information Systems (AIS) has a role to play here. It aims to represent all IS academics and their research (whatever topic, research approach or part of the globe it was carried out). We suggest that it should suggest – no, make as a ‘fait accompli’ – that there are 10 ‘A’ class journals in IS and name them. The list should encompass the discipline as a whole, not simply one culture and one point of view.

4. A fourth suggestion might involve a form of ‘positive discrimination’, Top journals could have a variety of sections reflecting the different sorts of IS research undertaken and publish the best papers in each category, so that worthy papers of different genres are not fighting it out for a spot but living together on an equal basis in the same issue.

In IS, given the state of the art and its tenuousness as a discipline, we argue that to ensure its future development and success we must let many flowers bloom (Avison and Nandhakumar, 1995).