Abstract
This brief comment is a response to the BH Editors timely editorial concerning the issues surrounding peer review. Building on their work, it discusses reviewer education, reviewer recruitment, and the role of associate editors/editors-in-chief. From these perspectives, it suggests ways in which we might prop up the current system – currently creaking under the demands made of it – rather than do away with it in favour of some elusive alterative. It concludes by raising the related issue of what constitutes a ‘contribution’ and suggests a future editorial might look beyond theoretical contributions as the ’holy grail’ of academic publishing.
To start with a piece of (hopefully constructive) feedback, I have to say that the humorous opening to this important and timely editorial doesn’t hit the right note for me. The combination of performative pressures, work intensification and increasing mental health issues, which academia as a whole is suffering currently, make anything connected with these issues no laughing matter. With that relatively minor caveat, however, I am grateful to the authors for raising the issue of peer review and for inviting me to join the debate. At IJMR, where I am one of three Editors-in-Chief, we require three reviews for each submission, and recognise all the challenges to and failings of the peer review system which our colleagues at Business History have so ably set out. I find the historical perspective here – and specifically the recency of peer review as a standard modus operandi for ‘quality’ journals – particularly helpful in allowing us to contemplate potentially radical change to a system that is, if not already broken, then certainly reaching its limits. So, from the perspective of a general management review journal, what thoughts can I add to the debate? There are three aspects of the topic upon which I would like to comment: reviewer education, reviewer recruitment, and the role of associate editors/editors-in-chief.
Reviewer education, and to a lesser extent recruitment, presents something of a paradox. The apprenticeship model of bringing early career researchers into the fold via conference paper reviewing and/or review ‘trials’ may be a good learning experience for the would-be reviewers, but often (and almost inevitably) results in poorer quality feedback for the authors on the receiving end. Thus what is good in terms of capacity building may not be fit for purpose in the short run. The alternative of running reviewer development workshops at conferences is time consuming for journal editors and doesn’t always reach the broader audience that we might hope for. Sharing reviews and decision letters with reviewers, whilst useful to some extent, is arguably too late as reviewers are already entering the process before they are prepared to do so. Perhaps a better way forward is to bring the skill of reviewing into the curriculum for doctoral research training as a productive way of benefiting the doctoral students (as future members of the academic community) at the same time as ensuring that by the time they are reviewing ‘for real’ they have a better understanding of what it takes to add value in this role. In my own institution, we host termly ‘PhD Summits’, in which doctoral students present their work, review the work of others to provide written and oral feedback, and receive feedback from an experienced colleague as a ‘discussant’ for their paper. This format offers an opportunity for practical skills development sandwiched between new and experienced sources of substantive critical feedback. Could this, and similar formats, be usefully embedded within doctoral programmes in recognition that research is not just about completing your PhD, but about becoming an active member in the wider academic community?
Reviewer recruitment is another knotty problem, and one which has – in my experience at least – not really bounced back from the severe problems experienced during Covid, when colleagues had so many other important and urgent demands on their time. There appear to be a number of ‘solutions’ to this problem, each with its own downside. One is to reduce the number of reviews per paper – either on an ad hoc basis when a paper is particularly hard to allocate or as a standard practice – with the consequent issues of quality and consistency which may result. A second is to just keep going down the list of possible reviewers – and accept the fact that the less close they are to the topic area, the less useful the resultant review is likely to be. And a third is for associate editors to step in as reviewers when either recruitment fails or those who agree to review fail to deliver. This latter is, for me, the least favourable outcome, since it results in reduced objectivity and synthesis in the post-review decision letter at the same time as (in a generalist journal such as IJMR, at least) increasing the burden on associate editors to be ‘expert’ in a wide range of topics. I would suggest that neither post publication reviewing nor open pre-publication reviewing are an effective solution here, as being either too late to ensure the publication of quality research or too contentious to arrive at a satisfactory conclusion in a workable number of iterations. Some journals are already ‘leaning’ on their editorial boards by introducing service agreements, in terms of a commitment to review a certain number of papers per year, but these are in practice unenforceable, and need to be pitched with care if they are not to be off-putting and hence counter-productive. Once again, I suspect the remedy lies at an institutional level, where the activity of reviewing needs to be recognised and valued as an integral part of what it means to be an academic. Since many journal editors are still struggling to attain this kind of recognition for their roles (certainly in terms of workload allocation) I am not hopeful that this shift will occur any time soon.
Finally, I believe the role of associate editors and – more particularly – editors, is vital in balancing these issues and building a workable way forward at the same time as maintaining journal quality. For me, there are three principal areas where editors can make a real difference to the peer review process. The first is – harsh, but true – by making tough desk reject decisions that ensure that only submissions with a good chance of making it to publication within a viable number of iterations are passed out to reviewers in the first place. This is a tricky issue in the face of a desire to expand authorship to those not currently still ‘learning their trade’, such as early career researchers, and those on the wrong end of EDI issues, but can be balanced by the provision of ‘value add’ rejection letters that provide meaningful feedback on how to improve their chances elsewhere. At the same time, I think some of the burden of assessing bias, ethical issues and conflicts of interest need to rest with the editor, too. Whilst I am yet to be convinced that we have a handle on the capabilities of AI in relation to ‘authorship’ of academic papers, I can see it as a tool to support editors in surfacing some of these pitfalls that it currently falls to reviewers to spot. Most importantly, editors need to have an overview of – and be reflexive about their role in – what kind of research gets published in their journal, and in counter-balancing reviewer preferences in this regard. Journals needs to have a clear and transparent remit in terms of methodologies, types of contribution and relevant topic areas, which need to be enacted by their editors, but at the same time editors need to both model and enforce an openness to paradigms and perspectives that differ from their own. Associate editors play a key role here, too, in synthesising the views of reviewers and offering authors a steer where these may stem from personal viewpoints rather than aligning with the journal’s remit.
This latter point strays into the murky waters of judgement criteria in management journals. In most ‘top’ management journals, these criteria are strongly weighted towards theory development, to the near exclusion of other types of contribution. I began my academic career within an ethnomethodological research paradigm – a foundational discipline which aims to explicate, rather than explain, the accomplishment of mundane practices and the ways in which they are made inter-subjectively accountable by members of a setting. That this might constitute a contribution was rarely recognised, and even more rarely valued. Whilst the ‘practice turn’ in management scholarship has gone some way towards bringing ethnomethodology in from the cold, the pressure to develop theory has if anything increased in recent years. Whilst the issue of what constitutes a contribution is well beyond the scope of the present editorial, and beyond the remit of individual reviewers, perhaps Business History would be well placed to raise this thorny issue in a future editorial?