Why review?

Over several editorials in this journal the peer-review publication system has been scrutinised. Initially, we published “Where to start?” (Banister, 2014), when idea for a (review) paper emerges and continued with “What to review” (Givoni, 2014), “How to write a literature review” (Van Wee & Banister, 2016) and “Why book reviews” (Kaplan, 2014). Banister (2015) criticised the “Journal Impact Factors and Paper Citations” concept that nowadays drives paper “production”. Following the same line, this editorial looks at what is the backbone of academic publishing – the peer-review process – and asks Why Review? Blind peer review is critical for progressing research, for disseminating the results and it is therefore critical for advancing state-of-the-art, but it remains the bottleneck of the publication process, often lying hidden behind the scenes, unnoticed.

We all want to publish our papers, preferably in “good”, reputable journals that have a high Impact Factor. When submitting our papers, we expect to get valuable and detailed comments and in reasonable time. At the same time, not all of us agree and happy to review others’ papers and to provide valuable and detailed comments in a reasonable time. The paradox is that we all want to see our own papers published but we are far less willing to review papers. Yet every single paper requires three comprehensive reviews.

This editorial claims that this inconsistency, or paradox, does not need to exist. It outlines the benefits alongside the responsibilities (or external benefits) of every researcher to take part in the peer-review process. It calls for more involvement of researchers in the review process and it focuses on the author–reviewer relationship, with the editor mediating between the two sides. It does not touch on the role of the publisher who gets the input (submitted papers) and labour (reviewers’ comments) for free and then sells the output at high price. Perhaps this is something that ought to be discussed in future editorials.

The benefits of and responsibility to review papers can be assessed through considering the personal and community (social) value of reviewing – both of these factors ought to play a role when receiving an invitation to review a paper.

On the personal side, reviewing a paper is an opportunity to be exposed to recent research and to be up to date with a subject, including recent literature which might have been missed or overlooked before. It is a chance to be exposed to new ideas, perspectives, methodologies, results, and different thinking, and it is an opportunity to critically assess one’s own research with respect to what others are doing. For young researchers especially, the above is a particularly important element, as well as a chance to “learn the trade” (of paper publication), but from the other side. Assessing others’ writing and communication style is an opportunity for self-reflection on one’s own writing and it is a good exercise in critical thinking and writing. To add to the above, reviewing a paper for a reputable journal is also important for showing engagement in the field and for the CV. Lastly, it is also a chance to read a paper that otherwise you would likely never read. In this respect, new researchers can actively put themselves forward as reviewers by contacting the editor(s).

The statistics for any journal show that the majority of papers are rejected, as they are found not to be suitable for publication (perhaps suggesting that often papers are submitted too soon, as authors try their luck). This only takes away some of the benefits mentioned above, and critically thinking what makes a paper unsuitable for publication should help to improve one’s own writing. When attending conferences it is often the case that many presentations are not relevant, not good, or simply not offering anything interesting to take home, but this does not stop us from attending conferences. Similarly with paper reviews, we can expect that not all papers we review will provide the same personal benefits, but this should not stop us from accepting the next invitation to review.

Many papers are seen as being potentially suitable for publication, but require major revisions and a re-review by the same reviewers before a publication decision can be made. Being invited to re-review a paper is a unique opportunity to assess one’s own review against the other (blind) reviews and against the comments and responses from the author(s). Authors usually give credit to the reviewers for original and helpful comments that (in their view) have helped improve the paper while vigorously defending their approach when disagreeing with the reviewers. To be able to see what other reviewers objectively thought about the paper is a rare chance for an unbiased reflection on one’s own understanding and assessment of research (the paper).

With experience and after several more papers reviewed, the benefits from reviewing papers might decline, but they still exist. The “social”, community responsibility to review papers should then play an increasing role in a decision to accept an invitation to review. If research is to be advanced, it must be published, and if state-of-the-art is to be pushed forward, research must be high-quality and offer some added value. All this requires the research to be (blind) reviewed. In this sense, the benefits of reviewing a paper can be considered as external benefits.

A one paragraph review of a paper is not really helpful to anyone. Much more than the actual recommendation (reject, major/minor revisions or accept), it is the content and quality of the review that matters. A good review should assist the author(s) to improve the paper and it should assume that a substantial effort and time has been put into the research and writing of the paper. It must be critical and point out the main weaknesses and limitations of the paper, but in a supportive way. The review should ideally give a very brief description of the paper’s subject and aim, to ensure all sides (reviewer, author, and editor) are on the same page, and then progress through three levels. The general level ought to outline the main strengths and weaknesses of the paper with respect to the subject, the research question posed, the intended contribution (added value), the methodology chosen and the delivery (structure and writing). Attention should also be given to the extent to which a paper meets the specific journal scope. Next, more specific, but still general, comments should be given with respect to different sections and parts of the paper. For example, was the methodology clearly presented and in sufficient detail to understand the analysis? Were the conclusions based on the results presented and clearly emerging from them? Finally, at the third level, there is room for specific comments, often alerting the authors to unclear sentences or paragraphs, typos, overlooked references, unclear (or unnecessary) tables and figures, etc.

A useful review takes time – time to carefully read the paper and time to write the review. This is valuable time that otherwise could have been used for publishing the next paper. But the benefits (personal and community) of the review must be kept in mind. Conscious of this time dilemma, editors have a role to play in trying to make the process easier and friendlier for reviewers. The “automation” of the review process and the on-line platforms used to submit the papers and to handle the review process has made the life of editors much easier, by lowering their workload substantially and allowing them to focus on the review and not the processing of it. This “automation” is essential for keeping on top of the status of the increasing number of papers submitted for consideration. But it can impersonalise the process and create a distance between the editor and the reviewers with a risk of alienating the reviewers and lowering their motivation to provide a detailed and on time review. One of the skills editors are required to develop is to find the right balance between using the automated process and their own personal involvement, and they should recognise the need for some direct communication with reviewers, as well as authors. With several hundred papers submitted per year keeping that balance is challenging.

The current (blind) peer-review system no doubt has some limitations, but it works and it serves its purpose and a better alternative is not yet available. It deserves however more attention and more open debate on how to preserve its functionality and quality. The hidden, anonymous nature of the review system, alongside the increasing work load on reviewers, authors and editors can generate “short cuts”. For reviewers this might mean accepting fewer papers to review, providing short and uncritical reviews, and delaying the response to invitation or the submission of the review. For authors it often means not following the submission guidelines, especially not anonymising the papers (and when this is done, not including a separate title page with all the details of the authors). In turn, the workload on editors is increased and they might settle for fewer reviewers, for sending a paper for review even if it is not anonymous, and/or for a very long review process. We all need to cherish the peer-review process, whether we are reviewers, authors and of course editors.