Nonfinancial conflict of interest in peer-review: Some notes for discussion

ABSTRACT

Strong beliefs can influence the way we deal with emotionally charged topics. Researchers, editors, and reviewers are not an exception. Declaring such nonfinancial conflict of interest when handling or reviewing submitted articles is often obligatory; however, the declaration is not a license to submit a biased review with personal insults or to break the journal’s guidelines. This kind of poor practice can be a clear sign of the seriousness of conflict of interest. In this article, I argue that hostile, unethical, and biased behavior of reviewers and editors often arises from a serious nonfinancial conflict of interest, which should not be ignored or undermined.

KEYWORDS:

• Peer review
• conflict of interest
• editors
• authors

Introduction

During the last decade, there has been an extensive discussion about nonfinancial conflicts of interest (COI) in science. Some argue that nonfinancial interests do not constitute a COI (Bero and Grundy 2016) while others emphasize the dangers of neglecting them (Wiersma, Kerridge, and Lipworth 2018a).

On one side of the debate, nonfinancial COIs are considered too complex and difficult to detect and manage; less damaging than financial COIs; of lower ethical priority compared with financial COI; represent a red herring for financial COIs. It is further argued that as no one is fully neutral and free of biases, anything and everything can potentially be labeled a COI; not all intellectual interests represent a COI; a diversity of perspectives is needed for the advancement of science; and disclosure of some interests may raise ethical and privacy concerns (Bero 2017; Bero and Grundy 2016; Goldberg 2020; Rodwin 2018; Wiersma, Kerridge, and Lipworth 2018a, 2018b).

According to the other view, “dismissing nonfinancial COIs is naive, empirically unfounded, and dangerous” (Wiersma, Kerridge, and Lipworth 2018b, 1). Both grant applicants and reviewers are concerned about nonfinancial COIs (Abdoul et al. 2012), although there is no consensus among experts in the field about what constitutes a nonfinancial COI (Grundy et al. 2020). A vast literature on cognitive biases provides enough evidence to justify concerns that strong beliefs, values, etc. can interfere with our work as scientists. If we exclude fabrication, p-hacking, HARKing, and the file-drawer effect, which are difficult to discover, the work of authors can be largely scrutinized in the context of their COI as their published work, public appearances, and their names are publicly known (Bero 2017; Rodwin 2018). On the other hand, it is much more difficult to detect a COI among peer-reviewers and address them transparently or publicly (Table 1).

Table 1. Detectability of COIs among authors, editors, and reviewers.

	Authors	Editors	Peer-reviewers	Grant reviewers
Names of involved persons	Names are public	Names are public on a journal’s website although names of handling editors are not always listed alongside each published paper	Names are always known to handling editors, but only in open review to authors	Names are known to the funding administration. The names of panelists, but not reviewers, are sometimes shared with applicants
Disclosure statements	Disclosure statements often public although not necessarily complete	Disclosure statements are often publicly unavailable even for financial COI	Disclosure statements are often available only to editors	Disclosure statements are available only to the funding administration
Who and how can raise COI issues	-> A reader can discover and raise COI issues which if confirmed could lead to published corrections or retractions	-> A reader can discover and raise COI issues. This might lead to the firing of an editor and examination of the peer-review process of some articles (e.g., authored by the editor), but it seems unlikely the journal/publisher will go through all submissions handled by the editor	-> Authors can raise COI issues only if they know who reviewed their papers. However, even then a discussion about COI will be limited as authors cannot publicly reveal the names of reviewers and share the content of their reviews in full	-> The funding administration can search for possible COIs not reported by panelists and reviewers

In this commentary, I focus on nonfinancial COI among journal editors and peer-reviewers. I base my discussion on my experiences and observations as an author, peer reviewer, and associate editor in the field of traffic safety. This is an applied discipline, which research findings have direct influence on policy making. Consequently, debates often go beyond academics and involve various stakeholders and parties with different interests.

For example, issues related to bicycle helmets are among the most controversial in traffic safety research (Goldacre and Spiegelhalter 2013; Olivier et al. 2014). Although biomedical (Cripton et al. 2014; McIntosh, Lai, and Schilter 2013), computer simulation (Fahlstedt, Halldin, and Kleiven 2016; McNally and Whitehead 2013), and case-control studies (Høye 2018; Olivier and Creighton 2017) show bicycle helmets’ efficacy and effectiveness in mitigating head injuries, many are skeptical, pointing out their limitations in crashes occurring at higher speeds and those involving motor vehicles. Furthermore, some (e.g., Adams and Hillman 2001) have proposed that wearing a helmet increases the risk of being involved in a crash because of risk compensation (i.e., helmet wearing-> feeling safer->taking more risks->crash more likely). Others are concerned that promotion and legislation of helmets leads to less cycling (Robinson 1998, 2003), which is a healthy activity (Mueller et al. 2015; Oja et al. 2011), and that less cycling would reduce the overall health benefits on a population level.

Heated debates on any of these issues are frequent especially on social media, where researches perceived as pro-helmet are often accused of producing fraudulent research, working for the benefit of the car industry, or, in the best case, only mocked for wasting resources on an unimportant issue such as bicycle helmets. Personal insults are common. This is nothing unusual for “debates” on social media (Kelland 2019). More problematic is when strong personal interests interfere with the desired objectivity of peer-review process.

Strong personal beliefs as a conflict of interest

Strong personal beliefs are often mentioned as a potential nonfinancial conflict of interest regarding “emotionally charged topics” such as human stem cells, abortion, evolution, gun control, marriage laws, etc. (Grundy et al. 2020; Rockwell 2006). Bicycle helmets, as one of the most controversial issues in traffic safety research, clearly fall within this category.

A well-known and respected active travel researcher said in a recent interview “I have been a hater of cars from my entire life. And that was what I was actually motivated my entire academic career” (Bernstein, & Reid, 2021, 2:49). Although the negative impact of cars on the environment and health is well researched and documented, it can hardly be put in the same category as cancer for example. There is a broad scientific consensus that no benefits can be associated with cancer, thus, hating cancer seems appropriate, although such researchers should be conscious about their strong motivation and the negative effects it can have on their work (e.g., observer bias). The same can hardly be said about cars as a machine. Openly stating that hate toward cars preceded and motivated an entire academic career is a clear sign of a biased approach to the topic of one’s research. A legitimate question arises here: if you hate the topic of your research, be it a car or perhaps a bicycle helmet, how will you as an editor or reviewer deal with a paper that says anything positive about cars or helmets?

Participation in heated scientific debates as a conflict of interest

Participation in heated scientific debates has also been mentioned as a possible source of conflict of interest (Rockwell 2006). The reason for this participation might be related to strong personal beliefs, including hate, as discussed previously, but it can simply be related to the researcher’s work in general or even to specific article(s).

An interesting situation concerns journals’ treatment of commentaries that focus on a particular paper. Some journals do not peer review commentaries, while others treat them in the same way they would the submission of a research paper. Inviting the authors of the target article to review a commentary might seem logical because these authors are experts and obviously highly motivated to provide a detailed assessment of the commentary. However, the inviting editor must be mindful of their biased perspective and carefully evaluate their peer review. If such reviews contain personal insults or question the authors’ motivation, they should always be discarded because they are clear signs of a serious COI. Journals have several different ways of handling such submissions:

* EiC accepts/rejects a commentary without peer review, and invites the authors of the target article to respond after possible acceptance

* EiC invites the authors of the target article to respond, and then makes a decision whether they will publish together both commentary and response

* EiC sends a commentary to an external peer review and after possible acceptance invites the authors of the target article to respond

* In addition to external reviewers, EiC includes the authors of the target article as peer reviewers and based on their review (not a formal response) makes a decision on the submitted commentary.

All of these approaches have pros and cons; however, it is of the utmost importance that the journal’s guidelines should be clear about the procedure especially if the authors of the target articles are routinely invited to provide a peer review of a commentary. A recent COPE document offers some guidelines about “handling of post-publication critiques” (COPE Council 2021).

Who is qualified to be a peer reviewer? Personal interests are not the proof of expertise

A reviewer should be an expert in the research field with at least some “proof” of having enough specialist knowledge needed to evaluate a manuscript. The proof is a reasonable publication record in the research field. In one of my recent submissions, the journal invited a non-researcher and the founder of a cycling advocacy group to review our paper in which, among other issues, we exposed this reviewer’s careless citation of an alleged finding from a secondary source. One could argue whether the single publication (a blog text) that this reviewer had constitutes a good enough proof of the required specialist knowledge.

Recruiting peer reviewers has been increasingly difficult in recent years in some journals (e.g., Fox, Albert, and Vines 2017), so it is possible that even the best journals sometimes adopt the take-what-you-can-get strategy. Unfortunately, the most motivated and willing reviewers could well be those who might benefit if a paper remains unpublished. The editors should be mindful of this possibility especially if the tone or content of the review supports it. The purpose of peer-review is not to ”kill the paper,” but to evaluate it in terms of its strengths and limitations.

Who can be an editor? Conflicts of commitments

Here I offer an example from my field of research regarding the connecting between the Journal of Transport and Health (JTH) and the organization called Transport and Health Study Group – THSG, which among other goals aims “To promote a more balanced approach to cycle safety and oppose cycle helmet legislation” [my underline]. Given that the journal’s EiC and several editorial board members belonged also to THSG, concerns about JTH’s editorial decisions being influenced by THSG’s goals were expressed in a blog post. It asked “ … will a paper that says anything positive about helmets or helmet legislation (backed by evidence) be peer-reviewed without this goal influencing the decision to publish? Also, will authors shy away from citing peer-reviewed articles that demonstrate evidence supportive of helmets or helmet laws?” (Olivier 2014)

In a public discussion the EiC replied: “In summary, many journals are associated with organizations but the society does not influence the journal. The BMJ does not take note of BMA policy” (Mindell 2014). However, this has provoked the following response (Churches 2014): “Yes, it is true that the British Medical Association has policies, and no, those policies don’t affect the editorial policies or decisions of the British Medical Journal, which the BMA publishes. But a key difference is that office bearers and policy-makers in the BMA are NOT the same people as the editors and editorial staff of the BMJ.”

The loyalty to one’s own advocacy group and own journal (i.e., science) might sometimes be in conflict. This is what is usually called “conflicts of commitments” (Wiersma, Kerridge, and Lipworth 2018a). To somewhat exaggerate this, imagine an EiC who belongs to an organization that is against gun control and promises they will treat equally submissions that are perceived as supportive or opposed to gun control. It is extremely difficult to prove that editors chose reviewers they knew would be negative (or positive) toward a particular paper or that someone’s bias has led to an unfair peer review; however, offensive language and knowingly violating journal rules should be taken very seriously if there is also another reason (e.g., belonging to an advocacy group) for suspecting serious bias.

How should publishers deal with official complaints – are nonfinancial COIs of secondary importance?

As Dalton (2001, 103) writes “ … it is often difficult to distinguish between genuine complaints and paranoia. Many editors can relate tales of anguished authors convinced that a reviewer had abused the process.” However, if a complaint contains evidence of a serious breach of journal guidelines (e.g., knowingly violating the journal’s double-blind policy), publishers should respond formally.

Some may argue the case of knowingly breaching the journal’s privacy rule is a minor or even an irrelevant matter considering how difficult it is to get an adequate response from publishers in more serious cases of “plagiarism, image, data, or ethical concerns” (Bik 2020). It is indeed possible that publishers give greater priority to more serious cases due to time constraints and simply ignore those perceived as minor. On the other hand, the way of communication can sometimes be perceived as unduly aggressive or unprofessional, leading to a label of a “difficult author” who can be ignored (see Smart 2020 for a discussion about difficult authors).

Small research fields and nonfinancial COIs

Bero and Grundy (2016, 4) assert that, “ … while personal relationships can lead to conflicts of interest, unlike conflicts of interest arising from financial ties, their effects rarely extend beyond the immediate situation.” However, this might not be true when a research field is small and narrow with a limited number of highly respected researchers who might shape the field both through their research work and also as peer-reviewers or editors. For example, one of my bicycle helmet papers was recently reviewed by the same reviewer for two journals indicating that the research field is indeed small. As someone who has extensively published on the topic of driver fatigue, I was also invited to review the same paper for two different journals. In small research fields, it is of the utmost importance to have a diverse editorial board, change EiCs in reasonable time frames, and perhaps not to select them from the same advocacy group.

The current “About the journal” section of mentioned JTH includes no information about the connection between the journal and THSG. The current EiC has also failed to mention he is on the board of THSG (EiC’s biography n.d.). However, on their website THSG states that “The THSG works with: The Journal of Transport and Health” (THSG n.d.). It is, therefore, unclear what the exact relationship between two is. Does, for example, THSG nominate EiCs and the publisher only approves them or does THSG have no influence other than through members who have dual roles both in the group and the journal? It is unclear why this relationship is not transparently described on the journal’s webpage.

Is transparency the key for nonfinancial COIs?

In a recent addendum, the editors of The Lancet (2021) wrote: “In line with guidance from the International Committee of Medical Journal Editors, medical journals ask authors to report financial and non-financial relationships that may be relevant to interpreting the content of their manuscript. There may be differences in opinion as to what constitutes a competing interest. Transparent reporting allows readers to make judgments about these interests.” However, readers can make such judgments only if a manuscript is published and only for the authors if the names of the handling editor and reviewers are not made public (Table 1). Furthermore, if reviewers do not sign their reviews, then it is impossible for authors to detect the possible peer-reviewers’ COIs and report them to editors. In a study examining COIs among grant reviewers for the American Institute of Biological Sciences, it was found that 35% of identified conflicts were self-reported, while 65% were identified by the institute staff (Gallo, Lemaster, and Glisson 2016).

How to recognize undeclared COI or seriousness of declared COI?

The problem of hostile and unprofessional reviews/reviewers has long been recognized (Lanier 2021; Resnik and Elmore 2016; Silbiger and Stubler 2019). The reasons for such behavior are naturally multiple and complex. Some reviewers might fail to spot their own unintended inappropriate language due to lack of time and pressure to comply with deadlines. Poor language and communication skills as well as cultural differences might produce unforeseen interpretations. Various personality traits such as aggressiveness, narcissism, and lack of self-awareness may also play a role.

However, I argue that the hostile, unethical, and biased behavior of reviewers and editors often arises from a serious nonfinancial COI, which then remains unrecognized or even ignored despite clear evidence of biased behavior. For example, using an editorial to mock your “opponents”; preventing these “opponents” from responding to this incorrect information; knowingly ignoring the journal’s guidelines; accepting peer-reviews with insults and questioning others’ motivation for writing their papers are all potential signs of serious COI. Even if nonfinancial COIs are self-reported, the declaration is not a license to submit a biased review with personal insults or to break the journal’s guidelines.

How to manage non-financial COI?

Bero and Grundy (2016, 3) argue that “if the only solution for a particular type of interest is recusal because the interest cannot be eliminated, this is not a conflict of interest but rather part of the researcher’s professional role or personal identity.” This is perhaps too strict a rule that does not take into account that the benefit-to-cost ratio of a recusal will be high and will not vary significantly across different roles which have power (Table 1). For example, declining to review a manuscript because of a non-financial COI (which does not have to be specified) will hardly produce any harm to a researcher (invitations to review are declined on a daily basis), while the benefit might be considerable. Even if a research field is small, there will be other qualified persons willing to perform the reviewing task. The same applies to grant reviewers. Similarly, all journals should already have in place a procedure to transfer manuscripts to other editors, so the difficulty of editors exempting themselves from dealing with certain submissions will be negligible. A problem, however, might arise if there is no diversity among editorial board members (e.g., if most belong to the same advocacy group).

This kind of pragmatic approach (low cost, high benefit) can easily be applied to those involved in peer-review. For authors, the situation might be somewhat more complicated as they must obviously be more specific about any interests that can cloud their objectivity than a reviewer who declines to review a specific paper. Further problems might arise if a journal uses a double-blind peer review where providing details about personal interests might reveal an author’s identity. Nevertheless, as mentioned previously, once a paper has been published, it can be scrutinized in the context of the authors’ overall activities, which are publicly known.

The main problem with non-financial COI, as stated in this paper, concerns those who have power in the peer-review process. The system that relies on the peer-reviewers’ honesty or their understanding of what constitutes a COI might not be fully efficient especially if the authors and larger academic audience do not even know about peer-reviewers’ identity. Furthermore, it is well known that most of the journals’ guidelines and rules regarding COI apply to authors and not to editors and reviewers (Resnik, Konecny, and Kissling 2017). Several major journals have recently “tightened rules on non-financial conflicts” (Nature editorial 2018) and require that all involved (authors, reviewers and editors) must declare possible competing interests (Nature n.d.). Whether this will improve the peer-review process remains to be seen.

Concluding remarks

In a situation where prestigious journals have a less than 10% acceptance rate, it is very unlikely that a paper without highly supportive reviews will be accepted. Reviews submitted by researchers or non-researchers with a serious (nonfinancial) conflict of interest are never going to be “highly supportive.” It is also unlikely that editors who hate a certain research topic or have conflict of commitments will be free of bias when selecting potential reviewers of papers perceived to be in conflict with their strong attitudes or goals of their advocacy group. In conclusion, ignoring or downplaying the nonfinancial COI of editors and peer-reviewers, especially regarding controversial issues in small research fields, is indeed dangerous and harmful to the scholarly community.

Disclosure statement

The author has been involved in a dispute with the editors of the mentioned Journal of Transport and Health regarding two of his submissions to this journal on the controversial topic of bicycle helmets. The present work has not received any direct funding. However, the author’s current projects (Road traffic suicides and Road user identities) are supported by several organizations, including the Automobile and Touring Club of Finland (ATCF), the Finnish Crash Data Institute (OTI), the Henry Ford foundation, Katsastustoiminnan Tukisäätiö, Kuorma-autoliikenteen Volvo Säätiö, Liikenneturva, Liikenneturvallisuuden Edistämissäätiö, Rahtarit, Liikennelääketieteen säätiö, and the Transport Workers’ Union AKT. Given that anti-helmet (law) advocates often claim that perceived pro-helmet researchers work for the benefit of the car industry, this funding might be perceived as COI.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.