Firstly, the title is wrong. There is no scientific bias since bias is, well, … unscientific. In fact, science can be defined as the least biased and the most productive method of obtaining knowledge.
Now, as an editor of an academic journal one has the luxury of testing some of one’s thoughts about bias. An editor becomes in fact the ultimate judge of scientific merit, a sort of regulator of what deserves to be accepted as good science. In addition, like all forms of power, this one regulates social integration, i.e. careers, and by extension, livelihoods. As I leave European Societies to the capable hands of a new team, I would like to share some of my conclusions over these six years with every reader. In fact, these conclusions might perhaps serve as bearing points for a healthy academic life in general.
1) Getting out of the pool. One of the ideas that seem to come naturally to every editor is to organise a pool of reviewers who will assess submitted articles, often in the form of an enlarged editorial board. Old biases die harder than new ones. The idea of the reviewer pool probably comes from the distant times of small scientific communities whose members had no efficient means of communicating. Although that time passed a century ago, the idea remained, most probably thanks to the comfort of having people you know and appreciate make a commitment to you. This sounds perfect. They are much more likely to accept your requests and you will know that you can trust their judgement. In other words, perfect double-sided bias.
Selection bias is the first to come to mind. Choosing people is choosing attributes. Of course, there are good and bad selections, depending on competence, pluralism and open-mindedness. Still, knowing or getting to know these people involves your values and you cannot undo that.
Selected bias is something we never think about. Those who are selected have a sense of being appreciated and respected. This often leads to the slippery path of prestige and self-importance. Most significantly, it leads to a sense of entitlement when the requests to review are repeated, as will be if the recipients are in a reviewer pool. Now, because you have repeatedly requested people for their assessment, you are less likely to reject that assessment. The more you do so, the more the reviewers will confirm the validity of their views. We all get a mutually reinforced sense of being right when our judgements are implemented. And believing that we are right is the quintessence of bias.
Ganging-up is not far. We have all known these moments when we are tempted to look at our students as an undifferentiated mass. It is a permanent struggle not to do so. A pool of reviewers in any form reaches rather soon a sense of ‘in-group’ comfort. When that is challenged, e.g. by authors who feel their articles’ assessments are wrong or unfair, it is very difficult to trouble that comfortable environment and put in doubt the established mode of operation. After all, these people have given over and over again their free time for your journal. You cannot just reject their contribution now. On top of it, they may turn against you for doing so. It is best to discreetly hold your ground and suggest that the article be submitted to another journal. There are many, so what you suggest is not really that bad, you are bound to think. Thus, you are now far, very far, from practising science.
2) Steering clear of recognition. Making decisions on other people’s work makes you think that you are somehow particularly qualified to do so. Also, if you succeed in making your journal more attractive or influential, you think that you deserve acknowledgement. All this can easily lead to the pursuit of a plan of influence instead of knowledge for your journal or – if you are weak – for yourself too.
The Halo effect is an obvious trap. You may genuinely think that inviting reputed colleagues to submit to your journal is a sound quality strategy. When you do so, you put yourself in an irreversibly biased position, since you are likely to avoid the rejection of their articles, consciously or unconsciously. That is very easy to do for an editor; you just assign the article to reviewers who are unlikely to disagree with the perspective of the author.
The Matthew effect has operated so much in science that everybody should be aware of it by now. However, you face it frequently and in many forms as an editor. Despite the fact that sociology articles are not usually co-authored by large teams, one can try and look if those who have done some of the work are relegated to a thanking footnote, or if a recently defended thesis leads to an article of unbalanced attribution of authorship, to put it mildly. Also, entire domains of work are often simplistically attributed to one person, thus perpetuating that effect. It is unsettling to think that we sociologists keep transmitting “big men” (and “big women”) traditions but we do, and countering that tendency is part of an editor’s role.
Season tickets are convenient but unhelpful. If a colleague does good work and they have published with your journal, they may reasonably think that they should continue to submit their articles to it. Although this does not seem compromising, it will most probably generate bias in itself, since readers will think that “this is the kind of subject or author” that the journal is interested in. Despite the bibliometric benefit that you may obtain, it is best to discourage colleagues from repeatedly submitting to the same journal. My experience is that they will understand, if you do that for the reasons that I mentioned here.
3 Treating disagreement as irrelevant. Scientific knowledge is not about consensus. To the extent that sociology can be scientific, assessing research must be done within the framework of reference that the specific researchers have chosen. It is belittling to think of your readers as being at risk of not understanding, as much as you do, the limitations of an article and the relativity of its conclusions.
Wholesale bias, instead of point-by-point examination, is a universal academic pitfall. The simple way to counter that bias is to separate method, analysis and conclusion and try to find a combination of reviewers who are likely to focus primarily on a different part of the article; e.g. ask someone from a cognate discipline to check methodology. Not liking a sound piece of work is confusing power with truth, a poisonous mixture when it comes to science.
Schools of thought may have been helpful when science was about a few individuals working independently. Contemporary interconnectedness renders them akin to structural limitations in one’s perspective, that is, regulators of bias. There is today, in the social sciences too, such seamless continuity between approaches, that maintaining symbolic frontiers can only be justified by relations of power. There are many journals whose primary goal is to maintain such frontiers and I am happy to say that European Societies is not one of them.
4 Anticipation bias is probably one of the most pernicious forms of distorting good judgement and the least admitted, even to ourselves. It is the current expression of the primeval need to know in advance that we are not at risk of being doubted, suspected or accused. Steering clear of controversy is the strongest silent temptation. Few of us will rock a boat in which we are comfortably seated, but this is exactly part of a good editor’s remit.
Flowing with the current is the expression of the Matthew effect in thematic areas. You must have noticed how many people are interested in areas that become influential, to the point of dissecting the slightest details of what we already know for certain, instead of exploring vast unknown aspects of human sociality. In many cases, this is simply ‘jumping on the band wagon’; regulating that undue fervour can in the most extreme cases be part of an editor’s job.
Inconvenient inquiries are among the best tests for the quality of an editor. To take a mild case, imagine that you receive an article which shows through the admitted methods of the discipline that most members of a specific racial minority believe that the negative stereotypes that concern them are justified. And suppose that you courageously forward this to reviewers who come back with nitpicking criticisms in order to have the article rejected, and tell you in their confidential comments that this is dangerous because it can be misinterpreted and could blow wind in the sails of racist discourses. You must find the courage to think the obvious, i.e., that this is a sociological journal and your colleagues are quite capable of understanding what the work is about and how stereotypes work. You do not need to protect anyone from knowledge, but you do need to protect them from silencing inconvenient knowledge.
5 Stifling run-of-the-mill approaches may sound particularly severe but cannot be avoided. We have reached the point where the pressure for producing publications is disconnected from both intellectual interest and practical relevance.
Paint-by-numbers articles. Careers are based on publications. Producing sound work that is not groundbreaking may be helpful. So, it increasingly happens that authors apply similar methods to similar datasets with regard to different variables; or, in qualitative work, the same problématique to similar social environments. Although this is not dishonouring, it cannot be a scientific priority either. Thus, requiring a degree of novelty should probably become an additional criterion for academic research. For example, we know by now that poverty generates disadvantages in every area, from nourishment to education; perhaps, looking at what generates poverty at the level of legitimate discriminations (e.g. the possession of skills) should be our current focus, instead of adding to what we all recognise as an established principle. If not, we just favour professional bias towards self-serving work, which is generally less useful to science than looking at new possibilities.
Self-referential influence is a particular problem when chasing after bibliometrics. Specialised journals tend to generate higher impact factors as authors in the same domain tend to cite each other. Essentially, this means that there is benefit to agreeing with others within the same sub-disciplinary paradigm. An editor’s role is probably more useful when he or she asks submitting authors to go well beyond such agreement in order to be considered for publication. Reviewers should also be made aware that an unfamiliar approach may lead to rejection much easier than a familiar one and cautioned against that tendency.
6) Survival vs ethics. Now, is it possible to do all the above and not be led to the pillory?
All these forms of bias do not exist without reason. The apes that we are probably reached their current state thanks to such biases, which are sophisticated products of evolution. They certainly have helped our ancestors preserve themselves on innumerable occasions. But they did not evolve to help us practice science. Science is an ‘un-natural’ pursuit and that is why it regularly meets with resistance. Social science is even more unnatural as it involves our certainties about ourselves. So, it is a healthy sign if an editor meets with dilemmas and challenges. Risk-taking is part of the job and a certain price might need to be paid for it. However, when one chooses independent reviewers and takes their views seriously into account, that social price is mitigated. Colleagues understand then that the response that they get from a journal is not biased even when they think it is wrong.
All in all, this is a job in which you survive not because you are liked but because you are trusted. As you go along and explain your modest capacity to help produce fair judgements via a community of peers, you will be pardoned for your mistakes and respected for your efforts. And that is the best way of surviving when one is not hungry for power and recognition.