“Choice-supportive bias” in science: Explanation and mitigation

ABSTRACT

Mistakes in scientific research, their origins, and the possible solutions have always been noteworthy. On the other hand, researchers in some fields have focused on cognitive biases and their interventions in thinking and information processing. In this article, by bridging between these two issues, i.e., scientific research mistakes and cognitive bias, it will be first shown that scientific research requires scientists’ choices in various stages such as subject selection, data treatment, theory determination, and technological design. Then, we show the involvement of choice-supportive bias in scientific inquiry as one of the potential cognitive limitations of science. Finally, three ways are discussed to mitigate the effects of this bias in science: more scores to self-criticism publications, recording of the shortcomings of the research by the researcher before publishing, and emphasizing control processes before researcher choices in scientific inquiry.

KEYWORDS:

• Research integrity
• mistake
• negligence
• misconduct
• cognitive bias
• post-purchase rationalization

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

1. There are several types of dual process theories. The details and technical properties of these theories do not always match exactly; nevertheless, there are clear family resemblances (Stanovich and West 2000).

2. Other names: system 1, implicit (tacit) system, intuitive system, emotional system, experiential system, or heuristic system.

3. Other names: system 2, explicit system, deliberate system, rule-based system, rational system, or analytic system.

4. For example, publication bias is defined as “the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or the strength of the study findings” (Ekmekci 2017). In other words, either the researcher tries to present an article whose results are of interest to reviewers; or editors and reviewers publish articles based on the findings, not on the methods and their scientific values. One sign of this bias is the tendency to send or publish articles with positive results. This tendency provides the possibility of some scientific misconduct and ignores some potentially useful research studies. It should be noted that there may be many sources to publication bias, some of which are not cognitive but are systemic (see, e.g., Ioannidis 2005).

5. A historical example is illustrative. Observational data for the Uranus and Mercury orbits did not match Newton’s prediction. Scientists had two choices: theory change or making auxiliary assumptions (to observation interpretation). In the first case, an auxiliary assumption (the existence of a new planet) was made and scientists succeeded (Neptune’s discovery). In the second case, they put forward the auxiliary assumption but failed until relativistic physics was introduced and explained this observation (Chalmers 2013, 73, 130).

6. A well-known historical case can well demonstrate that data validity depends on many external factors. To test Einstein’s theory of general relativity, Eddington and his group observed the 1919 eclipse simultaneously at two points on earth. They suppose that each of these observations could confirm or disprove the theory. Eddington went to South Africa and sent a group to Brazil. Eddington’s telescopes had low quality, and perchance, he encountered inappropriate weather conditions (clouds) on the experiment’s day. They took two photographs in South Africa and eighteen ones in Brazil. Eddington’s photos confirmed relativity and other photos rejected relativity. But because Eddington believed in the correctness of the relativity theory, he relied on more uncertain data and preferred the two photographs to eighteen opposing images and stated that the observational data confirm the relativity theory. On the other hand, he was a distinguished scientist in the scientific community, especially in general relativity. Eddington was said to be the only one who understood relativity after Einstein. Therefore, the scientific community accepted his view (Collins and Pinch 2012; Almassi 2009).

7. A well-known historical case can well illustrate the role of choice in technology design. In the 1990s, CFC12 gases were banned for implementation in refrigeration units due to their environmental harms. Engineers came up with a new design. In addition to having the necessary thermodynamic properties, the alternative gas should meet the following values: safety (low flammability), health (non-toxicity), and also environmental friendliness (low potential to destroy the ozone layer and generate heat). After research, engineers achieved a set of hydrocarbon-based gases visible in the form of a triangle. Methane was located at the top of the triangle. As we went down and left, chlorine atoms were mostly substituted for hydrogen atoms, and the more we went down and right, fluorine atoms were replaced for more hydrogen atoms. It is worth noting that there were many options (gases) in this triangle, all of which satisfied the thermodynamic requirements, but they had different side effects. Choosing a gas from each part of the triangle was accompanied by weighting one value and ignoring other ones. Therefore, there was an inevitable and permanent compromise among the three values considered in this design, namely “safety,” “health,” and “environmental friendliness.” As each one increased, the two other ones decreased (Van de Poel and Royakkers 2011, 178). Engineers eventually had to choose a gas from several options.

8. For example, after buying a car based on a set of criteria, the purchaser over-trusts his/her purchase and rationalizes his/her choice. The purchaser is sure that the car passed more criteria than it actually did. It should be noted that the first choice may be rational or irrational but supporting the choice, just because it was his/her choice, is irrational. From the dual-process point of view, feelings and thoughts which are arisen from the automatic system mode of thinking may be effective during decision making processes, especially under uncertainty (Kahneman 2011). Chooser overconfidence in his/her previous choices is a cognitive bias that can be effective during new choices.

9. It is worth mentioning that it is not impossible in science to support a false idea or reject a right idea. There are many cognitive processes (in the automatic system) that may help support a false idea or reject a right idea, such as confirmation bias. In this bias, one’s tendency to research, interpretation, attention, and information reminding is in a manner that confirms his/her previous assumptions and beliefs or at least does not oppose it (Plous 1993; Del Vicario et al. 2017). In other words, people are affected by this bias when they unjustifiably pay attention only to information that confirms their previous beliefs and does not oblige themselves to change their previous beliefs. Here, Phlogiston theory in chemistry (Chalmers 2013, 72, 106) or other well-known challenging cases in science (Collins and Pinch 2012) can be considered good examples.