Reliability and External Validity of Neurobiological Experiments

Abstract

Reliability and external validity are two fundamental values that pose incompatible constraints on neurobiological experiments. The more reliability an experimental result achieves, the less external validity it earns, and vice versa. In this article, I propose an externalist interpretation of external validity: the external validity of an experimental result depends not only on how much complexity is built into an experimental design, but also on the relationship between the experimental result and other related experiments. This externalist interpretation, which explains how a neurobiological experiment can be reliable and externally valid simultaneously, suggests a new way to understand the epistemology of neurobiological experimentation.

Acknowledgements

I thank John Bickle, Jacqueline Sullivan, and two anonymous referees of this journal for their invaluable comments on earlier versions of this paper.

Notes

1 It is an externalist interpretation in the sense that it allows factors external to an experiment, i.e. the results of other relevant experiments, to affect the external validity of the experiment's result.

2 Hacking suggests that his coincidence argument is a special case of an abductive argument, or inference to the best explanation. He says, ‘This argument from coincidence may seem like a special case of the cosmic accident argument … Theories explain diverse phenomena, and it would be a cosmic accident if a theory were false and yet correctly predicted the phenomena. We “infer to the best explanation” that the theory is true. The common cause of the phenomena must be the theoretical entities postulated by the theory.’ (Hacking 1983, 201–202).

3 A similar protein inhibition experiment had also been conducted by Richter, Wolf, and Engelmann (2005) using slightly different protocols. Most of the differences between Richter, Wolf, and Engelmann (2005) and Kogan, Frankland, and Silva (2000) will be discussed in the following portion of the article.

4 Realists may object that epistemic external validity is nothing more than a consequence of ontological external validity. They suggest that an experimental result can help us to explain our phenomenon of interest outside laboratory only if the mechanism it represents actually exists. This claim, however, is not supported by the history of science. Theories involving nonexistent or not-yet-proved-to-exist mechanisms or theoretical entities can be found to make important contribution to the explanation of our world. For example, the concept of gene was coined by Wilhelm Johannsen in 1909. Before the discovery of the double-helix structure of DNA by Francis Crick and James Watson in 1953, scientists had no idea what a gene actually is or whether it exists. However, this did not stop them from using this concept to give successful explanations of the phenomenon of heredity and the mechanism of natural selection. How gene should be defined is still a controversial issue among contemporary philosophers of biology. (For an overview of how the concept of gene has changed in the history of genetics, see Rheinberger and Müller-Wille 2009.) However, no one will question the power of a biological explanation simply because it involves the concept of gene. Laudan (1981) gives a list of theories that gave successful explanations and were non-referential with respect to many of their central concepts. Examples include the crystalline spheres of ancient and medieval astronomy, the humoral theory of medicine, the effluvia1 theory of static electricity, etc. These examples at least give us good reasons to believe that a conceptual distinction between the ontological and epistemological senses of the external validity of an experimental result is not ill-founded and could be beneficial in understanding the explanatory power of experimental results.