In praise of descriptive research

In 2016, we asserted that “the replication crisis” in the human sciences was not a single problem, but rather a set of problems loosely related to the reliability of the claims that human scientists had made about their work (Bulbulia, Spezio, Sosis, & Wildman, 2016). We discussed several proposals for addressing the reliability problem such as pre-registration, meta-analysis, and the use of probabilistic statistics. We urged that, despite their individual merits, an indiscriminate application of such proposals risks falling into an all-or-nothing ideal of scientific progress. According to this all-or-nothing image, a hypothesis is either supported or it is not supported. We argued that science does not work that way. Every result is provisional; and even when results fail to meet a customary standard we can learn from them. We maintained there is no substitute for careful reporting and a clear accounting of inferential strategies (Bulbulia et al., 2016, p. 276)

Here, we expand on these claims by arguing for the importance of descriptive research. The distinction between “theoretical/confirmatory research” and “descriptive/exploratory research” is common across the human sciences but a narrow preference for research at the theoretical/confirmatory end of the spectrum should be challenged on at least three grounds.

First, theories and descriptions are interdependent. Not only are theories grounded in observations, every theory prefers certain types of observations over others. In his autobiography, Darwin wrote:

We spent many hours in Cwm Idwal, examining all the rocks with extreme care, as Sedgwick was anxious to find fossils in them; but neither of us saw a trace of the wonderful glacial phenomena all around us; we did not notice the plainly scored rocks, the perched boulders, the lateral and terminal moraines. Yet these phenomena are so conspicuous that … a house burnt down by fire did not tell its story more plainly than did this valley. If it had still been filled by a glacier, the phenomena would have been less distinct than they are now. (Darwin 1887, p. 25)

As Darwin points out, it is not just that theories guide observations; significant aspects of nature remain unobservable without theories. As the evolutionary cultural science of religion develops beyond the handful of theoretical paradigms that guide present practices, we anticipate that a rich and diverse array of descriptions will inform observations about religious cognition and culture.

Second, data exploration generates new understanding. John Tukey pointed out half a century ago that we can learn a great deal from plotting and describing data (Tukey, 1970). Tukey’s ideas have since resurfaced in various efforts to promote sanity in inference (Gelman, 2004). We applaud and encourage descriptive presentations of data, and invite authors to document carefully how exploratory efforts might have prompted them to new questions, insights, and analyses.

Third, psychological science needs experiments to describe the mind. One of our favorite studies in the history of psychological science is a 1908 paper by Yerkes and Dodson, “The Relation of Strength of Stimulus to Rapidity of Habit-formation.” This is the study that famously discovered that there is an inverted U-shape curve between arousal and performance: at both the low and high levels of arousal, performance is low, yet in the middle level of arousal performance is optimal. Did Yerkes and Dodson set about to test the theory of a U-shaped law? Not at all. The purpose of their 1908 study was to investigate how habits are acquired: “In connection with a study of various aspects of the modifiability of behavior in the dancing mouse a need for definite knowledge concerning the relation of strength of stimulus to rate of learning arose. It was for the purpose of obtaining this knowledge that we planned and executed the experiments which are now to be described” (R.M. & Dodson, 1908, p. 459 emphasis added). Few studies have been as generative of psychological insights as Yerkes and Dodson’s 1908 paper, which has even driven research in the evolutionary psychology of religion (e.g., Xygalatas et al., 2013). Where would the great new generative observations arrive in an intellectual climate that was not supportive of descriptive studies?

We as editors agree that Religion Brain & Behavior should promote research at the descriptive/exploratory end of the research spectrum, provided that authors clarify its relevance and importance. As always, the central focus of an original research article should be to set forth what an audience might learn from the research, and what research might follow.