Towards a complete understanding of memory: theoretically important failures to reject the null hypothesis

One of the problems facing all scientific disciplines nowadays is the general requirement that in order to be published in prominent peer-reviewed outlets, the data being presented must contain statistically significant findings. One argument put forward in favour of this approach to publishing is that without statistically significant findings (at a minimum), there is no advance in science. Indeed, failures to reject the null hypothesis and failed replications have been routinely excised from many top publication outlets given their inherent ambiguity.

One serious and negative consequence of these publishing practices is what many have termed the “file-drawer” problem – that is, failures to replicate well-researched effects go unnoticed by the scientific community and remain in laboratory file cabinets. The important issue here is that if there are a sufficient number of failures to replicate that do not receive scientific attention, then studies that have found a particular effect become, unbeknownst to most of us, simply the “tip-of-the-iceberg” and just provide a skewed and unbalanced idea of the presence or absence of an effect in the population. In other words, as scientists we are not privy to the extent to which there is a failure to replicate and we must simply rely solely on the significant findings that have been published, coming to believe that these effects are real and robust.

Of course, failures to replicate must have as an impeccable set of standards for methodological and statistical rigor as those studies that have found these effects. Even then, if we do not have access to these studies, any attempt at meta-analyses to examine the true extent of an effect using only published research presents an incomplete picture of the robustness of such an effect. This problem has been circumvented by asking authors of meta-analyses to contact researchers to see whether they have “file-drawer” data sets that should also be considered for inclusion in such analyses. Although this can go some way to solving this problem, with the resurgence of analyses that improve inferences about null effects (e.g., Bayesian inference, see Wagenmakers et al., 2018; equivalence testing, Lakens, McLachie, Isager, Scheel, & Dienes, in press), it may now be a prudent time to invite researchers to contribute well-controlled experiments that fail to replicate well-known effects to the memory literature.

To this end, the editors of Memory welcome contributions to a special issue devoted to showcasing empirical examples of theoretically relevant null effects in memory research edited by Mark L. Howe and Henry Otgaar. The goal of this special issue is to provide examples of critical hypotheses, rigorously tested, that fail to confirm well-known memory phenomena (e.g., directed forgetting, eyewitness identification, false memory effects, reconsolidation effects, suggestibility, survival processing advantage, testing effects, think/no-think suppression effects, effects of stress and trauma on memory). This call builds on recent advances in Memory to encourage preregistered work using, for example, Registered Reports, and studies using such an approach are especially, but not exclusively, welcome. Indeed, submissions are not restricted solely to recently collected data. Our preference is to have exact replications where possible but we will also consider conceptual replications.

Authors are invited to submit proposals for manuscripts (Abstracts of 500 words) that address these issues and that utilise appropriate statistical procedures that address issues concerning failures to reject the null hypothesis (e.g., Bayesian inference and equivalence testing) by 1 June 2019. Please submit these Abstracts via email to both guest editors ([email protected] and [email protected]) and be sure to include all authors names and contact information. The Abstract should include the specific analytic tool(s) used to support null effects as well as the theoretical importance of the null effects. Abstracts will be reviewed by the guest editors and an additional referee where appropriate. All authors will be informed of the editors’ decision as to whether a full submission will be invited by 1 September 2019 and the deadline for submission of the completed manuscript will be 15 January 2020. Invited submissions are not guaranteed acceptance and will undergo a thorough external review as per the usual peer review process. Reviews will be solicited from authors of the original research that led to the discovery of these effects in memory. These reviewers will be invited to submit a commentary to accompany the main article if accepted. Before submission, authors should read the guidelines for articles published in Memory located at https://www.tandfonline.com/action/authorSubmission?journalCode=pmem20&page=instructions. We anticipate that the special issue will be published online in the first part of 2020 and will appear in the July–August issue in 2020.