Turning negative into positives! Exploiting ‘negative’ results in Brain–Machine Interface (BMI) research

ABSTRACT

Results that do not confirm expectations are generally referred to as ‘negative’ results. While essential for scientific progress, they are too rarely reported in the literature – Brain–Machine Interface (BMI) research is no exception. This led us to organize a workshop on BMI negative results during the 2018 International BCI meeting. The outcomes of this workshop are reported herein. First, we demonstrate why (valid) negative results are useful, and even necessary for BMIs. These results can be used to confirm or disprove current BMI knowledge, or to refine current theories. Second, we provide concrete examples of such useful negative results, including the limits in BMI-control for complete locked-in users and predictors of motor imagery BMI performances. Finally, we suggest levers to promote the diffusion of (valid) BMI negative results, e.g. promoting hypothesis-driven research using valid statistical tools, organizing special issues dedicated to BMI negative results, or convincing institutions and editors that negative results are valuable.

KEYWORDS:

• Negative results
• hypothesis
• models
• theory
• publication
• guidelines
• BCI
• BMI

Acknowledgments

The authors would like to acknowledge all the participants of the workshop at the Asilomar BCI meeting 2018, which is at the origin of the present paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

https://standards.ieee.org/industry-connections/neurotechnologies-for-brain-machine-interfacing.html.

1. In this manuscript, when we mention scientific results, whether positive or negative, we generally (unless stated otherwise) assume that they are scientifically valid, i.e. that they originate from rigorous scientific studies, and are free of bias and confounding factors. Indeed, for both positive and negative results to be relevant and useful, they naturally first need to be scientifically valid. Thus, for the sake of conciseness, in the remainder of this manuscript, both the terms ”negative results” and ”positive results” refer to ”valid negative results” and ”valid positive results”.

2. Well-known historical examples of perpetuated fallacies include when physicists believed the Earth to be flat or that the Sun orbited around the Earth.

3. Some works in that direction include [66,67] for BMI or [68,69] for Neurofeedback.

4. https://en.wikipedia.org/wiki/List_of_academic_journals_by_preprint_policy. Note that for some journals it matters which copyright license you publish your preprint under (e.g. at the time of writing, IOP publishing does not allow preprints that have been published under a Creative Commons license.)

5. http://www.sherpa.ac.uk/romeo/search.php.

Additional information

Funding

F.L. was partly supported by the European Research Council, with project BrainConquest grant [ERC-2016-STG-714567] and the French National Research Agency with project REBEL grant [ANR-15-CE23-0013-01]. R.C. was supported partially by the IEEE Brain Initiative and the IEEE Standards Association Industry Connections Program⁶.

https://standards.ieee.org/industry-connections/neurotechnologies-for-brain-machine-interfacing.html.

. C.J. was partially supported by the EPFL-Inria International Lab as well as by the Swiss National Foundation (SNF) for scientific research.