Two sides of the same coin: adaptation of BCIs to internal states with user-centered design and electrophysiological features

Figures & data

Figure 1. Checklist of the user-centered design (UCD) step-by-step process cycle adapted for brain-computer interface (BCI) optimization intended for end-users with exemplary questions for each step [based on 11,37]. It has been noted that the UCD implies a careful definition and selection of the targeted end-users. An algorithm for such selection was recently suggested [38]. On a further note, another workshop at the 8th International BCI Meeting 2021 focused on ideas ‘Toward an international consensus on user characterization and BCI outcomes in settings of daily living’ with an accompanying database, summarizing user factors and outcome measures: https://www.notion.so/6c11535322d04977a14cfaa60ba5494f?v=4c92db74d5854f30bef197b8a9cdd327.

Table 1. Overview of usability and BCI-relevant aspects (with examples for relevant standardized metrics) as well as their assessment times with potential end-users (with exemplary studies using these measures) and their assessment times in the selected early-stage BCI development study example with healthy non-end-users [22], based on earlier work [11,37]

Usability aspects	BCI-relevant aspects [with examples for relevant standardized metrics]	Assessment times with potential end-users [with exemplary studies using these measures]	Assessment times in the selected early stage BCI development study example with healthy non-end-users [22]
Effectiveness (how accurate and complete a user can accomplish BCI control)	Accuracy [% correct responses] [preferably measured online via concrete task results vs. offline via estimation from previously collected data)	Each session [many studies, extensively discussed by 39]	Over each session
Efficiency (relating the costs invested by the user, i.e. effort and time, to effectiveness]	Information transfer rate [ITR) [bits/min]	Each session [many studies, extensively discussed by 39]	Over each session
	Adjustment of the ITR: Utility metric [bits/min = 0 if effectiveness < 50%]	Each session [40]	Applied in one occasion of a temporarily demotivated participant (with an online accuracy score of < 30%]
	Subjective Workload [NASA-TLX]	Each session/task [6]	End of each session
Satisfaction (perceived comfort and acceptability while using the BCI)	General aspects of assistive technology [QUEST 2.0, expandable by four additional BCI-related items operationalizing reliability, learnability, speed, and esthetic design]	End of prototype testing [27,40–44]	-(not included due to time constraints)
	Overall satisfaction [visual analog scale ranging from 0–10]	Each session [45,46]	End of each session
	Interview [semistructured; free]	End of prototype testing [47]	“Mini interview” at the end of each session (“Could you imagine using a BCI for communication? What problems would need to be addressed? What would be major improvements? Did you notice anything else?”)

Notes. NASA-TLX = NASA-Task Load Index [53,54]. QUEST = Quebec User Evaluation of Satisfaction with Assistive Technology [82].

Table 2. Overview of selected aspects to potentially optimize BCI user motivation and performance as implemented in the selected study example [22], based on earlier study design guidelines [31,32] and the idea to use elements inspired by a popular science fiction movie franchise (pictures, sounds, etc.), in which characters can use the powers of their mind to positively interact with their environment via non-muscular pathways (‘Star Wars’), to create an overarching and motivating theme [48]

Suggested properties of a good instructional design	Corresponding design aspects in the study example [22]
Feedback
Non-evaluative and supportive feedback that conducts to a feeling of competence	Positive feedback sound after successful BCI use (not given after nonsuccessful BCI use) as well as more elaborated non-evaluative and supportive feedback from the experimenter during breaks
Engaging feedback and environment	Positive ‘Star Wars’ feedback sound (from a popular robot character, based on a mix of harmonic electronic and cheerful baby sounds)
Explanatory and specific feedback	Adjustable breaks during sessions, allowing time for verbal feedback by the experimenter as well as intermediate analysis of potential problems
Instruction
Goals should be clearly defined	Careful briefing and debriefing (explanation of paradigm and research purpose, suggesting strategies for successful use, concluding analysis of potential problems)
The meaning of the feedback should be explained	Illustration and explanation of the study specific BCI signal and ways of subjectively influencing it
The skill to be learned should be demonstrated	Demonstration of (previously recorded) successful BCI use of a end-user (also to illustrate the practical relevance of this line of research)
Task
Motivation and positive emotions promote learning	‘Star Wars’ theme (absolving ‘mission’ that were inspired by movie scenes), creation of a positive atmosphere (little refreshment breaks)
Need for autonomy and work at user’s own pace, adaptation of training procedure to the user	Adjustable breaks during sessions, allowing time to drink or snack little refreshments, eventually chance to try an alternative BCI version (auditory vs. tactile)
Need for variability over tasks and problems, progressive and adaptive tasks	Varying ‘missions’ (each one to a new ‘Star Wars’ movie scene), new BCI calibration at the beginning of each session (allowing users to reach online accuracies > 70% without guaranteeing 100%)

Notes. Originally, these suggestions were not meant for P300-based BCIs as utilized in the selected study example [22], since it seemed that they did not require human training [32]. However, this was before the relevance of training effects could be shown for auditory and tactile P300-based BCIs [22,49–52,84].

Figure 2. An example of the brain-computer interface (BCI) control cycle and how adaptation to the internal state of the user could be included. The user generates a control signal that is decoded by the BCI and provided as feedback to the user. In this example, the BCI is adapted by applying user-centered design (UCD) to determine the preferences and abilities of the user via self-report measures (such as the NASA-Task Load Index [NASA-TLX; 53,54], the System Usability Scale [SUS; 85,86], the BCI version of the Questionnaire for Current Motivation in Learning and Performance Situations [QCM-BCI; 18,35,36], visual analogue scales (VAS), or using an interview approach) and also via the detection of electrophysiological signal features (such as electroencephalographic oscillations, signal diversity and connectivity) by applying machine learning to determine the current state of the user.

Kübler A, Holz EM, Riccio A, et al. The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications. PloS One. 2014;9(12):e112392.

 PubMed Web of Science ®Google Scholar

Kübler A, Nijboer F, and Kleih S. Chapter 26—Hearing the needs of clinical users. In: editors Ramsey NF, Del J, and Millán R. Handbook of Clinical Neurology. Oxford, UK: Elsevier; 2020. Vol. 168. 353–368.

 Google Scholar

Kübler A, Holz EM, Sellers EW, et al. Toward independent home use of brain-computer interfaces: a decision algorithm for selection of potential end-users. Arch Phys Med Rehabil. 2015;96(3):S27–S32.

 PubMed Web of Science ®Google Scholar

Ziebell P, Stümpfig J, Eidel M, et al. Stimulus modality influences session-to-session transfer of training effects in auditory and tactile streaming-based P300 brain–computer interfaces. Sci Rep. 2020;10(1):11873.

 PubMed Web of Science ®Google Scholar

Wolpaw JR, Birbaumer N, McFarland DJ, et al. Brain–computer interfaces for communication and control. Clin Neurophysiol. 2002;113(6):767–791.

 PubMed Web of Science ®Google Scholar

Zickler C, Halder S, Kleih SC, et al. Brain Painting: usability testing according to the user-centered design in end users with severe motor paralysis. Artif Intell Med. 2013;59(2):99–110.

 PubMed Web of Science ®Google Scholar

Riccio A, Holz EM, Aricò P, et al. Hybrid P300-Based Brain-Computer Interface to Improve Usability for People With Severe Motor Disability: electromyographic Signals for Error Correction During a Spelling Task. Arch Phys Med Rehabil. 2015;96(3):S54–S61.

 PubMed Web of Science ®Google Scholar

Zickler C, Riccio A, Leotta F, et al. A Brain-Computer Interface as Input Channel for a Standard Assistive Technology Software. Clin EEG Neurosci. 2011;42(4):236–244.

 PubMed Web of Science ®Google Scholar

Holz EM, Botrel L, Kaufmann T, et al. Long-term independent brain-computer interface home use improves quality of life of a patient in the locked-in state: a case study. Arch Phys Med Rehabil. 2015a;96(3):S16–S26.

 PubMed Web of Science ®Google Scholar

Holz EM, Botrel L, Kübler A. Independent home use of Brain Painting improves quality of life of two artists in the locked-in state diagnosed with amyotrophic lateral sclerosis. Brain-Comput Interfaces. 2015b;2(2–3):117–134.

 Web of Science ®Google Scholar

Vasilyev A, Liburkina S, Yakovlev L, et al. Assessing motor imagery in brain-computer interface training: psychological and neurophysiological correlates. Neuropsychologia. 2017;97:56–65.

 PubMed Web of Science ®Google Scholar

Grier RA (2015). How High is High? A Meta-Analysis of NASA-TLX Global Workload Scores. Proceedings of the Human Factors and Ergonomics Society Annual Meeting Los Angeles, CA, US, 59( 1),1727–1731.

 Google Scholar

Hart SG. NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2006;50(9):904–908.

 Google Scholar

Demers L, Weiss-Lambrou R, Ska B. The Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0): an overview and recent progress. Technol Disabil. 2002;14(3):101–105.

 Google Scholar

Chavarriaga R, Fried-Oken M, Kleih S, et al. Heading for new shores! Overcoming pitfalls in BCI design. Brain-Comput Interfaces. 2017;4(1–2):60–73.

 Web of Science ®Google Scholar

Lotte F, Larrue F, Mühl C. Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design. Front Hum Neurosci. 2013;7:568.

 PubMed Web of Science ®Google Scholar

Lotte F, Renard Y, and Lécuyer A (2008 05/02/2022, September 19). Self-Paced Brain-Computer Interaction with Virtual Worlds: a Quantitative and Qualitative Study “Out of the Lab”. 4th international Brain Computer Interface Workshop and Training Course. https://hal.inria.fr/inria-00304340

 Google Scholar

Allison BZ, Kübler A, Jin J. 30+ years of P300 brain–computer interfaces. Psychophysiology. 2020;57(7):e13569.

 PubMed Web of Science ®Google Scholar

Brooke, John. ”Sus: a ”quick and dirty” usability.” Usability evaluation in industry 189.3 (1996).

 Google Scholar

Bangor A, Kortum P, Miller J. Determining What Individual SUS Scores Mean: adding an Adjective Rating Scale. J Usability Stud. 2009;4(3):10.

 Google Scholar

Kleih SC, Nijboer F, Halder S, et al. Motivation modulates the P300 amplitude during brain–computer interface use. Clin Neurophysiol. 2010;121(7):1023–1031.

 PubMed Web of Science ®Google Scholar

Nijboer F, Furdea A, Gunst I, et al. An auditory brain–computer interface (BCI). J Neurosci Methods. 2008;167(1):43–50.

 PubMed Web of Science ®Google Scholar

Rheinberg F, Vollmeyer R, Burns BD. QCM: a questionnaire to assess current motivation in learning situations. Diagnostica. 2001;47(2):57–66.

 Web of Science ®Google Scholar