Toward a Hybrid BCI: Self-Paced Operation of a P300-based Speller by Merging a Motor Imagery-Based “Brain Switch” into a P300 Spelling Approach

ABSTRACT

This study presents the self-paced operation of a brain–computer interface (BCI) speller, which can be voluntarily turned on/off by merging a motor imagery (MI)-based brain switch into a P300-based BCI speller. From an off state (idle state), the users can generate a “control signal” by consciously changing the cognitive state differential from the idle state to turn on a P300-based spelling system when he or she wants to spell words. With the system turned on, the user can spell words, and then, the spelling system can be voluntarily turned off and switched to the initial state using a command. In this paradigm, the participants tried to perform the two different cognitive tasks sequentially, rather than simultaneously, and multiple EEG components were processed sequentially. The practicability and effectiveness of the proposed approach were validated by eleven participants, and all of them achieved a satisfactory performance. For the P300 speller, they achieved an average PITR of 42.61 bits/min. The preliminary results indicated that the proposed hybrid BCI system with different mental strategies operating sequentially is feasible and has potential applications for practical self-paced control.

Acknowledgments

The authors would like to thank the subjects who volunteered to participate in the experiments.

Funding

This work is supported by the National Basic Research Program (Grant No 2015CB351706) and the National Natural Science Foundation of China (Grant No 91320202 and 91420302).

Additional information

Funding

This work is supported by the National Basic Research Program (Grant No 2015CB351706) and the National Natural Science Foundation of China (Grant No 91320202 and 91420302).

Notes on contributors

Yang Yu

Yang Yu was born in Liaoning, China in 1987. He received his BSc degree from Tianjin University in 2010 and his MSc degree from the National University of Defense Technology, China, in 2012, where he is currently working toward his PhD degree. His research interests include brain–computer interface and neuroimages.

Zongtan Zhou

Zongtan Zhou was born in Henan, China in 1969. He received BSc, MSc, and PhD degrees from the National University of Defense Technology, China, in 1990, 1994, and 1998, respectively. He was promoted to Professor in 2007. His research interests include brain–computer interface, cognitive neuroscience, image/signal processing, computer/biologicalvision, and neural networks.

Jun Jiang

Jun Jiang was born in Zhejiang, China in 1987. He received BSc, MSc, and PhD degrees from the National University of Defense Technology, Changsha, China in 2009, 2011 and 2016, respectively. His research interests include brain–computer interfaces and machine learning.

Erwei Yin

Erwei Yin was born in Heilongjiang, China in 1985. He received his BSc degree from Jilin University in 2008, and he received MSc and PhD degrees from the National University of Defense Technology, China in 2010 and 2015, respectively. His research interests include brain–computer interfaces and machine learning.

Kunjia Liu

Kunjia Liu was born in Shandong, China in 1994. She received her BSc degree from National University of Defense Technology in 2016, where she is currently working toward her MSc degree. Her research interests include brain–computer interfaces and machine learning.

Jingjun Wang

Jingjun Wang was born in Chongqing, China in 1991. He received the B.Sc from Beihang University, China in 2010. He is currently working toward the M.Sc degree in the National University of Defense Technology. His research interests include brain computer interface.

Yadong Liu

Yadong Liu was born in Gansu, China in 1977. He received BSc and PhD degrees from the National University of Defense Technology, Changsha, China in 2000 and 2006, respectively. His research interests include image/signal processing, computer/biological vision, system identification and control, and cognitive science.

Dewen Hu

Dewen Hu was born in Hunan in 1963. He received the BSc and MSc degrees from Xi’an Jiaotong University in 1983 and 1986, respectively. He got his PhD degree from the National University of Defense Technology in 1999. His research interests include cognitive science, brain–computer interface, and neural networks.