Development of a ternary hybrid fNIRS-EEG brain–computer interface based on imagined speech

References

• Nicolas-Alonso LF, Gomez-Gil J. Brain computer interfaces, a review. Sensors. 2012;12(2):1211–1279.
   PubMed Web of Science ®Google Scholar
• Sereshkeh AR, Trott R, Bricout A, et al. Online EEG classification of covert speech for brain–computer interfacing. Int J Neural Syst. 2017;27(08):1750033.
   PubMedGoogle Scholar
• Yousefi R, Rezazadeh Sereshkeh A, Chau T. Online detection of error-related potentials in multi-class cognitive task-based BCIs. Brain-Comput Interfaces. 2019;6:1–12.
   Web of Science ®Google Scholar
• Yousefi R, Sereshkeh AR, Chau T. Development of a robust asynchronous brain-switch using ErrP-based error correction. J Neural Eng. 2019;16:066042.
   Web of Science ®Google Scholar
• DaSalla CS, Kambara H, Sato M, et al. Single-trial classification of vowel speech imagery using common spatial patterns. Neural Networks. 2009;22(9):1334–1339.
   PubMed Web of Science ®Google Scholar
• Cramer SC, Lastra L, Lacourse MG, et al. Brain motor system function after chronic, complete spinal cord injury. Brain. 2005;128(12):2941–2950.
   PubMedGoogle Scholar
• Conson M, Sacco S, Sarà M, et al. Selective motor imagery defect in patients with locked-in syndrome. Neuropsychologia. 2008;46(11):2622–2628.
   PubMed Web of Science ®Google Scholar
• Schultz T, Wand M, Hueber T, et al. Biosignal-based Spoken Communication : A Survey. IEEE/ACM Trans Audio Speech Lang Process2015;14(8):1–15.
   Google Scholar
• Schultz T, Wand M, Hueber T, et al., Biosignal-based spoken communication: a survey. IEEE/ACM Trans Audio Speech Lang Process. 2017; 25:2257–2271
   Web of Science ®Google Scholar
• Brigham K, Kumar BVKV, “Subject identification from Electroencephalogram (EEG) signals during imagined speech,” IEEE 4th International conference of biometrics theory, Appl Syst BTAS 2010, Washington, DC, 2010.
   Google Scholar
• Wang L, Zhang X, Zhong X, et al. Analysis and classification of speech imagery EEG for BCI. Biomed Signal Process Control. 2013;8(6):901–908.
   Web of Science ®Google Scholar
• Torres-García AA, Reyes-García CA, Villaseñor-Pineda L, et al. Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification. Expert Syst Appl. 2016;59:1–12.
   Web of Science ®Google Scholar
• Kübler A, Mushahwar VK, Hochberg LR, et al. BCI Meeting 2005 - Workshop on clinical issues and applications. IEEE Trans Neural Syst Rehabil Eng. 2006;14(2):131–134.
   PubMed Web of Science ®Google Scholar
• Scholkmann F, Wolf M, Wolf U. The effect of inner speech on arterial CO2 and cerebral hemodynamics and oxygenation: A functional NIRS study Oxygen Transport to Tissue XXXV. In:  S Van Huffel et al editors. Advances in experimental medicine and biology; vol 789; 2013. p. 81–7.
   Google Scholar
• Gallegos-Ayala G, Furdea A, Takano K, et al. Brain communication in a completely locked-in patient using bedside near-infrared spectroscopy. Neurology. 2014;82:1930–1932.
   PubMed Web of Science ®Google Scholar
• Hwang H-J, Choi H, Kim J-Y, et al. Toward more intuitive brain–computer interfacing: classification of binary covert intentions using functional near-infrared spectroscopy. J Biomed Opt. 2016;21(9):091303.
   PubMed Web of Science ®Google Scholar
• Chaudhary U, Xia B, Silvoni S, et al. Brain–computer interface–based communication in the completely locked-in state. PLoS Biol. 2017;15(1):1–26.
   Web of Science ®Google Scholar
• Sereshkeh AR, Yousefi R, Wong AT, et al. Online classification of imagined speech using functional near-infrared spectroscopy signals. J Neural Eng. 2018;16(1):016005.
   PubMed Web of Science ®Google Scholar
• Ahn S, Jun SC. Multi-modal integration of EEG-fNIRS for brain-computer interfaces – current limitations and future directions. Front Hum Neurosci. 2017;11(October):1–6.
   PubMedGoogle Scholar
• Chiarelli AM, Zappasodi F, Di Pompeo F, et al. Simultaneous functional near-infrared spectroscopy and electroencephalography for monitoring of human brain activity and oxygenation: a review. Neurophotonics. 2017;4(04):1.
   Google Scholar
• Choi I, Rhiu I, Lee Y, et al. A systematic review of hybrid brain-computer interfaces: taxonomy and usability perspectives. PloS one, 2017;12(4),e0176674..
   Google Scholar
• Hong KS, Khan MJ. Hybrid brain-computer interface techniques for improved classification accuracy and increased number of commands: A review. Front Neurorobot. 2017;11(Jul):35.
   Google Scholar
• Mognon A, Jovicich J, Bruzzone L, et al. ADJUST: an automatic EEG artifact detector based on the joint use of spatial and temporal features. Psychophysiology. 2011;48(2):229–240.
   PubMed Web of Science ®Google Scholar
• Subasi A. EEG signal classification using wavelet feature extraction and a mixture of expert model. Expert Syst Appl. 2007;32(4):1084–1093.
   Web of Science ®Google Scholar
• Sereshkeh AR, Trott R, Bricout A, et al. Eeg classification of covert speech using regularized neural networks. IEEE/ACM Trans Audio Speech Lang Process. 2017;25(12):2292–2300.
   Web of Science ®Google Scholar
• Adeli H, Zhou Z, Dadmehr N. Analysis of EEG records in an epileptic patient using wavelet transform. J Neurosci Methods. 2003;123(1):69–87.
   PubMed Web of Science ®Google Scholar
• Ocak H. Automatic detection of epileptic seizures in EEG using discrete wavelet transform and approximate entropy. Expert Syst Appl. 2009;36(2 PART 1):2027–2036.
   Web of Science ®Google Scholar
• Subasi A. Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients. Expert Syst Appl. 2005;28(4):701–711.
   Web of Science ®Google Scholar
• Yamashita Y, Maki A, Koizumi H. Wavelength dependence of the precision of noninvasive optical measurement of oxy-, deoxy-, and total-hemoglobin concentration. Med Phys. 2001;28:1108–1114.
   PubMed Web of Science ®Google Scholar
• Lotte F, Congedo M, Lécuyer A, et al. A review of classification algorithms for EEG-based brain-computer interfaces. J Neural Eng. 2007;4(2):R1–R13.
   PubMed Web of Science ®Google Scholar
• Bharne PP. Classification techniques in brain computer interface : a review. An International Journal of Engineering & Technology (AIJET). 2015;2(1):1–7. http://www.aijet.in/ eISSN: 2394-627X
   Google Scholar
• Naseer N, Hong K-S. fNIRS-based brain-computer interfaces: a review. Front Hum Neurosci. 2015;9(January):1–15.
   PubMedGoogle Scholar
• Strother SC, Rasmussen PM, Churchill NW, et al. Stability and Reproducibility in fMRI Analysis. Pract Appl Sparse Model. 2012 (2009).
   Google Scholar
• Guo Y, Hastie T, Tibshirani R. Regularized linear discriminant analysis and its application in microarrays. Biostatistics, 2007;8(1):86–100.
   Google Scholar
• Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Appears in the International Joint Conference on Articial Intelligence (IJCAI), Montreal, Quebec, Canada, 1995.
   Google Scholar
• Ledoit O, Wolf M. Honey, I shrunk the sample covariance matrix. J Portf Manag. 2004;30(4):110–119.
   Web of Science ®Google Scholar
• Combrisson E, Jerbi K. Exceeding chance level by chance: the caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. J Neurosci Methods. 2015;250:126–136.
   PubMed Web of Science ®Google Scholar
• Koessler L, Maillard L, Benhadid A, et al. Automated cortical projection of EEG sensors: anatomical correlation via the international 10-10 system. Neuroimage. 2009;46(1):64–72.
   PubMed Web of Science ®Google Scholar
• Walsh B, Tian F, Tourville JA, et al. Hemodynamics of speech production: an fNIRS investigation of children who stutter. Sci Rep. 2017;7(1):1–13.
   PubMedGoogle Scholar
• Fazli S, Mehnert J, Steinbrink J, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. Neuroimage. 2012;59(1):519–529.
   PubMed Web of Science ®Google Scholar
• Blokland Y, Spyrou L, Thijssen D, et al. Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: an offline study in patients with tetraplegia. IEEE Trans Neural Syst Rehabil Eng. 2014;22(2):222–229.
   PubMed Web of Science ®Google Scholar
• Morioka H, Kanemura A, Morimoto S, et al. Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information. Neuroimage. 2014;90:128–139.
   PubMed Web of Science ®Google Scholar
• Shin J, Müller KR, Schmitz CH, et al. Evaluation of a compact hybrid brain-computer interface system. Biomed Res Int. 20172017;2017:1–11.
   Google Scholar
• Shin J, Kwon J, Im C-H. A ternary hybrid EEG-NIRS brain-computer interface for the classification of brain activation patterns during mental arithmetic, motor imagery, and idle state. Front Neuroinform. 2018;12(February):1–9.
   PubMedGoogle Scholar
• Dornhege G, Del Millán JR, Hinterberger T, et al. Toward brain-computer interfacing. Neural Inf Process Ser. MIT press; 2007 (December 2014):507.
   Google Scholar
• Yin X, Xu B, Jiang C, et al. A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. J Neural Eng. 2015;12(3):36004.
   Web of Science ®Google Scholar
• Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–1359.
   Web of Science ®Google Scholar