Augmenting the Instant Messaging Experience Through the Use of Brain–Computer Interface and Gestural Technologies

REFERENCES

• Birbaumer , N. , Kubler , A. , Ghanayim , N. , Hinterberger , T. , Perelmouter , J. , Kaiser , J. , … and Flor , H. 2000 . The thought translation device (TTD) for completely paralyzed patients . IEEE Transactions on Rehabilitation Engineering , 8 : 190 – 192 .
   PubMedGoogle Scholar
• Brave , S. and Dahley , A. 1997 . inTouch: A medium for haptic interpersonal communication . Extended Abstracts on Human Factors in Computing Systems , : 363 – 364 .
   Google Scholar
• Cernea , D. , Olech , P.S. , Ebert , A. and Kerren , A. 2011 . EEG-based measurement of subjective parameters in evaluations . Proceedings of the Fourteenth International Conference on Human Computer Interaction International , : 279 – 283 .
   Google Scholar
• Chanel , G. , Kierkels , J.J.M. , Soleymani , M. and Pun , T. 2009 . Short-term emotion assessment in a recall paradigm . International Journal of Human-Computer Studies , 67 : 607 – 627 .
   Web of Science ®Google Scholar
• Chanel , G. , Rebetez , C. , Betrancourt , M. and Pun , T. 2008 . Boredom, engagement and anxiety as indicators for adaptation to difficulty in games . Proceedings of the Twelfth International Conference on Entertainment and Media in the Ubiquitous Era , : 13 – 17 .
   Google Scholar
• Chanel , G. , Rebetez , C. , Bétrancourt , M. and Pun , R. 2009 . Emotion assessment from physiological signals for adaptation of game difficulty . Systems, Man and Cybernetics, Part A: Systems and Humans , : 1 – 12 .
   Google Scholar
• Csikszentmihalyi , M. 1990 . Flow: The psychology of optimal experience , New York , NY : Harper Perennial .
   Google Scholar
• Derks , D. , Bos , A. E. and von Grumbkow , J. 2008 . Emoticons and online message interpretation . Social Science Computer Review , 26 : 379 – 388 .
   Web of Science ®Google Scholar
• El Kaliouby , R. and Robinson , P. 2004 . FAIM: Integrating automated facial affect analysis in instant messaging . Proceedings of the Ninth International Conference on Intelligent User Interfaces , : 244 – 246 .
   Google Scholar
• Fabri , M. 2006 . Emotionally expressive avatars for collaborative virtual environments (Unpublished doctoral thesis) , Leeds , , UK : Leeds Metropolitan University .
   Google Scholar
• Fabri , M. and Moore , D. 2005 . Is empathy the key? Effective communication via instant messaging . Proceedings of the Eleventh International Conference on Networking Entities. ,
   Google Scholar
• Gibert , G. , Pruzinec , M. , Schultz , T. and Stevens , C. 2009 . Enhancement of human computer interaction with facial electromyographic sensors . Proceedings of the Twenty-First Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7 , : 421 – 424 .
   Google Scholar
• Gilleade , K. , Dix , A. and Allanson , J. 2005 . Affective videogames and modes of affective gaming: Assist me, challenge me, emote me . Proceedings of the Digital Games Research Association: Changing Views – Worlds in Play, June 16--20, 2005, Vancouver, Canada. ,
   Google Scholar
• Gürkök , H. and Nijholt , A. 2012 . Brain–computer interfaces for multimodal interaction: A survey and principles . International Journal of Human-Computer Interaction , 28 : 292 – 307 .
   Web of Science ®Google Scholar
• Holzapfel , H. , Fuegen , C. , Denecke , M. and Waibel , A. 2002 . Integrating emotional cues into a framework for dialogue management . Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces , : 141 – 148 .
   Google Scholar
• Khalili , Z. and Moradi , M. H. 2009 . Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of EEG . Proceedings of the 2009 International Joint Conference on Neural Networks , : 1571 – 1575 .
   Google Scholar
• Kim , K. H. , Bang , S. W. and Kim , S. R. 2004 . Emotion recognition system using short-term monitoring of physiological signals . Medical and Biological Engineering and Computing , 42 : 419 – 427 .
   PubMed Web of Science ®Google Scholar
• Ko , M. , Bae , K. , Oh , G. and Ryu , T. 2009 . A study on new gameplay based on brain-computer interface . Proceedings of the Digital Games Research Association: Breaking New Ground: Innovation in Games, Play, Practice and Theory, September 1--4, 2009, London, UK. ,
   Google Scholar
• Lewis , C. and Fabos , B. 2005 . Instant messaging, literacies, and social identities . Reading Research Quarterly , 40 : 470 – 501 .
   Web of Science ®Google Scholar
• Li , Y. , Zhang , J. , Su , Y. , Chen , W. , Qi , Y. , Zhang , J. and Zheng , X. 2009 . P300 based BCI messenger . Proceedings of the International Conference on Complex Medical Engineering, 1–5. ,
   Google Scholar
• Lievesley , R. , Wozencroft , M. and Ewins , D. 2011 . The Emotiv EPOC neuroheadset: an inexpensive method of controlling assistive technologies using facial expressions and thoughts? . Journal of Assistive Technologies , 5 : 67 – 82 .
   Google Scholar
• Lotte , F. , Congedo , M. , Lecuyer , A. , Lamarche , F. and Arnaldi , B. 2007 . A review of classification algorithms for EEG-based brain-computer interfaces . Journal of Neural Engineering , 4 : R1 – R13 .
   Web of Science ®Google Scholar
• Luor , T. , Lu , H. P. , Wu , L. L. and Tao , Y. H. 2010 . The effect of emoticon in simplex and complex task-oriented communication—An empirical study of instant messaging . Computers in Human Behavior , 26 : 889 – 895 .
   Web of Science ®Google Scholar
• Mandryk , R. L. and Atkins , M. S. 2007 . A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies . International Journal of Human-Computer Studies , 65 : 329 – 347 .
   Web of Science ®Google Scholar
• McFarland , D. J. and Wolpaw , J. R. 2008 . Brain–computer interface operation of robotic and prosthetic devices . Computer , 41 ( 10 ) : 48 – 52 .
   Google Scholar
• Morris DiMicco , J. , Lakshmipathy , V. and Fiore , A. T. 2002 . Conductive chat: Instant messaging with a skin conductivity channel . Proceedings of the Conference on Computer Supported Cooperative Work. pp. 193--194. ,
   Google Scholar
• Mugler , E. , Bensch , M. , Halder , S. , Rosenstiel , W. , Bogdan , M. , Birbaumer , N. and Kübler , A. 2008 . Control of an Internet browser using the P300 event-related potential . International Journal of Bioelectromagnetism , 10 : 56 – 63 .
   Google Scholar
• Nam , C. S. , Li , Y. and Johnson , S. 2010 . Evaluation of P300-based brain-computer interface (BCI) in real-world contexts . International Journal of Human–Computer Interaction , 26 : 621 – 637 .
   Web of Science ®Google Scholar
• Nam , C. S. , Schalk , G. and Jackson , M. M. 2010 . Current trends in brain computer interface (BCI) research and development . International Journal of Human–Computer Interaction , 27 : 1 – 4 .
   Web of Science ®Google Scholar
• Neviarouskaya , A. 2008 . AffectIM: An avatar-based instant messaging system employing rule-based sensing , Tokyo , , Japan : University of Tokyo . doctoral thesis
   Google Scholar
• Neviarouskaya , A. , Prendinger , H. and Ishizuka , M. 2007 . Recognition of affect conveyed by text messaging in online communication . Proceedings of the 2nd International Conference on Online Communities and Social Computing , : 141 – 150 .
   Google Scholar
• Nijboer , F. , Hoffmann , U. , Morin , F. , Carmien , S. , Leon , E. and Koene , R. 2009 . Affective brain–computer interfaces: Psychophysiological markers of emotion in healthy persons and in persons with Amyotrophic Lateral Sclerosis . Workshop on Affective Brain-Computer Interfaces and IEEE International Workshop on Social Signal Processing , : 1 – 11 .
   Google Scholar
• Nijboer , F. , Sellers , E. , Mellinger , J. , Jordan , M. , Matuz , T. , Furdea , A. , … and Kubler , A. 2008 . A P300-based brain-computer interface for people with amyotrophic lateral sclerosis . Clinical Neurophysiology , 19 : 1909 – 1916 .
   Google Scholar
• Nijholt , A. 2009 . BCI for games: A “state of the art” survey . Proceedings of the Seventh International Conference on Entertainment Computing , : 225 – 228 .
   Google Scholar
• Oakley , I. and O'Modhrain , M. S. 2002 . ContactIM: Exploring asynchronous touch over distance . Proceedings of the Conference on Computer Supported Cooperative Work, November 16--20, 2002, New Orleans, LA. ,
   Google Scholar
• Pennebaker , J. W. , Booth , R. and Francis , M. 2007 . Linguistic inquiry and word count: LIWC2007 operator's manual , University of Texas, Austin, Texas .
   Google Scholar
• Petrantonakis , P. C. and Hadjileontiadis , L. J. 2009 . On modeling user's EEG response during a human–computer interaction: A mirror neuron system-based approach . Proceedings of the Fourth European Conference of the International Federation for Medical and Biological Engineering , : 1241 – 1245 .
   Google Scholar
• Prendinger , H. and Ishizuka , M. 2005 . The empathic companion: A character-based interface that addresses users’ affective states . International Journal of Applied Artificial Intelligence , 19 : 267 – 285 .
   Web of Science ®Google Scholar
• Rebolledo-Mendez , G. , Dunwell , I. , Martinez-Miron , E. , Vargas-Cerdan , M. , Freitas , S. , Liarokapis , F. and Garcia-Gaona , A. 2009 . Assessing Neurosky's usability to detect attention levels in an assessment exercise . Proceedings of the Thirteenth International Conference on Human-Computer Interaction , : 149 – 158 .
   Google Scholar
• Rebsamen , B. , Guan , C. , Zhang , H. , Wang , C. , Teo , C. L. , Ang , M. and Burdet , E. 2010 . A brain-controlled wheelchair to navigate in familiar environments . Transactions on Neural Systems and Rehabilitation Engineering , 18 : 590 – 598 .
   PubMed Web of Science ®Google Scholar
• Sellers , E. W. , Vaughan , T. M. and Wolpaw , J. R. 2010 . A brain–computer interface for long-term independent home use . Amyotrophic Lateral Sclerosis , 11 : 449 – 455 .
   PubMed Web of Science ®Google Scholar
• Smith , R. W. and Gordon , S. A. 1996 . Pragmatic issues in handling miscommunication: Observations of a spoken natural language dialog system . Workshop on Detecting, Repairing, and Preventing Human-Machine Miscommunication , : 21 – 28 .
   Google Scholar
• Takahashi , K. 2004 . Remarks on emotion recognition from bio-potential signals . Proceedings of the Second International Conference on Autonomous Robots and Agents , : 186 – 190 .
   Google Scholar
• Tan , D. and Nijholt , A. 2010 . “ Chapter 1: Brain–computer interfaces and human-computer interaction ” . In Brain–computer interfaces , Edited by: Tan , D. and Nijholt , A. 3 – 19 . London , , UK : Springer Verlag .
   Google Scholar
• Thompson , E. R. 2007 . Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS) . Journal of Cross-Cultural Psychology , 38 : 227 – 242 .
   Web of Science ®Google Scholar
• Van Boxtel , A. 2010 . Facial EMG as a tool for inferring affective states . Proceedings of Seventh Edition of the International Conference on Methods and Techniques in Behavioral Research , : 104 – 108 .
   Google Scholar
• Voida , A. , Newstetter , W. C. and Mynatt , E. D. 2002 . When conventions collide: The tensions of instant messaging attributed . Proceedings of the Conference of Human Factors in Computing Systems , : 187 – 194 .
   Google Scholar
• Wolpaw , J. R. , Birbaumer , N. , McFarland , D. J. , Pfurtscheller , G. and Vaughan , T. M. 2002 . Brain–computer interfaces for communication and control . Clinical Neurophysiology , 113 : 767 – 791 .
   PubMed Web of Science ®Google Scholar
• Yeo , Z. 2008 . Emotional instant messaging with KIM . Extended Abstracts on Human Factors in Computing Systems , : 3729 – 3734 .
   Google Scholar
• Zander , T. O. , Kothe , C. , Welke , S. and Roetting , M. 2008 . “ Enhancing human-machine systems with secondary input from passive brain-computer interfaces ” . In Brain–computer interfaces , Edited by: Tan , D. and Nijholt , A. 181 – 199 . London , , UK : Springer .
   Google Scholar