New and emerging access technologies for adults with complex communication needs and severe motor impairments: State of the science

References

• Akcakaya, M., Peters, B., Moghadamfalahi, M., Mooney, A.R., Orhan, U., Oken, B., … Fried-Oken, M. (2014). Noninvasive brain-computer interfaces for augmentative and alternative communication. IEEE Reviews in Biomedical Engineering, 7, 31–49. doi:10.1109/RBME.2013.2295097
   PubMedGoogle Scholar
• Andresen, E.M., Fried-Oken, M., Peters, B., & Patrick, D.L. (2015). Initial constructs for patient-centered outcome measures to evaluate brain–computer interfaces. Disability and Rehabilitation: Assistive Technology, 11, 548–557. doi:10.3109/17483107.2015.1027298
   PubMed Web of Science ®Google Scholar
• Bacher, D., Jarosiewicz, B., Masse, N.Y., Stavisky, S.D., Simeral, J.D., Newell, K., … Hochberg, L.R. (2015). Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabilitation and Neural Repair, 29, 462–471. doi:10.1177/1545968314554624
   PubMed Web of Science ®Google Scholar
• Ball, L., Fager, S., & Fried-Oken, M. (2012). Augmentative and alternative communication for people with progressive neuromuscular disease. Physical Medicine & Rehabilitation Clinics, 23, 689–699. doi:10.1016/j.pmr.2012.06.003
   PubMed Web of Science ®Google Scholar
• Ball, L., Nordness, A., Fager, S., Kersch, K., Pattee, G., Beukelman, D. (2010). Eye gaze access of AAC technology for persons with amyotrophic lateral sclerosis. Journal of Medical Speech Language Pathology, 18, 11–23. Retrieved from https://www.atia.org/wp…/Research_Article-Eye_Gaze_Access_ALS_4AF258B.doc
   Web of Science ®Google Scholar
• Beukelman, D.R., Fager, S., Ball, L., & Dietz, A. (2007). AAC for adults with acquired neurologic conditions: A review. Augmentative and Alternative Communication, 23, 230–242. doi:10.1080/07434610701553668
   PubMed Web of Science ®Google Scholar
• Beukelman, D.R., & Mirenda, P. (2013). Augmentative and alternative communication: Supporting children and adults with complex communication needs (4th ed.). Baltimore, MD: Paul H. Brookes.
   Google Scholar
• Biswas, P., & Langdon, P. (2011). A new input system for disabled users involving eye gaze tracker and scanning interface. Journal of Assistive Technologies, 5, 58–66. https://doi.org/10.1108/17549451111149269
   Google Scholar
• Biswas, P., & Langdon, P. (2013, August 29–31). A new interaction technique involving eye gaze tracker and scanning system. Proceedings of the 2013 Conference South Africa, 67–70. doi:10.1145/2509315.2509322
   Google Scholar
• Blackstone, S. (1995). Repetitive strain injury and AAC. Augmentative Communication News, 8, 1–4. Retrieved from http://www.augcominc.com/newsletters/?fuseaction=newsletters&C=ACN
   Google Scholar
• Blackstone, S.W., Williams, M.B., & Wilkins, D.P. (2007). Key principles underlying research and practice in AAC. Augmentative and Alternative Communication, 23, 191–203. doi:10.1080/07434610701553684
   PubMed Web of Science ®Google Scholar
• Blankertz, B., Dornhege, G., Lemm, S., Krauledat, M., Curio, G., & Müller, K.R. (2006). The Berlin brain-computer interface: Machine learning based detection of user specific brain states. IEEE Transactions on Biomedical Engineering, 12, 581–607. doi:10.1109/TBME.2008.923152
   Google Scholar
• Brumberg, J.S., Burnison, J.D., & Pitt, K.M. (2016, July). Using motor imagery to control brain-computer interfaces for communication. In D. Schmorrow & C. Fidopiastis (Eds.), Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience. Toronto, Canada: Springer International Publishing Switzerland.
   Google Scholar
• Brumberg, J.S., Pitt, K.M., & Burnison, J.D. (2018). A non-invasive brain–computer interface for real-time speech synthesis: The importance of multimodal feedback. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26, 874–881. https://doi.org/10.1109/TNSRE.2018.2808425
   PubMed Web of Science ®Google Scholar
• Brumberg, J.S., Pitt, K.M., Mantie-Kozlowski, A., & Burnison, J.D. (2018). Brain–computer interfaces for augmentative and alternative communication: A tutorial. American Journal of Speech-Language Pathology, 27, 1–12. doi:10.1044/2017_AJSLP-16-0244
   PubMed Web of Science ®Google Scholar
• Caballero-Morales, S.O., & Trujillo-Romero, F. (2014). Evolutionary approach for integration of multiple pronunciation patterns for enhancement of dysarthric speech recognition. Expert Systems with Applications, 41, 841–852. doi:10.1016/j.eswa.2013.08.014
   Web of Science ®Google Scholar
• Chakrabarti, S., Sandberg, H.M., Brumberg, J.S., & Krusienski, D.J. (2015). Progress in speech decoding from the electrocorticogram. Biomedical Engineering Letters, 5, 10–21. doi:10.1007/s13534-015-0175-1
   Web of Science ®Google Scholar
• Chavarriaga, R., Fried-Oken, M., Kleih, S., Lotte, F., & Scherer, R. (2017). Heading for new shores! Overcoming pitfalls in BCI design. Brain-Computer Interfaces, 4, 1–14. doi:10.1080/2326263X.2016.1263916
   Web of Science ®Google Scholar
• Christensen, H., Cunningham, S., Fox, C., Green, P., & Hain, T. (2012). A comparative study of adaptive, automatic recognition of disordered speech, In INTERSPEECH 2012 Annual Conference of the International Speech Communication Association, 1776-1779. Retrieved from https://pdfs.semanticscholar.org/68b0/b9f6141172939bb778ccfb8afdf63c977f86.pdf
   Google Scholar
• Combaz, A., Chatelle, C., Robben, A., Vanhoof, G., Goeleven, A., Thijs, V., …Laureys, S. (2013). A comparison of two spelling brain-computer interfaces based on visual p3 and SSVEP in locked-in syndrome. PloS One, 8, 1–4. Retrieved from https://doi.org/10.1371/journal.pone.0073691
   Web of Science ®Google Scholar
• Engelke, C.R., & Higginbotham, D.J. (2013). Looking to speak: On the temporality of misalignment in interaction involving an augmented communicator using eye-gaze technology. Journal of Interactional Research in Communication Disorders, 4, 95–122. doi:10.1558/jircd.v4i1.95
   Google Scholar
• Fager, S., Bardach, L., Russell, S., & Higginbotham, J. (2012). Access to augmentative and alternative communication: New technologies and clinical decision-making. Journal of Pediatric Rehabilitation Medicine, 5, 53–61. doi:10.3233/PRM-2012-0196
   PubMedGoogle Scholar
• Fager, S., Beukelman, D., Fried-Oken, M., Jakobs, T., & Baker, J. (2012). Access interface strategies. Assistive Technology, 24, 25–33. https://doi.org/10.1080/10400435.2011.648712 doi:10.1080/10400435.2011.648712
   Web of Science ®Google Scholar
• Fager, S., Beukelman, D., Jakobs, T., & Hosom, J.P. (2010). Evaluation of a speech recognition prototype for speakers with moderate and severe dysarthria: A preliminary report. Augmentative and Alternative Communication, 26, 267–277. https://doi.org/10.3109/07434618.2010.532508
   PubMed Web of Science ®Google Scholar
• Fager, S.K., Jakobs, T., & Sorenson, T. (2018, January). Multimodal input to enhance access to technology. Paper presented at the annual Assistive Technology Industry Association (ATIA) conference. Orlando, FL.
   Google Scholar
• Fager, S.K., Sorenson, T., Butte, S., Nelson, A., Banerjee, N., & Robucci, R. (2017). Integrating end-user feedback in the concept stage of development of a novel sensor access system for environmental control. Disability and Rehabilitation: Assistive Technology, 19, 1–7. https://doi.org/10.1080/17483107.2017.1328615
   Google Scholar
• Farwell, L.A., & Donchin, E. (1988). Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology, 70, 510–523. doi:10.1016/0013-4694(88)90149-6
   PubMedGoogle Scholar
• Fried-Oken, M., Beukelman, D.R., & Hux, K. (2012). Current and future AAC research considerations for adults with acquired cognitive and communication impairments. Assistive Technology, 24, 56–66. doi:10.1080/10400435.2011.648713
   Web of Science ®Google Scholar
• Fried-Oken, M., Fager, S., & Jakobs, T. (2016, July). New directions in access to augmentative and alternative communication technologies for persons with minimal movements. Paper presented at Rehabilitation Engineering Society of North America’s annual convention. Arlington, VA.
   Google Scholar
• Fried-Oken, M., Jakobs, E., & Jakobs, T. (2018, January). Smart Predict: AAC app that integrates partner knowledge into word prediction. Paper presented at the annual Assistive Technology Industry Association (ATIA) conference. Orlando, FL.
   Google Scholar
• Haghi, M., Thurow, K., & Stoll, R. (2017). Wearable devices in medical Internet of things: Scientific research and commercially available devices. Healthcare Informatics Research, 23, 4–15. https://doi.org/10.4258/hir.2017.23.1.4
   PubMed Web of Science ®Google Scholar
• Hamidi, F., Baljko, M., Livingston, N., & Spalteholz, L. (2010). CanSpeak: A customizable speech interface for people with dysarthric speech. In: K Miesenberger, J. Klaus, W. Zagler, & A. Karshmer A. (Eds.), Computers helping people with special needs. ICCHP 2010. Lecture notes in computer science (Vol.6179). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14097-6_97
   Google Scholar
• Hanson, E.K., Beukelman, D.R., & Yorkston, K.M. (2013). Communication support through multimodal supplementation: A scoping review. Augmentative and Alternative Communication, 28, 310–321. doi:10.3109/07434618.2013.848934
   Google Scholar
• Hasan, M.S., & Hongnian, Y. (2017). Innovative developments in HCI and future trends. International Journal of Automation and Computing, 14, 10–20. https://doi.org/10.1007/s11633-016-1039-6
   Web of Science ®Google Scholar
• Hawley, M., Enderby, P., Green, P., Cunningham, S., Brownsell, S., Carmichael, J., … Palmer, R. (2007). A speech-controlled environmental control system for people with severe dysarthria. Medical Engineering & Physics, 29, 586–593. doi:10.1016/j.medengphy.2006.06.009
   PubMed Web of Science ®Google Scholar
• Herff, C., Heger, D., De Pesters, A., Telaar, D., Brunner, P., Schalk, G., & Schultz, T. (2015). Brain-to-text: Decoding spoken phrases from phone representations in the brain. Frontiers in Neuroscience, 9, 217. doi:10.3389/fnins.2015.00217
   PubMed Web of Science ®Google Scholar
• Higger, M., Quivira, F., Akcakaya, M., Moghadamfalahi, M., Nezamfar, H., Cetin, M., & Erdogmus, D. (2017). Recursive Bayesian coding for BCIs. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25, 704–714. doi:10.1109/TNSRE.2016.2590959
   PubMed Web of Science ®Google Scholar
• Higginbotham, D.J., Shane, H., Russell, S., & Caves, K. (2007). Access to AAC: Present, past, and future. Augmentative and Alternative Communication, 23, 243–257. doi:10.1080/07434610701571058
   PubMed Web of Science ®Google Scholar
• Higginbotham, D.J., & Wilkins, D.P. (1999). Slipping through the time stream: Social issues of time and timing in augmented interactions. In: D. Kovarsky, J. Duchan, & M. Maxwell (Eds.), Constructing (in)competence: Disabling evaluations in clinical and social interactions. Lawrence Erlbaum Associates: Mahwah, NJ. 49–82.
   Google Scholar
• Hoag, L.S.L.A., Bedrosian, J.L., McCoy, K.F., & Johnson, D. (2004). Trade-offs between informativeness and speed of message delivery in augmentative and alternative communication. Journal of Speech, Language, and Hearing Research, 47, 1270–1285. doi:10.1044/1092-4388(2004/096)
   PubMed Web of Science ®Google Scholar
• Hosom, J.P., Jakobs, T., Baker, A., & Fager, S. (2010). Automatic speech recognition for assistive writing in speech supplemented word prediction. Interspeech 2010, 2674–2677. Retrieved from https://www.isca-speech.org/archive/archive_papers/interspeech_2010/i10_2674.pdf
   Google Scholar
• Hux, H., Rankin-Erickson, J., Manasse, N., & Lauritzen, E. (2000). Accuracy of three speech recognition systems: Case study of dysarthric speech. Augmentative and Alternative Communication, 16, 186–196. doi:10.1080/07434610012331279044
   Google Scholar
• Judge, S., Robertson, Z., Hawley, M., & Enderby, P. (2009). Speech-driven environmental control systems: A qualitative analysis of users’ perceptions. Disability & Rehabilitation: Assistive Technology, 4, 151–157. doi:10.1080/17483100802715100
   PubMedGoogle Scholar
• Kaufmann, T., Holz, E.M., & Kübler, A. (2013). Comparison of tactile, auditory, and visual modality for brain-computer interface use: A case study with a patient in the locked-in state. Frontiers in Neuroscience, 7, 129. Retrieved from https://www.frontiersin.org/articles/10.3389/fnins.2013.00129/full
   PubMed Web of Science ®Google Scholar
• Kellis, S., Miller, K., Thomson, K., Brown, R., House, P., & Greger, B. (2010). Decoding spoken words using local field potentials recorded from the cortical surface. Journal of Neural Engineering, 7, 056007. doi:10.1088/1741-2560/7/5/056007
   PubMed Web of Science ®Google Scholar
• Kruger, E., Teasell, R., Salter, K., Foley, N., & Hellings, C. (2007). The rehabilitation of patients recovering from brainstem strokes: Case studies and clinical considerations. Topics in Stroke Rehabilitation, 14, 56–64. doi:10.1310/tsr1405-56
   PubMed Web of Science ®Google Scholar
• Kübler, A., Holz, E.M., Riccio, A., Zickler, C., Kaufmann, T., Kleih, S.C., … Mattia, D. (2014). The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications. PLoS ONE, 9, e112392. doi:10.1371/journal.pone.0112392
   PubMed Web of Science ®Google Scholar
• Lesher, G.W., Moulton, B.J., & Higginbotham, D.J. (1998). Techniques for augmenting scanning communication. Augmentative and Alternative Communication, 14, 81–101. doi:10.1080/07434619812331278236
   Google Scholar
• Lotte, F., Larrue, F., & Mühl, C. (2013). Flaws in current human training protocols for spontaneous brain-computer interfaces: Lessons learned from instructional design. Frontiers in Human Neuroscience, 7, 568. doi:10.3389/fnhum.2013.00568
   PubMed Web of Science ®Google Scholar
• McCane, L.M., Sellers, E.W., McFarland, D.J., Mak, J.N., Carmack, C.S., Zeitlin, D., … Vaughan, T.M. (2014). Brain-computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, 15, 207–215. doi:10.3109/21678421.2013.865750
   PubMed Web of Science ®Google Scholar
• McNaughton, D., Light, J., Beukelman, D.R., Klein, C., Nieder, D., & Nazareth, G. (2019). Building capacity in AAC: A person-centered approach to supporting participation by people with complex communication needs [Special Issue]. Augmentative and Alternative Communication,
   Google Scholar
• Mengistu, K.T., & Rudzicz, F. (2011, May 22–27). Adapting acoustic and lexical models to dysarthric speech. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4924–4927. Retrieved from http://www.cs.toronto.edu/∼frank/Download/Papers/mengistu_icassp11.pdf
   Google Scholar
• Moghadamfalahi, M., Orhan, U., Akcakaya, M., Nezamfar, H., Fried-Oken, M., & Erdogmus, D. (2015). Language-model assisted brain computer interface for typing: A comparison of matrix and rapid serial visual presentation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 23, 910–920. doi:10.1109/TNSRE.2015.2411574
   PubMed Web of Science ®Google Scholar
• Mugler, E.M., Patton, J.L., Flint, R.D., Wright, Z.A., Schuele, S.U., Rosenow, J., … Slutzky, M.W. (2014). Direct classification of all American English phonemes using signals from functional speech motor cortex. Journal of Neural Engineering, 11, 035015. doi:10.1088/1741-2560/11/3/035015
   PubMed Web of Science ®Google Scholar
• Murguialday, A.R., Hill, J., Bensch, M., Martens, S., Halder, S., Nijboer, F., … Gharabaghi, A. (2011). Transition from the locked in to the completely locked-in state: A physiological analysis. Clinical Neurophysiology, 122, 925–933. doi:10.1016/j.clinph.2010.08.019
   PubMed Web of Science ®Google Scholar
• NIDILRR Development Framework (2017). Retrieved from https://www.acl.gov/index.php/aging-and-disability-in-america/nidilrr-frameworks
   Google Scholar
• Nijboer, F., Sellers, E.W., Mellinger, J., Jordan, M.A., Matuz, T., Furdea, A., … Kübler, A. (2008). A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clinical Neurophysiology, 119, 1909–1916. doi:10.1016/j.clinph.2008.03.034
   PubMed Web of Science ®Google Scholar
• Oken, B.S., Orhan, U., Roark, B., Erdogmus, D., Fowler, A., Mooney, A., … Fried-Oken, M.B. (2014). Brain-computer interface with language model-electroencephalography fusion for locked-in syndrome. Neurorehabilitation and Neural Repair, 28, 387–394. doi:10.1177/1545968313516867
   PubMed Web of Science ®Google Scholar
• Peters, B., Mooney, A., Oken, B., & Fried-Oken, M. (2016). Soliciting BCI user experience feedback from people with severe speech and physical impairments. Brain-Computer Interfaces, 3, 47–58. doi:10.1080/2326263X.2015.1138056
   Web of Science ®Google Scholar
• Roark, B., Fowler, A., Sproat, R., Gibbons, C., & Fried-Oken, M. (2011). Towards technology-assisted co-construction with communication partners. In Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies, 22–31. Association for Computational Linguistics. Retrieved from https://pdfs.semanticscholar.org/942a/943ac4b46ec6d5a5c3e988e3e9a87cbf1ae9.pdf
   Google Scholar
• Roark, B., Fried-Oken, M., & Gibbons, C. (2015). Huffman and linear scanning methods with statistical language models. Augmentative and Alternative Communication, 31, 37–50. doi:10.3109/07434618.2014.997890
   PubMed Web of Science ®Google Scholar
• Sahadat, M.N., Alreja, A., & Ghovanloo, M. (2018). Simultaneous multimodal PC access for people with disabilities by integrating head tracking, speech recognition, and tongue motion. IEEE Transactions on Biomedical Circuits and Systems, 12, 192–201. doi:10.1109/TBCAS.2017.2771235
   PubMed Web of Science ®Google Scholar
• Scherer, R., Billinger, M., Wagner, J., Schwarz, A., Hettich, D.T., Bolinger, E., … Müller-Putz, G. (2015). Thought-based row-column scanning communication board for individuals with cerebral palsy. Annals of Physical and Rehabilitation Medicine, 58, 14–22. doi:10.1016/j.rehab.2014.11.005
   PubMed Web of Science ®Google Scholar
• Sellers, E.W., & Donchin, E. (2006). A P300-based brain-computer interface: Initial tests by ALS patients. Clinical Neurophysiology, 11, 538–548. doi:10.1016/j.clinph.2005.06.027
   Google Scholar
• Sellers, E.W., Ryan, D.B., & Hauser, C.K. (2014). Noninvasive brain-computer interface enables communication after brainstem stroke. Science Translational Medicine, 6, 257r7. doi:10.1126/scitranslmed.3007801
   Web of Science ®Google Scholar
• Sellers, E.W., Vaughan, T.M., & Wolpaw, J.R. (2010). A brain-computer interface for long-term independent home use. Amyotrophic Lateral Sclerosis, 11, 449–455. doi:10.3109/17482961003777470
   PubMed Web of Science ®Google Scholar
• Singh, G., Nelson, A., Robucci, R., Patel, C., & Banerjee, N. (2015). Inviz: Low-powered personalized gesture recognition using wearable textile capacitive sensor arrays. Proceedings of IEEE Pervasive Computing and Communications (PerCom), 198–206. doi:10.1109/PERCOM.2015.7146529
   Google Scholar
• Shah, S.G.S., Robinson, I., & Al Shawi, S. (2009). Developing medical device technologies from users’ perspectives: A theoretical framework for involving users in the development process. International Journal of Technology Assessment in Health Care, 25, 514–521. doi:10.1017/S0266462309990328
   PubMed Web of Science ®Google Scholar
• Shane, H., Blackstone, S., Vanderheiden, G., Williams, M., & DeRuyter, F. (2012). Using AAC technology to access the world. Assistive Technology, 24, 3–13. doi:10.1080/10400435.2011.648716
   Web of Science ®Google Scholar
• Vansteensel, M.J., Pels, E.G.M., Bleichner, M.G., Branco, M.P., Denison, T., Freudenburg, Z.V., … Ramsey, N.F. (2016). Fully implanted brain–computer interface in a locked-in patient with ALS. New England Journal of Medicine, 375, 2060–2066. doi:10.1056/NEJMoa1608085
   PubMed Web of Science ®Google Scholar
• Wilkinson, C.R., & De Angeli, A. (2014). Applying user centred and participatory design approaches to commercial product development. Design Studies, 35, 614–631. doi:10.1016/j.destud.2014.06.001
   Web of Science ®Google Scholar
• Wolpaw, J.R., Bedlack, R.S., Reda, D.J., Ringer, R.J., Banks, P.G., Vaughan, T.M., … Ruff, R.L. (2018). Independent home use of brain-computer interface by people with amyotrophic lateral sclerosis. Neurology, 91, e258–e267. doi:10.1212/WNL.0000000000005812
   PubMed Web of Science ®Google Scholar
• Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., & Vaughan, T.M. (2002). Brain-computer interfaces for communication and control. Clinical Neurophysiology, 113, 767–791. doi:10.1016/S1388-2457(02)00057-3
   PubMed Web of Science ®Google Scholar
• Young, V., & Mihailidis, A. (2010). Difficulties in automatic speech recognition of dysarthric speakers and implications for speech-based applications used by the elderly: A literature review. Assistive Technology, 22, 99–112. doi:10.1080/10400435.2010.483646
   PubMed Web of Science ®Google Scholar