Active Listening and Expressive Communication for Children with Hearing Loss Using Getatable Environments for Creativity

Abstract

This paper describes a system for accommodating active listening for persons with hearing aids or cochlear implants, with a special focus on children at an early stage of cognitive development and with additional physical disabilities. A system called the Soundscraper is proposed and consists of a software part in Pure data and a hardware part using an Arduino microcontroller with a combination of sensors. For both the software and hardware development it was important to always ensure that the system was flexible enough to cater for the very different conditions that are characteristic of the intended user group.

The Soundscraper has been tested with 25 children with good results. An increased attention span was reported, as well as positively surprising reactions from children where the caregivers were unsure whether they could hear at all. The sound synthesis methods, the gesture sensors and the employed parameter mapping were all simple, but they provided a controllable and sufficiently complex sound environment even with limited interaction. A possible future outcome of the application is the adoption of long-time analysis of sound preferences as opposed to traditional audiological investigations.

Acknowledgements

We would like to sincerely thank the children and parents who participated in this study and also the professionals who took part with great enthusiasm. We are grateful for kind contributions to the development of the Soundscraper from the Promobilia Foundation and the inspiration provided by The Swedish National Agency for Special Needs Education and Schools.

Notes

¹For audio and video examples, see http://www.speech.kth.se/∼kjetil/files/JNMR12.

²In the text, caregiver and technician represent two different roles that can be accomplished by e.g. special needs teachers, speech therapists, audiologists, or parents.

³ http://puredata.info

⁴ http://www.arduino.cc

⁵An unexplored variation could be to use discrete tones, assignable to scales, instead of continuous pitch.

⁶In addition to these we have made tests with game controllers, a full motion capture system, smartphone sensors, the Kinect camera and other devices, but not for the sessions reported here.

⁷Parts of the data from the tests, especially information on medical background—or conclusions drawn therefrom—are not available for dissemination due to their sensitive nature.

⁸There can be many explanations to why this was so, however these children did not have a common differentiation from the rest of the group.

⁹One example of this would be the sound of water signalling ‘bath time’.