A review of sonification solutions in assistive systems for visually impaired people

Abstract

Purpose

Visually impaired people (VIP) find it challenging to understand and gain awareness of their surroundings. Most activities require the use of the auditory or tactile senses. As such, assistive systems which are capable of aiding visually impaired people to understand, navigate and form a mental representation of their environment are extensively being studied and developed. The aim of this paper is to provide insight regarding the characteristics, as well as the advantages and drawbacks of different types of sonification strategies in assistive systems, to assess their suitability for certain use-cases.

Materials and methods

To this end, we reviewed a sizeable number of assistive solutions for VIP which provide a form of auditory feedback to the user, encountered in different scientific databases (Scopus, IEEE Xplore, ACM and Google Scholar) through direct searches and cross-referencing.

Results

We classified these solutions based on the aural information they provide to the VIP – alerts, guidance and information about their environment, be it spatial or semantic. Our intention is not to provide an exhaustive review, but to select representative implementations from recent literature that highlight the particularities of each sonification approach.

Conclusions

Thus, anyone who is intent on developing an assistive solution will be able to choose the desired sonification class, being aware of the advantages/disadvantages and at the same time having a fairly wide selection of articles from the representative class.

IMPLICATIONS FOR REHABILITATION

• The motivation behind this paper is to provide an overview of sonification strategies in the context of assistive systems for the visually impaired people.

• Whilst surveys and reviews which provide in-depth insights into assistive technologies and sonification exist, papers which provide a combined view of these topics are rather lacking.

• The analysis of the selected papers provides insight regarding the characteristics of different types of sonification strategies in assistive systems for visually impaired people and their suitability for certain use-cases.

Keywords:

• Sonification
• haptic feedback
• assistive systems
• visually impaired people
• navigation assistant
• environment perception

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work is supported by Romanian National Authority for Scientific Research (UEFISCDI), Program PN-III-P1-1.1-TE-2021-1505 (contract number: TE 147/2022): VIP SpaceNav, Smart buildings for the visually impaired - Platform for low cost, rapid configuration and safe navigation in indoor spaces.

Notes on contributors

Andrei Lăpușteanu

Andrei Lăpușteanu received his B.S. degree in computer science from Transilvania University of Brașov, Faculty of Electrical Engineering and Computer Science, in 2019. In 2021 he obtained his master’s degree in computer graphics from University POLITEHNICA of Bucharest, Faculty of Automatic Control and Computers. He is currently a Ph.D. candidate, focusing on psycho-acoustic and haptic representations of the 3D environment in assistive and entertainment solutions for the visually impaired. He is also a teaching assistant at National University of Science and Technology Politehnica Bucharest. His research interests include computer graphics, image and audio processing, video and audio games, computer vision, virtual and augmented reality.

Anca Morar

Anca Morar received the B.S. degree in computer science from University POLITEHNICA of Bucharest, in 2009, and the Ph.D. degree in computer science, in 2012, in the field of medical image analysis and visualization. She is currently an Associate Professor with the Computer Science and Engineering Department, Faculty of Automatic Control and Computers, National University of Science and Technology Politehnica Bucharest. Her research is focused on computer graphics, GPGPU, computer vision, and e-health.

Alin Moldoveanu

Alin Moldoveanu is currently a Full Professor with the Computer Science and Engineering Department, Faculty of Automatic Control and Computers (http://acs.pub.ro), National University of Science and Technology Politehnica Bucharest, where he teaches software engineering and virtual reality. His active research areas include virtual and augmented reality (exploring and applying immersion, sensory substitution, and distorted reality), e-Health (assistive and rehabilitative solutions and prevention of hospital acquired infections), and e-Learning (3D MMO mixed-reality campuses). He is also the Director or responsible for national or European research projects in these areas, such as Sound of Vision, TRAVEE, and HAI-OPS.

Maria-Anca Băluțoiu

Maria-Anca Băluțoiu received the B.S. degree in computer science from the POLITEHNICA University of Bucharest, in 2018, and the master’s degree in computer graphics, in 2020. She is currently a Ph.D. candidate, with focus on gamification of the educational process through the help of virtual reality, augmented reality and natural language processing. She is also a Teaching Assistant with the Computer Science and Engineering Department, Faculty of Automatic Control and Computers, National University of Science and Technology Politehnica Bucharest. Her research interests include computer graphics, computer vision, gamification, augmented reality, and virtual reality.

Florica Moldoveanu

Florica Moldoveanu is currently a Professor with the Department of Computer Science and Engineering with the National University of Science and Technology Politehnica Bucharest. She coordinates the master program computer graphics, multimedia and virtual reality at the Faculty of Automatic Control and Computers. She is also the President of the Health Level 7 Romania Association. Her research and teaching activity is focused on computer graphics, computer vision, software engineering, and e-health.