Intermodal Audio-Haptic Metaphor: Improvement of Target Search in Abstract Environments

Abstract

Applications requiring processing of complex environments present real challenges for simultaneous management of numerous constraints (multiple degrees of freedom, search criterion, etc.). Target searching is probably among the most critical tasks, consisting of finding configurations corresponding to various criteria (e.g., maximum, minimum, reference). During the search, users need to be aware of proximal results to compare values and make decisions. A new audio-haptic coupling strategy is proposed to improve the search for targets in complex environments, enabling the simultaneous use of both audio and haptic channels for value comparisons at different spatial configurations. This is accomplished through the use of tempo in both sensorial signals creating a connection between the two channels enabling an intuitive and efficient comparison. Including spatialized audio improves a user’s situation awareness. The benefit of this intermodal metaphor is evaluated for a 2D nonvisual and abstract environment. Results show improvements relative to simple haptic exploration.

Additional information

Notes on contributors

Mehdi Ammi

Mehdi Ammi is an associate professor at Paris-Sud University specializing in haptics for virtual reality and telepresence. In particular, he is interested in all aspects of haptic processes ranging from physiological mechanisms to the search for operational methodologies designed to integrate the haptic modality in different type of applications.

Brian F. G. Katz

Brian F. G. Katz is a researcher at LIMSI-CNRS and coordinator of the Sound & Space research theme. His fields of interest include spatial 3-D audio rendering and perception and room acoustics, with a background in physics and philosophy. He obtained his Ph.D. in Acoustics (1998) and HDR in engineering sciences (2011).