Assessing variability in audiovisual speech integration skills using capacity and accuracy measures

Abstract

Objective: While most normal-hearing listeners rely on the auditory modality to obtain speech information, research has demonstrated the importance that non-auditory modalities have on language recognition during face-to-face communication. The efficient utilization of the visual modality becomes increasingly important in difficult listening conditions, and especially for older and hearing-impaired listeners with sensory or cognitive decline. First, this report will quantify audiovisual integration skills using a recently developed capacity measure that incorporates speed and accuracy. Second, to investigate sensory factors contributing to integration ability, high and low-frequency hearing thresholds will be correlated with capacity, as well as gain measures from sentence recognition. Design: Integration scores were obtained from a within-subjects design using an open-set sentence speech recognition experiment and a closed set speeded-word classification experiment, designed to examine integration (i.e. capacity). Study sample: A sample of 44 adult listeners without a self-reported history of hearing-loss was recruited. Results: Results demonstrated a significant relationship between measures of audiovisual integration and hearing thresholds. Conclusions: Our data indicated that a listener's ability to integrate auditory and visual speech information in the domains of speed and accuracy is associated with auditory sensory capabilities and possibly other sensory and cognitive factors.

Key Words::

• Speech perception
• noise
• behavioral measures
• aging

Acknowledgements

The project described was supported by the INBRE Program, NIH Grant Nos. P20 RR016454 (National Center for Research Resources) and P20 GM103408 (National Institute of General Medical Sciences).

Notes

Declaration of interest: The authors report no conflicts of interest.

Notes

1. The α levels were set to the more conservative level of .025 to adjust for multiple comparisons.

2. We computed these same correlations with hearing thresholds, and the alternative gain measure comparing the difference between obtained AV, and p(A)+p(V)−p(A)*p(V). The correlation between low-frequency threshold and gain was non-significant (r(42) = .20, p = .19). However, the correlation for high-frequency was significant (r(42) = .52, p <.001).