A comparison of simultaneously-obtained measures of listening effort: pupil dilation, verbal response time and self-rating

Abstract

Objective

The aim of this study was to assess to what extent simultaneously-obtained measures of listening effort (task-evoked pupil dilation, verbal response time [RT], and self-rating) could be sensitive to auditory and cognitive manipulations in a speech perception task. The study also aimed to explore the possible relationship between RT and pupil dilation.

Design

A within-group design was adopted. All participants were administered the Matrix Sentence Test in 12 conditions (signal-to-noise ratios [SNR] of −3, −6, −9 dB; attentional resources focussed vs divided; spatial priors present vs absent).

Study sample

Twenty-four normal-hearing adults, 20–41 years old (M = 23.5), were recruited in the study.

Results

A significant effect of the SNR was found for all measures. However, pupil dilation discriminated only partially between the SNRs. Neither of the cognitive manipulations were effective in modulating the measures. No relationship emerged between pupil dilation, RT and self-ratings.

Conclusions

RT, pupil dilation, and self-ratings can be obtained simultaneously when administering speech perception tasks, even though some limitations remain related to the absence of a retention period after the listening phase. The sensitivity of the three measures to changes in the auditory environment differs. RTs and self-ratings proved most sensitive to changes in SNR.

Keywords:

• Listening effort
• response times
• pupillometry
• self-rating
• speech perception

Acknowledgements

The authors acknowledge Hörtech GmbH, Oldenburg, Germany, for providing the speech recordings of the Matrix Sentence Test in the Italian language.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 To explore if uninformative visual cue could affect auditory attention, participants were tested with and without references about the positions of the sources. The condition without references was obtained by using a curtain occluding the speakers. In order to avoid differences in the stimulus playback with and without the curtain, the curtain was used in both visual conditions and the visual cue provided via LEDs.

2 The R code for the statistical model of speech intelligibility was: m.intell = glmer(intell ∼ SNR*attention*vision+(1|subject)+(SNR + attention + vision|subject),data = data, family = binomial, glmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 100000)).

3 The R code for the statistical model of response times was: mod.RT = glmer(RT ∼ SNR*attention*vision+(1 |subject)+(SNR|subject), data = data, family = Gamma(link="log"), glmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 100000)).

4 The R code for the statistical model of self-ratings was: mod.ratings = clmm2(rating ∼ SNR*attention*vision,random = subject, data = data, Hess = TRUE)

5 The analysis was also performed with the traces normalized (e.g. within-trial mean scaling, Winn et al. 2018). As no differences emerged in the results of the two analyses, only the analysis with the baseline-adjusted pupil dilation data are presented here.

6 The R code for the statistical model of the pupil dilation was: m.pupil = lmer(pupil∼(ot1)*SNR*attention*vision+(ot1 |subject)+(ot1 |subject:SNR:attention:vision),data = data,control = lmerControl(optimizer="bobyqa"),REML = FALSE)

7 An analysis performed over a less conservative time window [0; 4 s] returned the same results.

Additional information

Funding

F.P. and C.Va. are supported by a grant from the Agence Nationale de la Recherche [ANR-16-CE17-0016]. F.P. is supported by grants from the Fondation Neurodis, Fondation MEDISITE and Italian Ministry for the University and Research [PRIN 20177894ZH]. C.Va. is supported by a grant from the Università Italo-Francese (UIF)/Université Franco-Italienne (UFI) and the Ermenegildo Zegna Founder's Scholarship. C.Vi. is supported by the Emilia-Romagna Regional Authority’s COMPRENDO project [Action POR FESR 2014-20 – Axis 1 Action 1.2.2 (2019)].