“Love looks not with the eyes”: supranormal processing of emotional speech in individuals with late-blindness versus preserved processing in individuals with congenital-blindness

ABSTRACT

Processing of emotional speech in the absence of visual information relies on two auditory channels: semantics and prosody. No study to date has investigated how blindness impacts this process. Two theories, Perceptual Deficit, and Sensory Compensation, yiled different expectations about the role of visual experience (or its lack thereof) in processing emotional speech. To test the effect of vision and early visual experience on processing of emotional speech, we compared individuals with congenital blindness (CB, n = 17), individuals with late blindness (LB, n = 15), and sighted controls (SC, n = 21) on identification and selective-attention of semantic and prosodic spoken-emotions. Results showed that individuals with blindness performed at least as well as SC, supporting Sensory Compensation and the role of cortical reorganisation. Individuals with LB outperformed individuals with CB, in accordance with Perceptual Deficit, supporting the role of early visual experience. The LB advantage was moderated by executive functions (working-memory). Namely, the advantage was erased for individuals with CB who showed higher levels of executive functions. Results suggest that vision is not necessary for processing of emotional speech, but early visual experience could improve it. The findings support a combination of the two aforementioned theories and reject a dichotomous view of deficiencies/enhancements of blindness.

KEYWORDS:

• Visual experience
• blindness
• emotional speech
• selective-attention
• brain plasticity

“Love looks not with the eyes, but with the mind. And therefore is wing'd Cupid painted blind”. Was Shakespeare correct in A Midsummer Night's Dream in ascertaining that blindness can improve the perception of others’ emotional states? Or is visual experience needed for the development of this skill?

Blindness has been found to have a profound effect on cognitive functioning and emotional processing (Silverstein et al., 2013; Valente et al., 2018). However, no clear consensus exists about how blindness impacts the processing of emotional speech (identification and selective-attention), a key function in daily social communication and interaction (Ben-David et al., 2016). Some studies, related to the Perceptual Deficit Hypothesis (Gori et al., 2014) and the Cross-Calibration Hypothesis (King & Carlile, 1993; Knudsen & Knudsen, 1989) suggest that the lack of early visual experience, an essential source of information for the development of social learning (Hobson & Bishop, 2003), may lead to reduced efficiency at processing emotional speech. This is evident in a comparison of individuals with Congenital Blindness (CB) versus Late Blindness (LB; Chen et al., 2022). Other studies, related to the Sensory Compensation Hypothesis (Chebat et al., 2020), show that brain plasticity and cortical reorganisation lead to intact (Gamond et al., 2017) or even enhanced (Sarzedas et al., 2023) emotional processing skills in blindness. Compared to sighted controls (SC), individuals with CB exhibit an increased activation of the amygdala in response to emotional auditory stimuli (Klinge et al., 2010), reflecting neural plasticity at early sensory processing stages (Föcker et al., 2012), and cortical reorganisation (Röder & Kekunnaya, 2021; Singh et al., 2018).

To better understand the role of vision and early visual experiences in the processing of emotional speech, and resolve these apparent contradictions, this study compared individuals with CB, LB, and SC on the identification and selective-attention of emotional speech.

Emotional processing

Early visual experience plays a critical role in the development of Theory of Mind (ToM; Minter et al., 1998), which involves identifying and understanding other people's emotional states and interpreting emotional cues conveyed through speech. Some studies found individuals with CB to experience delays in the development of ToM as compared to individuals with LB or SC (Roch-Levecq, 2006). For example, children with CB had difficulty identifying emotions based on voice characteristics (Minter et al., 1991). These findings relate to the Perceptual Deficit Hypothesis which assumes that damage to one of the senses leads to impairment of the development of related abilities and of the remaining senses (Bell et al., 2019). Accordingly, the CB brain is atrophied in most visual structures, mostly attributed to mechanisms of disuse (Merabet & Pascual-Leone, 2010). Such atrophy results in cognitive deficits in individuals with CB across modalities (as compared to SC; e.g. Cappagli et al., 2017). Specifically, the lack of early visual experience (CB) may impair the calibration of auditory (and visual) systems involved in processing emotional speech, leading to reduced proficiency (as compared to LB).

Other studies, conversely, indicate that individuals with blindness can exhibit compensatory mechanisms and enhanced performance in cognitive and emotional tasks. Prolonged brain plasticity and cortical reorganisation may underlie these compensations (Chebat, Harrar, et al., 2018), as related to the Sensory Compensation Hypothesis (Voss et al., 2010). Such compensations may lead to intact, or even supra-normal performance on verbal memory, working-memory, spatial abilities, auditory perception, and selective-attention (Amedi et al., 2003; Dormal et al., 2016). Notably, to improve tactile recognition of emotional facial expressions, individuals with CB have been found to recruit brain areas typically associated with visual recognition of facial expressions, (Kitada et al., 2013). These areas were also activated during emotional voice perception for individuals with CB. For example, in Klinge et al. (2010), participants with CB demonstrated superior auditory speaker identification when compared with participants with SC. The advantage was associated with stronger activation in the FFA, occipital cortex (visual regions), and the amygdala when processing emotional auditory stimuli. These compensatory mechanisms were indicated as more prominent for CB than for LB individuals, suggesting that the extent of cortical reorganisation could be associated with the onset and duration of blindness (Chebat, Harrar, et al., 2018; Chebat, Heimler, et al., 2018).

Emotional speech processing

In daily conversations, processing emotional speech is the main source for the apprehension of others’ emotional states, in the absence of visual information (Icht et al., 2021). Deriving emotional meaning in spoken language relies on the identification of information in two auditory channels: Semantics – what is said (the meaning of the words), and Prosody – how it is said (intonation). Typically developed young adults have been found to easily identify emotions based on semantic and prosodic information (Ben-David et al., 2019). The two channels were found to be interrelated, as individuals fail to selectively attend to one channel while ignoring the other. This suggests the involvement of executive functions (i.e. selective-attention, inhibition, and working-memory) in the daily processing of emotional speech (Ben-David et al., 2016).

The roles vision and visual experience play in the ability to process emotional speech remain elusive. As aforementioned, individuals with CB show reduced social competency (ToM) as compared to individuals with LB and SC, suggesting the critical role of visual experience (Green et al., 2004; Minter et al., 1998). CB limits opportunities for incidental social learning through observation (Greenaway & Dale, 2017), possibly restricting the development of social reasoning and cognition. Indeed, the functional profile of ToM-related brain regions is quantitatively weaker (even if qualitatively similar) in children with CB as compared to sighted children (Richardson et al., 2023). Additionally, individuals with CB were found to have difficulty integrating information across sensory channels (Röder, 2012; Scheller et al., 2021), a skill related to processing of emotional communication (Ben-David et al., 2016; Dor et al., 2022, 2024; Taitelbaum-Swead et al., 2022). However, testing the identification of emotional speech among individuals with blindness has yielded conflicting results.

Emotional prosodies in the absence of vision

Evidence in the literature is mixed. Some studies show that individuals with CB perform worse than individuals with LB and SC on the identification of emotional prosodies and vocalizations, supporting the idea that vision is necessary for the development of this skill. For example, Minter et al. (1991) found that children with CB had difficulty identifying emotions by voice, despite their ability to recognise non-emotional sounds. In contrast, adults with CB were not found to differ from SC in recognising negative prosodies and vocalizations on a dichotic listening task (Gamond et al., 2017). Finally, Klinge et al. (2010) suggest that individuals with CB experience increased brain activity for emotional vocalisation.

Emotional semantics in the absence of vision

Research on spoken-(non-emotional)-semantics processing supports the brain plasticity (sensory compensation) hypothesis, with supra-normal performance. Evidence suggests that individuals with CB have an advantage over SC in several speech-processing tasks: lexical-semantic, verbal memory, vowel perception, and syllable detection (Amedi et al., 2003; Topalidis et al., 2020). However, no study to date has directly tested the identification of emotional semantics (the emotional content of words and sentences) in blindness.

Executive functions and working-memory in the absence of vision

Speech processing requires working-memory, especially in complex speech tasks (Nitsan et al., 2019, 2022; Schwartz & Pell, 2012). The literature indicates that working-memory is preserved or even improved in blindness (Heled et al., 2022; Pasqualotto et al., 2013; Pigeon & Marin-Lamellet, 2015). For certain tasks (i.e. finding items) SC use vision, while individuals with blindness commonly use memory-based strategies (e.g. serial recall), serving as working-memory training (Arcos et al., 2022), with improved performance, indicating brain plasticity (Dormal et al., 2016; Röder et al., 2001).

The current study

This study aimed to investigate the impact of early visual experience or its absence on the ability of individuals with blindness to process emotional speech, focusing on both prosodic (tone of speech) and semantic (meaning of words) channels. We compare SC (continued visual experience) to groups of individuals with LB (early visual experience) and CB (no visual experience), on two abilities: (a) Identification of emotional prosody and spoken semantics, and (b) Selective-Attention to a single speech channel, while ignoring the other. The following predictions stem from the aforementioned literature: (1) Sensory Compensation: Individuals with blindness, in general, will be at least as good as (or even better than) SC, supporting the role of brain plasticity. Individuals with CB may outperform individuals with LB, due to prolonged plasticity. Alternatively, (2) Perceptual Deficit: Blindness should impair performance, due to the lack of continued visual experience. Individuals with LB will have an advantage over individuals with CB, supporting the role of early visual experience. (3) Executive functions: Working-memory was found to play a critical role in complex speech-processing tasks (Nitsan et al., 2019, p. 2022). Since it can be affected by blindness (Heled et al., 2022), working-memory might moderate possible group-differences in emotional-speech processing. Furthermore, we examine the effects of aging, gender, and residual light perception on the processing of emotional speech following findings in the literature (e.g. Ben-David et al., 2019, Lausen & Schachat, 2018; Chebat et al., 2020).

Methods

Participants

The sample size was calculated based on an a-priori power analysis (see Appendix A1). Seventeen adults with CB and 15 adults with LB were recruited by approaching Israeli organisations for the visually impaired and individuals with blindness. Twenty-one participants were recruited as SC by advertisement in social media. Table 1 indicates that the groups did not differ in average age or gender distribution, and all met the inclusion criteria. Appendix A2 provides full demographic data for participants with blindness.

Table 1. Demographic data.

	N	Age M, sd, [range]	Gender
CB	17	44.2, 9.8, [30–63]	8 males
LB	15	50.5, 12.4, [28–74]	7 males
2-group difference		t(30) = 1.598, p = .120	χ^2(1) = 0.000, p = .982
SC	21	46.6, 13.4, [28-72]	6 males
SC vs CB		t(34) = .389, p = .389	χ^2(1) = 1.380, p = .240
SC vs LB		t(39) = .627, p = .535	χ^2(1) = 1.242, p = .265
3-group difference		F(2, 50) = 1.090, p = .344	χ^2(2) = 1.776, p = .411
Inclusion criteria (as assessed by self-report questionnaires)	Native Hebrew speakers; Possessing a high school or a college diploma; Normal or corrected to normal vision; Within the age range of 18–31 years old; Scored below the cutoff of 25 on DASS-21 self-report (Henry & Crawford, 2005), ensuring no symptoms for stress, anxiety, and depression.

Ethics

The study was approved by the ethics committee of the last author’s academic institution and all participants gave their informed consent before the experiment commenced. Participants with blindness were read the informed consent form by an experimenter and orally consented to the study. Participants received the equivalent of USD 25 to compensate for the time they spent participating in the study.

Tools & materials

The following tools were used: The Auditory forward and backward digit span test (WAIS III, Wechsler, 1997; see Appendix A3), and the Hebrew version of the the Test of Rating of Emotions in Speech (T-RES; Ben-David et al., 2019), with the following emotions: Anger, Happiness, Sadness, and Neutrality Hebrew version of. Participants were presented with spoken sentences in which the emotional semantic and prosodic content appears in different combinations from trial to trial (Figure 1): (a) Congruent (matched), in which the same emotion is presented in both channels (Figure 1:gray cells); (b) Incongruent (mismatched), in which different emotions are presented in each channel (Figure 1:black cells); (c) Semantic baseline (Figure 1:white cells), in which the prosody is neutral, while the semantics present one of the three tested emotions; (d) Prosodic baseline (Figure 1:white cells), in which the semantics are neutral, while the prosody presents one of the three tested emotions.

Figure 1. Outline of the T-RES paradigm, as used in the current study, and the timeline of the study design.

In each trial, listeners were asked to rate the extent to which they agreed that the speaker conveyed a predefined emotion (Anger, Sadness, or Happiness), using a 6-point Likert scale. For example, “To what extent do you agree that the speaker conveys happiness? From 1 – strongly disagree to 6 – strongly agree”.

The session began with a practice task followed by two of the T-RES tasks presented in a counterbalanced order: (a) prosodic-rating, in which listeners are asked to rate the sentence based only on prosodic information (ignoring the semantics); and (b) semantic-rating, in which listeners are asked to rate the sentence based only on semantic information (ignoring the prosody). Each task consisted of three emotional rating blocks, anger-rating, happiness-rating, and sadness-rating, counterbalanced. In sum, each session totalled 90 trials (15 sentences X2 tasks X3 blocks). For more details, see Appendix A4.

Design & procedure

The experiment was divided into two separate sessions. In the first session (no more than 25 minutes), inclusion criteria were verified, and the digit span tasks were completed. In the second session, conducted 2–5 days later (for no more than 45 minutes), they performed the T-RES. Appendix A3 includes full details on administration procedure.

Statistical analyses

All analyses used mixed linear modelling, MLM. In the first stage, the tests were conducted across the three groups, CB, LB, and SC. We used average ratings as the dependent variable. The goal of each analysis was to conduct a 2-way interaction of Group and the test-specific variable (Emotion Identification or Selective-Attention). In the next stage, the MLM analysis was repeated, this time conducting all three possible two-group comparisons: (a) directly comparing CB and LB groups, (b) comparing CB and SC, and (c) comparing LB and SC. As a final stage, we examined how individual differences in working-memory (measured by digit span) and residual light perception impacted Identification and Selective-Attention in blind individuals (for full details, see Appendix A5).

Each analysis included a test-specific variable.

1. Identification was gauged as the difference between the average ratings of baseline sentences (Figure 1:white cells) that presented the target emotion in the attended channel versus sentences that did not. For example, identification of semantic anger is gauged as the difference between the rating of a semantically angry sentence spoken with neutral prosody (expected to be high, 6) and the average ratings of semantically sad and happy sentences spoken with neutral prosody (expected to be low, 1). Thus, the maximal score for Identification would be 5; lower scores indicate the extent of difficulty in discriminating between the three emotions.

2. Selective-Attention was gauged by comparing average ratings of congruent sentences (presenting the rated emotion in both channels; Figure 1:grey cells) with ratings of incongruent sentences (presenting the rated emotion in the target channel, and a different emotion in the other channel; Figure 2:black cells). For example, selective-attention to semantic anger was gauged as the difference between ratings of a semantically angry sentence spoken with angry prosody (congruent, expected to be high, 6) and the average ratings of semantically angry sentences spoken with sad or happy prosodies (incongruent). If selective-attention is perfect, the difference should be 0 (hence, ratings of incongruent sentences should be 6). Failures of selective-attention are indicated by difference scores higher than 0 (hence, ratings of incongruent sentences should be lower than 6).

Figure 2. Average and standard deviations (error bars) for the T-RES scores for the three study groups. The height of the respective columns depicts the extent of Identification (Panel A), or failures of Selective Attention (Panel B) to the target channel in the Late Blind (dark bars), the Congenitally Blind (gray bars) and the Sighted Control groups (white bars).

Results

All participants successfully completed the experimental tasks. To foreshadow the main findings, the LB group outperformed the SC and the CB groups in both Identification and Selective-Attention measures. However, the CB group did not differ from the SC in these measures. Average ratings for the comparison of the three groups are presented in Table 2. The complete MLM analyses are available in Appendix B.

Table 2. Average ratings for the comparison of the three groups.

	Late Blind		Congenitally Blind		Controls
	Identification (baseline sentences)
	Prosody	Semantic	Prosody	Semantic	Prosody	Semantic
Target-emotion-present	5.92 (0.13)	5.69 (0.13)	5.46 (0.13)	5.23 (0.12)	5.48 (0.12)	5.25 (0.12)
Target-emotion-absent	1.52 (0.14)	1.65 (0.13)	1.77 (0.13)	1.89 (0.12)	1.84 (0.12)	1.96 (0.11)
Average Identification	4.22 (0.21)		3.51 (0.19)		3.46 (0.18)
	Selective Attention
	Prosody	Semantic	Prosody	Semantic	Prosody	Semantic
Congruent	5.94 (0.08)	5.92 (0.08)	5.78 (0.07)	5.75 (0.07)	5.92 (0.07)	5.89 (0.07)
Incongruent	5.80 (0.17)	5.59 (0.18)	5.17 (0.16)	4.95 (0.17)	5.46 (0.15)	5.25 (0.17)
Average Selective Attention	0.23 (0.15)		0.70 (0.14)		0.56 (0.13)

Identification

Identification of emotions in baseline sentences for all groups is presented in Figure 2(a). Salient to visual inspection is the facility of all participant groups to identify the presented emotions in speech. Nevertheless, it appears that the LB group performed better at this task than the SC and the CB group.

The MLM analysis (Appendix B:left panel) revealed a significant main effect for Identification and a significant interaction of Identification and Group. Inter-group comparisons (see shaded row, Appendix B1) revealed that the source of this interaction was the improved performance of the LB group (M = 4.22, SD = 0.19). Indeed, Identification scores were significantly higher for the LB group than for the CB group (M = 3.51, SD = 0.19), and for the SC group (M = 3.46, SD = 0.18). No such group difference was observed when comparing the CB group and the SC group, with highly similar effects in both groups (see the upper rows in Table 2). Note, identification scores did not reach ceiling for either group (a score of 5; see Table 2).

Selective-attention to the prosodic or the semantic channel

The average Selective-Attention measure, as presented in Figure 2(b), indicates failures of selective-attention for all groups. All participants rated the target emotion higher in congruent than in incongruent combinations of prosody and semantics. The extent of failures in the LB group, however, was less severe than in the SC group and in the CB group. These findings, taken together with the results presented in Figure 2(a), suggest that LB individuals have supra-normal spoken-emotion identification and selective-attention abilities as compared to SC and CB individuals.

The MLM (see the left panel of Appendix B2) revealed that the Selective-Attention measure (i.e. the difference between congruent and incongruent stimuli favoring the former) differed between the three groups. As in the previous analysis (Identification), the inter-group analyses pointed to improved performance by the LB group (see shaded row, Appendix B2). That is, Selective-Attention failures were significantly lower for the LB group than for both the CB group and the SC group (M = 0.23, SD = 0.15, M = 0.70, SD = 0.14, and M = 0.56, SD = 0.13, respectively). No significant difference in Selective-Attention was found between the CB and the SC group. Follow-up analysis revealed that Selective-Attention failures were significant for the CB and the SC groups. However, no significant failures were noted for the LB group (see the bottom row, Appendix B2). Appendix C indicates that (faint) residual light perception was not found to significantly affect the results for either identification or selective attention. Finally, to appreciate the distribution of the raw data, Appendix D presents violin plots that accompany Figure 2(a,b).

The role of cognitive span

Executive functions were found in the literature to affect complex speech-processing tasks (Ben-David et al., 2016, 2019; Icht et al., 2021; Nitsan et al., 2019, p. 2022; Schwartz & Pell, 2012). Figure 3 suggests that digit span (as a gauge for executive functions) has a moderating effect on the difference between CB and LB individuals in Identification (as presented in Figure 3(A)), as well as in Selective-Attention (Figure 3(B)). Namely, a clear advantage for LB over CB groups is observed for individuals with a lower digit span (−1 SD from the mean for their respective group) in both measures. This advantage disappears when examining individuals with a higher digit span (+1 SD). The complete analyses are available in Appendix E.

Figure 3. The moderating effect of Digit Span scores on the differences between CB and LB groups in T-RES scores. Higher points on the Y axes represent better performance: better emotion Identification (Panel A) and better (smaller failures of) Selective Attention (Panel B), for CB (gray lines) and LB (dark lines) individuals. X axes represent the distribution of performance on the Digit Span test with relation to the respective group mean.

In the first stage, initial partial correlations were probed separately for the CB and LB groups, controlling for the possible effect of age. For the CB group, we found Digit Span and Emotion-Identification (averaged across target – emotions and channels) to be significantly correlated, while the correlation of Digit Span and Selective-Attention (averaged) was only approaching significance. However, for the LB group, none of these correlations reached significance. At the next stage, we ran two separate moderation models (using PROCESS model 01; Hayes, 2013). The dependent measure was either Emotion Identification or Selective-Attention, the independent variable was Group (0 = CB, 1 = LB), the moderator was Digit span (range: 11–29, M = 20.594, SD = 4.669), and age (range: 28–74, M = 47.125, SD = 11.384) served as a covariate.

A similar pattern was observed for both Emotion Identification and Selective Attention. A significant interaction between Group and Digit-Span was found for both measures. Analyses indicate that the conditional effect of Group on Emotion Identification and on Selective Attention was significant for individuals with low to medium digit-span scores (with better performance for the LB over the CB group). However, for individuals with higher digit span scores, the effect of Group did not reach significance in both measures.

Discussion

Blindness presents a unique opportunity to study the intricate relationship between sensory deprivation, sensory compensation, and executive functions on emotional processing. The current study explored the multifaceted effects of blindness on emotional-speech processing, focusing on the role of vision and early visual experience in the identification and selective-attention of emotional prosodies and semantics. We found supra-normal abilities for individuals with LB (with early visual experience) as compared to individuals with CB (without any visual experience) and SC (with continued visual experience), across the two separate speech processing tasks. We suggest that the sensory compensation and perceptual deficit hypotheses complement rather than contradict each other to explain the current results. As sensory compensation suggests, all individuals with blindness were found to perform at least as well as controls, supporting the role of cortical reorganisation in mechanisms of behavioural compensation in the auditory modality (Chebat et al., 2020). As perceptual deficit suggests, individuals with LB outperformed individuals with CB, supporting the role of early visual experience in the development of emotional skills (Gori et al., 2014). Digit span, as a gauge of executive functions, was found to moderate the advantage of LB over CB, suggesting that cognitive abilities may compensate for the lack of early visual experience.

The combined effect of perceptual deficit and sensory compensation hypotheses

The literature review presented two seemingly rivaling predictions, based on the sensory compensation and perceptual deficit hypotheses. Our data reject a simplified dichotomous view of either enhancements or deficiency of blindness, but rather suggest a combination of the two mechanisms, (Merabet & Pascual-Leone, 2010).

Following the sensory compensation hypothesis, individuals with CB can recruit visual areas (forming ∼35% of the cortical mantle) to perform a variety of cognitive tasks, using other remaining intact senses, such as touch or hearing (Kupers & Ptito, 2014). Certain adaptive brain mechanisms have also been found in people with LB, indicating post-critical-period brain plasticity (Chebat, Harrar, et al., 2018; Chebat, Heimler, et al., 2018). Indeed, both groups of individuals with blindness performed at least as good as SC (our first prediction), with emotional speech processing spared in blindness. In addition, this hypothesis predicts that longer experiences with blindness will increase the recruitment of brain areas in a task-specific, sensory-independent fashion. This process, known as a-modal task specificity, leads to an advantage in performance for individuals with CB over LB (Chebat, Harrar, et al., 2018; Chebat, Heimler, et al., 2018). Accordingly, studies from the Röder lab (Klinge et al., 2010; Röder et al., 2000) suggest that individuals with CB have an advantage specifically in speech processing. This latter part of the hypothesis was not supported in our data, as individuals with LB performed better than individuals with CB.

The complementary perceptual deficit hypothesis suggests that early visual experience plays a pivotal role in the accurate calibration of sensory input associated with processing emotional speech and other cognitive functions (Bell et al., 2019). Accordingly, in our study, individuals with LB outperformed individuals with CB, even after controlling for age-related differences (our second prediction). However, this hypothesis also implies that SC individuals continually rely on visual input for shaping and organising cognitive processes, across the life span. Thus, individuals with blindness are expected to experience impairment across multiple domains. In contrast to the latter part of the hypothesis, the present study did not reveal any deficits in performance for the CB or the LB groups, as compared to the SC group, in either task.

Our study thus suggests an integrated approach, which posits that rather than being mutually exclusive, sensory compensation and perceptual deficit mechanisms in individuals with blindness intertwine. While blindness can foster the recruitment of non-visual senses to preserve and improve emotional speech processing, the lack of early visual experience relatively impairs it.

Vision and emotion processing

Our data could contribute to the debate in the literature on the necessity of vision for ToM performance (Minter et al., 1991 vs. Sak-Wernicka, 2016). Continued visual experience was not found to be necessary for successful emotional-speech processing, as the performance of SC was not better than the performance of individuals with CB. However, early visual experience could improve this ability, as indicated by the enhanced performance of individuals with LB. The fact that emotional speech is usually accompanied by visual social cues (facial expressions and bodily stances) might explain these findings. Namely, individuals with LB acquire emotional-speech processing skills when visual cues are available. These cues can provide contextual information that aids in understanding and interpreting the emotional content of spoken language (Hassin et al., 2013). For example, a happy prosody is typically produced with a corresponding happy facial expression (e.g. a smile; Pell, 2005) providing redundancy of information that facilitates learning and processing. In the absence of visual cues, individuals with LB may still be able to use this internalised scheme. Individuals with CB, on the other hand, may rely mainly on auditory cues when acquiring emotional-speech processing skills. Further support for the role of early visual experience in emotional processing comes from research on autism spectrum disorder (ASD). Specifically, avoidance of eye contact at an early age, common in ASD, is assumed to impair social and emotional learning (Kliemann et al., 2010).

Executive functions as a moderating factor

The advantage of individuals with LB over individuals with CB was further nuanced by the moderating effect of executive functions, as gauged by the digit span task. This advantage was only found to be significant for individuals with lower levels of executive functions, while relatively higher levels of executive function eliminated this advantage. Executive functions were also found to provide a protective effect when testing individuals with ASD without intellectual disabilities. This unique group of individuals were found to perform as well as their controls on the same emotional-speech task used in the current study (T-RES, Ben-David et al., 2020). The authors attributed their spared emotional processing performance to preserved executive functions (Icht et al., 2021). Taken together, this points to the important role of executive functions, specifically working-memory, in processing emotional speech.

This insight could resonate with recent theoretical models on speech processing: FUEL (Pichora-Fuller et al., 2016) and the ELU (Rönnberg, 2003). Both models suggest that efficient processing depends on cognitive resources, task demands, and the listener's effort. Challenging listening conditions, such as distorted speech, call for the allocation of additional processing resources. Accordingly, the literature indicates that individuals with higher levels of executive functions are more efficient at processing speech in adverse conditions than their peers with lower levels of executive functions (Nitsan et al., 2019). In the current study, emotional prosodies altered the acoustic characteristics of the speech signal (Carl et al., 2022), possibly increasing the listening effort. As a result, we propose that individuals with CB could overcome the auditory challenge by deploying cognitive resources that are only available to individuals with higher levels of executive function.

Caveats and implications

The T-RES paradigm has a few limitations, such as the use of a single professional female actress, and the assessment of basic simple emotions (for a full list of limitations, see Ben-David et al., 2021). To expand the results and increase their generalisation, future studies may wish to examine other emotional speech-processing tasks and gauge more sources of individual differences (e.g. personality, Nagar et al., 2022) and increase the sample size to reveal effects that might have been obscured. This paradigm may also be less sensitive to changes in ToM, as it is focused on providing high external validity. Future studies should also consider testing brain activation to directly assess the extent of plasticity and sensory compensation (e.g. Chebat, Harrar, et al., 2018; Chebat, Heimler, et al., 2018). The current sample was limited to native Hebrew speakers. Further studies may wish to expand the generality of the results to other languages (see version of the T-RES in English and German; Ben-David et al., 2016; Shakuf et al., 2022). Finally, some participants in our sample of individuals with blindness had at least faint residual light perception. In separate analyses (see Appendix C), this factor was not found to significantly affect the results. However, the role of light perception should be further investigated in the formation of ToM.

In sum, processing of emotional speech appears to be spared in blindness and even improved by post-critical-period visual deprivation and adaptive mechanisms. Ergo, vision is not necessary for the processing of emotional speech, but early visual experience could improve it. Taken together, our study indicates the differential impact of blindness onset on emotional speech processing, pointing to the complex effect of blindness on cognitive performance, that does not necessarily confer to deficiency or enhancement.

Acknowledgements

We wish to thank: Adi Dgani and Lior Golan for their assistance in collecting the data for this study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data available on request from the corresponding author, due to privacy restrictions.

Additional information

Funding

This work was supported by a grant from Ariel University (Encouraging interdisciplinary research in the field of accessibility; # RA2000000338).