The history and future of neural modeling for cochlear implants

ABSTRACT

This special issue of Network: Computation in Neural Systems on the topic of “Computational models of the electrically stimulated auditory system” incorporates review articles spanning a wide range of approaches to modeling cochlear implant stimulation of the auditory system. The purpose of this overview paper is to provide a historical context for the different modeling endeavors and to point toward how computational modeling could play a key role in the understanding, evaluation, and improvement of cochlear implants in the future.

KEYWORDS:

• Auditory system
• binaural hearing
• cochlear implant
• computational model
• current spread
• hearing
• neural response
• temporal fine structure

1. Conceptual models of auditory physiology underlying the early development of cochlear implants

The initial development of cochlear implants (CIs) in the late 1950s to early 1980s was spearheaded by several different otolaryngologists around the world, leading interdisciplinary groups of researchers including physiologists, engineers, speech scientists, psychophysicists, audiologists, and more. For some historical perspectives on and by the early innovators, see Chouard (2015), Clark (2015), Eisenberg (2015), and Merzenich (2015). While computational models specifically of CI stimulation only started to appear in the literature toward the end of this period, conceptual models based on electrophysiological recordings and mathematical neural models from other domains played a crucial role in the earlier stages of development. As discussed in several of the historical perspectives listed above, an early question was whether any usable speech perception could be achieved with single-channel implants. The debate about the utility of single-channel implants was maintained for some time, in part due to practical and clinical issues—moving to a multichannel implant would lead to greater engineering and surgical complexity than a single-channel implant, and some remarkable achievements in perception of sound had been obtained with some patients using single-channel implants. However, a larger issue at hand was a scientific debate over a temporal versus a place code for sound frequencies in the auditory nerve. The tonotopic mapping produced by the resonant properties of the basilar membrane in the cochlea was known to impart frequency tuning to auditory nerve fibers (ANFs), such that the auditory nerve could support a place coding scheme for sound frequencies over the entire audible range (Kiang 1965). However, ANFs were also known to synchronize to the phase of acoustic tones at least up to frequencies of 4–5 kHz (Rose et al. 1967), providing support for a temporal coding scheme for low frequencies. Thus, it was determined that placement of multiple electrodes along the length of the cochlea had the potential to enable perception of different sound frequencies according to the place-code theory, but the electrical current-waveform frequencies or pulse timings had the potential to convey frequency information according to the temporal-code theory.

In addition to the concepts of temporal and place coding, early mathematical models of nerve fiber stimulation were influential in understanding the mechanisms behind the generation of neural action potentials (APs) and how stimulus parameters for extracellular stimulation might influence response properties. For example, the seminal work of Hodgkin and Huxley (1952) to derive a nonlinear mathematical model of the squid giant axon membrane laid a biophysical foundation for understanding nerve membrane properties such as the morphology of AP waveforms, a threshold potential for AP generation, refractoriness, anode break excitation, and accommodation (subthreshold adaptation). Earlier work by Weiss (1901) and Lapicque (1907) provided mathematical expressions to explain the strength-duration trade-off in the threshold current in terms of the linear membrane response to a finite-length stimulating pulse. Hill (1936) subsequently modified these descriptions to take into account accommodation.

2. Initial development of computational and mathematical models for CIs

With the advent of prototype multielectrode CIs in the late 1970s and early 1980s, it became apparent that spatial interactions from stimulation on different electrodes could affect perception. These consisted of both electric field interactions within the cochlea and physiological interactions at the neural level (White et al. 1984). The electrical field interactions, as well as a desire to target specific neural populations with stimulation from each electrode, led to the development of computational models of current spread in the cochlea that could be used to evaluate the effects of different electrode configurations (Black and Clark 1980). In the study of CI performance, the output of such models of current spread has often been used as a proxy for the pattern of neural excitation produced by a particular electrode configuration and stimulus pattern. However, the details of AP generation, as originally modeled for the unmyelinated squid giant axon by Hodgkin and Huxley (1952), may lead to patterns of neural activity that cannot be inferred directly from the pattern of current spread, particularly in the case of ongoing stimulation where multiple forms of temporal interaction can occur (Boulet et al. 2016). This motivated Colombo and Parkins (1987) to develop a biophysical model of mammalian ANF stimulation by a CI that was based on the original Hodgkin and Huxley (1952) formulation for describing voltage-gated ion channels but incorporated modifications made by several previous researchers to describe myelinated axons.

Early biophysical models focused on the details of the cell morphology and ion channel descriptions but, like the Hodgkin and Huxley (1952) model, were deterministic, despite substantial evidence that electrical stimulation produced stochastic responses in ANFs. Part of the reason for continuing to use deterministic models was that the amount of stochastic activity in an electrically stimulated ANF is substantially less than that of an acoustically stimulated fiber (Kiang and Moxon 1972). A second factor was the practicality that stochastic models have greater computational complexity. Therefore, only simplified single-ANF models incorporating stochastic activity were developed in the late 1970s and early 1980s. For example, White (1978) proposed a stochastic version of the “integrate-and-fire” style phenomenological model developed by Weiss (1901), Lapicque (1907), and Hill (1936), while Hochmair-Desoyer et al. (1984) investigated the behavior of a simplified biophysical ANF model incorporating a noise term.

3. Recent developments

Since these early models, computational approaches have found their way into implant research, although the number of researchers developing and using models has remained relatively small. We will start our summary at the periphery, where 3D models of the cochlea are used to describe the spatial distribution of the electric field potential evoked by the current flow from the electrode. In this special issue, Hanekom and Hanekom (2016) review the development and refinement of these models, which nowadays take into account various aspects of the cochlea’s complex 3D geometry, including the surrounding space. Considering this, it may seem surprising that the simple one-dimensional (1D) approach to assume the field potential to decay exponentially from the electrode is still frequently used (e.g., Fredelake and Hohmann 2012), because it makes only few assumptions and provides predictable input when, for example, theoretically investigating the interaction of stimulation pulses in the neuronal system. Also, because only parameters for electrode sensitivity and decay are needed, the approach lends itself to be fitted to individual patients (Cohen 2009b; Babacan et al. 2010). However, important effects like across-turn stimulation or the differentiation of stimulating the peripheral dendrite versus the central axon cannot be captured.

Three-dimensional (3D) approaches model the conductivity in the different sections and tissues of the cochlea, the modiolus, and the surrounding area and have thus predictive power to be used in the design of implant electrodes, as described in the reviews by Hanekom and Hanekom (2016) and by Kalkman et al. (2016) in this issue. First, a volume conduction description of the cochlea is obtained from images of the cochlea, for example, from histologic sectioning (Wong et al. 2016). These high-resolution images form the basis for the 3D model and they can be aligned to patient-individual images obtained from micro-computed tomography (µCT). Next, the image needs to be sectioned and the 3D model structure computed to gain volumetric descriptions of the different parts with similar conductivity, like the scalae, and the bounding bones. The electrode is added to the model and finite-element-model solvers are used to compute the electrical potential across 3D space (Frijns et al. 1996). Knowledge of the impedances of the various structures is crucial for reliable predictions, and is thus the focus of current research (Malherbe et al. 2015b). The field potential serves as excitation function for auditory nerve models and has hence been used as the main model result when investigating effects of electrode design and position (Frijns et al. 2001).

One model step further, the gradient of the field potential at the ANF’s nodes can be used as input to single-node models or phenomenological models to predict the firing probability of a nerve fiber to electric stimulation. Single-node models are reviewed by O’Brien and Rubinstein (2016) in this issue, where contemporary models build on the original Hodgkin-Huxley or Schwarz-Eikhof equations (Negm and Bruce 2014). The modeler’s focus is to predict stochastic effects, subthreshold accommodation and rate adaptation with the same set of parameters. Stochasticity can be modeled biophysically by opening and closing a finite number of sodium and potassium channels using Markov chain theory (Groff et al. 2009) or more phenomenologically by adding a noisy current (Morse and Evans 1999). Adaptation is modeled with a low-threshold potassium channel and a hyperpolarization-activated channel, or by changing the extracellular potassium concentration (O’Brien and Rubinstein 2016). Similarly, these aspects are also in the focus of recent phenomenological models where noisy thresholds, adaptation, and refractoriness are modeled through changes of the threshold, as reviewed by Takanen et al. (2016) in this issue.

Knowing the field potential along the axons also allows estimating the firing probability with compartmental models, as reviewed by O’Brien and Rubinstein (2016) in this issue. In these models, the axon is divided into nodes connected by multiple intermodal segments to model the differential effects of the electrical field, giving rise to multiple possible sites of spike initiation with differing latencies, variances, and thresholds (Colombo and Parkins 1987). The morphology of the nerve, its diameter, the position and thickness of the myelin shell, and, not least, its trajectory in the electric field affect spike generation and are increasingly refined by anatomical knowledge in more recent models. Degeneration of the nerve is a critical aspect in CI users and compartmental models have been used to investigate the effects of various stages of degeneration, from the dying of the peripheral dendrite to demyelination (Snel-Bongers et al. 2013).

Some models are particularly developed for animals and can serve as a validation tool within the limits of interspecies differences. Auditory nerve activity can be measured invasively in animals, as can electric field potentials, providing ground truth data for verification (Parkins and Colombo 1987; Frijns et al. 1996). In humans, only compound response measures of nerve activity are available either from the brainstem or from within the cochlea via neural response telemetry, which complicates verification and individual fitting of model activity on single fiber level.

A discussion of computational models for CIs would not be complete without a brief mention of psychophysical models in addition to the neural models that have been described thus far. While gaining a quantitative understanding of the physiological response to electrical stimulation is an important contribution from neural modeling to CI research, ultimately it is desirable to use computational models to produce quantitative predictions of the auditory percepts evoked by CIs and to use these models for the evaluation and improvement of implant designs. Psychophysical CI models vary in the degree to which they include aspects of neural processing. For example, Zeng and Shannon (1994) proposed a relatively simple model of neural processing at the level of the cochlea, auditory nerve, auditory brainstem, and central auditory system using linear, logarithmic, and power-law input-output functions to arrive at a parsimonious description of loudness growth for both acoustic and electrical hearing in the case of pure-tones and constant pulse trains from a single electrode. Similarly, Blamey et al. (2004) developed an excitation-current scale for CIs partly inspired by stochastic neural models of ANFs. To explain the loudness of more complex CI stimulation patterns, McKay and colleagues incorporated phenomenological descriptions of population refractory and spread-of-excitation effects in a psychophysical model (McKay and McDermott 1998; McKay et al. 2001). Litvak et al. (2007) also used an analytical model of current spread and neural activation to predict the effects of CI electrode configuration on loudness growth functions. In an inverse approach, a phenomenological model of electrical stimulation was used to process acoustic stimuli to estimate loudness with CIs across time and frequency for transmission in a vocoder for normal-hearing listeners (Babacan et al. 2010).

The advantage of only describing average ANF population responses in these types of psychophysical models, rather than individual ANF responses, is that they require relatively few parameters to fit to psychophysical data and are very computationally efficient. Bruce et al. (1999) proposed a psychophysical model of CI intensity perception for constant-intensity pulse trains that incorporates a phenomenological model of individual ANFs. A renewal-process implementation of the neural model can improve the computational efficiency (Bruce et al. 2000), but the number of parameters remains high, such that a large number of experiments are required to fit this type of model to an individual CI user (Cohen 2009a, 2009b, 2009c, 2009d, 2009e). The model of Bruce et al. (1999) was also extended by Xu and Collins (2003, 2004, 2005, 2007) to consider intensity perception under conditions of more complicated electrical stimulation patterns.

While the majority of CI psychophysical models have focused on loudness and intensity perception, a number of studies have also incorporated neural models to explore amplitude modulation perception (Xu and Collins 2007; Goldwyn et al. 2010b; O’Brien et al. 2016) and pitch perception (Saeedi et al. 2014) in CI users.

Since it is paramount for CIs to achieve optimal speech perception, recent approaches have used simple electric field models together with phenomenological or biophysical auditory nerve models as front ends to predict speech intelligibility with CIs. Fredelake and Hohmann (2012) have used the model by Hamacher (2004), integrated the output across multiple fibers and time, and used this as input to a statistical classifier to predict speech intelligibility.

Having CIs on both ears helps speech understanding and sound localization (Firszt et al. 2008) by processing of interaural level cues, whereas interaural temporal cues are less salient (Seeber and Fastl 2008). The neural mechanisms underlying binaural benefit, the potential reasons for the limited benefit from interaural time differences with CIs, the gaps of our understanding of binaural neural processing, and recent models are reviewed by Dietz (2016) in this issue. Culling et al. (2012) have developed a model to predict speech intelligibility benefit from bilateral CIs over the performance with monaural implants by computing the benefit gained from selecting the ear with the better signal-to-noise ratio, taking into account a good and a bad performing ear. Predicted benefit from bilateral CIs was as large as 18 dB. Biophysical models of the electrically stimulated binaural auditory system focus on lateral superior olive (LSO) processing of level differences between the ears, since sensitivity to timing cues is low (Colburn et al. 2009). From the LSO output, localized sound direction is estimated (Kelvasa and Dietz 2015).

4. Development and application of models today and outlook to the future

Models to compute the electric field distribution look back on a 20-year history of use for optimizing electrode design and location (Frijns et al. 1996). Models were helpful to disentangle the expected benefits of modiolus-hugging versus more laterally placed electrodes. For example, a placement close to the modiolus was shown to yield lower thresholds and more local stimulation from the nearby electrode, but it is detrimental to intended field interactions when stimulating simultaneously on multiple electrodes to focus the electric field (Hanekom 2001; Goldwyn et al. 2010a). More recently, complex algorithms to steer the current via multiple electrodes have been developed and their impact can be analyzed with 3D models of the cochlea which take into account local resistance changes of various anatomical features (Frijns et al. 2011; Malherbe et al. 2015b). As the underlying 3D models become more accurate and resistances better known, accuracy will improve in the near future. Inasmuch as these models are fitted to individual patients (Malherbe et al. 2015a), it can be envisaged that they can be used to predict optimal fitting and current focusing parameters for multielectrode stimulation at different stimulation levels during the patient’s fitting session with the audiologist where parameters are programmed into the device.

The important trend to fit models to individual patients has the potential to drastically change audiological practice and to increase the importance of models. Beyond the cochlea’s geometric information and electric conduction parameters, the neural elements of the model need to be individually fitted. Interesting parameters for fitting are the status of degeneration of the nerve, all the way to knowing the location and extent of dead regions. Several studies have investigated the potential of electrical compound action potentials (ECAPs) measured in the cochlea via neural response telemetry to be used for estimating thresholds and dead regions. Cohen (2009b) has presented a procedure to determine the spread of excitation from electrodes using ECAP measures and fitted a CI model with it. Bierer et al. (2011) and DeVries et al. (2016) used compound measures of nerve activity to assess the electrode-nerve interface further, including thresholds and dead regions. Even without knowledge of the individual 3D geometry, this information can be used to improve the fitting of CIs by optimizing the channel number and by switching off channels near dead regions, and models can help in this process (Goldwyn et al. 2010a).

For more sophisticated individual fitting, the neuron’s jitter and refractoriness should be known parameters which are affected by neural degeneration (O’Brien and Rubinstein 2016) and which crucially affect response timing. A predictable transmission of temporal information is crucial for binaural hearing; hence a recent modeling focus has been to better predict the stochastic nerve response pattern for single pulses and pulse trains as reviewed by Takanen et al. (2016) in this issue. An interesting stimulus for predictable spike timing is pseudomonophasic pulses where a stimulating monophasic pulse is followed by a charge-equalizing inverse-polarity pulse with longer-duration which itself cannot evoke a spike (Macherey et al. 2006). Horne et al. (2016) presented a phenomenological model that is able to predict thresholds and spike latency and jitter for monophasic, biphasic, and pseudomonophasic stimulation as a function the phase-duration and amplitudes with the same parameter set. For pulse trains, refractoriness, accommodation, and rate adaptation affect firing probability and spike timing in various ways (Boulet et al. 2016) and to date no phenomenological model exists which can reliably take all these aspects into account, particularly when pulse timing and amplitude of pulses in the pulse train vary as necessary for coding temporal fine structure for binaural hearing. Here, modelers rely on more data from physiological measurements. Biophysical models aimed at describing such temporal interactions will also benefit from more information about which particular ion channels are present at different positions along the length of ANFs (Negm and Bruce 2014; Boulet et al. 2016; O’Brien and Rubinstein 2016).

To date, commercial CI strategies do not particularly use models to predict the stimulation current and pulse timing; however, it is envisaged that this will change in the future. While standard CI processing relies on a linear filterbank, some strategies have been developed which use nonlinear cochlear models for the filtering stage in a CI processor, so far with mixed outcomes (Wilson et al. 2005; Harczos et al. 2013). At the next stage, current spread can be taken into account when computing the electrode stimulation pattern. With a strategy that considers masking patterns, which are related to current spread, before selecting the channels to be stimulated in an n-of-m strategy (Nogueira et al. 2005), slight performance gains and a significant improvement of battery life could be achieved (Buechner et al. 2008). In other words, if channels that are masked by a previous pulse (i.e., many nearby neurons are in their refractory state) are not stimulated, performance does not degrade, but rather “unnecessary” stimulation pulses can be omitted. This approach is taken further in recent strategies based on ideas from computational models, and we envisage that the use of computational neural models in implant strategies holds high potential. Babacan et al. (2010) present the Refractory-State-Coding strategy which considers the percentage of neurons available for stimulation according to their refractory state after a preceding stimulating pulse. The model is fitted to individual patients taking spread of excitation information from ECAP measures into account. In an evaluation of a related approach with normal hearing listeners listening to vocoded signals, somewhat improved vowel boundaries were observed (El Boghdady et al. 2016).

Last, we want to highlight the issues with coding monaural and binaural temporal information in the stimulating pulses of CIs and how model predictions can aid in the development of new strategies. Above about 300 pulses per second, CI users lose the ability to follow rate pitch (Baumann and Nobbe 2004) and to extract interaural time differences from pulse timings (Laback et al. 2015). These effects have not yet been investigated with binaural CI models. However, various approaches exist to code temporal information in electric pulse trains. Temporal information can be extracted from the instantaneous frequency (FAME: frequency amplitude modulation encoding; Nie et al. 2005), from signal amplitude peaks (PDT: peak derived timing; Hoesel and Tyler 2003), or from zero crossings of the signal (Zierhofer 2001) and serve as the time points for the stimulation pulses. Depending on the sound, the rate of these pulses can be high, limiting benefit due to the rate limitation. Further, perceived loudness depends on the pulse rate which is given by the periodicities in the sound, leading to an unwanted dependence of loudness on fundamental frequency. These issues could be addressed by stimulating with a rate sub-sampled from the frequency observed in the respective channel (Smith et al. 2013). However, while low rates are good for transmitting temporal information, high rates are beneficial to sound quality and speech understanding (Churchill et al. 2014). To overcome this dilemma, interaurally phase locked activity in the nerve can be evoked by introducing interaurally linked temporal jitter (Laback et al. 2015) or by introducing a temporal gap followed by a steep rising envelope flank (Monaghan and Seeber 2016) into otherwise high-rate pulse trains. Using computational neural models, pulse timings and amplitudes could be computed to evoke a specific pattern on the auditory nerve that incorporates the required temporal information, making a model-based design of strategies possible.

Turning from electrical to optical stimulation would change the way CIs operate, and it is hoped that this could overcome the issues associated with the severe electrical current spread in today’s CIs. The review by Weiss et al. (2016) in this issue discusses the prospects for optically stimulating CIs along with first models for optical stimulation of neurons after genetic transfection with light-sensitive channels and models for beam propagation in the cochlea. Three aspects appear most critical at present before optical CIs can become a clinical reality: (1) achieving a defined, selective stimulation of spiral ganglion cells with a focused light beam, (2) inserting light-sensitive channels with high sensitivity and sufficient temporal resolution into the cells, and (3) solving clinical issues of the safety of gene transfection and the use and implantation of optical stimulation systems. Current prospects are to achieve a spatial resolution with optical stimulation up to 10 times the resolution of current electrically stimulating CIs, which would be a significant improvement. However, resolution is limited by the pointing precision of the light sources, by beam width, by light scattering, by diffusion from propagation through the bone, and by neural survival. The temporal characteristics of the light-sensitive channels are highly debated, as their time constants are on the order of milliseconds and hence slow for auditory stimulation. However, the activation time constant of Channelrhodopsin2-channels is comparatively fast at 0.6 ms, while the deactivation time constant is as slow as 20 ms (Kleinlogel et al. 2011), together still allowing for sustained firing rates of up to 200 pulses per second. New types of rhodopsins might allow faster deactivation, and the search is ongoing. Another aspect is how to safely transfect the spiral ganglion cells to become light sensitive on a clinical scale. Operating with viruses for transfection in humans might entail risks which are even unknown at present. While optical stimulation bears high potential to solve some of the electrical CI’s most pressing problems, many development stages have first to be overcome, and modeling should play an important role in determining optimal approaches to optical stimulation.

Given the ever increasing computational power, we believe that computational models of CIs will make important contributions to the development of physical implants and stimulation strategies in the future, while also finding their way into the fitting process of CIs and potentially directly into stimulation strategies. We hope that this special issue of Network: Computation in Neural Systems will serve as an introduction and inspire researchers to use and develop computational models of cochlear implant stimulation.

Acknowledgments

We thank Dr. Marko Takanen for comments on an earlier version of the article.

Funding

This work was supported by the German Federal Ministry for Education and Research (BMBF 01 GQ 1004B) through the Bernstein Center for Computational Neuroscience Munich and by the Natural Sciences and Engineering Research Council of Canada (Discovery Grant 261736).

Additional information

Funding

This work was supported by the German Federal Ministry for Education and Research (BMBF 01 GQ 1004B) through the Bernstein Center for Computational Neuroscience Munich and by the Natural Sciences and Engineering Research Council of Canada (Discovery Grant 261736).