Review of outcomes and measurement instruments in cochlear implantation studies

The field of Cochlear Implantation (CI) continues to rapidly evolve, with professionals and industry exploring novel technologies, extended applications of existing technologies and other means of optimizing CI outcome. Such developments will have a financial impact upon healthcare systems and providers, and demand the development of evidence of efficacy, with evidence-based practice remaining the ultimate goal. Currently, effective comparison between CI technologies, populations, and evaluation of the added value of novel technologies is significantly hindered by heterogeneity in outcomes used and corresponding measurement instruments. Ultimately, this negatively impacts the shared decision-making (SDM) process between professionals and patients, because of the lack of definitive evidence.

A previous Editorial in this journal stressed the importance of developing agreed sets of outcomes (e.g. Core Outcome Sets (COS), http://www.comet-initiative.org) to standardize reporting of effectiveness, and enable the meaningful combination and comparison of data from multiple studies (Bruce et al., 2015). Other fields of medicine, such as Rheumatology and Gynecology, have embraced this approach to good effect (Boers et al., 2014; The CROWN Initiative, 2014).

To illustrate the extent of unmet need in the field of CI, we have conducted a review of outcome reporting in Cochlear Implants International (CIM). Studies published in this journal in 2015 (Vol 16: 1–6, S1, S2, S4) were included, with the exclusion of reviews with lack of new patient data, lack of effectiveness evaluation and those not involving CI recipients. All articles were reviewed for outcomes, corresponding choice of measurement instrument, time point of measurement, study design, study description, study sample, and age group of participants.

A total of 43 articles were reviewed, in which 17 different outcomes were used. Even greater variability was seen in the corresponding measurement instruments used, with a total of 72 instruments/tools recorded (Table 1). The most common outcome was speech perception (24/43 publications), with 25 measurement instruments being used for this outcome (Table 1). Furthermore, there was a lack of standardized time points for outcome measurement, with time points varying between 0 and 20 years after CI surgery. This wide variation in outcome domains and corresponding measurement instruments is in agreement with the findings of the systematic review of adult CI studies by Vila et al. who found 11 domains in only 8 studies (Vila et al., 2016). Although, only publications in CIM in 2015 were included, the inclusion of more volumes was deemed unlikely to lead to any major changes in our findings. Review of the first three issues of CIM in 2015 resulted in a total number of 15 outcomes, with only 2 outcomes added after analyzing the other issues.

Table 1 Overview of outcomes and measurement instruments, CIM 2015.

Outcome	No. of studies	Methods of measurement
Hearing	18	Aided thresholds Warble tones free field Electrical auditory brainstem response (EABR) Perceptual threshold (T-level) Loudest comfortable level (C-level) Fitting to outcomes expert (FOX) Stimulus recognition Visual analog scale (VAS) Pure tone audiometry Customized nucleus 5 and nucleus 6 questionnaires APHAB questionnaire Telephone profile Evoked compound action potentials (ECAP) Electrically evoked stapedius reflex threshold (ESRT)
Speech perception	24	Sound field speech perception test Hochmair–Schult–Moser (HSM) sentence test Sentence test with adaptive randomized roving levels (STARR) Fitting to outcomes expert (FOX) Bamford–Kowal–Bench (BKB) sentence test Open-set speech recognition Oldenburger Satztest (OlSa) list of 50 phonetically balanced French words’ (no further details) Freiburger monosyllables Arthur Boothroyd words Lafon words Marginal benefit from acoustic amplification sentences List 16 consonants (no further details) List 12 vowels (no further details) AzBio score Consonant-nucleus-consonant (CNC) Flemish recordings of the Dutch word test (NVA test) Monosyllabic word scores Bi-syllabic word testing Three-digit test Telephone profile Iowa closed-set sentence test Speech, Spatial, and Other Qualities of Hearing Scale for parents (SSQ-P) Adaptive McCormick Toy Test (MTT) The Chear Auditory Performance Test
Complications of surgery	12	Medical records reviewed Subjective rates of post-operative dizziness Pain score (1–10) Subjective description of pain
Social functioning	1	Focus group interviews with professionals
Academic functioning	1	Focus group interviews with professionals
Communication	1	Focus group interviews with professionals
Expressive language	2	Rosetti Infant Toddler Language scale Counting spoken word use and manual signs from 14 m video
Receptive language	2	MacArthur–Bates Communication Development Inventory: Words and Sentences Rosetti Infant Toddler Language scale
Compliance	4	Scale (poor-average-good) Medical records reviewed Diary Data logging Custom Sound Database Non-validated in house questionnaires
Auditory perception	5	Categories of Auditory Performance (CAP) scores Adaptive two-alternative forced choice staircase procedure Categories of Auditory Performance (CAP) – II scores
Fitting time	1	Fitting time noted
Listening effort	2	Meaningful Auditory Integration Scale (MAIS) Speech, Spatial, and Other Qualities of Hearing Scale for parents (SSQ-P)
Duration of surgery	1	Medical records reviewed
Quality of Life	4	Un-validated in house questionnaire SF-36 quality of life testing Hearing Handicap Inventory (HHI) for Adults Semi structured interview technique Glasgow Benefit Inventory
Tinnitus	2	Medical records reviewed
Sound localization	1	Speech, Spatial, and Other Qualities of Hearing Scale for parents (SSQ-P)
Hearing in difficult conditions	3	Hearing-In-Noise testing (HINT) Flemish sentences-in-noise test (LIST)

This review also highlighted the significant numbers of different measurement instruments used in CI studies. To an extent, this variation is inevitable, reflecting the need to use speech and language tests in the patient's native language. The nature of some measurement instruments allows for translation into different languages without affecting the validity of the measurement instrument. While other instruments (e.g. speech perception tests) may not be readily translated, as they are specific to the understanding of words and phrases in a particular language. In such circumstances, it may be useful to compare the proportion of CI patients within 1 and 2 standard deviations of the mean for normal hearing patients, when ‘non-translatable’ instruments are being compared. Alternatively, the design and use of novel measurement instruments not reliant upon understanding of language (e.g. phoneme tests), could address some of these difficulties. Similarly, speech understanding with digits may be an alternative to ‘traditional’ speech perception tests, with digits representing familiar stimuli known to people with even limited language ability, with testing having a smaller learning effect (Cullington and Aidi 2017).

The development and adoption of standardized sets of core outcomes has the potential to reduce the current levels of heterogeneity in outcome reporting for evaluations of new technologies and novel applications of existing cochlear implants. It should be noted that the use of a ‘general’ set of core outcomes may not be appropriate for all studies, as determined by the primary objective of the evaluation (e.g. music appreciation). Likewise, the benefits of CI in patients with additional needs (e.g. cognitive impairment) may not be best evaluated using a ‘general’ set of outcomes and measurement instruments. Benefit in this group of patients can be under-estimated if the chosen outcome domains and corresponding measurement instruments are inappropriate and/or unrealistic (Bruce et al., 2013).

Ultimately, the implementation of standardized sets of core outcomes and corresponding measurement instruments in effectiveness studies for CI in children, young people, and adults, would help to maximizing patient access and benefit from CI, by optimizing the evidence available to professionals and healthcare commissioners.