Remote check test battery for cochlear implant recipients: proof of concept study

Abstract

Objectives

(1) To investigate the remote check test battery, designed for self-administration by cochlear implant (CI) recipients, parents/caregivers, to determine if the results give adequate information for clinicians to decide the necessity of an appointment and to capture suggestions for improvement. (2) To gauge acceptance of remote monitoring by CI-recipients and their parents/caregivers.

Design

Prospective, multicentre, un-blinded, non-randomized, single-subject, repeated-measures evaluation. The test battery includes an implant-site photograph, impedance measurements, datalogs, questionnaires, speech perception and aided threshold tests. Clinicians reviewed test battery results, followed by a clinical appointment with each CI-recipient, and reported if the battery identified all the issues. Study sample: n = 93 CI-recipients (73 adults, 20 children) and 28 clinicians.

Results

The test battery identified 94% (615/656) of all issues. The test battery and clinician observations agreed in 99% (92/93) of cases on the need for a clinic visit. For 68% (63/93) of cases, the test battery identified all clinician observed issues. The majority (77%, 72/93) of recipients would be satisfied if clinic visits were based on their test battery results. A significantly high proportion agreed that remote monitoring was more convenient than clinic visits and could result in travel, time and cost reductions.

Conclusion

This is the first comprehensive test battery designed for CI-recipient remote monitoring.

Keywords:

• Hearing
• e-health
• telemedicine
• tele-audiology
• Digit Triplet Test
• Aided Threshold Test

Introduction

Recent data from the World Health Organisation (WHO) indicates that approximately 466 million adults and children have disabling hearing loss. The majority live in low and middle-income countries (WHO 2020) and may face reduced access to hearing health care.

Cochlear implantation is the preferred treatment for adults and children when hearing loss meets local cochlear implant (CI) candidacy criteria and hearing aids are not beneficial. CI recipients and hearing aid users require professional clinical management in order to obtain optimal benefit from their devices. Individuals with significant hearing loss living in developing as well as developed countries can experience limited contact with hearing professionals because most hearing aid dispensers and CI centres are located in medium to large cities; many of these individuals live in smaller, less urban areas and travel may be difficult for various reasons (Fabry 2010; Swanepoel et al. 2010; Slager et al. 2019). The current COVID-19 pandemic has created additional challenges in obtaining hearing health care as well as difficulties for professionals in providing safe care to their patients. During this public health crisis, various telehealth services have provided opportunities to serve individuals who live long distances from their providers ensuring safe ongoing care for all individuals (Clark et al. 2020). Studies have found that adult and paediatric CI recipients have received telemedicine follow-up services that are the same as or similar to the usual services provided in the clinic (Hughes et al. 2012; Hughes, Sevier, and Choi 2018; Slager et al. 2019). A remote care pathway can offer increased patient empowerment and improved hearing outcomes (Cullington et al. 2018).

Routine cochlear implant follow-up appointments conducted in a clinic may include CI re-programming and troubleshooting, counselling regarding device use and communication abilities/strategies, evaluation of speech perception, hearing thresholds and demonstration of additional or new equipment. During the appointment, the clinician assesses if there are implant or external device issues, and addresses hearing performance concerns by (1) examining the ear, implant site and external equipment, (2) performing impedance and/or neural response telemetry, speech and hearing performance measures, (3) reviewing datalogs/device settings and use, and (4) talking to recipients/caregivers about their experience and concerns. Clinicians use these evaluations to develop their management decisions, which could include referrals to other team members or other medical specialties (Müller and Raine 2013).

Clinical experience suggests that most implant recipients will require more frequent visits during the first year following implantation; however, after this clinical follow-up tends to be on a semi-annual or annual basis (Müller and Raine 2013; Hemmingson and Messersmith 2018) . Currently, prior to the in-person appointment, clinicians do not have a standard way to determine if any potential issue/s have arisen that require clinical action. Thus, follow-up visits are often scheduled routinely with a set amount of time, generally for an hour, to avoid missing care for anyone.

Howe and Mawman (2015) reported the results of their clinic’s audit to examine the clinical efficacy of routinely scheduling adult annual appointments versus a process that was patient-initiated. They found that most adults’ implant and functional hearing performance remained stable over time. After these audit results, their clinic adopted an optimised patient-directed system for clinical reviews that has been well accepted by their recipients. Gajadeera et al. (2017) studied electrical stimulation levels over an 8 to 10-year post-implant period for a large group of adult CI recipients. They discovered that the majority of recipients showed minor changes in threshold (T) and comfort (C) levels and dynamic range over this time period. They suggested that recipients who maintained consistent stimulation levels for several appointments after the first six months of device use could be seen less frequently. Children typically require more appointments compared to adults and their T and C levels have been shown to stabilise after the first few years (Hughes et al. 2001 and Incerti et al. 2018). Re-programming appointments could be at the recipient’s needs and/or request as opposed to a fixed schedule, which would help to maximise time and resources for other clinical services. For patients and their parent/caregiver, unnecessary in-person appointments may result in additional burdens of transportation time and costs, time away from work, loss of income, or school absence and missed tuition.

Ideally, clinicians could determine if there were issues that required clinical management without scheduling an in-person clinic appointment. The conceptual framework of the remote check battery was to provide a set of quick, self-administered tests that could be easily run at home by an adult or child implant recipient or their parent/caregiver. Test results would be sent automatically to the recipient’s clinician for review. It was hypothesised that based on self-assessment test results the clinician could decide if a recipient required an in-person follow-up appointment or not. If an appointment was required, it was hypothesised that the results and issues identified from the remote check battery would help the clinician plan for the type and length of the appointment. Portions of the proposed remote check battery are typically performed in person during routine clinical management. It was unknown if the proposed remote check battery for in-home use would provide the clinician with the right types of information on possible issues to determine actions for clinical management. Table 1 lists the clinical issues, clinical actions and tests included in the remote check battery to identify the clinical issues of focus.

Table 1. Issues the remote check battery is designed to identify.

Issue	Clinical action	Tests used in clinic	Tests used in remote check battery
Electrode faults	Deactivate electrodes	Impedance test	Impedance test
Poor usage	Counselling	Datalogs	Datalogs
Non auditory sensations	Programming	Questions	Questionnaire
Poor device usage due to training issue	Provide training	Datalogs, questions	Datalogs, questionnaire
Device issues	Replace parts	Questions	Questionnaire
Medical issues (skin flap / soreness etc.)	Ear nose throat referral	Visual inspection, questions	Photo, questionnaire
Poor sound quality	Programming	Questions, speech perception test, test for audibility	Questionnaire, Digit Triplet Test, Aided Threshold Test

This is a proof of concept study to evaluate the proposed self-assessment remote check battery targeted for in-home use. The goals of this study are to (1) evaluate if the test battery, when carried out by the recipients or parent/caregivers, provides sufficient information to the clinician to determine if a clinical visit is required, (2) identify any changes required to improve the remote check battery and (3) evaluate the acceptance of the concept of a remote check for recipients or parent/caregivers. Findings from this study were used to refine the remote check battery prior to commercial implementation on a smartphone app.

Materials and methods

Study design

This multi-center study used a prospective, un-blinded, non-randomized, single-subject, repeated-measures design where each subject served as his or her own control. Blinding of clinicians was not feasible due to their required judgment regarding the validity of the information provided via remote and in-person assessments on an individual patient basis. The study was conducted over 23 months from first subject enrolment to last patient visit and assessment. All tests were completed at the clinic during scheduled visits in a single session and presented no additional risks over routine clinical care. The study received national ethics approval from the East of England – Cambridgeshire and Hertfordshire Research Ethics Committee in the UK (REC reference 16/EE/0177) and RVEEH HREC (reference16/1305H) in Australia, and approval at each collaborating site in line with local requirements. The study was listed on the ISRCTN registry (ISRCTN13600782). The study sponsor was Cochlear Ltd.

Research participants

Ninety-eight CI recipients were enrolled across four sites in the United Kingdom and one site in Australia. All eligible recipients who had scheduled appointments for routine clinical management at one of the investigational sites were invited to voluntarily participate in the study; those that indicated they were interested were asked to come to the clinic an hour before their appointment time. The inclusion criteria were: implanted adults ≥18 years old or children ≥4 years old, with Nucleus^® 24, Nucleus^® Freedom or Nucleus^® 500 series cochlear implants; ≥3 months’ implant experience; familiarity with English vocabulary for digits zero to nine as used in the Digit Triplet Test (DTT); scheduled for a routine clinical appointment during the study period. The demographic details of the recipients enrolled are given in Table 2. Twenty-eight clinicians took part in the study.

Table 2. Demographic details of participating recipients.

	Adults	Children
Number of recipients	73	20
Gender
Male	36	7
Female	37	13
Mode of hearing
Unilateral	46	3
Bilateral	5	16
Bimodal	22	1
Cause of deafness
Congenital	10	12
Progressive	54	5
Sudden	7	2
Unknown	2	1
Age at test (years)^b	60 (19–87)	12 (6–17)
Age at onset of hearing loss (implanted ear) (years)^b	24 (0 to 70)	1 (0–12)
Duration of hearing loss (preimplant) (years)^b	30.4 (0.5–68.5)	5.1 (1.5–12)
CI-experience (yrs)^a^,b	5.1 (0.33–18)	5.5 (0.25–14)

^aBilateral cases – second ear implant date referenced; ^bmean(range).

Methods

Participants were enrolled after written informed consent was obtained. Tests were always administered in the same sequence; Figure 1 shows a flowchart of the sequence of procedures. Each enrolled participant or parent/caregiver administered the remote check battery on-site prior to their scheduled clinical appointment. Prior to seeing the recipient for their clinical appointment, the clinician reviewed the results of the remote check battery to identify any potential issues. Clinicians then made a judgement using the remote check data and their knowledge of the recipient and clinical expertise if the recipient would need to be seen in person at the clinic. Then, the clinician conducted the scheduled in-person appointment anyway. Afterwards, the clinician documented any new issues found during the appointment or whether the test battery had identified all issues.

Materials and evaluations

Rationale and description of remote check battery

The test battery was designed to use tests and questionnaires that could ultimately be completed on a mobile device. It comprises the following procedures (Figure 1): photographs of the skin over the implant site, impedance measurements, datalogs, The Digit Triplet Test (DTT), The Aided Threshold Test (ATT), and recipient or parent/caregiver questionnaires.

Figure 1. Study procedures in sequence administered.

Implant site photographs

In some cases, if the magnet is too strong, the transmitter coil can cause excessive pressure on the skin over the implant leading to skin irritation; medical and/or clinical action may be necessary. Visual inspection of the skin over the implant site can help detect such issues. In a study by Patricoski et al. (2003), two otolaryngologists reviewed video otoscopic still images of the tympanic membrane after tympanostomy tube placement and compared their findings to the later microscopic examination of the patients. Results indicated that a review of images was comparable to an in-person examination. In our study, photographs of the scalp over the implant site were taken with an iPad by the person accompanying the recipient, that is, the parent/caregiver, or if unaccompanied, by the investigator. To ensure clear and informative images, instructions were given to remove the coil and part the hair to expose as much skin as possible over the site of the implant’s magnet beneath the skin. Although no specific criteria were provided to indicate when a clinical visit would be necessary, typical concerns would include evidence of redness, swelling, or recipients’ descriptions of pain or discomfort in the area.

Electrode impedances and datalogs

The feasibility of collecting datalogs and completing impedance telemetry as self-tests has been demonstrated in commercially available products. The Nucleus Smart app automatically collects datalogs from CP1000 series sound processors. The CR230 Remote assistant with CP800 series sound processors can be easily used by recipients to complete impedance telemetry. However, the display of this data for clinician review has only been implemented in the Custom Sound software. In this proof of concept study, as it was important for the clinician to be able to easily review the data, these data were collected using Nucleus^® Custom Sound™ Suite software 4.4 or 5.0. Electrode impedances for all implants were measured via telemetry by the investigator datalogs were automatically downloaded when the clinician connected the recipient’s sound processor to the software.

Electrode impedances

A review of impedance test results is well established in clinical practice. Typically, electrode impedances are measured intraoperatively and at postoperative appointments by the clinician using Cochlear Custom Sound Suite. The software provides the range of impedances considered to be within normal limits. Electrodes with impedances below 565 Ohms have a short circuit, electrodes with impedances above 25–30 kOhms are open-circuit and these electrodes should be deactivated. Clinicians use this information when making program adjustments and managing possible clinical issues (Goehring et al. 2013).

Datalogs

Datalogs provide clinicians with important information about device use patterns and the possible impact of consistent use, or lack thereof, on hearing performance. Clinicians rely on this information for troubleshooting, program optimisation and counselling (Easwar et al. 2016; Cristofari et al. 2017; Easwar et al. 2018.). Specific criteria regarding when information from the datalogs would trigger clinical management were not given, rather the clinician used their clinical judgement and knowledge of the recipient’s history.

Digit Triplet Test

One test battery requirement was that the speech perception test be quick, easy and self-administered by adults and children, including those with poor speech perception or language abilities. The Digit Triplet Test (DTT) developed by Smits, Kapteyn, and Houtgast (2004), is a self-test, which uses monosyllabic digit triplets (e.g. 5-3-6) instead of sentences or monosyllabic words, was included in the test battery. It is an adaptive speech reception threshold (SRT) test where the signal-to-noise ratio (SNR) increases or decreases based on the response given to each test item. The SRT is the point where the respondent identifies half (50%) of the items correctly. Smits and collaborators assessed 38 (76 ears) listeners with hearing impairment and found the DTT correlated well with an existing Dutch sentences-in-noise SRT test (Plomp and Mimpen 1979; Smits, Kapteyn, and Houtgast 2004). Kaandorp et al. (2015) tested 12 individuals with normal hearing, 24 hearing aid (HA) users and 24 CI users on the Digits in Noise test (DIN) which is similar to the DTT. They found that results for the DIN and Dutch sentences in the noise test were highly correlated and they concluded that the DIN was an appropriate measure for monitoring the hearing performance of HA and CI users. Cullington and Agyemang-Prempeh (2017) tested 16 adult CI recipients with the DTT and compared results to the Bamford–Kowal–Bench sentences in quiet and noise (Bench, Kowal, and Bamford 1979). They determined that the DTT had no floor or ceiling effects and correlated well with BKB sentences in quiet and adaptive noise when tested in the clinic. Koopmans, Theo Goverts, and Smits (2018) showed that children with normal hearing, 4 years and older, could reliably complete the DIN.

In the remote check test battery, the DTT was administered using a custom-built iPad app that was calibrated to present signals via the personal audio cable to the Nucleus^® CP910 sound processor set to 100% accessory-only setting. Three digits in speech-weighted noise were presented in an adaptive paradigm with the signal and noise changing in tandem so that the overall output is 65 dBA. The participants enter the digit triplet they hear onto the iPad. An initial run was completed for practice and the scoring began with the next run. Two sets of eight triplets each were presented with an opportunity to take a break between the sets. The score is the mean SNR of the 16 triplets. Test results were deemed unreliable and the test was repeated if the standard deviation of the SNR was greater than 2.6 dB.

Aided Threshold Test

Clinicians routinely measure recipients’ warble tone thresholds in the sound field while they are listening with their CI. They compare these against predetermined criteria and may adjust program parameters based on the results (Vaerenberg et al. 2014). Unaided pure tone audiometry implemented for self-testing on an iPad has been shown to provide results equivalent to thresholds using standard audiometric equipment (Thompson et al. 2015). In our proof of concept study, the Shoebox Audiometry App (https://www.shoebox.md) was used to measure aided thresholds at 0.25, 0.5, 1, 2, 4, and 6 kHz. The app was calibrated to present signals to the CP910 sound processor via the personal audio cable. Calibration involved measuring the output from the processor placed in an anechoic test box in response to the calibration signal as a reference. The app was calibrated by adjusting it so that when the calibration signal was streamed to the processor via the personal audio cable, the output from the processor matched the reference measurement. The recipient pressed a button on the app to start listening and then moved the button to “yes” or “no” icons to indicate if they heard the signal. The responses were automatically assessed as unreliable if the recipient indicated that they heard the sound when there was no stimulus, they were then asked to repeat the test for that frequency. The aided audiogram for each recipient was available for the clinician to review.

Recipient/parent/caregiver questionnaires

During a clinic visit, recipients provide subjective feedback to their clinician regarding hearing performance or any device issues. This is one of the most important sources of information to determine what clinical activities may be required during an appointment. The clinician solicits the recipient’s feedback by asking questions related to areas where there could be possible issues requiring clinical attention. Standardised and/or in-house customised questionnaires are useful in discovering issues in a consistent manner that might require clinical action (Howe and Mawman 2015; Cullington and Agyemang-Prempeh 2017). For the remote check test battery, adults and caregivers of children completed a customised multiple-choice questionnaire with 20 and 18 questions, respectively; some questions included sub-questions. Questions covered topics such as medical issues related to their ear or implant area, sound quality and device related issues, concerns that could necessitate program changes, or areas where they could benefit from training on use of their device. Adults were queried about any changes in their participation in listening activities. Seven questions from the Speech Spatial Qualities (SSQ) of Hearing scale (Gatehouse and Noble 2004) were included to assess self-perceived hearing ability in a variety of situations. Finally, an open-ended question asked if there were concerns, they wanted to discuss with their clinician.

Recipients or parents/caregivers also completed a separate questionnaire to evaluate the acceptability of the remote monitoring concept. They responded using a five-point Likert scale to indicate their level of agreement with the questions and statements.

Clinician questionnaire

Clinicians completed a questionnaire detailing the clinical review of their recipient’s test battery results. During this review, clinicians recorded the course of action they would take for each issue identified in the battery. Then at the end of their scheduled in-person appointment, clinicians reported all the activities they completed during the session, for example, programming, counselling, ENT referral. They also indicated whether the information that led to a clinical action was captured in the remote check battery or resulted from new information discovered during the face-to-face session. Clinicians were asked to specify if they completed activity as part of their clinic’s standard procedure or because they had additional time.

Sample size

The primary objective of the study was to evaluate if the outcome of the test battery matched that of the in-person clinical appointment. Since only two outcomes were possible (matches/does not match), sample size estimation showed that a minimum sample size of 29 subjects was required. To evaluate the acceptance of the concept of remote monitoring for recipients or parents/carers, sample size estimation showed that a sample size of 67 subjects would represent the views of the general population with a 90% confidence level, and 10% margin of error. A total of 93/98 enrolled subjects completed the remote check test battery for analysis; hence the minimum sample size requirements were met.

Results

Ninety-eight subjects were enrolled. Data were available for analysis from 93 (95%) recipients including 73 adults and 20 children. Of the 98 enrolled individuals, one participant withdrew prior to completing any study procedures due to lack of time and four were discontinued because all study procedures could not be completed within the timeframe of their routinely scheduled visit.

Implant site photograph: Clear and useful photos could be taken for all 93 recipients. Based on the clinician’s assessment of the image, 17 adults and one child required clinical action: change of magnet strength (n = 6), counselling (n = 6), referral to ENT (n = 2), or monitoring for a period of time (n = 4).

Electrode impedances: In all but three cases (90/93, 97%), clinicians determined that electrode impedances were within normal limits. Clinicians decided that reprogramming was required for the three recipients with impedances outside normal limits.

Datalogs: Using their clinical experience, clinicians identified 69 issues from 39 recipients. In summary, the issues identified included: Time on air or Coil offs (9 recipients), indicating < 3 h/day “time on air” or a relatively high number of “coil offs”; Time in speech (13 recipients) considered low, ranging from 0.3 to 1.7 h/day; and Program use (14 recipients) suggesting excessive changes between programs. In 29 recipients, accessory usage issues were identified for non-use or use in only one ear for bilateral recipients. For these identified issues, clinicians advised the following actions: counselling (80%), program adjustments (19%), or replacing parts (1%).

Digit Triplet Test: Speech measures were completed for 89/93 (96%) recipients. Due to technical issues or time constraints, the DTT was not completed for the remaining four recipients. Three adults and two children had scores > 20 dB SNR suggesting the test was too difficult for them and their data were excluded from analysis. Data from 84 individuals, 66 adults and 18 children, were analysed. The adults’ average score was −4.30 dB SNR (range: −11.2 to 11.6 dB SNR). The children’s average score was −4.5 dB SNR (range: −10.3 to 8.5 dB SNR). Comparison of DTT with other speech tests routinely conducted at the clinic was not performed as it was not part of the objectives of this study. There was a moderate negative correlation (r = −0.47, p = 0.015) between the age and DTT score for children. Clinicians used clinical judgement to evaluate the DTT scores crosschecking with the reports of difficulties in speech perception in the questionnaire and aided thresholds along with the normative values of DTT from published literature to consider further action.

Aided Threshold Test: ATT was completed for 89/93 (95%) recipients. ATT could not be completed for two adults due to technical issues and for two bilaterally implanted children (aged 8 years and 12 years) in the second ear due to lack of cooperation. For eight other children, reliable responses were not obtained for all frequencies tested. One participant who had tinnitus had elevated thresholds (above 30dBHL); however, three other participants with tinnitus had aided thresholds within 30dBHL. Group means aided thresholds across all test frequencies for adults (n = 71) and children (n = 20) were 23.0 dB HL (range: 10 to 90 dB HL) and 21.8 dB HL (range: 10–70 dB HL) respectively. The criterion used to determine the need for clinical management varied across clinicians. Further action was suggested by clinicians for individuals with thresholds ranging from ≥10 dB to ≥45 dB HL at one or more frequencies. For the majority, 75% (61/81), further action was recommended for individuals with an aided threshold ≥30 dB HL at one or more frequencies.

Questionnaires: A completed recipient questionnaire was available for 93 recipients. Figure 2 provides a breakdown of the types of issues into several categories that were identified by the questionnaire, the number of issues per category for the total cohort, and per adult and child subgroups. The largest proportion of issues identified related to sound quality, followed by device-related and training issues for the total cohort and both subgroups. Questions relating to changes in behaviour were not queried via the parent’s/carers version of the questionnaire, while a small number of issues were reported for adult participants.

Figure 2. Types of issues identified by the recipient questionnaire. n = 93, 73 adults and 20 children.

Is the test battery sufficient to determine clinical action?

A total of 656 clinical issues were identified across the 93 recipients with the test battery identifying 94% (615/656) of all issues. The number of issues identified by the test battery was significantly higher than the number of issues missed (6%, 41/656) by the battery (p < 0.0001). In all but one case (92/93, 99%) the test battery reached the same conclusion as the clinician in determining whether the recipient required any clinical action. In 99% (91/92) of recipients, the battery and clinician identified one or more issues requiring action. In one case, no issues were identified by the battery or clinician. In only one case (1%, 1/93), the test battery did not identify any issues while the clinician identified two issues requiring attention.

Figure 3 shows the percentage of issues identified by the different components of the battery and those identified additionally by the clinician.

Figure 3. Percentage of total issues identified (n = 656) by the different components of the remote check battery, and the percentage of issues identified additionally by the clinician.

For 68% (63/93) of recipients, the test battery identified all issues that were found by the clinician. In 32% (30/93) of recipients, the clinician found at least one more issue not identified by the battery. For these 30 recipients, while the battery identified issues that would have resulted in a clinic visit for 29 recipients (97%), the clinician recognised additional issues that were missed by the battery. One-way Chi-square analysis found that the number of recipients (n = 63/93, 68%) where the test battery successfully identified all issues identified by the clinician was significantly higher than the number (n = 30/93, 32%) where the test battery failed to recognise one or more issues (p = 0.0009). When the results for adults and children were analysed separately, the cases of adults (48/73, 66%) and children (15/20, 75%) where the battery successfully recognised all issues were significantly higher than those where the clinician identified additional issues (p = 0.01 and p = 0.04 for adults and children, respectively).

Suggested improvements to the test battery

During the in-person appointments, clinicians identified 41 issues (in 30 recipients) that were not identified by the test battery; these included missing information, misunderstood questions, unclear information on accessories owned by the recipient, comfort level sweeps and emerging device issues.

Missing information: The test battery included tests to assess performance like ATT, DTT, SSQ. However, clinicians identified additional subjectively reported sound quality issues like “hissing sound” in 5 recipients and “sound not being loud enough” in 4 recipients. In addition, clinicians identified two cases where the Dry aid kit was not used and one case where help obtaining spare parts like a microphone cover was needed. Therefore, questions to be incorporated into the recipient questionnaire include queries related to general sound quality, sound not being loud enough, use of the Dry Aid kit and assistance in obtaining spare parts.

Based on the evaluation by clinicians in relation to one or two select cases the following proposed additions in a modified test battery were made: more detailed datalogs; compliance telemetry; and a photo of the surgical scar behind the ear.

Misunderstood questions: During the clinical session, clinicians found 11 issues related to questions in the questionnaire that were intended to capture the necessary information, however, recipients appeared to misunderstand the questions. When completing the questionnaire, recipients responded that “they did not have the issue” or something similar however during the clinical session the clinician identified that the issue was present. There were four problematic questions. The proposal was made to reword the questions in subsequent questionnaire versions to improve the reader’s interpretation.

Unclear information on accessories owned by the recipient: The clinicians identified the necessity for accessories such as the Aqua + or wireless accessories in 6 recipients. During the clinical appointment, the clinicians had to determine this need by eliciting feedback from recipients in person as it was not captured in the test battery. Furthermore, they did not have access to the professional portal where the list of accessories owned by the recipient is listed. Making the professional portal accessible to the clinician would provide useful information during the review of the test battery results.

Comfort level sweep: Albeit in only one case, during a comfort level sweep by the clinician, the recipient indicated some channels were softer than the others, although no self-report of any sound quality or performance issues was made via the test battery or during the in-person session. The proposal was made to consider the inclusion of a recipient-administered test at home to assess loudness levels for different frequencies. Such information may be helpful for the clinician to determine the need for program adjustments.

Emerging device issues: During the clinical session, clinicians inspected the external equipment to identify potential issues based on the physical status of components, for example, very hard or brittle cable. Parts were replaced for seven recipients. The recipient questionnaire identified 12 issues as actual device issues but did not identify future potential issues. No further amendments to the questionnaire were proposed.

Removing redundant questions: In order to remove redundancy, correlation analyses were performed to investigate multi-collinearities among answers to questions. Issues that could result in reduced/declining hearing ability were assessed by 18 different items, that is, electrode impedances, DTT, ATT, four different datalogs, and 11 questions. Redundancy in responses was observed for 3/152 (2%) comparisons with a strong correlation (>0.7) between responses. No recipient reported issues related to avoidance one to one conversations, or additional training needs specific to Dry aid kit use or charging batteries. As a result, these questions were considered for removal from a modified test battery.

Recipient and parents’/carers’ perceptions

Seventy-seven percent (72/93) of recipients and parents/caregivers reported they would be “satisfied” or “very satisfied” if their clinic visits were based on results from the self-administered remote test battery (p < 0.001). Reasons for satisfaction included: the convenience of remote monitoring; the ability to request an appointment if needed; and the continued involvement of their clinician. Ten recipients (11%) expressed they were neutral to the concept. The remaining 12% (11/93) responded they were “dissatisfied” or “very dissatisfied” with the concept of remote monitoring, nine of whom added they would prefer in-person appointments. Additional examination indicated that 78% (57/73) of adults and 85% (14/20) of parents/caregivers would be “satisfied” or “very satisfied” if their clinic visits were based on remote monitoring compared to those who would not be satisfied (p < 0.0001). Correlations between age, duration of use and satisfaction ratings were not significant.

On average recipients and parent/caregivers reported 2 h travelling to the clinic (range:15 min to 10 ¼ h) each time. There was no significant correlation between the amount of travel time and the individual satisfaction ratings. CI-recipients and parents/caregivers from the UK reported an average spend of £40.30 (range = £0 to £350) for transport, parking, time off work or other additional costs for each visit. CI-recipients and parents/caregivers from Australia reported an average spend of $20.80 (range = $0 to $55.00) for each clinic visit. There was no significant correlation between the actual amount of money spent and satisfaction ratings. Seventy-six percent (71/93) of respondents “agreed” or “strongly agreed” that remote monitoring could result in a significant reduction of personal time and money spent. There was a moderate statistically significant correlation between the perceived gains in time spent and expenditure and satisfaction ratings (r = 0.57, p < 0.0001). Seventy-seven percent (72/93) “agreed” or “strongly agreed” that remote monitoring was a more convenient option than travelling to the clinic. There was a moderate to strong statistically significant correlation between perceived convenience of remote monitoring and satisfaction rating (r = 0.67, p < 0.0001).

Discussion

Remote Check test battery comprising of six tests: ATT, DTT, photos of the implant site, Impedance test, datalogs, and a questionnaire, was hypothesised to be able to provide clinicians with sufficient information to decide if a recipient required an in-person follow-up appointment or not. This was a proof of concept study to assess the viability of the concept. This study used a collection of commercially available or research apps (for ATT, DTT, photos), Custom Sound software (for impedance and datalogs) and an online survey (for questionnaire). Most of the tests, except for the Custom Sound tests, could be self-administered. The vision was to learn from the outcomes of this study so that a purpose-built Remote Check app can be further developed.

The primary objective of this study was to evaluate if the test battery provided sufficient information for the clinician to determine if a recipient requires clinical management. In 92/93 recipients the test battery was successful in reaching the same conclusion as the clinician to the question of whether the recipient required any clinical actions or not.

The number of recipients for whom the test battery successfully identified all issues was significantly higher than the number of recipients where the test battery failed to identify all issues that were subsequently identified by the clinician during the in-person session (p = 0.0009).

The majority of recipients (91/93) enrolled in the study were observed to require some form of further clinical management as identified by the remote test battery and/or by the clinician. As the sample of CI-recipients was limited to individuals already scheduled for a clinical visit, either as part of clinical routine reviews or at their request, the study cohort likely represents a skewed subgroup of CI recipients with existing clinical needs. The accrued recipient group is therefore ideally suited to determine the sensitivity of the proposed remote test battery for its intended purpose in identifying issues via self-assessment. Future research to determine the specificity of the remote test battery in its final format may benefit from additional inclusion of recipients who are not scheduled for or already deemed in need of a clinic visit.

Most recipients were able to complete the ATT and DTT auditory assessments through self-assessment. The primary issues preventing task completion were lack of time or hardware issues. Elevated thresholds were observed for one of four participants who reported tinnitus. Clinicians typically employ pulsed or warble tones to overcome tinnitus issues in audiometry. In ATT, to emulate a pulse tone, the tone can be presented multiple times; however, this was not made clear to the recipients in this study. In the updated version of the test battery, instructions for ATT will need to make it clear that the tone can be presented multiple times when faced with tinnitus. Although the criterion used by clinicians varied greatly to determine if further clinical management was required based on aided thresholds only, the majority of recipients (75%) where further action was needed displayed aided thresholds ≥30 dB HL at one or more frequency. As we found in two children, cooperation to complete the tests may be an issue, especially when attention span, time of the day, or time to complete the test, among other factors, may influence their willingness and ability to complete the test measures. Therefore, attempts should be made to reduce the time necessary to complete the individual tests and the test battery, especially during a single session. Alternatively, tests such as the ATT could be completed at a separate, more amendable time to encourage the child’s willingness, alertness, and ability to cooperate with the test.

The majority of CI recipients completed the DTT as a self-administered speech in noise test, for 5/89 who attempted the DTT, the listening task was considered too difficult, as shown by their SNR (>20 dB) which is a relatively quiet listening condition. In three adults with an SNR > 20 dB, their duration of deafness prior to implantation ranged between 25 and 40 years. While clinically this is considered a long duration of deafness, examination of the variable duration of deafness and DTT SNR scores for the cohort showed no significant correlation. It is recommended that suitability to complete the DTT should ideally be determined by a screening or practice run DTT test. This may be considered in the clinic during the initial and early clinical visits immediately postimplant or remotely at a later stage. This may also serve as an intrasubject baseline to enable the assessment of clinically relevant changes over time.

Clinicians suggested that either baseline scores for each recipient or guidance on the interpretation of DTT and ATT results would be helpful in determining any action/s required for the individual. They also suggested ready access to additional complementary information, such as the individual’s hearing history would be beneficial when interpreting the results of the battery.

A commonly expressed concern among clinicians was the assessment of the sound processor microphones. The ATT and DTT tested by direct connect via the personal audio cable bypass the sound processor microphones and consequently do not assess them. As Cucis et al. (2019) have shown that the median microphone deterioration of hearing aids worn daily is 2 dB which can result in significant deterioration of speech perception over time. Typically, assessment of aided thresholds in the sound field includes the sound processor microphones by default. However, one may argue that standard sound field audiometry may not be sufficiently sensitive to detect microphone deterioration of 2 dB, given the typical clinical step size of ±5 dBHL. More sensitive methods to detect microphone deterioration that could be easily administered by the recipient as well as in the clinic would be clinically useful to monitor microphone function. The current best practice is to replace the microphone covers every three months, when they appear dirty, or there is a reduction in sound quality (Nucleus^® CP910 and CP920 Sound Processor User Guide).

Necessary changes to the test battery

The study also aimed to identify additional clinical metrics that may lead to potential improvements in the test battery for remote monitoring.

Clinicians identified a total of 41 issues that were not identified by the existing remote test battery. To bridge the gap between issues identified by the remote battery and the clinician during the in-person assessment, we identified missing information and questions that were misunderstood. The addition of new questions, revision of unclear wording in existing questions, the inclusion of compliance telemetry, more detailed datalogs, and an additional photo of the surgical scar behind the ear were recommended for inclusion. It is projected that incorporation of these recommended changes into an updated version of the proposed remote test battery would have the ability to detect 33 of the 41 missed issues for our study cohort. Thus, these modifications would lead to a potential improved detection rate of all reported issues observed for our study cohort, from 94% to 98% by the remote check battery.

While the test battery identified current device issues, it did not identify potential issues that could cause future issues, for example, potential issues identified through visual and physical inspection of device components by the experienced clinician. To identify emerging device issues, future best practice models could involve providing recipients with guidance to take responsibility for the timely detection of current and potential issues. Seven such issues were detected for our cohort by the clinician, but not by the current test battery. Improved datalogging feedback, if included in updated versions of the remote test battery, could assist in the detection of hardware issues during self-assessment.

A smartphone app that permits self-administration Remote Check test battery has been developed. The necessary changes identified from this study were implemented in the test battery. In the Remote Check app, the audio signals for ATT and DTT were streamed via blue tooth to the sound processor instead of routeing through a cable. A subsequent study (Maruthurkkara, Case, and Rottier 2021) evaluated the ease of use of self-test functions, test–retest reliability of the tests and compared the test results in streamed vs free field conditions.

Currently, adults and children visit their clinic regularly for mapping appointments so that medical, device, usage, training, speech perception or sound quality issues can be identified and resolved. Remote check test battery can be useful for identifying issues that are typically identified in mapping appointments so that unnecessary appointments can be reduced and the scheduled ones can be more focussed on the identified issues. Children with CI may also benefit from the reduced number or more focussed mapping appointments while continuing to have their habilitation appointments to monitor speech and language development as needed.

Conclusion

The proposed test battery is the first self-assessment battery designed primarily for at-home use by the CI-recipient, parent/caregiver to provide a comprehensive evaluation of potential issues experienced by the CI recipient, that informs the clinician regarding patient management, appointment scheduling, and required clinical actions. The test battery was successful in determining when a CI recipient required clinical management with a sensitivity of 94%, compared to clinician in-person assessment.

This study showed high acceptance of the concept of remote monitoring by the majority (77%) of recipients or parent/caregivers, who reported they would be satisfied or very satisfied if their clinic visits were determined based on remote monitoring. The majority of respondents recognised that the remote check battery has the potential to save time, reduce costs and increase the convenience of aftercare. Satisfaction ratings with the remote monitoring concept were moderate to strongly correlated with perceived improvement in convenience and time involved.

Results from this remote check battery investigation were used to develop a smartphone application that is currently being assessed by clinicians and CI recipients or parents/caregivers in several countries. The use of a remote check application in real life is an important step towards enabling CI recipients to monitor their progress and to help their clinicians determine and plan for clinical visits based on their needs. This may also further support global case management during times of recommended social distancing.

Authors’ contributions

SM was involved in study design, data analysis and write-up of the draft manuscript. HC was involved in ethics submission. AA, HC, JM, KA, and SJ were involved in data collection at their respective centres. All co-authors reviewed the draft manuscript and provided their approval for submission for publication.

Acknowledgements

The authors would like to thank the cochlear implant recipients and their families who participated in the study. The authors also thank Magdalena Margol-Gromada, Hannah Meakin, Nicola Hatton and Kenneth Munro for involvement in the conduct of the study. Thanks to Mridula Sharma, Josie Wyss and Anne Beiter for the review of the manuscript.

Declaration of interest

SM and KA are employees of Cochlear Ltd. HC performed paid consultancy work for Cochlear Europe from 2011 to 2017. No conflict of interests to disclose for AA, JM and SJ.

This study was sponsored by Cochlear Limited.