TabSINT: open-source mobile software for distributed studies of hearing

Abstract

Objective: The recent emphasis on outcomes-based medical research has motivated a need for technology that allows researchers and clinicians to reach a larger and more diverse subject population for recruitment and testing.

Design: This article reports on open-source mobile software (TabSINT) that enables researchers to administer customised hearing tests and questionnaires on tablets located across multiple sites. Researchers create and modify test protocols using text-based templates and deploy it to the tablets via a cloud-based repository or USB-computer connection. Results are exported locally to the tablet SD card and can also be automatically posted to a cloud-based database.

Results: Between 2014 and 2019, TabSINT collected 25,000+ test results using more than 200+ unique test protocols for researchers located worldwide.

Conclusions: TabSINT is a powerful software system with the potential to greatly enhance research across multiple disciplines by enabling access to subject cohorts in remote and disparate locations. Released open-source, this software is available to researchers across the world to use and adapt to their specific needs. Researchers with engineering resources can contribute to the repository to extend the capability and robustness of this software.

Keywords:

• Open-source software
• mobile applications
• medical information systems
• mobile learning
• biomedical informatics
• telemedicine

This article was based on a presentation at the 2019 Annual Conference of the National Hearing Conservation Association, and is part of the 10th dedicated supplement to hearing loss prevention

 

Introduction

The near ubiquitous adoption of mobile devices has presented researchers with remarkable new possibilities for collecting data on large populations of individuals in the places where they live, work, and socialise. The option to conduct large-scale studies on smartphones and other mobile devices has major appeal for researchers in the health, behavioural, and social sciences.

However, the barriers involved in actually implementing research protocols on mobile devices have limited their widespread use to either (a) very sophisticated research organisations that have the resources to programme and maintain their own custom applications (apps) and cloud-based server systems or (b) researchers with relatively simple research projects that rely primarily on the collection of survey or questionnaire-based responses.

This paper reports on TabSINT, an open-source software platform designed to facilitate the development of smartphone-based research protocols. TabSINT is directed towards users who possess intermediate technical skills but have neither the time nor resources to develop a custom application solely for the purpose of executing a single research protocol. TabSINT was designed to provide an infrastructure for handling the basic functions that are common in all smartphone-based research protocols (i.e. a common user interface, administrative controls for downloading new research protocols, and database capabilities for storing results and, when appropriate, transmitting them to a central server) while providing a flexible, text-based interface for developing new user-defined tests that are unique to the requirements of a specific user. TabSINT (which stands for Tablet Speech-in-Noise-Test) was originally designed to facilitate hearing research studies, so particular attention was focussed on the unique challenges of producing properly calibrated audio and multimedia signals that are accurate and consistent across different hardware platforms.

State of the art

A number of systems exist to create customised surveys or questionnaires, manage research study deployment and collected data, or administer audiometric-quality hearing tests. While many of these systems meet the needs of certain hearing studies independently, none provide all three capabilities in one consistent platform.

Surveys and questionnaires

Many commercial products exist for developing and deploying customised surveys and questionnaires on mobile applications. These products range from free, simple survey creators to fully featured research management platforms.

Researchers requiring simple surveys and questionnaires with minimal data security restrictions can leverage free web-based products like Survey Monkey, Survey Gizmo, and Qualtrics. These services provide simple tools to administer surveys to a disparate group of subjects and collect results in a tabular format on a hosted central server. These services, however, lack the ability to develop customised user interface components, handle multi-media files, and integrate with hardware peripherals to control and receive data. Data are generally hosted by the service provider without the ability to self-host or collect data entirely offline.

Research study platforms

Beyond basic survey software products, researchers can utilise more advanced software platforms developed specifically for running research projects with greater customisation and control. One popular platform is REDCap (Research Electronic Data Capture). REDCap provides a flexible and secure web application for building and managing online surveys and databases (Harris et al. 2009). It includes a comprehensive set of tools for collecting, managing, and analysing data in a variety of research contexts. REDCap is developed as a server application, meaning users must run and maintain their own server to utilise the software. While this is feasible at some academic institutions or larger companies, this severely limits the ability of the software to be widely accessed and used by smaller groups and private institutions.

Many other private software products and services exist to provide similar managed research study solutions, but these solutions can be costly and lack the ability to manage and present the media (i.e. calibrated audio recordings and videos) necessary for hearing studies.

Mobile hearing test systems

Several vendors have developed mobile applications to administer audiometric screening and diagnostic tests including HearScreen, uHear, EarTrumpet, HearingTest, and the ShoeBOX Audiometer. These applications generally rely on the internal sound card of the mobile device to process target media and a standard pair of headphones to present the sound to the user. While many of these applications have been developed and empirically researched as audiometers, few of these systems have the ability to administer general speech-in-noise tests (Brungart et al. 2017).

The ShoeBOX Audiometer system is an iPad-based hearing test system with support for testing with pure tones or speech stimuli (Rourke, Kong, and Bromwich 2016). While the ShoeBOX system is designed primarily as a tablet-based audiometer, it has the capability to administer calibrated speech-in-noise tests via its manual testing options. As an FDA approved device that is in compliance with ANSI standards, the ShoeBOX audiometer is designed to extend the capability of audiometers used in the clinic. However, clinical use of the ShoeBOX for measuring hearing thresholds requires initial and periodic recalibration of the specific combination of tablet and headphones used for testing, so the ShoeBOX system is not currently suitable for deployment of studies to pre-existing mobile devices. The software is also configured to be highly reliable for clinical applications, so it is not easily configured for specific study protocols that may require customised surveys as well as custom speech-in-noise tests and media.

Design

The aims of the present research were to implement and support a system to enable customised, distributed studies of hearing through a tablet-based interface. The system allows tests to be updated and modified remotely in near real time and can automatically submit data to a central cloud-based server, as shown in Figure 1.

Figure 1. TabSINT system design concept. Test protocols can be updated as needed and results can be automatically uploaded to a server or exported to the SD card of the tablet.

The overall system includes a mobile application that resides on a tablet, a cloud-based data server, and a test protocol that is generated by the researcher. The mobile application is implemented as a hybrid web application using standard web technologies so it can be easily deployed on a variety of platforms. The cloud-based data server can be one of two options: (1) a customised TabSINT web-server application that manages protocols, results, and calibrates media for specific tablets and headsets, or (2) a freely available cloud-based remote Git repository that hosts protocols for download to the tablet and receives uploaded results from the tablet. Git (https://git-scm.com/) is a powerful software utility used to manage and harmonise distributed file revisions. Using a Git-based platform enables automatic version control of protocols and results. The test protocol is a text-based configuration file that specifies the text, inputs, media, and logic for a specific study design.

TabSINT integrates these components to cover each part of the research iteration cycle: protocol design, protocol deployment, test administration, and results analysis. Protocols are designed by the researchers running a study by editing a text-based JavaScript Object Notation (JSON) template. The protocol is transferred to a tablet via a TabSINT web-application server, remote Git repository, or USB connection to the local Secure Digital (SD) card of a device. The TabSINT application processes and renders the text protocol into a set of interactive pages. The user steps through the protocol while the application records test data and results. Test results are automatically uploaded to the server, Git repository, or SD card of the tablet and are immediately available for analysis. The system is designed to provide researchers flexibility in deploying human research study protocols while minimising the effort required to administer the study.

TabSINT mobile application

The TabSINT mobile application is used to administer test protocols and manage test results from a mobile tablet or phone interface. The application is built as a web application and can be deployed natively to both Android and iOS operating systems. TabSINT can import protocols and export results to several types of back-end servers, as well as the local SD card of the tablet. These options provide the user flexibility when planning the management of data throughout a study or group of studies.

Application architecture

The TabSINT mobile application is implemented as an HTML5-based web application packaged for mobile devices using the open-source Apache Cordova library (Smutný 2012). Using Cordova, the application can be compiled and deployed to multiple mobile operating systems from a single code base. Cordova also provides support for plug-ins that allows the application to access device-specific utilities (i.e. camera, speakers, microphone). TabSINT leverages Cordova plug-ins to control the sound output level, play media, and communicate with external devices via Bluetooth.

While Cordova web applications have historically provided a suboptimal user experience compared to native applications, recent advances in mobile device processors and browser technology have made the performance indistinguishable (Charland and Leroux 2011). More importantly, this approach allows the application codebase to be reused across deployment targets and reduces the barriers required to edit and maintain the application code.

The TabSINT application is built on the AngularJS front-end framework (Balasubramanee et al. 2013). AngularJS is an open-source all-JavaScript framework maintained by Google that provides scaffolding to implement a single-page web application. AngularJS comes with a comprehensive testing framework that allows TabSINT to include a suite of automated quality control evaluations. These tests are automatically run at frequent intervals during development to ensure code stability.

Application user interface

When the TabSINT application is first opened, the user is presented with three navigation options: Admin View, Exam View, and Documentation. The Admin View contains app configuration options, hardware and software properties, and interfaces to manage test protocols and stored results. The Exam View is used to present and administer the current active protocol. The Documentation link opens the web-based app documentation in a local browser.

Admin view. The Admin View section allows a test administrator to configure the application, download and debug test protocols, and view and upload test results. The view is split into three pages: Configuration, Protocols, and Results. The Configuration page provides an interface to all the customisable properties of the application and presents details about the software and hardware running on a specific device.

The Protocols page provides an interface for adding and managing test protocols in the application. The user can choose to validate the protocol against the protocol schema before loading. TabSINT comes pre-installed with a “Feature Demo” protocol that includes examples of many different response areas and advanced techniques to help users develop new protocols.

The Results page of the Admin View provides an interface for viewing and managing results stored on the tablet. Test results are stored in the application’s persistent memory. Test results can be automatically uploaded or exported at the end of each test, or when the administrator manually chooses to output the results.

Exam view. The TabSINT Exam View renders the currently active protocol to the user. When a new test is started, TabSINT presents a ready page based on the top level properties of the protocol. Once the user begins the protocol, TabSINT presents the pages of the protocol in linear order, unless the protocol defines custom logic to control page flow. Figures 2 and 3 show an example protocol definition and the rendered exam view displaying a multiple-choice response area. Protocol pages consist of general informational text, user response areas, or custom defined response areas. At the completion of each page, the user’s response is appended to the results structure and the test moves to the next protocol page. Depending on the pre-defined logic of the protocol, the content presented on later pages can vary based on the user’s responses to earlier pages.

Figure 2. Example protocol definition for a multiple-choice response area using a JSON formatted text file (Figure 3 shows the user interface generated in TabSINT using this protocol).

Figure 3. Screenshot of a multiple choice response area within the exam view of TabSINT (Figure 2 shows the text-based protocol used to generate this user interface).

Figure 4. Schematic of the media processing algorithm used to calibrate and play back audio through wired headsets (Vic Firth or HDA 200) connected to TabSINT at user prescribed levels.

Figure 5. Number of tests recorded through the TabSINT system in four separate studies by the DoD since 1 January 2015.

Protocols can define custom “Navigation Menu” objects that will populate the menu in the top right icon of the Exam View. This allows the user to jump around the protocol to different sections without having to progress linearly through each protocol page. When a user reaches the end of a test, TabSINT navigates to a final test summary page. The test result is either automatically uploaded to the active server, or stored to the persistent application storage. Each test result is automatically backed up as a local text file to ensure that all results are available in the event of a server fault.

Data interfaces

TabSINT natively supports three different interfaces to import protocols and export test results: the TabSINT web-server application, remote Git repositories, and the SD Card of the device. The TabSINT web-server application was developed to provide an integrated TabSINT specific back-end for protocols, results, and protocol file processing. It employs an SQL database to store protocols and results which can then be queried remotely; it can also be extended to provide extra services, including sound card-specific calibration of media files and automated post-processing of results (as described later in this article).

The Git repository option allows for the remote management of protocols and results without having to develop and maintain a separate web server. TabSINT interfaces directly with GitLab (https://gitlab.com), a programme and web service for managing file repositories based on the Git version control system. Protocols are structured as single Git repositories hosted within GitLab. When configured, TabSINT will download protocol repositories from an instance of GitLab and store the files locally on the tablet. When changes are made to the repository, TabSINT only downloads the files that have changed since the last download. Test results are posted back to a separate “results” repository located within the same GitLab group. Users have the option to host their own GitLab instance, or use a free hosted version of GitLab.

The Device Storage option allows a user to load protocols and export results directly to the SD card of the tablet. Users select a directory on the tablet SD card to export results. Results are then automatically saved to this directory when a test is completed. Protocols and results can be transferred from the tablet to a computer via a USB computer connection. This option provides a simple way to use TabSINT when few mobile devices are needed and remote data access is not required or desired.

Integrations

TabSINT is designed to present customised test protocols with a combination of survey questions and audio stimulus. Audio is output through the headphone jack of the tablet, which can be plugged in to any compatible transducer (i.e. headset, audiometer). The level of the audio stimulus output to the transducer is controlled by protocol input parameters. TabSINT integrates with a custom server application (described in Section III) which processes protocol media to be presented at user specified sound pressure level (dB SPL) when connected to specific wired headsets, including the Vic Firth and HDA 200 headsets.

If the existing capabilities of TabSINT do not meet a user’s needs, the TabSINT application code includes an API to allow external contributors to build hardware and software that can interface directly with TabSINT source code. The API allows third-party users to incorporate custom features into TabSINT while leveraging the existing user interface and data management framework. The plug-in API allows external software to run custom code on events (i.e. test reset, page load) as well as include user interface elements in the Admin View and Navigation Bar that configure the plug-in or generate notifications.

As of December 2017, three plug-ins have been developed to integrate external features and hardware peripherals into the TabSINT framework. The public release of TabSINT maintained by Creare includes a plug-in to support a Creare-developed wireless (Bluetooth) headset designed to administer audiometric quality hearing tests outside of the sound booth (Meinke et al. 2017). Two additional plug-ins have been developed by external contributors to support a tablet-based sound level metre and a service to communicate via Bluetooth to a computer.

Open-source

The TabSINT source code is published open-source under the Apache v2 licence on GitLab. The repository is available at the URL https://gitlab.com/creare-com/tabsint. The source code on GitLab contains application source files, the command line interface, developer documentation, and the source code for the user guide. The README file in the repository provides instructions for setting up the development environment and installing the required dependencies.

Creare currently manages the TabSINT source code and releases updates on a monthly cycle. Releases are published alongside the repository directly on the GitLab. Each release is accompanied by a brief changelog documenting the changes in the new version.

Creare also manages a front-end website for TabSINT hosted at http://tabsint.org. This website provides a live demo of the application, access to a User Guide and Developer Guide, and links to Android releases. This website provides an easy resource for directing new users to TabSINT.

TabSINT server application

A custom Linux-based web-server application (“TabSINT Server”) can store and serve protocols and media, collect and present test results, and provide hardware-specific file processing for protocol media. The server architecture is implemented on a virtual Ubuntu Linux server running on Amazon Web Services, providing a basic LAMP stack of components (Linux, Apache, MySQL, and PHP). This architecture is designed to be flexible, secure, and provide simultaneous services to both the tablet application and the researchers via a web API.

The key feature of the TabSINT Server application is the automatic processing and calibration of protocol audio files. Based on predetermined characteristics of the tablet model and sound delivery device (i.e. a headphone model), the server modifies protocol audio files for playback through TabSINT at prescribed levels. Using this service, users can upload protocols containing uncalibrated media files and specify output levels in the protocol using units of dB SPL.

The processing algorithms support three distinct playback modes:

1. Arbitrary: The sound file is played with an output levelsuch that the A, C, or Z weighted Equivalent Continuous Sound Level (LEQ) of the output is equal to a user-specified target.

2. As-recorded: A reference sound file with a known, fixed LEQ (specified at upload time) accompanies the target sound file; this file is typically a recording made with a calibrated 94 dB, 1 kHz sound source. The device volume is calculated such that the output LEQ of the reference file would be equal to the known value, and then the target sound file is played at that volume.

3. WRT-reference: This playback mode is similar to the “As-recorded” method, except that the level of the reference file is specified at playback time instead of at upload time, and can vary between playbacks. This is useful for quickly adjusting the signal-to-noise ratio between two files, a common task in psychoacoustic testing.

The processing algorithms on the server calculate scale factors for each audio file based on the requested playback method and, if necessary, an input reference file. The audio files are then normalised and filtered for a specific sound delivery device response. The procedure generates a new set of protocol audio files and a JSON formatted calibration definition. The processed audio files and the calibration file are downloaded to the tablet together with a protocol.

Once on the tablet, the protocol requests playback of an audio file at a specific level and playback type. TabSINT rescales the requested level according to the playback type using the scale factors in the calibration file and then outputs the audio to the system at the adjusted level. TabSINT ensures the overall system output (the hardware volume control) is always set to 100% so that the audio is not inadvertently attenuated.

Figure 4 shows a schematic of the media processing algorithm from raw audio files to media played through the audio jack of the tablet.

TabSINT protocols

Researchers create and modify test protocols by editing a text-based JSON file. JSON is a human readable, language-independent format commonly used to describe data objects. Protocol files may be written from scratch or from templates included with the TabSINT software package. The syntax for writing a test protocol is defined and validated by an easily human readable JSON-Schema. TabSINT natively supports many different questionnaire and speech-in-noise response areas for use in protocols. Protocols may also generate specifically calibrated media when processed by the TabSINT Server application. Once the protocol is deployed to the tablet, TabSINT interprets the text protocol into a dynamic, interactive set of pages and response areas.

Protocol development

Protocols are developed in any standard text editor. The text in the protocol must conform to the JSON Data Interchange Standard (https://json.org) and be valid against the TabSINT Protocol Schema. To help new users get started quickly, the TabSINT source code includes protocol templates that provide examples of common survey inputs, hearing tests, and logic to control test progression.

Protocols consist of metadata, test properties and defaults, and an array of pages to present during a test. Each page entry defines the parameters of a single presentation, which may contain informational text or a user interactive response area. Page entries may also be defined as references to other sections of the protocol or nested sub-protocols. This architecture allows test protocols to be subdivided into many smaller sub-protocols and support complex navigational logic.

TabSINT includes support for multiple speech-in-noise response areas (Oldenburg Matrix Test, Modified Rhyme Test, etc.) and over 18 questionnaire types (multiple choice, button grids, Lickert scales, image maps, videos, etc.). These response areas contain many configurable options to customise the response area. The protocol schema defines the allowable inputs for all response areas. Figures 2 and 3 show the text version of a protocol response area and the same response area rendered within the TabSINT application.

TabSINT includes two advanced protocol features that expand the functionality to include customised content. First, TabSINT includes support for user-defined “Custom Response Areas” where the user defines the HTML and JavaScript of a page’s response area. The JavaScript functions defined in this response area have access to the previous test results and data defined by the test logic of TabSINT. This feature can be used to create alternative inputs, or to present custom information to users. TabSINT also includes support for “preprocessing functions” that will run before a page is loaded. The preprocessing function can dynamically change any of the properties on that specific protocol page based on previous test results and flags. This feature allows specific page content to change depending on a participant’s test history.

Deployment

Since 2014, the TabSINT system has been used in a large number of studies within the Department of Defence (DoD) as well as in industry and academia (U.S. and International). As of January 2019, the TabSINT system has processed over 25,000 tests, using 120 different tablets and over 200 separate protocols (some of these are evolving versions of single study protocols). Table 1 provides examples of different users, applications and protocols developed by the TabSINT community over the past five years. These numbers illustrate very clearly the impact of the technology on the audiology community at large. New research teams are adopting TabSINT because these tools lower the barrier of access to expanded and distributed studies of hearing, allowing researchers to reach a more diverse set of subjects at reduced cost. TabSINT has now been translated into three languages (French, Japanese, and Spanish).

Table 1. Applications of TabSINT for research.

Administrator	Application	Protocol
Creare	Headset comfort/fit evaluation	Survey questions about the comfort and fit of the Creare Headset.
Industry	Pharmaceutical development	Hearing tests and questionnaires to evaluate eligibility in drug development trials.
Academic research	Hearing loss epidemiology	Demographic survey question and audiology screening using the Creare Headset.
	Hearing testing outside the clinic	Audiology evaluation using the Creare Headsets performed at: – State fair – Manufacturing plant – Physician offices – Schools – Rural health clinic – Nursing home – Outpatient chemotherapy treatment centres
Public health	Screening for school age children in low-resource setting	Audiology evaluation using the Creare Headset of school age children at school.
Department of defence	Hearing injury prevention (HIP)	Survey questions to evaluate the efficacy of hearing injury prevention educational materials.
	Functional impairments associated with hearing loss (FIT)	Hearing tests and questionnaires to develop fitness for duty metrics.
	Ecological hearing evaluation	Speech recognition tests and background noise measurements in ecological environments (e.g. restaurant).
	Prevalence	Hearing tests and questionnaires to measure prevalence of auditory processing difficulties.

Department of defense (DoD)

Currently, the primary user of TabSINT is still the DoD. Figure 5 shows the number of individual test results uploaded to the server between 2015 and 2017 for four different studies that used TabSINT as the primary data collection system: over 11,000 test results have been uploaded across 21 separate sites located across the country. This figure illustrates how TabSINT can be used to reach many sites and implement a variety of human studies of hearing that may differ significantly in their design yet can use the same equipment for delivery. For example, the Hearing Injury Prevention (HIP) study is essentially a survey, the Functional Impairments Associated with Hearing Loss (FIT) study includes several speech-in-noise tests and a functional hearing test, and the CAPD study includes a variety of tests aimed at evaluating central auditory processing function.

Future development

TabSINT is under active development to increase the capability and accessibility of the software ecosystem. In the short term, new features will include a set of tools to streamline results processing and analysis. Currently, each exam result is output from TabSINT as a flat text file in a JSON format. While the JSON format allows for a flexible, machine-readable structure, many researchers do not have access to resources for analysing these files. To enable wider access to results, new MATLAB and Python utilities can generically translate TabSINT result files into a flattened comma delimited format (CSV). CSV files are compatible with many common statistical analysis tools, including Microsoft Excel.

Future development will also include a more automated version of TabSINT that is available to users at home (“Take Home TabSINT”). This version of TabSINT will contain a minimal user interface capable of running a single test protocol repeatedly. The user will be prompted to run through the protocol at certain time intervals with results output to the local SD card or uploaded in the background to a cloud server.

To enable “Take Home TabSINT” and other extensions, TabSINT will be modified to meet the rigorous security and privacy standards dictated for clinical mobile health applications. While the current iteration of TabSINT supports strong security controls to minimise data exposure including data encryption at rest and public-key encryption of results, the data storage structure and external communications protocol will be audited and hardened to address security and privacy risks outlined in the HIPAA guidelines for mobile health applications (Jain and Shanbhag 2012; Department of Health and Human Services 2013).

Conclusion

The TabSINT software system provides an out-of-the-box mobile solution to meet the needs of hearing researchers with a flexible questionnaire engine, cloud-based data management, and playback of calibrated media. While systems exist to meet some of these needs, TabSINT provides a singular, cohesive framework for managing and deploying widespread studies of hearing. TabSINT is built using open web technologies and released open-source on GitLab to allow researchers and developers to access, contribute, and improve the features of the software. Future iterations of the software will allow researchers to collect data from subjects at home and allow clinicians to collect data outside of a research context.

TabSINT is providing researchers the ability to reach a much more diverse subject pool and to conduct human studies with large sample sizes, thereby greatly improving the quality of the data. In the period of January 2014 through the end of 2018, TabSINT has collected data from over 25,000 subjects at many different sites around the world. The data obtained to date within the DoD is affecting new DoD-wide hearing standards that will greatly impact service members and their ability to continue, or not, in their chosen military occupational specialty when they suffer from hearing loss. Within the society at large, TabSINT has already been used to reach out to patients who would not otherwise have access to hearing health care or be able to participate in hearing research. TabSINT could also extend to applications beyond hearing; for example, neurocognitive tasks and vision testing.

Acknowledgments

In particular, we gratefully acknowledge the support and contributions of the Audiology and Speech Centre at the Walter Reed National Military Medical Centre and the Department of Defence Hearing Centre of Excellence in the development and extensive testing of this software.

We would like to thank Hector Galloza (2gari), Rob Chambers (Creare), Brendan Flynn (Creare) Jaclyn Schurman (University of Maryland), and the many other engineers and audiologists at Creare and Walter Reed for their support in the development of this system.

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Funding

This work was supported by the U.S. Army Medical Research Materiel Command and the Army Public Health Command under SBIR Phase III Awards #W81XWH-13-C-0194, #W81XWH-16-C-0160, and #W81XWH-17-C-0218 to Creare LLC.