Harnessing automatic speech recognition to realise Sustainable Development Goals 3, 9, and 17 through interdisciplinary partnerships for children with communication disability

Abstract

Purpose

To showcase how applications of automatic speech recognition (ASR) technology could help solve challenges in speech-language pathology practice with children with communication disability, and contribute to the realisation of the Sustainable Development Goals (SDGs).

Result

ASR technologies have been developed to address the need for equitable, efficient, and accurate assessment and diagnosis of communication disability in children by automating the transcription and analysis of speech and language samples and supporting dual-language assessment of bilingual children. ASR tools can automate the measurement of and help optimise intervention fidelity. ASR tools can also be used by children to engage in independent speech production practice without relying on feedback from speech-language pathologists (SLPs), thus bridging the long-standing gap between recommended and received intervention intensity. These innovative technologies and tools have been generated from interdisciplinary partnerships between SLPs, engineers, data scientists, and linguists.

Conclusion

To advance equitable, efficient, and effective speech-language pathology services for children with communication disability, SLPs would benefit from integrating ASR solutions into their clinical practice. Ongoing interdisciplinary research is needed to further advance ASR technologies to optimise children’s outcomes. This commentary paper focusses on industry, innovation and infrastructure (SDG 9) and partnerships for the goals (SDG 17). It also addresses SDG 1, SDG 3, SDG 4, SDG 8, SDG 10, SDG 11, and SDG 16.

Keywords:

• Sustainable Development Goals (SDGs)
• good health and well-being (SDG 3)
• industry, innovation and infrastructure (SDG 9)
• partnerships for the goals (SDG 17)
• communication disability
• automatic speech recognition (ASR)
• assessment
• therapy

Introduction

Communication disability in childhood is common, impacting approximately 1 in 10 children (McGregor, 2020). Communication disability is a collective term referring to children with impairments in speech and language, such as speech sound disorder (SSD) and developmental language disorder (DLD) (cf. Hussain et al., 2018). Interdisciplinary partnerships are necessary to drive innovative solutions if we are to tackle the longstanding global challenges of providing these children with equitable and sufficient access to high-quality speech-language pathology services to accelerate learning. Without timely and adequate help, young children with communication disability face increased risks of poor outcomes relevant to multiple Sustainable Development Goals (SDGs) (United Nations, 2015) including no poverty (SDG 1) (Olusanya et al., 2006), good health and well-being (SDG 3) (St Clair et al., 2019), quality education (SDG 4) (Lewis et al., 2019), and decent work and economic growth (SDG 8) (Conti-Ramsden et al., 2018). They also face a risk of reduced inequalities (SDG 10) socially, economically, and politically (Murphy et al., 2018), inequalities in access to adequate, safe, and affordable housing (Tually et al., 2011) relevant to sustainable cities and communities (SDG 11), and inequalities in access to and experience of the justice system (Snow, 2019) relevant to peace, justice and strong institutions (SDG 16). In this commentary we highlight how two SDGs – industry, innovation and infrastructure (SDG 9) and partnerships for the goals (SDG 17) – could advance speech-language pathologists’ (SLPs) practice and in doing so, mitigate the risk of poor outcomes across multiple SDGs for children with communication disability. Our focus is on clinical applications of automatic speech recognition (ASR) – technologies that have emerged through innovation, and interdisciplinary partnerships.

What are automatic speech recognition (ASR) technologies?

Automatic speech recognition (ASR) technologies convert human speech signals to text. ASR is an interdisciplinary field within computer science encompassing signal processing, natural language processing (NLP), machine learning (ML), and artificial intelligence (AI). Many commercial ASR systems (e.g. Google Cloud Speech-to-Text) additionally feature conversational agents (e.g. Google Assistant) utilising a mixture of ASR, NLP, and Text-To-Speech (TTS) or speech synthesis to realise human-machine conversation (Allouch et al., 2021). Current state-of-the-art ASR systems are designed for clear adult speech and are limited to low-noise conditions.

Applications of ASR in the delivery of speech-language pathology services for children

Time and clinical resources are finite in practice, and clinical decisions on the best use of time and resources can be fraught with compromise. In this section, we explore how innovations in ASR could minimise such compromises for three challenges in practice.

Innovations in ASR to help ensure equitable, efficient, accurate assessments

SLPs have looked to ASR to reduce the prohibitive time demands of language sampling transcription (Miller et al., 2016), with promising ASR solutions emerging, at least in particular languages. For example, Fox et al. (2021) discovered that ASR (using Google Cloud Speech-to-Text) was more accurate than SLPs in orthographically transcribing narrative samples from English-monolingual school-age children with DLD in real time. Interdisciplinary teams comprising SLPs and engineers have also developed ASR tools that can identify and classify errors in English-speaking children’s speech (e.g. Tabby Talks; Shahin et al., 2015).

Worldwide, multilingualism is the norm and for assessments to be valid, SLPs must consider children’s culture and the language(s) they speak and hear (Mdlalo et al., 2019). When access to professional interpreters is limited or absent, SLP services for multilingual children become inequitable (Newbury et al., 2020). Albudoor and Peña (2022) investigated whether assessment of bilingual children’s responses on an expressive language assessment task could be automated with ASR. They found moderate agreement between human and ASR (Google Cloud Speech-to-Text) transcription for English-Spanish bilingual children with typical language acquisition (n = 59) and children with DLD (n = 25). Given the rapid pace and development of ASR technology, Albudoor and Peña (2022) suggest that accurate language assessment for children is possible using ASR, irrespective of the language(s) spoken, thus contributing to good health and well-being (SDG 3) for all children with communication disability.

Innovations in ASR to measure intervention fidelity

Measurement of fidelity (i.e. determining if an intervention was implemented as intended) is challenging and often unreported (Baker et al., 2022). It typically involves SLPs retrospectively completing intervention fidelity checklists, reviewing recordings of intervention sessions, or relying on extra personnel to count intervention elements in real time (e.g. Alt et al., 2021; Munro et al., 2021). Such approaches to measurement are inefficient and do not expedite optimal fidelity in real time. An intervention approach where real-time measurement of fidelity is critical is Vocabulary Acquisition and Usage for Late Talkers (VAULT) treatment (Alt et al., 2021). VAULT is an input play-based intervention in which SLPs need to say predetermined treatment words at a prescribed dose (e.g. say 90 productions of put, baby, and chair totalling 270 inputs) during a 30-minute session (Alt et al., 2021). Such a dose is not feasible for SLPs to count as they engage toddlers in varied play activities. Although ASR tools that convert speech-to-text can successfully support physicians’ clinical documentation (Latif et al., 2021), such tools do not count words. One ASR tool that does count adult words spoken during adult–child interactions is the Language ENvironment Analysis (LENA^TM) system (Wang et al., 2020). However, LENA^TM counts all adult words spoken rather than specific predetermined words. Six SLPs who are authors of the current paper (EB, RH, SM, MA, KT, NM) have been conducting research into VAULT. They contacted engineers, data scientists, and linguists with expertise in ASR to design a solution. Through this interdisciplinary partnership, the Speech Therapy Word Tracker (STeWoT) prototype tool was developed.

The STeWoT prototype was developed in Python and interacts with existing commercial ASR technology, either Microsoft Azure (Microsoft, 2022) or Google Cloud (Google, 2022). As depicted in Figure 1, SLPs’ talk during a VAULT session can be converted to text. This text can then be analysed in real time to yield a count (on a PC screen) of the occurrence of the pre-determined treatment words. In this way, real-time feedback is received on input dose. Early trials of role-play VAULT session activities have shown promising alignment with a human count (see Table I). Pending further trials of the prototype tool during VAULT sessions, the tool not only has the potential to automate efficient and precise measurement of fidelity, but automate the collection of assessment, baseline, treatment, and generalisation data for children and adults with communication disability.

Figure 1. Speech Therapy Word Tracker (STeWoT) Tool workflow.

Table I. Comparison of machine versus human count of speech-language pathologist’s production of treatment words during two VAULT^a treatment role-play session activities.

	Count of treatment words during an activity^b in session one			Count of treatment words during an activity^c in session two
Human or Machine	Put	Baby	Chair	Daddy	Cook	Meat
Human (real-time)	37	30	30	28	37	40
Human (recording)	36	32	29	28	41	41
Microsoft Azure	38	34	29	26	44	44
Google Cloud	36	31	25	25	40	31

^aVocabulary Acquisition and Usage for Late Talkers. ^bActivity duration 5:32 minutes. ^cActivity duration 5:34 minutes.

Innovations in ASR to increase intervention intensity

Intensity is a multidimensional construct encompassing the parameters of dose within a session plus the frequency, duration, and total number of sessions (Warren et al., 2007). Intervention outcomes can be compromised when the intensity received is inadequate, such as children being seen for fewer sessions than empirically recommended (e.g. Glogowska et al., 2000), under-dosing of trials within a session, and sessions being less frequent than required (Hegarty et al., 2021). This gap between recommended and received intensity is long-standing and recalcitrant (Law & Conti-Ramsden, 2000), with SLPs calling for time-saving evidence-based tools (Hegarty et al., 2021).

Interdisciplinary teams have been using ASR to bridge the gap between recommended and received intensity. For instance, a team of SLPs, engineers, and computer scientists worked with children and their carers to develop a therapeutic video game – Apraxia World (more recently renamed Say Bananas!) – to provide real-time feedback on children’s speech accuracy using ASR technology (Hair et al., 2021). The team designed the game to motivate and engage children in long-term independent speech practice. The emerging evidence suggests it can help increase session dose, frequency, and total number of sessions. In a pilot study of 10 children, improvements in the children’s speech were considered comparable to face-to-face SLP services (Hair et al., 2021). The inherent automation of the tool also has the potential to optimise and measure fidelity of implementation. Indeed, automated feedback can be more accurate and consistent compared to adults’ feedback, given adults’ tendency to be lenient (Hair et al., 2021).

Across the examples highlighted, some ASR applications such as Say Bananas! are available for clinical use, while others such as the STeWoT tool are prototypes at the frontier of scientific investigation paving the way for future clinical application. A systematic review would be needed to provide an in-depth understanding of the breadth of tools available and in development. Additionally, the example ASR solutions in this commentary emerged from Minority world countries such as Australia and the United States, where speech-language pathology services exist. Although it may seem fanciful for children living in Majority world countries with limited or non-existent speech-language pathology services to benefit from ASR technologies, equity of access and opportunity for their use could become a reality as children living in these countries have increased access to computers, the Internet and telepractice (Zahir et al., 2021).

Summary and conclusion

Innovative solutions are possible when interdisciplinary partners collaborate to solve vexing challenges in clinical practice. To advance equitable, efficient, and effective speech-language pathology services for children with communication disability and realise children’s good health and well-being (SDG 3), SLPs are encouraged to integrate evidence-based ASR solutions into practice. Going forward, further interdisciplinary research is needed to advance ASR solutions. The groundswell of interest in ASR also needs to be matched by practice-based research considering clinical reach, utility, and effectiveness.

Declaration of interest

The authors declare that they developed the prototype of STeWoT tool mentioned in the manuscript. The authors alone are responsible for the content and writing of this article.