Speech-driven mobile games for speech therapy: User experiences and feasibility

Abstract

Purpose: To assist in remote treatment, speech-language pathologists (SLPs) rely on mobile games, which though entertaining, lack feedback mechanisms. Games integrated with automatic speech recognition (ASR) offer a solution where speech productions control gameplay. We therefore performed a feasibility study to assess children’s and SLPs’ experiences towards speech-controlled games, game feature preferences and ASR accuracy.

Method: Ten children with childhood apraxia of speech (CAS), six typically developing (TD) children and seven SLPs trialled five games and answered questionnaires. Researchers also compared the results of ASR to perceptual judgment.

Result: Children and SLPs found speech-controlled games interesting and fun, despite ASR–human disagreements. They preferred games with rewards, challenge and multiple difficulty levels. Automatic speech recognition–human agreement was higher for SLPs than children, similar between TD and CAS and unaffected by CAS severity (77% TD, 75% CAS – incorrect; 51% TD, 47% CAS, 71% SLP – correct). Manual stop recording yielded higher agreement than automatic. Word length did not influence agreement.

Conclusion: Children’s and SLPs’ positive responses towards speech-controlled games suggest that they can engage children in higher intensity practice. Our findings can guide future improvements to the ASR, recording methods and game features to improve the user experience and therapy adherence.

Keywords:

• speech-controlled games
• mobile therapy apps
• ASR applications
• ASR in games
• childhood apraxia of speech

Declaration of interest

No potential conflict of interest was reported by the authors.

Supplementary material available online

Additional information

Funding

This work was made possible by NPRP grant #[8-293-2-124] from the Qatar National Research Fund (a member of Qatar Foundation). Ballard was supported by Australian Research Council Future Fellowship FT120100355. The statements made herein are solely the responsibility of the authors.