Speaker Identification in Interactions between Mothers and Children with Down Syndrome via Audio Analysis: A Case Study in Mexico

Abstract

In this work, we aim at identifying the speaker in interactions between mothers and children with Down syndrome (DS) using audio. We collected audio from a session in which children with DS solved puzzles, and their mothers were by their side. We generated a dataset by manually annotating human speech activity and non-speech. We used machine learning to perform four experiments, including individual and generalized models achieving on average F1-scores of 0.74 (five-class model) and 0.84 (two-class) for individual models and 0.69 (five-class) and 0.82 (two-class) for the generalized models. Our results can be helpful to behavioral researchers, therapists, and those interested in better understanding how mother–child interactions unfold in naturalistic settings.

Notes

Acknowledgements

We thank José Carlos Grandío, J. Eduardo Montoya, and Guillermo Ochoa from the Sonora Institute of Technology (ITSON), who participated in the audio annotation stage of this work.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 See https://www.lena.org/technology/.

2 See https://projets-lium.univ-lemans.fr/spkdiarization/.

3 See https://www.audacityteam.org/.

4 See https://www.nltk.org/.

5 See https://python-speech-features.readthedocs.io.

6 See https://hmmlearn.readthedocs.io/en/latest/tutorial.html.

Additional information

Funding

This work has been partially funded by the Sonora Institute of Technology (ITSON) through the PROFAPI program.

Notes on contributors

Carlos R. Flores-Carballo

Carlos R. Flores-Carballo is a Software Engineering student at the Sonora Institute of Technology (ITSON), México. His research interests are machine learning, data analysis, and mobile computing.

Gabriel A. Molina-Arenas

Gabriel A. Molina-Arenas is a Software Engineering student at the Sonora Institute of Technology (ITSON), México. His research interests are machine learning, data analysis, and mobile computing.

Adrian Macias

Adrian Macias is a full professor at the Dept. of Computing and Design at the Sonora Institute of Technology (ITSON), México. He holds a MsC in Computer Science from the Center for Scientific Research and Higher Education of Ensenada, Mexico (CICESE Research Center). His Research Interests are Technologies in support of Down Syndrome Children.

Karina Caro

Karina Caro is an assistant professor at the Autonomous University of Baja California (UABC), Mexico, where she directs the Technology for Social Good Research Lab (Tech4Good Lab). She received her Ph.D. in Computer Science from the Center for Scientific Research and Higher Education of Ensenada, Mexico (CICESE Research Center).

Jessica Beltran

Jessica Beltran is an assistant professor at the Applied Mathematics Research Center (CIMA) from the Autonomous University of Coahuila (UAdeC), México. She received her Ph.D. in Computer Science from the Center for Scientific Research and Higher Education of Ensenada, Mexico (CICESE). Her interests include machine learning and data analysis.

Luis A. Castro

Luis A. Castro is a full professor at the Dept. of Computing and Design at the Sonora Institute of Technology (ITSON), México. He holds a PhD in Informatics from the University of Manchester, UK. He is the former president of the Mexican Association on Human-Computer Interaction (AMexIHC).