Parselmouth for bioacoustics: automated acoustic analysis in Python

Figures & data

Figure 1. A plot of the fundamental frequency estimated through Parselmouth, overlaid on two seal call spectrograms, shows Praat’s ability to correctly track the fundamental frequency, even in the presence of noise. Automated generation of such plots was used to swiftly assess the tracking quality, as to ensure correctness of the results.

Code fragment 1: A short Python function wraps the functionality to estimate the median fundamental frequency using Parselmouth. Note the seamless interaction between Praat functionality (e.g. snd_part.to_pitch(…)), pure-Python syntax (e.g. the function definition or if-statement), and other Python libraries (i.e. NumPy’s np.nanmedian and SciPy’s scipy.stats.iqr) (see Supplemental online material, calculate_pitch.py, https://figshare.com/articles/dataset/Parselmouth_for_bioacoustics_automated_acoustic_analysis_in_Python/24307391?file=42678629).

Code fragment 2: The use of Python and Parselmouth renders stimulus generation reusable and reproducible. Python functions help to compartmentalise the code, and Python’s built-in data structures (i.e. lists and dictionaries) allow the programmer to reuse existing programming experience and paradigms in combination with Parselmouth’s functionality (see Supplemental online material, generate_playbacks.py, https://figshare.com/articles/dataset/Parselmouth_for_bioacoustics_automated_acoustic_analysis_in_Python/24307391?file=42678626).

Code fragment 3: Parselmouth’s formant analysis and SciPy’s convex hull calculations are combined in a handful of lines of Python code to calculate the 2-dimensional formant range of synthesised speech fragments (see Supplemental online material, plot_for mant_triangles.py, https://figshare.com/articles/dataset/Parselmouth_for_bioacoustics_automated_acoustic_analysis_in_Python/24307391?file=42678623).

Figure 2. A scatterplot of the values of the first two formants tracked by Praat, accompanied by the convex hull encompassing all the points. A single Python script performed the formant analysis through Parselmouth, calculated the convex hull with SciPy, and plotted the result with the matplotlib library. To demonstrate the advantages of having scripted the analysis, we swiftly recreated the plots with new data and tweaked the figure’s aesthetics (cfr. Rasilo and Jadoul 2020).

Table 1. A wide variety of research fields and studies have made use of Parselmouth. A majority of these studies use Parselmouth to perform acoustic feature extraction (AFE).

Macroarea & Title of the paper	Keywords (as reported in the original paper)	Parselmouth functionality
Bioacoustics
Large-scale unsupervised clustering of Orca vocalizations: a model for describing Orca communication systems (Poupard et al. 2019)	none	Acoustic feature extraction (AFE): pitch curve extraction
Scaredy-cat: Assessment of individual differences in response to an acute everyday stressor across development in the domestic cat (Urrutia et al. 2022)	stress social separation individual differences vocalisation thermography Felis silvestris catus	AFE: mean and standard deviation of fundamental frequency
Ontogeny of vocal rhythms in harbor seal pups: an exploratory study (Ravignani et al. 2019)	bioacoustics pinnipeds rhythm timing vocal development	AFE: maximum-intensity peaks, inter-peak intervals
Machine learning
Neural analysis and synthesis: Reconstructing speech from self-supervised representations (Choi et al. 2021)	speech analysis speech synthesis voice conversion self-supervised information perturbation	Speech manipulation and resynthesis (formant shifting, pitch randomisation, PSOLA)
What do audio transformers hear? Probing their representations for language delivery & structure (Singla et al. 2022)	interpretability probing pre-trained acoustic representations	AFE: mean pitch, local jitter, local shimmer, voiced to unvoiced ratio
Speech pathology & medicine
Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood (Schultebraucks et al. 2022)	computer vision deep learning depression digital biomarker emergency department landmark feature posttraumatic stress resilience voice analysis	AFE: audio expressivity index, audio intensity, fundamental frequency, harmonics-to-noise ratio, glottal to noise excitation ratio, voice frame core, formant frequency variability, intensity variability, pitch variability, normalised amplitude quotient
Artificial intelligence-based voice assessment of patients with Parkinson’s disease off and on treatment: machine vs. deep-learning comparison (Costantini et al. 2023)	speech Voice Parkinson’s disease artificial intelligence deep learning CNN SVM L-Dopa F0	AFE: 96 vocal formants-related features
Vocal markers from sustained phonation in Huntington’s disease (Riad et al. 2020)	Huntington’s disease phonation pathological speech processing dysarthria modulation power spectrum	AFE: airflow insufficiency, aperiodicity, irregular vibration of vocal folds, signal perturbation, increased noise, vocal tremor, articulatory deficiency (in combination with pre-existing ‘tremor’ Praat script; Brückl 2012)
Do you have COVID-19? An artificial intelligence-based screening tool for COVID-19 using acoustic parameters (Vahedian-Azimi et al. 2021)	none	AFE: fundamental frequency, fundamental frequency perturbation, harmonicity, vocal tract function, airflow sufficiency, and periodicity
Formant-aware spectral analysis of Sustained vowels of pathological breathy voice (Ikuma et al. 2023)	voice disorder breathiness acoustic measurements linear regression harmonic modeling vowel effect	AFE: fundamental frequency, acoustic breathiness index
Emotional analysis & prosody
Gender, candidate emotional expression, and voter reactions during televised debates (Boussalis et al. 2021)	none	AFE: average per-second fundamental frequency
Modelling individual and cross-cultural variation in the mapping of emotions to speech prosody (van Rijn and Larrouy-Maestri 2023)	none	Pre-processing: sentence segmentation
Does the Lombard effect improve emotional communication in noise? – Analysis of emotional speech acted in noise (Zhao et al. 2019)	Lombard effect noise emotional speech database recording auditory detection acoustic analysis	AFE: fundamental frequency, sound intensity, harmonics-to-noise ratio, first and second formants, spectral tilt
Human-robot interaction & social robotics
Tell me more! Assessing interactions with social robots from speech (Laban et al. 2020)	social robots human-robot interaction self-disclosure voice assistant voice analysis embodiment	AFE: mean pitch, mean harmonicity, mean intensity, energy, duration
Towards a real-time measure of the perception of anthropomorphism in human-robot interaction (Tsfasman et al. 2021)	multi-modal human-robot interaction prosody acoustic-prosodic entrainment	AFE: pitch and RMS-intensity
Informal caregivers disclose increasingly more to a social robot over time (Laban et al. 2022)	caregiving informal caregivers human-robot interaction HRI social robot self-disclosure intervention ecological momentary intervention EMI stress	Speech segmentation: extracting disclosure duration
Music & song
Large-scale iterated singing experiments reveal oral transmission mechanisms underlying music evolution (Anglada-Tort et al. 2023)	cultural transmission oral transmission cultural evolution iterated learning social learning singing music evolution melody pitch perception online experiments	AFE: pitch (from song)
Acoustic analysis of the influence of warm-up on singing voice quality (Półrolniczak and Kramarczyk 2023)	signal analysis singing voice singing quality acoustic analysis warm-up	AFE: jitter and shimmer

Nunes CEP, Nevard L, Montealegre-Z F, Vallejo-Marín M. 2021. Variation in the natural frequency of stamens in six morphologically diverse, buzz-pollinated, heterantherous Solanum taxa and its relationship to bee vibrations. Bot J Linn Soc. 197(4):541–553. doi: 10.1093/botlinnean/boab044.

 Web of Science ®Google Scholar

Poupard M, Best P, Schlüter J, Symonds H, Spong P, Lengagne T, Soriano T, Glotin H. 2019. Large-scale unsupervised clustering of Orca vocalizations: a model for describing Orca communication systems. 2nd International Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots; Aug 29–30; London, UK.

 Google Scholar

Urrutia A, Bánszegi O, Szenczi P, Hudson R. 2022. Scaredy-cat: assessment of individual differences in response to an acute everyday stressor across development in the domestic cat. Appl Anim Behav Sci. 256:105771. doi:10.1016/j.applanim.2022.105771.

 Web of Science ®Google Scholar

Ravignani A, Kello CT, de Reus K, Kotz SA, Dalla Bella S, Méndez-Aróstegui M, Rapado-Tamarit B, Rubio-Garcia A, de Boer B. 2019. Ontogeny of vocal rhythms in harbor seal pups: an exploratory study. Curr Zool. 65(1):107–120. doi: 10.1093/cz/zoy055.

 PubMed Web of Science ®Google Scholar

Choi HS, Lee J, Kim W, Lee J, Heo H, Lee K. 2021. Neural analysis and synthesis: reconstructing speech from self-supervised representations. Adv Neural Inf Process Syst. 34:16251–16265.

 Google Scholar

Singla YK, Shah J, Chen C, Shah RR 2022. What do audio transformers hear? Probing their representations for language delivery & structure. 2022 IEEE International Conference on Data Mining Workshops (ICDMW); November; 28 Nov 2022–01 Dec 2022; Orlando, FL, USA; p. 910–925. doi: 10.1109/ICDMW58026.2022.00120.

 Google Scholar

Schultebraucks K, Yadav V, Shalev AY, Bonanno GA, Galatzer-Levy IR. 2022. Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood. Psychol Med. 52(5):957–967. doi: 10.1017/S0033291720002718.

 PubMed Web of Science ®Google Scholar

Costantini G, Cesarini V, Di Leo P, Amato F, Suppa A, Asci F, Saggio G. 2023. Artificial intelligence-based voice assessment of patients with Parkinson’s disease off and on treatment: machine vs deep-learning comparison. Sensors. 23(4):2293. doi: 10.3390/s23042293.

 PubMed Web of Science ®Google Scholar

Riad R, Titeux H, Lemoine L, Montillot J, Bagnou JH, Cao XN, Bachoud-Lévi AC. 2020. Vocal markers from sustained phonation in Huntington’s disease. Proc. Interspeech 2020; p. 1893–1897. https://dblp.org/rec/conf/interspeech/RiadTLMBCDB20.html.

 Google Scholar

Brückl M. 2012. Vocal tremor measurement based on autocorrelation of contours. INTERSPEECH 2012 ISCA's 13th Annual Conference Portland; Sep 9–13; Portland, OR, USA; p. 715–718. doi: 10.21437/Interspeech.2012-223.

 Google Scholar

Vahedian-Azimi A, Keramatfar A, Asiaee M, Atashi SS, Nourbakhsh M. 2021. Do you have COVID-19? An artificial intelligence-based screening tool for COVID-19 using acoustic parameters. J Acoust Soc Am. 150(3):1945–1953. doi: 10.1121/10.0006104.

 PubMed Web of Science ®Google Scholar

Ikuma T, McWhorter AJ, Oral E, and Kunduk M. 2023. Formant-aware spectral analysis of Sustained vowels of pathological breathy voice. J Voice. doi:10.1016/j.jvoice.2023.05.002.

 PubMedGoogle Scholar

Boussalis C, Coan TG, Holman MR, Müller S. 2021. Gender, candidate emotional expression, and voter reactions during televised debates. Am Polit Sci Rev. 115(4):1242–1257. doi: 10.1017/S0003055421000666.

 Web of Science ®Google Scholar

van Rijn P, Larrouy-Maestri P. 2023. Modelling individual and cross-cultural variation in the mapping of emotions to speech prosody. Nat Hum Behav. 7(3):1–11. doi: 10.1038/s41562-022-01505-5.

 PubMed Web of Science ®Google Scholar

Zhao Y, Ando A, Takaki S, Yamagishi J, Kobashikawa S. (2019). Does the Lombard Effect Improve Emotional Communication in Noise? — Analysis of Emotional Speech Acted in Noise. Proc. Interspeech September 15–19, 2019, 3292–3296, doi: 10.21437/Interspeech.2019-1605.

 Google Scholar

Laban G, George JN, Morrison V, Cross ES. 2020. Tell me more! Assessing interactions with social robots from speech. Paladyn. 12(1):136–159. doi: 10.1515/pjbr-2021-0011.

 Google Scholar

Tsfasman M, Saravanan A, Viner D, Goslinga D, De Wolf S, Raman C, Oertel C. 2021. Towards a real-time measure of the perception of anthropomorphism in human-robot interaction. Proceedings of the 2nd ACM Multimedia Workshop on Multimodal Conversational AI; p. 13–18.

 Google Scholar

Laban G, Morrison V, Kappas A, Cross ES. 2022. Informal caregivers disclose increasingly more to a social robot over time. CHI '22: CHI Conference on Human Factors in Computing Systems; 29 April 2022–5 May 2022; New Orleans, LA, USA; p. 1–7. doi: 10.1145/3491101.

 Google Scholar

Anglada-Tort M, Harrison PM, Lee H, Jacoby N. 2023. Large-scale iterated singing experiments reveal oral transmission mechanisms underlying music evolution. Curr Biol. 33(8):1472–1486.e12. doi: 10.1016/j.cub.2023.02.070.

 PubMed Web of Science ®Google Scholar

Półrolniczak E, Kramarczyk M. 2023. Acoustic analysis of the influence of warm-up on singing voice quality. J Voice. doi: 10.1016/j.jvoice.2023.02.017.

 PubMedGoogle Scholar