Exploring the effect of lexicality and listener experience on gradient ratings of Swedish sibilant fricatives

ABSTRACT

This study explores the influence of lexicality on gradient judgments of Swedish sibilant fricatives by contrasting ratings of initial fricatives in words and word fragments (initial CV-syllables). Visual-Analogue Scale (VAS) judgments were elicited from experienced listeners (speech-language pathologists; SLPs) and inexperienced listeners, and compared with respect to the effects of lexicality using Bayesian mixed-effects beta regression. Overall, SLPs had higher intra- and interrater reliability than inexperienced listeners. SLPs as a group also rated fricatives as more target-like, with higher precision, than did inexperienced listeners. An effect of lexicality was observed for all individual listeners, though the magnitude of the effect varied. Although SLP’s ratings of Swedish children’s initial voiceless fricatives were less influenced by lexicality, our results indicate that previous findings concerning VAS ratings of non-lexical CV-syllables cannot be directly transferred to the clinical context, without consideration of possible lexical bias.

KEYWORDS:

• Visual-analogue scale
• rating scales
• speech assessment
• fricatives
• speech perception

Introduction

Within the first few years of life, typically developing children manage the tremendous feat of mastering speech to such a level that they are intelligible to strangers at 4–5 years of age (e.g. Hustad et al., 2021). Age of acquisition of specific speech sounds is characterised by huge individual variation, although the order of acquisition is constrained by, for example, oro-motor maturation and phoneme frequency in the ambient language/s (e.g. Li & Munson, 2016). Sibilant fricatives, such as /s/ and /ɕ/ are produced with complex tongue configurations and are reported to be relatively late-acquired across many structurally different languages (e.g. McLeod & Crowe, 2018). Studies of voiceless sibilant fricatives in the process of acquisition indicate that young children gradually differentiate between the sibilants of their ambient language(s) and show increased contrast with age (see e.g. Holliday et al., 2015; Wikse Barrow et al., in review). Young children’s sibilant fricatives are often less distinct and show overlap in acoustic features that distinguish adult sounds, resulting in productions that may be perceived as somewhere ‘in-between’ the target sibilants. Fine phonetic variation that lies within adult phoneme boundaries, such as the ‘in-between’ alluded above, can be referred to as covert contrast. Covert contrasts are systematically conveyed by the child but will nonetheless often go undocumented in transcription of speech, given the categorical nature of the transcription task (see Munson et al., 2010 for a first-rate treatise). However, given fine-grained perceptual tasks, listeners have been shown to be sensitive to fine phonetic detail in child speech. The Visual Analogue Scale (VAS) is an example of such a gradient rating scale, consisting of a horizontal line with two anchors, upon which a listener marks their perception of the sound presented. VAS has been used in many studies concerning covert contrast in child speech (Munson & Urberg Carlson, 2016; Munson et al., 2010; Munson, Johnson, et al., 2012; Munson, Schellinger et al., 2012; Schellinger et al., 2017; Strömbergsson et al., 2015) and is recommended for use in clinical practice by some (e.g. Munson, Schellinger, et al., 2012). In the clinical setting, speech-language pathologists (SLPs) are faced with highly variable speech input from children with speech difficulties. Indeed, children with Speech Sound Disorder (SSD) are a distinctly heterogeneous group with respect to aetiology, coexistent language difficulties and speech profiles (Dodd, 2014). Assessment of speech is essential in the clinical care of children with SSD (e.g. Rvachew & Brosseau-Lapré, 2017; Skahan et al., 2007; Wikse Barrow et al., 2023) and VAS provides an interesting alternative to the categorical tool that is transcription of speech. However, questions remain regarding possible lexical bias in VAS ratings, which may threaten the clinical validity of the rating scale. It is well established that lexicality of speech stimuli affects perception of speech. In his seminal study from 1980, Ganong explored the effect of lexical knowledge on speech perception by enlisting participants to rate acoustic continua between /d/ and /t/-initial real words and non-words (e.g. ‘dask’ and ‘task’), for which the voice onset time (VOT) of the initial sound varied systematically. He showed that listeners were more likely to categorise acoustically ambiguous stimuli as real words, as opposed to non-words, particularly when VOT values were close to phoneme boundaries (Ganong, 1980). The ‘Ganong effect’ refers to this lexical effect, namely that lexical knowledge influences auditory perception, making listeners more likely to make phonetic classifications that result in real words rather than non-words.

As for previous investigations of phonetic detail in child speech using VAS, most studies have used CV-syllables as stimuli for perceptual evaluations (see e.g. Ancel et al., 2023; Munson et al., 2010; Meyer & Munson, 2021; Munson, Johnson, et al., 2012; Strömbergsson et al., 2015, although see Coniglio et al., 2022; Harel et al., 2017 for exceptions). In most cases, this has resulted in stimuli consisting of word fragments without lexical meaning. With the Ganong effect in mind, it is possible that VAS ratings of fine phonetic variation could be affected by stimuli lexicality, that is, meaningless syllable or meaningful word.

Although there are obvious benefits of basing perceptual evaluations on cut-out CV syllables in cross-linguistic speech perception research (see e.g. Li et al., 2011, who explored perception of English and Japanese children’s voiceless sibilants, through a speeded classification task), the use of such stimuli does not necessarily reflect perceptual evaluation conducted in the clinic. Indeed, clinical speech samples are most often elicited through picture naming, repetition or spontaneous speech (e.g. McLeod & Baker, 2014; Skahan et al., 2007; Wikse Barrow et al., 2023), resulting in whole-word productions containing lexical information that could bias perceptual ratings. However, it has been suggested that experienced clinicians might be less vulnerable to lexical bias than inexperienced listeners (Meyer & Munson, 2021; Munson, Johnson, et al., 2012). Although this hypothesis has not been explicitly tested, experience has previously been shown to impact VAS ratings of children’s speech; Munson, Johnson, et al. (2012), compared ratings elicited from speech-language pathologists (SLPs) with those from inexperienced listeners in a category goodness rating task. Participants rated the initial consonants of open syllable pairs beginning with /s/-/θ/, /t/-/k/ or /d/-/g/ via a digital VAS. Overall, SLPs’ ratings were more clearly associated with acoustic features of the stimuli, showed higher intrarater reliability and less bias towards more frequent sounds compared to the ratings of the untrained listeners (Munson, Johnson, et al., 2012).

In a recent online experiment, aiming to investigate whether length of clinical experience influenced ratings of child speech, Meyer and Munson (2021) utilised a 9-point scale to explore the same contrasts as Munson, Johnson et al. (2012) as well as /s/-/ʃ/. They found that more experienced SLPs tended to be more categorical in their ratings than less experienced SLPs and inexperienced listeners. The authors speculate that the observed difference in response categoricity could be due to long-term use of phonetic transcription, which might strengthen categorical percepts (Meyer & Munson, 2021).

A recent exception to syllable-based VAS investigations can be found in Coniglio et al. (2022) study on perception of /l/ by three listener groups with varying experience, SLPs, SLP students and inexperienced listeners. They used whole words as stimuli in their three-way (/j/-/l/-/w/) VAS, also presenting the intended word so as to ‘ … simulate the actual clinical setting’. (p. 396), and found that more experienced listeners used the VAS more continuously than inexperienced listeners. The authors interpreted their results as evidence for more gradient scale use in SLPs, and express support for clinical use of VAS in ratings of /l/. However, how the findings in Coniglio et al. (2022) relate to previous work is unclear, as both the scale, the target sounds and the instructions to listeners differ. Moreover, the effect of lexical context on experienced and inexperienced listeners’ VAS ratings is, as of yet, unknown.

Aims

In the current study, we investigate how experienced (SLPs) and inexperienced listeners rate Swedish children’s initial fricatives when presented in lexical and non-lexical contexts, using VAS. We explore the rating behaviour and reliability of ratings for individuals and for each listener group. The effects of listener experience and lexicality, as well as possible interactions between them, are modelled using Bayesian mixed-effects beta regression. The present study thus attempts to replicate and expand upon the findings from Munson, Johnson et al. (2012) and Meyer and Munson (2021), as well as to explore possible interactions between lexicality and listener experience.

Research questions

Do experienced and inexperienced adult listeners differ in their gradient category goodness ratings of children’s voiceless fricatives? Specifically, does listener experience influence the reliability of ratings?

Does lexicality affect experienced and inexperienced adult listener ratings of children’s voiceless fricatives?

Method

Speech material

Recording

The speech material was selected from recordings of 26 children between 3 and 8 years of age, who performed an audio-prompted picture naming task. Recordings were performed in a quiet room at the Department of Linguistics at Stockholm University or in the child’s home, with a caregiver present. A Sennheiser HSP-2 head-mounted microphone, placed approximately 2 cm from the corner of the child’s mouth, with a sampling rate of 48 kHz was used for recordings. For details regarding the recording procedure, see Wikse Barrow et al., (in review). A set of two minimal word pairs and two near-minimal word pairs (i.e. words with minimal initial CV-syllables) were selected for the present study; [suːl]-[ɕuːl], [sʉːɹ]-[ɕʉːɹ], [sɑːɹa]-[ɕɑːt] and [siːda]-[ɕiːlo]. See Table 1 for a description of speech stimuli.

Table 1. Orthographic and phonetic transcriptions of the word pairs and extracted syllables used in the perceptual evaluation.

Word pair	English gloss	Phonetic transcription, words	Phonetic transcription, extracted syllables
“sol” – “kjol”	sun – skirt	[suːl] – [ɕuːl]	[su] – [ɕu]
“sur” – “tjur”	sour – bull	[sʉːɹ] – [ɕʉːɹ]	[sʉ] – [ɕʉ]
“Sara” – “tjat”	Sara – nagging	[sɑːɹa] – [ɕɑːt]	[sɑ] – [ɕɑ]
“sida” – “kilo”	page – kilo	[siːda] – [ɕiːlo]	[si] – [ɕi]

Stimuli selection

To select appropriate and varied stimuli, all productions of the relevant words were extracted through a Praat (Boersma & Weenink, 2024) script and normalised for amplitude. The extracted words were then rated independently by the Authors CWB and SS, and the initial sound was categorised as one of the four transcription categories: [s], [ɕ], [s:ɕ] when the sound was deemed intermediate, but more similar to /s/ and [ɕ:s] when the sound was deemed intermediate but more similar to /ɕ/. This procedure is similar to the intermediate transcription categories suggested by Stoel-Gammon (2001) although the target sound was not taken into consideration in the present analysis. Productions that included disturbances or background noise, and productions that were not appropriately described on the /s/-/ɕ/ continua (e.g. [θ]) were excluded.

The speech stimuli selected for the perception task included 9–12 words from each transcription category (i.e. [s], [ɕ] and intermediate; [s:ɕ] or [ɕ:s]), for each word pair, resulting in a total of 126 words. Note that [ɕ] for /s/ but not [s] for /ɕ/ substitutions occurred in the material, making the stimuli unbalanced in this respect. Age and gender of the speakers were balanced as best as possible. Once the choice of words was finalised, the initial syllable of each word (i.e. the initial fricative and 150 ms of the following vowel) was extracted via a Praat script, thus generating 126 CV-syllables to parallel the 126 whole words. Vowel onset was marked at the start of periodicity and/or low-frequency energy in the spectrogram. Note, however, that some productions had long and diffuse fricative-vowel transitions, and the vowel onset annotation is therefore less reliable for these tokens. In total, 252 tokens (126 syllables) from 25 children (10 female) between three and eight were included in the perceptual task. See Table 2 for an overview of the speech stimuli included. Due to an error in the wordlist preparation, one syllable was rated by only 22 listeners. This syllable and the associated word were excluded from the analysis and the summary in Table 2. As such, a total of 250 tokens, excluding duplicate presentations, were included in the analysis.

Table 2. An overview of the speech stimuli included in the perceptual evaluation. Note that both words and the extracted syllables are presented, although the fricative was always assigned to the same transcription category regardless of lexical context.

Transcription category	Speaker age Intended fricative										Total
	3 years /s/ /ɕ/		4 years /s/ /ɕ/		5 years /s/ /ɕ/		6 years /s/ /ɕ/		7 years /s/ /ɕ/
[s]	18	8	20	2	18	2	4	0	18	0	90
[s/ɕ]	2	12	12	20	6	10	8	8	2	0	80
[ɕ]	0	6	0	28	0	20	0	10	0	16	80

To provide a coarse acoustic description of the speech stimuli, the first spectral moment (spectral centre of gravity; CoG) of each initial fricative was calculated and is shown in Figure 1. CoG is related to place of articulation in lingual fricatives, and has previously been used to quantify the acquisition of the English sibilant contrast /s/-/ʃ/ (see e.g., Holliday et al, 2015; Romeo et al., 2013). Note, however, that spectral moments analysis has been criticized as the method only provides a gross description of the spectral envelope, and the moments have an ambiguous relation to fricative articulation (see discussion in Shadle, 2023). However, for the present purposes, a course-grained description of acoustics was deemed acceptable.

Figure 1. Spectral centre of gravity (CoG) of word-initial fricatives across transcription categories [ɕ], [s/ɕ] and [s]. Each point shows the value of one sound (total = 125).

First, the fricatives were segmented in Praat (Boersma & Weenink, 2024); a stable region of frication, visible in the spectrogram (0–10 kHz) from the whole-word view was annotated. The boundaries of the segment were set conservatively, excluding gradual onsets and vowel transitions. A 30 ms (rectangular) window from the middle of the annotated fricative segment was extracted with a Praat script. The excised sounds were run through an R (R Core Team, 2024) script, to generate multi-taper spectra (k = 8, n = 4) and calculate the CoG (range = 0.3–15 kHz), with the spectRum (Reidy, 2016) package. One fricative segment included a brief mid-fricative silence, and for this sound, the fricative region closest to the vowel was used for acoustic analysis.

As indicated by the distributions of CoG in Figure 1, there is overlap between transcription categories as well as substantial variation within categories (primarily for /s/). Nevertheless, the general increasing trend with increasingly anterior sounds indicates that the stimuli chosen reflect the anticipated acoustic distribution for the three transcription categories.

Perceptual evaluations

Participants

Inexperienced listeners (n = 27, 14 self-reported male and 13 self-reported female) were recruited through social networks, and clinically active speech-language pathologists (SLPs) (n = 18) were recruited through social media, email and telephone. All participants were between 19 and 58 years of age, had Swedish as their first language and reported a typical hearing. All SLPs were required to work clinically with children with SSD. SLP students were not included in this study. None of the inexperienced listeners reported having studied phonetics or speech-language pathology. Table 3 provides a summary of participant demographics, including age, work experience (for SLPs) and time spent with children between 1 and 6 years of age or children with SSD.

Table 3. Description of participant demographics for the two listener groups (inexperienced listeners and SLPs).

Group	Age	Do you have children?	How often do you see children 1–6 years?
Inexperienced listeners	Mean: 33 years Min: 19 years Max: 58 years	Yes: 6 No: 21	Every day:1 Every week: 4 Every month: 3 More seldom/never: 19
	Work experience	How often do you see children with SSD?
Speech-language pathologists	<1 year: 4 1–5 years: 5 6–10 years: 6 11–15 years: 2 > 15 years: 1	Every day: 9 Every week: 6 Every month: 3 More seldom/never: 0

Ethical considerations

The present study is part of a PhD project that involves recordings of children as well as perceptual judgments presented herein. The project has been approved by the Swedish Ethics Review Authority (registration numbers: 2019–02854, 2020–03306, 2022–02168–02). Informed written consent was collected from all listeners prior to data collection. Participants were also informed that they could withdraw from the study at any time without consequences and were compensated for their time with one cinema ticket.

Procedure

The perceptual task was administered through PsychoPy (Peirce et al., 2019) on a laptop computer. The sounds were presented at a fixed level via BeyerDynamic DT 770 PRO headphones. All participants performed the listening task in a quiet room, bar one who sat in a quiet location outdoors. The test leader (Author LO) was available for questions throughout the experiment.

Participants were instructed to rate the first sound of each syllable or word on a digital VAS, by clicking on the location that corresponded to their perception of the sound. The VAS ranged from ‘s’ to ‘tj’ (i.e. the most common orthographic representation of /ɕ/). For an illustration of the digital VAS used in the present study, see Figure 2. The click location of each response corresponds to a whole number between 0 (i.e. ‘s’) and 100 (i.e. ‘tj’). The written instructions ‘Listen and rate the first sound’ (Swe: Lyssna och bedöm det första ljudet) were printed above the scale throughout the experiment. Participants were encouraged to use the entire scale and to rely on their first impression of the initial sound for their judgment. Sounds were presented only once, although the test leader could repeat a sound in cases of intermittent background noise. Prior to the presentation of the test items, a training block containing six adult productions (three syllables), was presented, to ensure that all participants understood the procedure.

Figure 2. The digital VAS used in the present study.

The speech stimuli were presented in two blocks, separated by a short break. Words and their constituent syllables were presented in different blocks. Participants were counterbalanced across four conditions with different pseudo-random wordlists. To explore intrarater reliability, 9.5% of the child recordings (12 words, 12 syllables) were duplicated. Duplicates were presented in the opposite block to their twin, and the second presentation of each sound was used only for analysis of reliability. In addition to the child recordings, 31 adult tokens (18 syllables) were included in the test blocks as perceptual anchors. The adult tokens were produced by two female native speakers of Swedish from the Svealand region (Author CWB and SS), and were recorded in the anechoic chamber of the Phonetics Laboratory at Stockholm University.

Analysis

The ratings were summarised and analysed in R (R Core Team, 2024). Adult speech tokens and responses with reaction times of less than 200 ms after onset of audio presentation were removed prior to analysis. The second presentation of the duplicates was only included in the analysis of intrarater reliability. A total of 11,245 ratings of the 250 sounds were used in analyses of rating behaviour and in the statistical model.

The raw responses were tallied and visualised through a variety of plots in R, to gauge the visible effects of lexical context, experience and individual variation. Anonymised data and code to generate statistics, tables and figures presented in this paper (including model evaluation and additional visualisations of the data), are publicly available at https://doi.org/10.5281/zenodo.11943307.

Reliability analysis

Intra- and interrater reliability were calculated with Intraclass Correlation (ICC), a two-way model of consistency. For intrarater reliability, ratings of the first and second presentation of the duplicated sounds were used. For interrater reliability, all ratings excluding the second presentations of the duplicated items were used (i.e. two ratings from each participant for each sound; from syllable and word contexts). Interrater reliability was calculated for all listeners (n = 45) as well as within the groups; SLPs (n = 18) and inexperienced listeners (n = 27).

Bayesian mixed-effects Beta regression

Bayesian mixed-effects beta regression was used to analyse VAS responses. Beta regression recognised that the VAS is bounded on both sides (Smithson & Verkuilen, 2006), which can help reduce issues with the assumption of homoscedasticity that would come with linear regression. Beta regression also enables the study of effects, not only on the mean of VAS responses but also in their variability through the phi parameter (for an application in the domain of language research, see Babel, 2022). The phi parameter corresponds to precision, which is inversely related to the variance (i.e. high precision corresponds to highly consistent responses). Though a Bayesian formulation of beta regression wass not critical for the present purpose, it was chosen as standard weakly regularising priors facilitates model convergence, while keeping statistical inferences conservative. Moreover, Bayes Factors (BF) as a measure of empirical support provides an intuitive alternative to null hypothesis significance testing (for a review, see Wagenmakers, 2007, for an accessible introduction, see McElreath, 2018 and for a tutorial on Bayesian models directed at researchers in the phonetic sciences, seeVasishth et al., 2018).

To facilitate interpretation, VAS responses were transformed from [s]-to-[ɕ] ratings into ratings of typicality or category goodness, for the intended fricative; responses for intended /s/ tokens were transposed by subtracting them from 100. Therefore, higher VAS responses designated a more ‘target-like’ rating for both intended fricative targets (i.e. /s/ or /ɕ/). All responses were then scaled to a range from 0.00004 to 0.99996 to remove 0 and 1 values (Smithson & Verkuilen, 2006).¹

Beta regression analysed both the mean and precision of these transformed VAS responses as a function of participants’ experience (weighted effect coded: 18 = SLP vs. 27 = inexperienced listener) and lexicality (weighted effect coded: 125 = word vs. 125 = syllable), and their interaction. The analysis employed the maximal random effects structure justified by the design, including random by-participant and by-item intercepts as well as slopes for the full factorial of experience and lexicality. Following recommendations in the statistical literature, we used weakly regularising priors – specifically, the same settings as employed in Xie et al. (2021): Student priors (3, 0, 2.5) for all population-level effects, Cauchy priors (0, 2.5) for the standard deviations of group-level effects, and LKJ (1) priors for the correlation matrix of group-level effects.

Model syntax

VASresponse ~ 1 + SLP * lexical_context +(1 + lexical_context | participant) +(1 + lexical_context * SLP | item),phi ~ 1 + SLP * lexical_context +(1 + lexical_context | participant) +(1 + lexical_context * SLP | item),family = Beta(link = “logit”, link_phi = “log”))

The model was fit with the brms library (Bürkner, 2017) in R, with 1000 warm-up and 4000 posterior samples for each of four chains (i.e. a total of 16 000 posterior samples). All indicators suggested convergence (e.g. all Řs $\le$ 1.001). We report the results of Bayesian hypothesis testing using brms’s hypothesis function (Bürkner, 2017).

Note that the minimal/near-minimal word pair environment could affect perceptual ratings, particularly in the whole word context. However, substitutions were unbalanced across minimal and near-minimal word pairs (8 [s] for /ɕ/ substitutions in minimal word pairs and 4 in near-minimal word pairs). To avoid over-parameterisation of the model and possible interference from substitution errors in the speech material, minimal word pair status was not included in the model.

Results

Visualisations of the click locations show a wide range of VAS responses, although many tokens were classified as close to one of the two scale endpoints (i.e. [s] or [ɕ]). Figure 3 shows the density distributions of the VAS responses for the three transcription categories [s], [ɕ] and in-between (i.e. [s/ɕ]). As is revealed in Figure 3, the sounds that were transcribed as [s] or [ɕ] were to a high degree rated as very [s]-like and [ɕ]-like, respectively. The ratings of the in-between transcription category [s/ɕ] were instead dispersed across a larger range of scales, although an increase in ratings close to the intended fricative endpoint is visible, as well as a small bump in the middle of the scale.

Figure 3. Density distributions of VAS responses across all participants, separated by transcription category ([s], [ɕ] and in-between [s/ɕ]).

Plots of individual participants’ ratings were visually inspected to gauge the variation of VAS responses. Figure 4 shows individual responses for three participants, revealing differences in categoricity and use of the VA scale. Most participants used the whole scale, but some showed a visually salient categorical pattern (n = 2, both inexperienced), with few ratings across the middle of the scale, and many located at the scale endpoints (see e.g. the rightmost panel of Figure 4). Most listeners exhibited a pattern similar to the participant in the left and middle panels of Figure 4, with the majority of ratings for the [s] and [ɕ] transcription categories close to the end of the scale adjacent to the intended fricative label and a decreasing number of responses towards the middle. The ‘in-between’ category elicited more varied responses across the whole scale with few ratings at the very endpoints. Note, however, that no statistical quantification of rating categoricity was performed. Visualisations of all individual listeners’ ratings are available at 10.5281/zenodo.11943307.

Figure 4. Distribution of three listeners’ ratings across the three transcription categories ([s], [ɕ] and in-between [s/ɕ]). Each point corresponds to one rating (total=250 per listener) and density is illustrated by the opacity of the points.

Reliability

Intrarater reliability was estimated through ICC (model: two-way, type: consistency), calculated across the duplicated sounds for each individual listener and is presented in Figure 5. ICC scores ranged from 0.57 (95%-CI: 0.23–0.79; poor-to-good reliability) to 0.98 (95%-CI: 0.95–0.99; excellent reliability) with an overall mean of 0.83 (good reliability).

Figure 5. Violin plot of intrarater reliability (ICC) for SLPs and inexperienced listeners. Each point represents one individual listeners’ ICC score.

On a group level, SLPs were more consistent than inexperienced listeners, although many inexperienced listeners had ICC scores at the same level as SLPs. ICC scores from less (5 years or less of work experience) and more experienced (6 years or more of work experience, ranging up to 15 + years) SLPs were plotted, but no clear differences came to light; less experienced SLPs had both the highest and lowest scores, and variation within the group was large.

To investigate interrater reliability, ICC (two-way model, consistency) was calculated, including only the first repetition of all sounds. Overall ICC including all ratings and all listeners was 0.729 (95%-CI: 0.693–0.764, F(244,10736) = 122, p < 0.001). Comparing the two groups, the interrater reliability for SLPs (ICC = 0.79, 95%-CI: 0.759–0.82, F(248,4216) = 68.8, p < 0.001) was higher than that for inexperienced listeners (ICC = 0.697, 95%-CI: 0.658–0.736, F(245,6370) = 63, p < 0.001). This corresponds to good reliability for SLPs, and moderate-to-good reliability for inexperienced listeners (Koo & Li, 2016).

Effects of lexicality and experience

Table 4 summarises the results of Bayesian hypothesis tests over Bayesian mixed-effects beta regression. As a guide for interpretation; BFs > 10 are considered to be strong support and BFs > 30 as very strong support (Raftery, 1995; Schönbrodt et al., 2017). The posterior probability presented in Table 4 reflects the proportion of posterior samples that support the hypothesis. A model summary and plots of posterior distributions for all population-level effects can be found in the online material (10.5281/zenodo.11943307).

Table 4. Summary of Bayesian hypothesis tests. For each effect, the estimate $\hat{\beta }$, its standard error (SE), directional 90% Credible Intervals (CrI), Bayes Factors (BF), and posterior probabilities are listed. As is the standard for Beta regression, estimates of effects on response means are expressed in log-odd units and estimates of effects on response precision are expressed in log units.

Hypothesis	Estimate $\widehat{(\mathit{\beta }})$	SE ($\hat{\mathit{\beta }}$)	CrIlower	CrIupper	BF	Posterior probability
Mean ($\mathit{\mu }$)
Experience (SLP vs mean) > 0	0.21	0.09	0.07	0.35	155.9	0.99
Lexicality (Word vs mean) > 0	0.38	0.05	0.31	0.46	>16000	1
Lexicality × Experience (Word vs mean; SLP vs mean) < 0	−0.27	0.08	−0.39	−0.14	7999	1
Precision ($\mathit{\varphi }$)
Experience (SLP vs mean) > 0	0.46	0.21	0.12	0.80	72.1	0.99
Lexicality (Word vs mean) > 0	0.16	0.05	0.08	0.25	1453.6	1
Lexicality × Experience (Word vs mean; SLP vs mean) < 0	−0.11	0.09	−0.27	0.04	7.5	0.88

We found strong evidence for main effects of both experience and lexicality (see estimates of effects on response mean and precision in Table 4). SLPs gave overall higher (more prototypical) ratings than inexperienced listeners (BF = 155.9) and fricatives in words were rated as more target-like than fricatives in syllables (BF > 16,000). These main effects exhibited a credible interaction (BF = 7999), as shown in Table 4. Simple slope analyses, not shown here, found that lexicality had credible effects in the same direction for both groups of participants ($\hat{\beta }$ Estimate was 0.49, 95%-CrI: 0.38–0.61 for inexperienced listeners and 0.23, 95%-CrI: 0.1–0.35 for SLPs). The effect was, however, reduced for SLPs (as indicated by the interaction).

As shown in Table 4, all effects found for the mean of VAS responses were also found for the precision of VAS responses: SLPs had more precise (less variable) responses than inexperienced listeners (BF = 72), and VAS ratings of fricatives in word context were more precise than VAS ratings of fricatives in syllable contexts (BF = 1453.6). As was the case for means, the effect of lexicality on precision was reduced for SLPs (although the effect of the interaction on the precision of ratings received weaker support; BF = 7.5).

The Bayesian regression used herein also enables assessment of the extent to which the effects reported above held within each participant and/or item. Hypothesis testing revealed that the effect of lexicality on the mean of responses was credible in all participants (BFs from 11.3 to >16000, although all but two had BFs > 24) but not all items (BFs from 0.08 to >16000). The latter is expected given that the fricatives varied in terms of prototypicality; highly prototypical sounds are expected to elicit higher VAS ratings regardless of whether appearing in a whole word or in a word fragment. The effect of the experience was not found to hold within each item either (BFs from 0 to 15,999).

Discussion

In the present study, we explored experienced (SLPs) and inexperienced listeners’ ratings of Swedish children’s word-initial voiceless sibilant fricatives, specifically investigating whether the lexical context in which a sound stimulus is presented affects the VAS rating of said sound. We begin by addressing our research questions in relation to previous research on VAS ratings of child speech, then discuss listener experience and consider the possible clinical implications of our findings. Finally, limitations of the current work and future directions of research are discussed.

Do experienced and inexperienced adult listeners differ in their gradient category goodness ratings of children’s voiceless fricatives? Specifically, does listener experience influence the reliability of ratings?

On a group level, no large differences regarding the use of scale were visible. However, individual variation was apparent, with different response patterns across individual listeners in both groups. Most listeners used the whole scale, although a couple showed more categorical patterns. Collectively, SLPs had higher intra- and interrater reliability than inexperienced listeners, reflected in both the ICC score and the phi parameter of the beta regression model. These findings are expected and are consistent with Munson, Johnson, et al. (2012), regarding higher intrarater reliability for SLPs as compared to inexperienced listeners, despite different measures of reliability being used.

Does lexicality affect experienced and inexperienced adult listeners ratings of children’s voiceless fricatives?

The Bayesian mixed-effect Beta regression model showed an effect of lexicality for both groups (SLPs and inexperienced listeners) and for all individual listeners. An interaction between lexicality and experience was found, such that the effect of lexicality was more pronounced in the inexperienced group of listeners. In other words, SLPs’ ratings of children’s initial voiceless fricatives were less influenced by the lexical status of the stimuli presented. Moreover, an effect of both experience and lexicality on precision of VAS ratings was revealed, reflected in the phi parameter of the model. SLPs had higher precision (i.e. lower variance) than inexperienced listeners and words were rated with higher precision than syllables/word fragments.

Overall, these results lend support for the hypothesis expressed in Munson, Johnson et al. (2012), specifically that the percepts of listeners with more experience should be more stable across levels of lexical complexity (i.e. word fragments vs. whole words), compared to those of less experienced listeners.

Moreover, we found that SLPs rated initial voiceless fricatives as more target-like than did inexperienced listeners, in line with findings from Meyer and Munson (2021) who discovered that more-experienced SLPs used scale endpoints (corresponding to target-like ratings) to a higher extent than did less-experienced SLPs and inexperienced listeners.

Notably, however, the difference between inexperienced and experienced listeners’ VAS ratings was larger for syllables than for whole words. Inexperienced listeners are undoubtedly more familiar with whole words compared to word fragments, so this difference is perhaps not unexpected. However, it could indicate that the effect of listener experience is exaggerated in previous work. That is, had we conducted an experiment to parallel previous procedures, including only CV-syllables (Ancel et al., 2023; Meyer & Munson, 2021; Munson et al., 2010; Munson, Johnson, et al., 2012; Strömbergsson et al., 2015), we might have found a more pronounced effect of listener experience.

On surface level, the present results may seem contradictory to Coniglio et al. (2022), who reported a more gradient use of the VA scale among clinically experienced listeners compared to less experienced listeners. However, as the task and analysis in their study differ to ours in many respects, results cannot easily be compared and differences in rating categoricity may be due to study-specific features (e.g. stimuli or task design).

A note on experience

As described above, a clear effect of experience was found in the present study. However, the definition of experience used herein is broad and ambiguous, resulting in a number of uncertainties related to the effects observed. That is to say, we do not know whether the ‘SLP effect’ is driven by experience with children (with highly variable speech) and/or experience with transcription of speech and systematic listening.

Indeed, although the inexperienced listener group in the present study was highly heterogeneous with respect to time spent with children (see Table 3), most had limited experience interacting with young children.

In a recent study, exploring different types of experience in perception of child speech, Yu et al. (2023) compared accuracy on a speech-in-noise transcription task including child speech (isolated words) from four listener groups with varying experience interacting with young children. They found no experience-based advantage for child speech intelligibility (i.e. mothers of young children and early childhood educators did not perform better than undergraduate students), although SLPs were more accurate than the other listener groups. However, as SLPs performed better than the other groups for both adult and child speech, Yu et al. (2023) speculate that their performance might be related to a ‘task advantage’. Hence, listener experience is a multifaceted construct, and further exploration into how different aspects of experience contribute to listener ratings of child speech, in different listening tasks, is warranted.

A clinical perspective

The overarching raison d’être of speech therapy is arguable to reach and maintain functional communication and intelligible speech for the client/patient. Thus, it is reasonable for clinicians to assess speech in naturally occurring forms, for example words. However, words come accompanied by lexical bias, among others, and SLPs need to be able to shift their focus from communication to fine phonetic detail in children’s speech, in order to capture gradient development in speech sound contrast acquisition and to choose appropriate targets in speech intervention (see e.g. Macken & Barton, 1980, Munson, 2010; Tyler et al., 1993). While there are examples of whole-word stimuli in VAS perception studies (e.g. Coniglio et al., 2022; Harel et al., 2017), much previous work has utilised CV-syllables. The present study indicates that clinically active SLPs are, as a group, less affected by lexicality of the sound stimuli they are rating, compared to laypeople (denoted herein as inexperienced listeners). This supports the validity of VAS ratings as a clinically viable assessment tool. Nevertheless, a lexical effect was present for all individual listeners, with higher (more prototypical) and more consistent ratings for whole word stimuli. The clinical impact of this effect is not known, although it is conceivable that lexical bias could reduce the listener’s sensitivity to fine phonetic detail. Our results indicate that lexical bias should be considered when attempting to apply findings from previous studies on speech perception using VAS (Ancel et al., 2023; Munson & Urberg Carlson, 2016; Munson et al., 2010; Munson, Johnson, et al., 2012, 2012; Schellinger et al., 2017; Strömbergsson et al., 2015) to the clinic.

Limitations & future research

One possible caveat of the current design is that the stimuli used were natural speech, chosen to reflect a perceptual variation across the /s/-/ɕ/ continua. This choice hopefully mirrors some of the variation that SPLs encounter in the clinic but limits experimental control of the physical characteristics of the speech stimuli. It is unclear how much of the rating behaviour observed in the present study is influenced by the specific stimuli used herein. To increase generalisability, more research with other speech material and speech sound contrasts is needed.

With respect to the lexicality of stimuli, the excised word fragments/CV sequences used do sound slightly strange. However, the format was explicitly chosen to increase comparability across vowel contexts and to mirror previous VAS investigations (e.g. Julien & Munson, 2012). Additionally, the use of whole-word productions introduces auxiliary information, such as f₀, voice quality and additional speech errors, which could also bias perceptual ratings. However, as speaker-specific characteristics vary substantially among SLP clients, rich and variable stimuli are vital to explore the validity of clinical assessment instruments.

The acoustic characteristics of the stimuli might also affect ratings, and although we present a course acoustic description of the sounds used herein, we do not relate acoustic characteristics to listener percepts. As such, we did not explore the accuracy of the VAS ratings provided. We argue that acoustic-perception investigations should include all potentially relevant acoustic cues, especially when exploring subtle phonetic variation, and should ideally explore both individual differences and group trends in listeners’ cue-weighting. Considering the complexity of fricative acoustics, we believe such a study to be beyond the scope of the present article. Nevertheless, the lack of acoustic investigation does lead to some open questions about the effects of experience and lexicality presented above. For example, do ratings of high prototypicality correspond to acoustically canonical tokens? Is acoustic gradience mirrored in the gradience of VAS ratings? Do these patterns differ between the listener groups? Future studies of the relationship between perception and acoustic characteristics of Swedish children’s voiceless fricatives are welcome and would advance our knowledge concerning cue-weighting and cross-linguistic tendencies in speech perception.

Finally, the group sizes for SLPs and inexperienced listeners were unbalanced due to difficulties recruiting clinicians. The difference in size was taken into account in the effect weighting of the model and the number of SLPs is comparable to previous work (e.g. Coniglio et al., 2022; Munson, Johnson, et al., 2012). Nevertheless, perceptual experiments with larger and more balanced groups are needed to further evaluate the effects found here. Online perception experiments, where large groups are easier to recruit, could be a valuable avenue for future research.

Conclusion

The present study reveals differences in the reliability and the effect of lexicality on experienced (SLPs) and inexperienced listeners’ ratings of Swedish children’s voiceless fricatives. Overall, SLPs had higher intra- and interrater reliability for VAS ratings, although individual variability was sizeable. The Bayesian mixed-effects beta regression and subsequent hypothesis testing lends strong support for an effect of lexicality on VAS ratings for both groups, which holds also for all individuals. The results of this investigation indicate that while SLPs are less affected by stimuli lexicality, the findings of previous research concerning VAS ratings of non-lexical initial syllables cannot be directly transferred to the clinical context, without consideration of possible lexical bias. Questions concerning the impact of level and type of experience that may affect gradient ratings of fine phonetic detail in child speech remain open.

Author contribution

Conceptualisation and design, Data & experiment preparation: CWB & SS.

Data collection: LO.

Statistical analysis: CWB with input from FJ (as described in Acknowledgements).

Data Curation, Visualisations, Writing – Original Draft: CWB

Writing – Review & Editing: CWB & SS.

All authors read and approved the final version of the manuscript.

Acknowledgments

Sincere thanks to Professor Florian Jaeger for generously sharing his knowledge and time and for providing feedback on a previous version of the manuscript. Prof. Jaeger hosted a workshop on statistics at Stockholm University, during which the foundation of the Bayesian model in this paper was built. Nevertheless, any mistakes in model implementation and interpretation are CWBs. Thank you also to Marcin Włodarczak for his help with coding mishaps.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data (anonymous ratings) and code used to generate the results presented in this paper are publicly available at 10.5281/zenodo.11943307. Audio is not shared.

Additional information

Funding

The author(s) reported that there is no funding associated with the work featured in this article.

Notes

^1. This ‘shrinkage’ or ‘squeezing’ is required for beta regression, which does not allow values of 0 and 1 (21.6% of our data). The degree of shrinkage constitutes a researcher degree of freedom. To reduce the risk of anti-conservativity, we committed to the same degree of shrinkage as in previous work (Smithson & Verkuilen, 2006).