Inter-rater reliability in classification of canonical babbling status based on canonical babbling ratio in infants with isolated cleft palate randomised to Timing of Primary Surgery for Cleft Palate (TOPS)

ABSTRACT

Canonical babbling (CB) is commonly defined as present when at least 15% of all syllables produced are canonical, in other words a canonical babbling ratio (CBR) ≥0.15. However, there is limited knowledge about inter-rater reliability in classification of CB status based on CBR and inter-rater differences in assessment of CBR. We investigated inter-rater reliability of experienced Speech Language Therapists (SLTs) on: classification of CB status based on CBR ≥ 0.15, CBRs and the total number of syllables per infant used to calculate CBR.

Each infant (n = 484) was video-recorded at a clinical site in play interaction with their parent as part of the randomised controlled trial Timing of Primary Surgery for Cleft Palate. Each recording was subsequently assessed by three independent SLTs, from a pool of 29 SLTs. They assessed the recordings in real time.

The three assessing SLTs agreed in classification of CB status in 423 (87.4%) infants, with higher complete agreement for canonical (91%; 326/358) than non-canonical (77%; 97/126). The average difference in CBR and total number of syllables identified between the SLT assessments of each infant was 0.12 and 95, respectively.

This study provided new evidence that one trained SLT can reliably classify CB status (CBR ≥ 0.15) in real time when there is clear distinction between the observed CBR and the boundary (0.15); however, when the observed CBR approaches the boundary multiple SLT assessments are beneficial. Thus, we recommend to include assessment of inter-rater reliability, if the purpose is to compare CBR and total syllable count across infants or studies.

Trial registration number here: www.clinicaltrials.gov, identifier NCT00993551

KEYWORDS:

• Canonical babbling status
• canonical babbling ratio
• agreement
• assessment variation
• isolated cleft palate

Acknowledgments

Collaborators of the TOPS Speech Group:

Liz Albery, Bristol, UK; Helene Søgård Andersen, Copenhagen, Denmark; Malin, Appelqvist, Uppsala, Sweden; Pia Bodling, Linköping, Sweden; Melanie Bowden, Manchester, UK; Karin Brunnegård, Umeå, Sweden; Stephanie Van Eeden, Newcastle, UK; Josefin Enfalt, Umeå, Sweden; Ana Paula Fukushiro, Bauru, Brazil; Cristina Guedes de Azevedo Bento Goncalves, Bauru, Brazil; Line Dahl Jørgensen, Copenhagen, Denmark; Jorunn Lemvik, Oslo, Norway; Louise Leturgie, Århus,Denmark; Eva Liljerehn, Uppsala, Sweden; Natalie Lodge, Edinburgh, UK; Siobhan McMahon, Liverpool, UK; Haline Coracine Miguel, Bauru, Brazil; Kathryn Patrick, Manchester, UK; Ginette Phippen, Salisbury, UK; Silvia Helena Alvarez Piazentin-Penna, Bauru, Brazil; Lucy Southby, Oxford, UK; Ann-Sofie Taleman, Linköping, Sweden;. Jorid Tangstad, Bergen, Norway; Renata Yamashita, Bauru, Brazil.

The authors wish to thank Joan Bogh Nielsen, Århus, Denmark; Lucy Rigby, Newcastle, UK; Nina-Helen Pedersen, Bergen, Norway, and Ragnhild Aukner, Oslo, Norway for contributing to data collection and analyses. We also wish to thank the participating clinical sites, the Data Coordinating Center, Liverpool, UK, and the Administrative Coordinating Center, Manchester, UK, for help with data collection, management of the twelve months visit data and preparation of the speech material.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website

Notes

¹ In 66 (4.5%) of the 1452 assessments (3*484) an infant was assessed by an SLT from their own site.

Additional information

Funding

This publication was made possible by Grants Number R21DE15128, U01DE018664 and U01DE018837 from the Institute of Dental and Craniofacial Research (NIDCR). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIDCR.