Reliability of data on percent consonants correct and its associated quality indicator in the Swedish cleft lip and palate registry

Abstract

Background

Data in national health care quality registries must be valid and reliable in order to enable open comparisons of results.

Aim

To assess the reliability of data on percent consonants correct (PCC) and its associated quality indicator ≥86% correct consonants in the Swedish quality registry for patients born with cleft lip and palate (CLP) registry.

Methods

Six independent speech-language pathologists re-assessed the audio recordings of 96 five-year-olds with PCC data in the CLP registry. Target consonants of a single-word picture-naming test were phonetically transcribed, and PCC was calculated. The reliability of PCC data was assessed with the intraclass correlation coefficient (ICC). The reliability of the quality indicator ≥86% correct consonants was assessed with point-by-point percentage agreement and Cohen’s kappa.

Results

Intra- and inter-judge agreement for PCC was excellent with ICCs above 0.9, and so was the agreement of data from the CLP registry and the six judges’ re-assessments. The percentage agreement between all judges and the CLP registry for the quality indicator ≥86% correct consonants was poor (67%). However, in 88% of the cases, results from four judges and the CLP registry agreed, corresponding to good agreement. The mean of all kappa values for six judges and the CLP registry corresponded to good agreement (0.72).

Conclusions

The results indicate the PCC data in the CLP registry and the quality indicator ≥86% correct consonants to be reliable. When differences in outcome between treatment centres are detected, the raw data collected should always be re-examined before drawing definitive conclusions.

Keywords:

• Percent consonants correct
• cleft lip and palate
• reliability
• quality registry

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Kristina Klintö

Kristina Klintö is a SLP, Associate Professor, and project manager for the Swedish CLP registry. She is responsible for speech variables, and collection and analysis of speech registry data from Skåne University Hospital. She performed perceptual assessment and was responsible for statistical calculations and results. She also wrote the first draft of the manuscript.

Emilie Hagberg

Emilie Hagberg is a SLP, and PhD student. She is responsible for speech variables, and collection and analysis of speech registry data from Karolinska University Hospital. She performed perceptual assessment and contributed to the writing of the manuscript.

Christina Havstam

Christina Havstam is a SLP, and Associate Professor. She is responsible for speech variables, and collection and analysis of speech registry data from Sahlgrenska University Hospital. She contributed to the writing of the manuscript.

Cecilia Nelli

Cecilia Nelli is a SLP and Master of Science. She is responsible for speech variables, and collection and analysis of speech registry data from Linköping University Hospital. She performed perceptual assessment and contributed to the writing of the manuscript.

Åsa Okhiria

Åsa Okhiria is a SLP, and PhD student. She is responsible for speech variables, and collection and analysis of speech registry data from Uppsala University Hospital. She performed perceptual assessment and contributed to the writing of the manuscript.

Karin Brunnegård

Karin Brunnegård is a SLP, and PhD. She is responsible for speech variables, and collection and analysis of speech registry data from University Hospital of Umeå. She performed perceptual assessment and contributed to the writing of the manuscript.