Reliability of data on percent consonants correct and its associated quality indicator in the Swedish cleft lip and palate registry

Figures & data

Table 1. Interpretation of levels of kappa (k) statistics, intraclass coefficients (ICCs), and percentage agreement according to Cicchetti [28].

Levels of k or ICC	Levels of observed agreement (%)	Levels of clinical or practical significance
<.40	<70	Poor
.40–.59	70–79	Fair
.60–.74	80–89	Good
.75–1.00	90–100	Excellent

Table 2. Intrajudge agreement for the dichotomised quality indicator ≥86% correct consonants as calculated with Cohen’s kappa and percentage agreement, and percent consonants correct (PCC) based on 59 (PCC 59) and 30 (PCC 30) target consonants respectively as calculated with intraclass correlation coefficient (ICC).

	Quality indicator			PCC 59		PCC 30
	%	k	95% CI	ICC	95% CI	ICC	95% CI
SLP1–SLP1	97	.93	.79–1	.98	.97–.99	.97	.93–.99
SLP2–SLP2	86	.67	.38–.97	.99	.97–.99	.97	.93–.98
SLP3–SLP3	90	.79	.57–1	.98	.95–.99	.94	.86–.97
SLP4–SLP4	97	.93	.78–1	.97	.94–.99	.96	.92–.98
SLP5–SLP5	93	.86	.68–1	.97	.93–.99	.95	.9–.98
SLP6–SLP6	93	.86	.67–1	.98	.96–.99	.96	.92–.98

CI: confidence interval; SLP: speech-language pathologist.

Table 3. Agreement for the quality indicator ≥86% correct consonants, when pair-wise comparisons of reassessments by speech-language pathologists (SLP1, SLP2, SLP3, SLP4, SLP5, and SLP6) and data from the Swedish Quality Registry for cleft lip and palate registry (QR) were performed, as calculated with percentage agreement and Cohen’s kappa.

	%	k	95% CI
SLP1–SLP2	78	.58	.42–.74
SLP1–SLP3	95	.89	.8–.98
SLP1–SLP4	94	.86	.76–.97
SLP1–SLP5	93	.85	.74–.96
SLP1–SLP6	83	.67	.53–.82
SLP2–SLP3	83	.67	.53–.82
SLP2–SLP4	78	.58	.42–.74
SLP2–SLP5	81	.63	.48–.79
SLP2–SLP6	82	.64	.49–.8
SLP3–SLP4	89	.76	.62–.89
SLP3–SLP5	92	.83	.72–.94
SLP3–SLP6	84	.69	.55–.83
SLP4–SLP5	93	.85	.74–.96
SLP4–SLP6	79	.59	.43–.75
SLP5–SLP6	87	.73	.6–.87
QR-SLP1	89	.77	.64–.9
QR-SLP2	79	.59	.43–.75
QR-SLP3	90	.79	.66–.91
QR-SLP4	87	.72	.58–.86
QR-SLP5	90	.79	.66–.91
QR-SLP6	80	.61	.45–.76
Mean	–	.72	.58–.85

CI: confidence interval.

Cicchetti DV. The precision of reliability and validity estimates re-visited: distinguishing between clinical and statistical significance of sample size requirements. J Clin Exp Neuropsychol. 2001;23(5):695–700.

 PubMed Web of Science ®Google Scholar