Agreement Between Standard and ICD-10-Based Injury Severity Scores

Figures & data

Table 1 Strength of Agreement Using Kappa Statistics

Value of κ	Strength of Agreement
< 0.20	Poor
0.21–0.40	Fair
0.41–0.60	Moderate
0.61–0.80	Good
0.81–1.00	Very good

Note: Adapted with permission from Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159. Biometrics © 1977 International Biometric Society.¹⁸

Table 2 Demographics and Injury Characteristics for the TARN Trauma Population

	N = 4308
Age, median [IQR] (years)	44 [29.5;60]
Male sex, n (%)	3019 (70)
Mechanism of injury, n (%)
Blunt	3944 (92)
Vehicle incident/collision	2236 (57)
Fall, > 2 meters	768 (19)
Fall, < 2 meters	336 (8.5)
Blows/blast/crush	174 (4.4)
Burn	12 (0.3)
Other	418 (11)
Penetrating	362 (8)
Stabbing	257 (71)
Shooting	105 (29)
ISS, median [IQR]	16 [9;25]
> 15, n (%)	2317 (54)
No. of ICD-10 injury diagnoses, median [IQR]	4 [2;6]

Abbreviations: TARN, Trauma Audit and Research Network; ISS, Injury Severity Score; IQR; Interquartile Range; ICD-10, International Classification of Diseases 10th revision.

Table 3 Summary Statistics of the Four Different Methods for Determining ISS

	ISS-TARN	ISS-TQIP	ISS-NIS	ISS-Map
Median	16	10	17	9
IQR	9–25	2–25	5–26	4–16
< 15 (%)	46	54	43	75
> 15 (%)	54	46	57	25

Abbreviations: ISS, Injury Severity Score; TARN, Trauma Audit and Research Network; TQIP, Trauma Quality Improvement Program; NIS, National Inpatient Survey; IQR, Interquartile Range.

Figure 1 Bland-Altman plot comparing ISS-TQIP and ISS-TARN. (A) The difference between ISS-TQIP and ISS-TARN as a function of mean ISS ((ISS-TQIP + ISS-TARN)/2). Bias (red line): −4.1 (95% CI: −4.5 – −3.7). Limits of agreement (blue lines): −27.7 to 19.5 (95% CI: [−28.3 - −27.0]; [18.8–20.1]). (B) The difference between ISS-TQIP and ISS-TARN as a function of mean ISS ((ISS-TQIP + ISS-TARN)/2) for a subset of trauma patients with mean ISS > 15 (n=2012). Bias (red line): −2.5 (95% CI: −3.1 – −1.8). Limits of agreement (blue lines): −32.1 to 27.1 (95% CI: [−33.2 – −30.9]; [26.0–28.3]). (C) The difference between ISS-TQIP and ISS-TARN as a function of ISS-TARN. Bias (red line): −4.1 (95% CI: −4.5 – −3.7). Limits of agreement (blue lines): −27.7 to 19.5 (95% CI: [−28.3 – −27.0]; [18.8–20.1]). (D) The difference between log(ISS-TQIP) and log(ISS-TARN) as a function of mean log(ISS) ((log(ISS-TQIP) + log(ISS-TARN))/2). Proportional bias (red line): 0.57 (95% CI: 0.55–0.59), ie ISS-TQIP is on average 43% lower than ISS-TARN. Proportional limits of agreement (blue lines): 0.07–4.30 (95% CI: [0.07–0.08]; [4.08–4.54]), ie ISS-TQIP ranges from being 93% lower to 330% higher than ISS-TARN.

Abbreviations: ISS, Injury Severity Score; TQIP, Trauma Quality Improvement Program; TARN, Trauma Audit and Research Network.

Figure 2 Bland-Altman plot comparing ISS-NIS and ISS-TARN. (A) The difference between ISS-NIS and ISS-TARN as a function of mean ISS ((ISS-NIS + ISS-TARN)/2). Bias (red line): 1.1 (95% CI: 0.7–1.4). Limits of agreement (blue lines): −23.5 to 25.7 (95% CI: [−24.2 - −22.9]; [25.0–26.3]). (B) The difference between ISS-NIS and ISS-TARN as a function of mean ISS ((ISS-NIS + ISS-TARN)/2) for a subset of trauma patients with mean ISS > 15 (n=2358). Bias (red line): 4.0 (95% CI: 3.4–4.5). Limits of agreement (blue lines): −24.0 to 31.9 (95% CI: [−25.0 – −23.0]; [30.9–32.9]). (C) The difference between ISS-NIS and ISS-TARN as a function of ISS-TARN. Bias (red line): 1.1 (95% CI: 0.7–1.4). Limits of agreement (blue lines): −23.5 to 25.7 (95% CI: [−24.2 – −22.9]; [25.0–26.3]). (D) The difference between log(ISS-NIS) and log(ISS-TARN) as a function of mean log(ISS) ((log(ISS-NIS) + log(ISS-TARN))/2). Proportional bias (red line): 0.89 (95% CI: 0.87–0.92), ie ISS-NIS is on average 11% lower than ISS-TARN. Proportional limits of agreement (blue lines): 0.14–5.61 (95% CI: [0.14–0.15]; [5.34–5.89]), ie ISS-NIS ranges from being 86% lower to 461% higher than ISS-TARN.

Abbreviations: ISS, Injury Severity Score; NIS, National Inpatient Survey; TARN, Trauma Audit and Research Network.

Figure 3 Bland-Altman plot comparing ISS-map and ISS-TARN. (A) The difference between ISS-map and ISS-TARN as a function of mean ISS ((ISS-map + ISS-TARN)/2). Bias (red line): −7.3 (95% CI: −7.6 – −7.0). Limits of agreement (blue lines): −28.0 to 13.4 (95% CI: [−28.5 – −27.4]; [12.9–14.0]). (B) The difference between ISS-map and ISS-TARN as a function of mean ISS ((ISS-map + ISS-TARN)/2) for a subset of trauma patients with mean ISS > 15 (n=1724). Bias (red line): −11.8 (95% CI: −12.4 – −11.2). Limits of agreement (blue lines): −36.8 to 13.2 (95% CI: [−37.9 – −35.8]; [12.2–14.2]). (C) The difference between ISS-map and ISS-TARN as a function of ISS-TARN. Bias (red line): −7.3 (95% CI: −7.6 – −7.0). Limits of agreement (blue lines): −28.0 to 13.4 (95% CI: [−28.5 – −27.4]; [12.9–14.0]). (D) The difference between log(ISS-map) and log(ISS-TARN) as a function of mean log(ISS) ((log(ISS-map) + log(ISS-TARN))/2). Proportional bias (red line): 0.60 (95% CI: 0.58–0.61), ie ISS-map is on average 40% lower than ISS-TARN. Proportional limits of agreement (blue lines): 0.13–2.68 (95% CI: [0.13–0.14]; [2.58–2.79]), ie ISS-map ranges from being 87% lower to 168% higher than ISS-TARN.

Abbreviations: ISS, Injury Severity Score; TARN, Trauma Audit and Research Network.

Table 4 Proportional Bias for the Three Conversion Tool ISSs Compared to ISS-TARN Split into Three Groups According to the Size of the Bias, Including the Age Distribution and Proportion of Patients with Severe Head Injury for Each Proportional Bias Group

	ISS-TQIP n = 4308	ISS-NIS n = 4308	ISS-Map n = 4308
Reasonable bias^a (≤ ±15%), n (%)	717 (17)	852 (20)	934 (22)
Age, mean (sd) [years]	46.9 (19.2)	46.5 (19.0)	45.5 (18.9)
Head AIS3+, n (%)	382 (53)	380 (45)	206 (22)
Moderate bias^a (±15–50%), n (%)	999 (23)	1257 (29)	1057 (25)
Age, mean (sd) [years]	48.2 (19.9)	47.3 (19.4)	45.9 (19.3)
Head AIS3+, n (%)	567 (57)	638 (51)	465 (44)
Extreme bias^a (> ±50%), n (%)	2592 (60)	2199 (51)	2317 (54)
Age, mean (sd) [years]	45.3 (18.9)	45.5 (19.2)	46.7 (19.3)
Head AIS3+, n (%)	534 (21)	465 (21)	812 (35)

Notes: ^aThe proportional bias was calculated as ([conversion tool ISS – ISS-TARN]/mean ISS)*100%, where conversion tool ISS is ISS-TQIP, ISS-NIS, and ISS-map, respectively, and mean ISS is the mean between ISS-TARN and the respective conversion tool ISSs.

Abbreviations: ISS, Injury Severity Score; TARN, Trauma Audit and Research Network; TQIP, Trauma Quality Improvement Program; NIS, National Inpatient Survey, sd; standard deviation, AIS, Abbreviated Injury Scale.

Table 5 Kappa Statistics for ISS Dichotomized at 9, 15, and 25, Respectively, for the Three Conversion Tool ISSs Compared to ISS-TARN

ISS Cut-Off	ISS-TQIP	ISS-NIS	ISS-Map
9	0.39	0.46	0.36
15	0.43	0.44	0.29
25	0.33	0.37	0.11

Abbreviations: TARN, Trauma Audit and Research Network; ISS, Injury Severity Score; TQIP, Trauma Quality Improvement Program; NIS, National Inpatient Survey.

Dansk Traumeregister Årsrapport 2018 (The Danish Trauma Registry Annual Report 2018); 2019. Available from: https://www.sundhed.dk/content/cms/19/97019_dansktraumeregister-aarsrapport_2018_offentligversion.pdf. Accessed December 10, 2021.

 Google Scholar