An observational cohort study of the performance of the REDS score compared to the SIRS criteria, NEWS2, CURB65, SOFA, MEDS and PIRO scores to risk-stratify emergency department suspected sepsis

Figures & data

Figure 1. Patient flow diagram.

Table 1. The baseline characteristics of the study population.

	All (n = 2594 Median [IQR] or number (percentages)	Survivors (n = 2262) Median [IQR] or number (percentages)	Non-survivors (n = 332) Median [IQR] or number (percentages)	Difference between survivors and non-survivors
Patient characteristics
Age (years)	73 [57–84]	72 [56–83]	80 [72–88]	p < .0001
Sex (male)	1380 (53.2%)	1189 (52.6%)	192 (57.8%)	p = .08
Respiratory rate (breaths/minute)	23 [19–28]	22 [19–28]	25 [20–32]	p < .0001
Oxygen saturation (%)	96 [93–97]	96 [93–98]	94 [89.5–96]	p < .0001
Fraction of inspired oxygen (FiO2)	0.21 [0.21–0.21]	0.21 [0.21–0.21]	0.21 [0.21–0.28]	p < .0001
Heart rate (beats/minute)	102 [87–118]	102 [88–118]	101 [83–118]	p = .32
Systolic blood pressure	127 [110–144]	127 [111–145]	119 [102–143.5]	p < .0001
Temperature (degrees centigrade)	37.2 [36.6–38.2]	37.3 [36.6–38.3]	36.8 [36.4–37.6]	p < .0001
Altered mental state (new)	923 (35.6%)	748 (33.1%)	175 (52.7%)	p < .0001
Refractory hypotension	79 (3%)	51 (2.3%)	28 (8.4%)	p < .0001
Blood results
WCC (×10^9/l)	11.8 [8.2–16]	11.8 [8.2–15.9]	12.0 [8.7–16.3]	p = .33
Platelets (×10^9/l)	250 [188–328]	251 [189–328]	243 [177–325]	p = .09
International normalized ratio (INR)	1.2 [1.1–1.4]	1.2 [1.1–1.3]	1.2 [1.1–1.6]	p < .0001
Urea (mmol/l)	7.1 [4.8–11.4]	6.7 [4.6–10.1]	12.7 [7.5–20.7]	p < .0001
Creatinine (micromol/l)	90 [69–129]	88 [68–122]	115.5 [77–187]	p < .0001
Bilirubin (micromol/l)	10 [6–15]	9 [6–15]	10 [7–16]	p = .15
Albumin (g/l)	32 [28–36]	32 [28–36]	28 [23–32]	p < .0001
C-reactive protein (mg/l)	69 [23–152]	67 [22–146]	92 [33–188]	p = .0001
Lactate (mmol/l)	1.6[1.1–2.5]	1.5 [1.1–2.3]	2.2 [1.4–3.9]	p < .0001
Co-morbidities
Dementia	393 (15.2%)	309 (13.7%)	84 (25.3%)	p < .0001
Malignancy	395 (15.2%)	340 (15%)	55 (16.7%)	p = .46
-with metastases	240 (9.3%)	203 (9%)	37 (11.1%)	p = .22
NH residency or live-in carer or minimum  TDS care package	729 (28.1%)	568 (25.1%)	161 (48.5%)	p < .0001
Long-term oxygen therapy (LTOT)	48 (1.9%)	36 (1.6%)	12 (3.6%)	p = .001
Community or previous in-hospital DNAR orders	276 (10.6%)	197 (8.7%)	79 (23.8%)	p < .0001
Any of the above co-morbidities	1248 (48.1%)	1015 (44.9%)	233 (70.2%)	p < .0001
Source of infection/sepsis
Respiratory	1065 (41.1%)	872 (38.5%)	193 (58.1%)	p < .0001
Urogenital	440 (17%)	416 (18.4%)	24 (7.2%)	p < .0001
Abdomen	178 (6.7%)	161 (7.1%)	17 (5.1%)	p = .20
Soft tissue	156 (6.0%)	143 (6.3%)	13 (3.9%)	p = .11
Ear, nose and throat	36 (1.4%)	36 (1.6%)	0 (0%)	p = .01
Central nervous system	21 (0.8%)	19 (0.8%)	2 (0.6%)	p = 1.0
Bone	12 (0.5%)	10 (0.4%)	2 (0.6%)	p = .66
Device	12 (0.5%)	12 (0.5%)	0 (0%)	p = .38
Unknown	311 (12%)	268 (11.8%)	43 (12.9%)	p = .02
Not infection	363 (14%)	325 (14.4%)	38 (11.4%)	p = .18
Scores
REDS	2 [2–4]	2 [1–3]	4 [3–5]	p < .0001
SIRS	2 [2–3]	2 [2–3]	2 [2–3]	p = .74
NEWS2	5 [4–8]	5 [4–7]	8 [5–10]	p < .0001
CURB65	2 [1–3]	2 [1–3]	3 [2–4]	p < .0001
SOFA	2 [1–3]	1 [0–3]	3 [2–5]	p < .0001
MEDS	7 [5–9]	6 [3–8]	10 [7–12]	p < .0001
PIRO	8 [5–11]	8 [5–11]	11.5 [9–14]	p < .0001
Treatments
Time to antibiotics (minutes)	102 [55–171.5]	105 [56–172]	82 [50–158]	p = .002
Volume of intravenous fluid (ml)	1000 [500–1000]	1000 [500–1000]	1000 [500–1500]	p = .16
Admission to the intensive care unit	223 (8.6%)	174 (7.7%)	55 (16.6%)	p < .0001
Hospital length of stay (days)	5 [2–12]	5 [2–12]	5 [2–15]	p = .96
Arrival by ambulance	2160 (83.3%)	1844 (81.5%)	316 (95.2%)	p < .0001

IQR: inter-quartile range; NH: nursing home; DNAR: do not attempt resuscitation; TDS: ter die sumendum (three-times-a-day); REDS: risk-stratification of emergency department suspected sepsis; SIRS: systemic inflammatory response syndrome; NEWS2: national early warning score 2, CURB65: confusion urea respiratory rate blood pressure 65 (years); SOFA: sequential organ failure assessment; MEDS: mortality in emergency department sepsis; PIRO: patient infection response organ.

Figure 2. Receiver operator characteristic (ROC) curves for the REDS and comparator scores, for in-hospital mortality. REDS: risk-stratification of emergency department suspected sepsis; SIRS systemic inflammatory response syndrome; NEWS2: national early warning score 2; SOFA: sequential organ failure assessment; MEDS: mortality in emergency department sepsis; PIRO: patient infection response organ; CURB65: confusion urea respiratory rate blood pressure 65 (years); AUROC: area under receiver operator characteristic curve for the REDS score 0.73 (95% confidence interval [CI] 0.72–0.75); SIRS criteria 0.51 (95% CI 0.49–0.53); NEWS2 0.69 (95% CI 0.67–0.70), CURB65 0.71 (95% CI 0.69–0.72); SOFA score 0.74 (95% CI 0.72–0.76), MEDS score 0.75 (95% CI 0.73–0.75); PIRO 0.74 (95% CI 0.72–0.75).

Table 2. The AUROC curves, the cut-off points and the test characteristics of the REDS and comparator scores.

Score	H-L test	AUROC (95% CI)	Significance of difference in AUROC curve compared with REDS score	Cut-off score	Sensitivity percentage (95% CI)	Specificity percentage (95% pI)	PPV percentage (95% CI)	NPV percentage (95% CI)	Accuracy percentage (95% CI)
REDS	p = .61	0.73 (0.72–0.75)	Not applicable	≥3	80.1 (75.4–84.3)	55.3 (53.2–57.3)	20.8 (19.7–22)	95 (93.8–95.9)	58.4 (56.5–60.4)
				≥5	35.2 (30.1–40.6)	89.5 (88.2–90.8)	33.1 (29.0–37.4)	90.4 (89.7–91.1)	82.6 (81.1–84.0)
				≥7	15.1 (11.4–19.4)	98.1 (97.5–98.7)	54.4 (44.5–63.8)	88.7 (88.3–89.2)	87.5 (86.2–88.8)
SIRS	p = .02	0.51 (0.49–0.53)	p < .0001	≥2^a	76.5 (71.6–81.0)	24.7 (23.0–26.5)	13.0 (12.3–13.7)	87.8 (85.4–89.8)	31.3 (29.6–33.2)
NEWS2	p = .41	0.69 (0.67–0.70)	p = .005	≥8	51.5 (46.0–57.0)	77.8 (76.0–79.5)	25.4 (23.0–28.0)	91.6 (90.7–92.5)	74.4 (72.7–76.1)
				≥7^a	59.9 (54.4–65.3)	67.7 (65.7–69.6)	21.4 (19.7–23.2)	92.0 (91.0–92.9)	66.7 (64.8–68.5)
				≥5^a	81.0 (76.4–85.1)	40.9 (38.9–43.0)	16.8 (15.9–17.6)	93.6 (92.1–94.9)	46.0 (44.1–48.0)
				≥3^a	97.6 (95.3–98.9)	5.8 (4.9–6.8)	13.2 (13.0–13.4)	94.2 (89.0–97.1)	17.5 (16.1–19.1)
				3 in one vital sign^a	86.1 (82.0–89.9)	31.4 (29.5–33.4)	15.6 (14.9–16.3)	93.9 (92.1–95.3)	38.4 (36.5–40.3)
				≥5 or 3 in one vital sign^a	91.0 (87.4–93.8)	24.6 (22.9–26.5)	15.1 (14.5–15.6)	94.9 (92.9–96.3)	33.1 (31.3–35.0)
CURB65	p = .14	0.71 (0.69–0.72)	p = .05	≥3	63.3 (57.8–68.5)	68.1 (66.2–70.0)	22.6 (20.8–24.4)	92.7 (91.6–93.6)	67.5 (65.7–69.3)
SOFA	p = .22	0.74 (0.72–0.76)	p = .61	≥3	63.0 (57.5–68.2)	72.9 (71.0–74.7)	25.4 (23.4–27.5)	93.1 (92.1–93.9)	71.6 (69.8–73.3)
ΔSOFA	N/A	N/A	N/A	≥2^a	63.0 (57.5–68.2)	70.1 (68.1–72.0)	23.6 (21.8–25.5)	92.8 (91.8–93.7)	69.2 (67.3–70.9)
MEDS	p = .02	0.75 (0.73–0.77)	p = .32	≥10	56.9 (51.4–62.3)	80.8 (79.1–82.4)	30.3 (27.7–33.0)	30.3 (27.7–33.0)	77.7 (76.1–79.3)
				≥8^a	74.1 (69.0–78.7)	58.4 (56.4–60.5)	20.7 (19.5–22.1)	93.9 (92.7–94.9)	60.5 (58.5–62.3)
PIRO	p = .3	0.74 (0.72–0.75)	p = .85	≥10	66.3 (60.9–71.3)	66.8 (64.8–68.7)	22.7 (21.0–24.4)	93.1 (92.1–94.0)	66.7 (64.9–68.5)

^aOther cut-off points advocated in the literature.

H-L: Hosmer–Lemeshow; AUROC: area under receiver operator characteristic curve; CI: confidence interval; REDS: risk-stratification of emergency department suspected sepsis; SIRS: systemic inflammatory response syndrome; NEWS2: national early warning score 2; CURB65: confusion urea respiratory rate blood pressure 65 (years); SOFA: sequential organ failure assessment; MEDS: mortality in emergency department sepsis; PIRO: patient infection response organ.

Table 3. The AUROC curves, the cut-off points and the test characteristics of the REDS and comparator scores after exclusion of patients with missing variables.

Score	HL test	AUROC (95% CI)	Significance of difference in AUROC curve compared with REDS score	Cut-off Score	Sensitivity percentage (95% CI)	Specificity percentage (95% CI)	PPV percentage (95% CI)	NPV percentage (95% CI)	Accuracy percentage (95% CI)
REDS	p = .67	0.74 (0.72–0.76)	N/A	≥3	82.9 (77.9–87.2)	51.8 (49.4–54.2)	21.4 (20.2–22.6)	95.0 (93.6–-96.2)	56.1 (53.8–58.3)
				≥5	38.3 (32.5–44.4)	88.0 (86.3–89.5)	33.4 (29.2–38.0)	90.0 (89.1–90.9)	81.2 (79.4–82.9)
				≥7	16.7 (12.5–21.7)	97.7 (96.8–98.3)	52.9 (42.8–62.8)	88.1 (87.6–88.7)	86.6 (85.0–-88.1)
SIRS	p = .02	0.51 (0.48–0.52)	p < .0001	≥2^a	77.0 (71.5–81.9)	22.4 (20.4–24.4)	13.6 (12.8–14.4)	86.0 (82.9–88.6)	29.8 (27.8–31.9)
NEWS2	p = .12	0.68 (0.66–0.70)	p = .004	≥8	53.5 (47.4–59.6)	75.7 (73.6–77.8)	25.9 (23.3–28.6)	91.2 (90.1–92.2)	72.7 (70.7–74.7)
CURB65	p = .33	0.70 (0.68–0.72)	p = .049	≥3	64.7 (58.7–70.4)	67.0 (64.7–69.2)	23.6 (21.7–25.7)	92.3 (91.1–93.4)	66.7 (64.5–68.8)
SOFA	p = .69	0.74 (0.72–0.76)	p = .71	≥3	65.1 (59.0–70.8)	71.0 (68.8–73.1)	2.6.2 (24.0–28.4)	92.8 (91.6–93.8)	70.2 (68.1–72.2)
ΔSOFA	N/A	N/A	N/A	≥2^a	65.1 (59.0–70.8)	68.0 (65.7–70.2)	24.3 (22.3–26.4)	92.5 (91.3–93.6)	67.6 (65.5–69.6)
MEDS	p = .21	0.75 (0.73–0.76)	p = .57	≥10	57.6 (51.7–63.6)	79.9 (77.9–81.7)	31.1 (28.2–34.2)	92.3 (91.2–93.2)	76.8 (74.9–78.7)
PIRO	p = .33	0.73 (0.71–0.75)	p = .84	≥10	66.5 (60.6–72.2)	65.1 (62.8–67.4)	23.2 (21.3–25.1)	92.5 (91.2–93.6)	65.3 (63.2–67.4)

^aOther cut-off points advocated in the literature.

H-L: Hosmer–Lemeshow; AUROC: area under receiver operator characteristic curve; CI: confidence interval; REDS: risk-stratification of emergency department suspected sepsis; SIRS: systemic inflammatory response syndrome; NEWS2: national early warning score 2; CURB65: confusion urea respiratory rate blood pressure 65 (years); SOFA: sequential organ failure assessment; MEDS: mortality in emergency department sepsis; PIRO: patient infection response organ.

Figure 3. The percentage distribution of the REDS score through the study population and the percentage distribution of the REDS score through the deceased population. REDS score: risk-stratification of emergency department suspected sepsis score. Percentages have been rounded to the closest 1%.

Data availability statement

All data used is included in the manuscript.