Evidence for Use of Validated Sepsis Screening Tools in the Prehospital Population: A Scoping Review

Figures & data

Table 1. Line-by-line search terms used to retrieve articles assessing prehospital identification of sepsis using EWS.

Line #	Search term
Embase
1	'sepsis’/exp OR sepsis:ti,ab OR septic*:ti,ab OR 'septic shock’:ti,ab OR 'blood poisoning’ OR 'pyemia’ OR 'phyohemia’
2	'emergency services*' OR 'emergency medicine*' OR 'emergency personnel*' OR 'first responders*' OR 'paramedics*' OR 'emergency service’:ti,ab OR 'emt*':ti,ab OR 'ambulance’:ti,ab OR 'prehospital’:ti,ab OR 'pre-hospital’:ti,ab
3	'identif*':ti,ab OR 'detect*':ti,ab
4	'early warning score’:ti,ab OR 'ews’:ti,ab OR 'sofa’ OR 'qsofa’ OR 'modified early warning score’ OR 'modified shock index’ OR 'shock index’ OR 'robson’ OR 'prehospital early sepsis detection’ OR 'modified robson screening tool’ OR 'sirs’ OR 'prehospital severe sepsis’ OR 'sepsis score’ OR 'bas 90-30-90′
5	#1 AND #2 AND #3 AND #4
6	#6 AND [english]/lim AND [adult]/lim
CINAHL
1	Sepsis OR Blood poisoning OR pyemia OR pyemia OR pyohemia OR septic
2	emergency services OR emergency medicine OR emergency personnel OR first responders OR paramedics OR emergency service OR emt OR emts OR ambulance or paramedic or prehospital OR pre-hospital
3	Identif OR detect
4	Early Warning Score OR EWS OR SOFA OR qSOFA OR Modified Early Warning Score OR Modified shock index OR Shock Index OR Robson OR Prehospital Early Sepsis Detection OR modified robson screening tool OR SIRS OR prehospital severe sepsis OR sepsis score OR BAS 90-30-90
5	1 and 2 and 3 and 4
6	Limit 5 to English language, adults 19+ years, humans
OVID-Medline
1	"Sepsis".mp. or Blood poisoning.ti,ab. or pyemia.ti,ab. or pyemia.ti,ab. or pyohemia.ti,ab. or sepsis.ti,ab. or septic*.ti,ab. or septic shock.ti,ab.
2	(emergency services* or emergency medicine* or emergency personnel* or first responders* or paramedics* or emergency service*).mp. or emt.ti,ab. or emts.ti,ab. or ambulance.ti,ab. or paramedic*.ti,ab. or prehospital.ti,ab. or pre-hospital.ti,ab.
3	(Identif* or detect*).ti,ab.
4	(Early Warning Score or EWS).ti,ab. or SOFA.mp. or qSOFA.mp. or Modified Early Warning Score.mp. or Modified shock index.mp. or Shock Index.mp. or Robson.mp. or Prehospital Early Sepsis Detection.mp. or modified robson screening tool.mp. or SIRS.mp. or prehospital severe sepsis.mp. or sepsis score.mp. or BAS 90-30-90.mp.
5	1 and 2 and 3 and 4
6	Limit 5 to (english language and humans and "all adult (19 plus years)")
PubMed
1	"Sepsis"[Mesh] OR Blood poisoning[tiab] OR pyemia[tiab] OR pyemia[tiab] OR pyohemia[tiab] OR sepsis[tiab] OR septic*[tiab]
2	emergency services[mesh] OR emergency medicine[mesh] OR emergency personnel[mesh] OR first responders[mesh] OR paramedics[mesh] OR emergency service*[tiab] OR emt[tiab] OR emts[tiab] OR ambulance[tiab] or paramedic*[tiab] or prehospital[tiab] OR prehospital[tiab]
3	Identif*[tiab] OR detect*[tiab]
4	Early Warning Score[tiab] OR EWS[tiab] OR SOFA OR qSOFA OR Modified Early Warning Score OR Modified shock index OR Shock Index OR Robson OR Prehospital Early Sepsis Detection OR modified robson screening tool OR SIRS OR prehospital severe sepsis OR sepsis score OR BAS 90-30-90
5	1 and 2 and 3 and 4
6	Limit 5 to English language, adults 19+ years, humans

Figure 1. Flowchart for selection of articles assessing validated EWS use to identify prehospital sepsis.

Table 2. Study characteristics.

Year*	First author	Study type	Country	EWS(s) evaluated
2014	Wallgren	Retrospective cross-sectional	Sweden	BAS 90-30-90, Robson
2015	Bayer	Retrospective analysis	Germany	PRESEP
2015	Polito	Retrospective cohort	USA	PRESS
2016	Smyth	Systematic Review	USA	BAS 90-30-90, CIS, MEWS, PRESS, PRESEP, Robson
2016	Williams	Systematic Review	USA	BAS 90-30-90, MEWS, PRESEP, Robson
2017	Dorsett	Retrospective chart review	USA	Robson, qSOFA
2017	Tusgul	Retrospective	Switzerland	SIRS, qSOFA
2018	Barbara	Retrospective chart review	USA	qSOFA
2018-2	Jouffroy	Retrospective observational	France	MEWS, mRST, PRESEP, qSOFA
2018-5	Jouffroy	Retrospective observational	France	qSOFA
2018	Hunter	Retrospective cohort	USA	qSOFA
2019	Althunayyan	Retrospective cohort	Saudi Arabia	MSI, SI
2019	Shu	Retrospective observational	USA	qSOFA
2019	Silcock	Retrospective cohort	United Kingdom	NEWS, qSOFA
2019	Smyth	Retrospective cohort	United Kingdom	SEPSIS
2020	Jouffroy	Retrospective observational	France	SI
2020-2	Lane	Validation study	Canada	BAS 90-30-90, CIP, HEWS, MBIS, MEWS, NEWS, PHANTASI, PITSTOP, PreSAT, PRESEP, PRESS, PSP, qSOFA, Robson, SEPSIS, SIRS
2020-6	Lane	Retrospective cohort	Canada	SIRS, qSOFA
2021-6	Jouffroy	Prospective	France	PSPoT
2021-8	Jouffroy	Retrospective	France	SIGARC
2021	Usul	Retrospective cohort	Turkey	MEWS, qSOFA
2021	Wallgren	Prospective cohort	Sweden	NEWS2, RETTS
2022	Jouffroy	Retrospective Analysis	France	SI

BAS = Blood Pressure Andningsfrekvens (meaning “respiratory rate” in Swedish); CIP = Critical Illness Prediction Score; HEWS = Hamilton Early Warning Score; MBIS = Mecklenburg Bacterial Infection Scale; MEWS = Modified Early Warning Score; mRST = Modified Robson Screening Tool; MSI = Modified Shock Index; NEWS = National Early Warning Score; NEWS2 = National Early Warning Score 2; PHANTASI = Prehospital Antibiotics Against Sepsis; PITSTOP = Paramedic Initiated Treatment of Sepsis Targeting Out-of-Hospital Patients; PRESAT = Prehospital Sepsis Assessment Tool; PRESEP = Prehospital Early Sepsis Detection; PRESS = Prehospital Severe Sepsis; PSP = Prehospital Sepsis Project; PSPOT = Prehospital Shock Precautions on Triage; qSOFA = Quick Sequential Organ Failure Assessment; RETTS = Rapid Emergency Triage and Treatment System; SEPSIS = Screening to Enhance Prehospital Identification of Sepsis; SI = Shock Index; SIRS = Systemic Inflammatory Response Syndrome.

* Year formatted as YYYY-M referring to year and month of publication.

Table 3. Summary of primary conclusions.

First author*	Primary conclusions
Althunayyan	No cut off value of MSI or SI was found to have optimal accuracy for the prediction of sepsis related outcomes
Barbara	EMS patients with positive qSOFA screens were more likely to be septic upon disposition to the ED
Bayer	PRESEP could be a valuable and feasible tool and should be validated prospectively
Dorsett	qSOFA has very low sensitivity in the prehospital setting for early identification of sepsis. Early detection of disease requires more sensitivity than specificity, and qSOFA may not be the most appropriate tool
Hunter	EtCO^2 performed slightly better than qSOFA as a predictor of mortality and sepsis diagnosis
Jouffroy 2018-2	No system can stand alone to define sepsis and improve the early recognition of septic patients
Jouffroy 2018-5	qSOFA is not optimal for triaging patients in the prehospital setting and predicting ICU admission
Jouffroy 2020	SI is associated with day 28 mortality, SI > 0.9 appears to be a simple tool to help physicians to define increased risk of mortality
Jouffroy 2021-6	PSPoT is associated with in hospital mortality of septic shock patients cared for by EMS
Jouffroy 2021-8	SIGARC could potentially contribute to providing a more appropriate therapeutic pathway for patients which could result in decreased mortality in patients with septic shock
Jouffroy 2022	Prehospital hemodynamic delta SI among septic shock patients is associated with 28-day mortality
Lane 2020-6	qSOFA is the simplest of all tools. CIP, NEWS, and qSOFA identified patients with low probability of sepsis at low scores and high probability of sepsis at high scores
Lane 2020-2	qSOFA provides more information to help clinicians differentiate those patients with uncomplicated infection from those with life-threatening sepsis
Polito	PRESS was internally validated but needs to be externally validated
Shu	qSOFA has low evidence for prehospital use in the current triage system
Silcock	NEWS performed better than qSOFA in identifying risk of adverse outcomes in the prehospital setting
Smyth 2016	Sepsis recognition within EMS is highly variable, suggests EWS may help improve sepsis recognition as they demonstrate similar sensitivity with greater specificity
Smyth 2019	SEPSIS score is significantly associated with high risk of severe illness or death from sepsis, may assist clinicians to identify patients in need of treatment
Tusgul	qSOFA, SIRS, and SEPSIS gave suboptimal performance for early recognition of prehospital septic patients
Usul	Neither MEWS nor qSOFA had optimal accuracy for predicting prehospital sepsis, but qSOFA was more valuable in determining prognosis
Wallgren 2014	Robson showed superior sensitivity than BAS 90-30-90, implementation of EWS could lead to increased prehospital identification of sepsis
Wallgren 2021	NEWS2 may be better than RETTS in identifying prehospital sepsis
Williams	Predictive accuracy, clinical utility, and generalizability of many prehospital EWS remain uncertain

CIP = Critical Illness Prediction Score; EtCO²= End Tidal Carbon Dioxide; MSI = Modified Shock Index; NEWS = National Early Warning Score; PRESEP = Prehospital Early Sepsis Detection; PRESS = Prehospital Severe Sepsis; PSPOT = Prehospital Shock Precautions on Triage; qSOFA = Quick Sequential Organ Failure Assessment; SI = Shock Index.

* First author formatted as Author YYYY-M referring to year and month of publication.

Table 4. Summary of classification statistics outcomes.

EWS	First author*	Sensitivity	Specificity	PPV	NPV	AUC
BAS 90-30-90
	Bayer	0.62	0.83	0.51	0.89	–
	Lane 2020-2	0.57	0.79	0.43	0.87	–
	Smyth 2016	0.62	0.83	0.51	0.89	–
	Wallgren 2014	0.70	–	–	–	–
	Williams	0.70	–	–	–	–
CIS
	Smyth 2016	0.76	–	–	–	–
HEWS
	Lane 2020-2	0.85	0.41	0.28	0.91	–
MBIS
	Lane 2020-2	0.44	0.77	0.34	0.83	–
MEWS
	Bayer	0.74	0.75	0.45	0.91	–
	Jouffroy 2018-5	0.85	0.33	0.41	0.80	–
	Lane 2020-2	0.53	0.77	0.39	0.86	–
	Smyth 2016	0.74	0.75	0.45	0.91	–
	Usul	0.76	0.44	–	–	0.63
mRST
	Jouffroy 2018-5	1.00	0.16	0.39	1.00	–
MSI
	Althunayyan	0.90	0.76	0.98	0.39	0.91
mSIRS
	Silcock	–	–	–	–	0.74
NEWS2
	Wallgren 2021	0.74	0.61	0.60	0.74	0.67
PHANTASI
	Lane 2020-2	0.20	0.88	0.32	0.80	–
PITSTOP
	Lane 2020-2	0.02	1.00	0.65	0.79	–
PreSAT
	Lane 2020-2	0.49	0.71	0.32	0.83	–
PSPoT
	Jouffroy 2021-6	–	–	–	–	0.62
PRESEP
	Bayer	0.85	0.86	0.63	0.95	–
	Jouffroy 2018-5	0.92	0.29	0.41	0.88	–
	Lane 2020-2	0.49	0.76	0.36	0.84	–
	Smyth 2016	0.85	0.86	0.63	0.95	–
	Williams	0.85	0.86	0.66	0.95	–
PRESS
	Lane 2020-2	0.11	0.98	0.56	0.80	–
	Polito	0.86	0.47	0.19	0.96	0.82
	Smyth 2016	0.85	0.47	0.19	0.96	–
PSP
	Lane 2020-2	0.42	0.77	0.33	0.83	–
qSOFA
	Barbara	–	–	0.67	–	–
	Dorsett	0.16	0.97	–	–	–
	Hunter	0.68	0.40	–	–	–
	Jouffroy 2018-2	0.68	0.75	0.31	0.93	–
	Jouffroy 2018-5	0.62	0.16	0.29	0.44	–
	Jouffroy 2021-6	–	–	–	–	0.62
	Lane 2020-2	0.40	0.99	0.66	0.55	–
	Lane 2020-6		–	–	–	0.51
	Shu	0.42	0.93	0.32	0.95	–
	Silcock	–	–	–	–	0.68
	Tusgul	0.32	0.98	0.32	0.95	–
	Usul	0.84	0.37	–	–	0.85
RETTS
	Wallgren 2021	0.83	0.45	0.55	0.76	0.57
Robson
	Dorsett	0.47	0.80	–	–	–
	Bayer	0.95	0.43	0.32	0.97	–
	Lane 2020-2	0.75	0.54	0.31	0.83	–
	Smyth 2016	0.95	0.43	0.32	0.97	–
	Wallgren 2014	0.92	–	–	–	–
	Williams	0.95	0.43	0.32	0.90	–
SEPSIS
	Lane 2020-2	0.26	0.94	0.55	0.82	–
	Smyth 2019	0.37	0.98	0.49	0.98	–
	Tusgul	0.50	–	–	–	–
SI
	Althunayyan	0.85	0.81	0.98	0.32	0.90
SIGARC
	Jouffroy 2021-8	0.97	0.07	0.31	0.85	–
SIRS
	Jouffroy 2018-2	0.87	0.43	0.22	0.94	–
	Lane 2020-2	0.45	0.72	0.31	0.83	–
	Tusgul	0.69	–	–	–	–

‘–’ = Not reported; BAS = Blood Pressure Andningsfrekvens (meaning “respiratory rate” in Swedish); CIP = Critical Illness Prediction Score; HEWS = Hamilton Early Warning Score; MBIS = Mecklenburg Bacterial Infection Scale; MEWS = Modified Early Warning Score; mRST = Modified Robson Screening Tool; MSI = Modified Shock Index; mSIRS = Modified Systemic Inflammatory Response Syndrome; NEWS = National Early Warning Score; NEWS2 = National Early Warning Score 2; PHANTASI = Prehospital Antibiotics Against Sepsis; PITSTOP = Paramedic Initiated Treatment of Sepsis Targeting Out-of-Hospital Patients; PRESAT = Prehospital Sepsis Assessment Tool; PRESEP = Prehospital Early Sepsis Detection; PRESS = Prehospital Severe Sepsis; PSP = Prehospital Sepsis Project; PSPOT = Prehospital Shock Precautions on Triage; qSOFA = Quick Sequential Organ Failure Assessment; RETTS = Rapid Emergency Triage and Treatment System; SEPSIS = Screening to Enhance Prehospital Identification of Sepsis; SI = Shock Index; SIRS = Systemic Inflammatory Response Syndrome.

* First author formatted as Author YYYY-M referring to year and month of publication.

Table 5. Expert consultation questions and answers.

Question	Answer
What are your thoughts on the current Sepsis-3 definition and use of qSOFA in the prehospital setting?	“When it was derived from our hospital dataset, you know, it kind of looked okay. qSOFA has been validated in a couple of other EMS systems and it performed, or ED, sorry ED systems and I think at least in prehospital systems as well, and generally it performed poorly.” -E2**
What are your thoughts on using other common screening tools such as SIRS or NEWS or ETCO2?	“There’s nothing glorious about any of them.” -E2 “I don’t think anybody’s really figured out a great tool that accurately identifies these patients in a prehospital population.” -E2
Why do we continue to use qSOFA if most literature says otherwise?	“We don’t… It’s not used at all clinically, it’s just used as a research tool.” -E1* “qSOFA was not really designed to identify sepsis. It was designed to predict in hospital mortality… It has been used for different purposes and in different settings but that wasn’t even what it was designed to do.” -E2
Are there any efforts that you know of to update and/or create a tool that would be more beneficial?	“We had tried to a number of years ago. In fact, we, um, Chris Seymour and the department of Critical Care who essentially validated the Sepsis-3 definition, had, uh, and coined qSOFA, had, we had even gotten some tablets to create a screening tool on a tablet that the paramedics would use. And then we could essentially validate a score and stuff like that. And unfortunately, we got stuck on the fact that any, you know, usually you can find something that retrospectively is good enough that you can then validate it prospectively or try to implement it and see how well it works. We’ve really not found something that is all that helpful or all that, you know, whose performance characteristics are good enough to really try and implement in the field in a reliable way.” -E2 If you try to identify a tool that is just detect sepsis in general from prehospital data then I think that’s a fool’s errand. And I think it’s because there’s just, there’s too much noise…You’ll never get a signal to noise ratio with the tools that we have.” -E1
What do you think are the biggest gaps in prehospital sepsis identification and how can we address this?	“The pressing gap is can we identify patients in septic shock…And the product of that might be a gap analysis. So to say where have we not looked or where we should look.” -E1 “You know the reality of this is any issue is something that many people have thrown many different datasets at trying to figure out, trying to create a score, trying to validate a score, you know, unfortunately none of those have been terribly successful at finding it… I think that a true prehospital score or one that’s useful, is one that’s probably going to require something that we don’t do now.” -E2
What constitutes “good clinical judgment” and who is qualified to give that?	‘There’s no definition for that. The clinician you’re talking about or whichever you want to talk about, that’s the clinician’s judgment. in the out of hospital setting that’s going to be an EMT or paramedic. in the in-hospital setting that clinician’s judgment may be the nurse at triage, it could be the physician that’s evaluating the patient, it just depends on the level of evaluation that you’re talking about.” -E2 “What the clinician’s judgment is just actually just heuristic. It is pattern recognition of a certain set of a data points.” -E1 “Your clinical judgment is the most important thing, and the tool is supplemental.” -E1
Do you think better education for prehospital providers would help paramedics and EMT’s come up with a better clinical impression or clinical judgment, or do you think a more protocolized method for EMS is the way to go?	“Right now, most paramedics in the United States have the equivalent of an associate degree in training. Somewhere between 1200 and 2000 hours of training. It’s an obvious statement that if you made it a bachelor’s degree or made it a master’s degree like it is in Canada or like it is in Australia, they’re going to be better. They’re going to have more clinical experience; they’re going to have better clinical judgment.” -E1
What do you think could be a focus of study in the future? The machine learning algorithm based on variability and vital sign data, not just the static measurements, certainly is a focus of your research with the DoD. Do you think that holds value in the prehospital setting for identifying septic shock or is it still kind of too broad a measurement and sepsis and is so multifactorial in a way and it can affect so differently based on their innate and underlying mechanics. Is that also just kind of shooting in the dark?	“So, the physiologic data is a biologic data. And in trauma, it’s super accurate… I think sepsis would be a harder nut to crack, but certainly using those granular vital sign parameters is probably the best, one of the better, tools that we have available to use now. Because even though we don’t use them in current clinical practice we can still extract them.” -E1
Do you think [the previous answer] is viable from a prehospital perspective especially given how variable prehospital services can be in terms of their own resources depending on where you are in the country?	“There are certainly services that still won’t be able to generate the data, but um, the other problem with the machine learning data is unless you have it linked to outcomes and unless you know what interventions… I think the struggle isn’t so getting the machine data, the struggle is getting the clinical annotation and linking it to outcomes.” -E1

*E1 = Expert 1, **E2 = Expert 2.