The impact of introducing the early warning scoring system and protocol on clinical outcomes in tertiary referral university hospital

Abstract

Purpose

The aim of this study was to evaluate the impact of a hospital protocol in response to patient deterioration in general wards, stratified using the national early warning score (NEWS), on primary patient outcomes of in-hospital mortality and percentage of patients transferred to the intensive care unit (ICU).

Patients and methods

We conducted a prospective observational cohort study among adult medical patients admitted to a university hospital in Bangkok. A 4-month pre-protocol period (November 2015 to February 2016) was assigned to a control group and a protocol period (March 2016 to June 2016) was allocated to a protocol group. On admission, vital signs (respiratory rate, pulse rate, systolic blood pressure, and temperature), oxygen saturation, presence of oxygen supplementation, and neurological status were used to calculate NEWS. Patients were categorized as low, moderate, or high risk based on the NEWS. During protocol period, when patients’ conditions are critical and they are at imminent risk, the NEWS detects the event and triggers a systematic response. The response enables closed monitoring and early treatment by expert physicians to rapidly stabilize and triage the patient to a location where services meet the patient’s needs. Primary outcomes were compared between the pre-protocol and protocol groups using historical controls for the intervention, which is the availability of NEWS to staff and an associated escalation pathway.

Results

A total of 1,145 patients were included in the analysis: 564 patients in the pre-protocol group and 581 in the protocol group. The mean NEWS of patients at admission was higher in the protocol group than in the pre-protocol group (2.4±2.4 vs 1.77±2.158; P<0.001). There was no significant difference for in-hospital mortality and percentage of patients transferred to ICU between the groups. Among 95 (8.3%) patients at moderate risk, in-hospital mortality and ICU transfer percentage were lower in the protocol group than in the pre-protocol group (2.9 vs 15.4%; P=0.026; RR 0.188, 95% CI 0.037%–0.968% and 8.7 vs 26.9%; P=0.021; RR 0.322, 95% CI 0.12–0.87, respectively).

Conclusion

Implementing the NEWS with the hospital protocol did not change the overall patient’s outcomes.

Keywords:

• health service administration
• mortality
• patient safety
• quality improvement
• rapid response system
• early warning score

Introduction

Acute deterioration and serious adverse events (SAEs) such as sudden cardiac arrest, intensive care unit (ICU) admission, and death in patients on general medical wards are preceded by subtle changes in basic physiological parameters such as vital signs, oxygen saturation, and mental status. Recent studies have reported that evidence of clinical deterioration detectable by the worsening of those parameters is often present several hours before the occurrence of SAEs.^1–3 Inappropriate management factors that lead to patient deterioration are poor clinical monitoring, inadequate interpretation of changes in physiological parameters, failure to stabilize patients, and delay in ICU transfer.^2,4

Tools for the early recognition of patients at risk for deterioration are urgently needed to improve patient’s outcomes.⁴ The national early warning score (NEWS), which uses physiological parameters for scoring, is beneficial for predicting patient deterioration.⁵ The NEWS comprises a simple method to calculate scores based on physiological parameters such as respiratory rate, pulse rate, systolic blood pressure, oxygen saturation, the presence of oxygen supplementation, and neurological status.⁵ Nevertheless, the positive effect of the NEWS on patient safety outcomes remains to be established. To date, few studies are available describing specific interventions according to different values of the NEWS for improving patient’s outcomes.⁶

The concept of patient safety in the general ward setting comprises two key elements: early detection of and rapid response to deterioration.⁷ We hypothesized that with the detection of patient deterioration using the NEWS, early intervention and prompt management by a medical team might improve patient’s outcomes. Thus, we aimed to evaluate the impact of a hospital protocol in response to patient deterioration, stratified by the NEWS, on patient’s outcomes of in-hospital mortality and percentage of ICU transfers.

Patients and methods

The research ethics board committee of Ramathibodi Hospital reviewed and approved this study and waived the need for informed consent owing to the purely observational character and qualitative research (ID 11-58-11). We conducted a prospective observational cohort study in adult patients admitted to a general medical ward of Ramathibodi Hospital in Bangkok. A 4-month pre-protocol period (from November 2015 to February 2016) was assigned as the control group. The protocol period (the protocol group) was from March 2016 to the end of June 2016.

Preparation of the hospital protocol for response to patient deterioration stratified by NEWS was started in October 2016. At that time, head of the Department of Medicine (Chomsri Kositchaiwat) supported the project as an urgent priority for patient safety. The clinical lead team planned the protocol implementation and conducted an evidence-based review to identify the most appropriate NEWS and activation criteria. The team also developed data collection forms.

Patients

We included all adult patients aged over 18 years who were admitted or transferred to the general medical ward from either the emergency department or ICU. Diagnoses were grouped as per the ICD-10 according to primary diagnosis relating the leading disease during hospital stay. Patients who were admitted for palliative care only were excluded from this study.

Pre-protocol period

During the pre-protocol period, vital signs (respiratory rate, pulse rate, systolic blood pressure, and temperature), oxygen saturation, the presence of oxygen supplementation, and neurological status were recorded at admission and every 4 h on a paper chart by the nursing staff. The attending physicians were unaware of the NEWS of patients. When patients exhibited signs of deterioration, the ward nurse judged the extent of deterioration and contacted an attending physician. The stepwise approach followed a traditional hierarchy of first-year resident, senior resident, medical fellow, and then the attending physician, with several cycles of assessment and management. The ICU team passively received patients when the attending physician transferred the patient to the ICU, according to the ICU admission criteria. The ICU admission criteria comprise hemodynamic instability, cardiac arrest, status epilepticus, or acute respiratory failure. Independent physicians collected the data without interfering with clinical decisions.

Hospital protocol implementation

Figure 1 shows the protocol structure, which was adapted from National Early Warning Score: Standardising the Assessment of Acute Illness Severity in the NHS, modified by the Royal College of Physicians in 2012.^8,9 When patients’ conditions become critical and they are at imminent risk, the NEWS detects the event and triggers a systematic response. The response provides closed monitoring and early treatment by expert physicians to rapidly stabilize and triage the patient to a location where services meet the patient’s needs.

Figure 1 The hospital protocol structure for response to the deterioration of patients.

Abbreviations: ICU, intensive care unit; NEWS, national early warning score.

On admission, vital signs were used to calculate the NEWS. Table 1 shows hospital scoring of the NEWS. Patients were classified according to the NEWS at admission as being at low risk (score ≤4), moderate risk (scores 5–6), and high risk (score ≥7). Either essential management or ICU transfer was provided to patients based on the hospital protocol (Table 2). The protocol was followed by primary care nurses, medical residents, medical fellows, and attending physicians participating in patient care. In cases where the patient’s condition continued to decline after the initial management, and for those who met the criteria for ICU admission, the patient was transferred to the ICU. The primary physician notified the critical care or pulmonary fellow when ICU beds were unavailable; the ICU team would organize a bed to rapidly transfer the patient. However, if the patient had a do not resuscitate order, the protocol would be terminated and the patient would not be transferred to ICU or undergo cardiopulmonary resuscitation. Table 2 demonstrates the protocol for clinical response and intervention for a NEWS trigger. The primary endpoints were compared between the pre-protocol and protocol periods.

Table 1 The National early warning scoring (NEWS)

Physiological parameters	3	2	1	0	1	2	3
Respiration rate (breaths per min)	≤8		9–11	12–20		21–24	≥25
SpO2 (%)	≤91	92–93	94–95	≥96
Any supplemental oxygen?		Yes		No
Temperature (°C)	≤35.0		35.1–36.0	36.1–38.0	38.1–39.0	≥39.1
Systolic BP (mmHg)	≤90	91–100	101–110	111–219			≥220
Heart or pulse rate (beats per minute)	≤40		41–50	51–90	91–110	111–130	≥131
Level of consciousness using the AVPU system				A			V, P or U

Notes: SpO₂, estimate of arterial oxygen saturation. BP, blood pressure. Reprinted from Resuscitation, 84(4), Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI, The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death, 465–470, 2013, with permission from Elsevier.⁵

Abbreviations: A, alert; P, pain; U, unresponsive; V, verbal.

Table 2 Protocol for clinical response and intervention for NEWS trigger

Levels of severity	Scores 0–1	Scores 2–4	Scores 5–6, alert sign 1 time (regardless of other scores)	Scores ≥7 and 5–6 or alert sign 2 times
Nurse response	Routine	Routine	Move the bed near the nurse counter for close monitoring	Transfer to ICU
Doctor response	Routine	Routine	First-year resident within 30 min	Both first-year and senior residents within 15 min
Frequency of monitoring	Every 8 h	Every 4 h	Every 2 h	Continuous monitoring
Notify attending staff	Routine (daily)	Routine (daily)	Within 8 h	Immediate

Abbreviations: ICU, intensive care unit; NEWS, national early warning score.

Data collection and outcome measures

Data were obtained from chart reviews and using NEWS data collection forms by investigators who were not involved in patient management. The primary outcome was in-hospital mortality and percentage of patients transferred to ICU. The secondary outcome was CPR rate in the general ward and the time to ICU transfer. The period from when a patient first scored a predefined high-risk value of NEWS to ICU transfer was calculated for each patient who was transferred to the ICU.

Statistical analyses

Statistical analyses were performed using IBM SPSS Version 22 (IBM Corporation, Armonk, NY, USA). Continuous variables were reported as mean ± SD and tested using a two-tailed t-test for independent samples. For variables not distributed normally, the median with IQR was reported. A nonparametric test was used to compare non-normally distributed variables. Categorical variables were compared with the chi-squared and Fisher’s exact tests. Univariate logistic regression was used to examine the association of NEWS with in-hospital mortality. A P-value of <0.05 was considered statistically significant.

Results

A total of 1,161 patients were included in the study. Sixteen patients were excluded from analysis because they were admitted for palliative care only. Therefore, 1,145 patients were included in the analysis: 564 patients in the pre-protocol group and 581 patients in the protocol group. The mean patient age was 65±16.12 years, and 53.3% of patients were male. There were significant associations between in-hospital mortality and percentage of ICU transfer with risk stratification by NEWS (P<0.001 in both groups; Figure 2). At univariate logistic regression analysis, NEWS was associated with in-hospital mortality (OR 1.43; 95% CI 1.27–1.61).

Figure 2 The association between in-hospital mortality and percentage of ICU transfer with risk.

Note: Stratification by NEWS.
Abbreviations: ICU, intensive care unit; NEWS, national early warning score.

The NEWSs of patients on admission in the pre-protocol group were lower than those in the protocol group (1.77±2.16 vs 2.4±2.4; P<0.001). Regarding ICD-10 diagnosis, neoplasm was more common in the pre-protocol group, whereas cardiovascular and respiratory diseases were more common in the protocol group. The percentage of patients with moderate and high risk was greater in the protocol group than in the pre-protocol group (11.9 vs 4.6% in the moderate-risk group and 7.2 vs 5.1% in the high-risk group, respectively; P<0.001; Table 3). Regarding primary outcomes, there was no significant difference in the overall in-hospital mortality and percentage of patients transferred to ICU between the groups (2.0 vs 2.6%, P=0.47, and 3.9 vs 5.7%, P=0.16, respectively). The percentage of ICU transfer was higher among high-risk patients in the protocol period compared with those in the pre-protocol period, but this was not statistically significant (17 [40.5%] vs 6 [20.7%], respectively, P=0.08). Only three patients received cardiopulmonary resuscitation: one patient in the pre-protocol group and two patients in the protocol group.

Table 3 Baseline characteristics and bedside observations for NEWS on admission

Parameters	Pre-protocol group (N=564)	Protocol group (N=581)	P-value
Age (years), mean (SD)	55.64 (17.54)	59.61 (16.86)	0.64
Male sex, n (%)	317 (56.2)	293 (50.4)	0.05
ICD diagnosis group, n (%)			0.00
Infectious diseases	28 (40)	42 (60)	0.11
Neoplasm	368 (58.2)	264 (42.8)	0.00
Cardiovascular diseases	33 (28.9)	81 (71.1)	0.00
Respiratory diseases	31 (32)	66 (38)	0.00
Gastrointestinal diseases	36 (51.4)	34 (48.6)	0.71
Neurologic diseases	13 (56.5)	10 (43.5)	0.48
Urologic diseases	16 (39)	25 (61)	0.18
Bone and connective tissue	7 (43.8)	9 (56.3)	0.66
Hematological disease	12 (63.2)	7 (36.8)	0.22
Endocrine disease	5 (33.3)	10 (66.7)	0.21
Skin and subcutaneous tissue disease	5 (55.6)	4 (44.4)	0.70
Other diagnoses	10 (25.6)	29 (74.4)	0.00
NEWS at admission, mean (SD)	1.77 (2.16)	2.4 (2.4)	0.00
Patients stratified by NEWS, n (%)
Low risk	509 (90.2)	470 (80.9)	0.00
Moderate risk	26 (4.6)	69 (11.9)
High risk	29 (5.1)	42 (7.2)

Abbreviation: NEWS, national early warning score.

Interestingly, among 95 (8.3%) patients at moderate risk, the in-hospital mortality and percentage of ICU transfer in the protocol period were lower than in the pre-protocol period (2.9 vs 15.4%; P=0.03; RR 0.19, 95% CI 0.04–0.97, and 8.7 vs 26.9%; P=0.02; RR 0.32, 95% CI 0.12–0.87, respectively; Table 4). The median time to ICU transfer in the pre-protocol group was longer than that in the protocol group but was not statistically significant (23 h [2.87, 43.75] vs 8 h [2.5, 13]; P=0.2).

Table 4 Outcomes of the national early warning scoring system and protocol

	Pre-protocol group	Protocol group	P-value
Rate of intensive care unit transfer, n (%)
Total	22 (3.9)	33 (5.7)	0.16
Low risk	9 (1.8)	10 (2.1)	0.68
Moderate risk	7 (26.9)	6 (8.7)	0.02
High risk	6 (20.7)	17 (40.5)	0.08
In-hospital mortality, n (%)
Total	11 (2)	15 (2.6)	0.47
Low risk	4 (0.8)	7 (1.5)	0.23
Moderate risk	4 (15.4)	2 (2.9)	0.03
High risk	3 (10.3)	6 (14.3)	0.62

Discussion

The main findings of our study can be summarized as follows. First, we found that higher NEWS on admission was associated with increased in-hospital mortality and ICU transfer rates, which is in line with recent studies recommending NEWS as a potential triage tool for medical patients.^5,8 Second, implementing the NEWS with the hospital protocol did not change overall patient outcomes.

Several assessment tools based on vital signs have been developed for the early screening of patients at risk of clinical deterioration. A recent meta-analysis demonstrated that introducing an early warning score chart did not significantly improve mortality or SAEs;^4,10 however, those results may be due to the use of modified forms of the early warning score together with different thresholds and methodology, making it difficult to compare between studies.

The NEWS at admission is associated with death and ICU transfer within 2 days after admission. The NEWS also performs better than other scoring systems for assessing medical risk in the hospital setting.⁵ Nevertheless, its impact on patient outcomes has not been investigated. We hypothesized that the NEWS is not only a simple and effective screening tool, but when coupled with a rapid response hospital protocol, NEWS could also help to decrease patient SAEs, especially in situations of hospital overcrowding.¹¹

Delayed response has been identified as an important factor in higher mortality and ICU transfer among hospitalized patients.¹² The main aim for applying a triaging system to the patients is to detect earlier patients at the risk of deterioration and provide appropriate management, which improves those patients’ conditions or ensures they receive care at an appropriate site of care.

Our present study, selecting a real-world problem, compared between the systematic and nonsystematic processes (historical control) to triage patients in a medical general ward. Nevertheless, implementing the NEWS with the hospital protocol did not change overall patient outcomes. The higher percentage of moderate- and high-risk patients enrolled in the protocol period could be why the overall in-hospital mortality did not decrease, compared with the pre-protocol, despite a reduction in mortality in the moderate-risk group. Furthermore, according to the ICD-10 diagnosis, neoplasm was more common in the pre-protocol group, while cardiovascular and respiratory diseases were more common in the protocol group. These findings suggested that the patients’ baseline characteristics were different between groups.

In high-risk patients, we predicted the increase in the patient ICU transfer rate in the protocol group because when patients are defined as high risk, the protocol mandates early ICU admission. We did not see decreased mortality in the high-risk group or in patients overall after implementation of the protocol. This finding probably caused the similar overall in-hospital mortality between both periods. Furthermore, higher percentage of ICU transfer in high-risk patients observed during protocol period led to greater imbalance between ICU bed demand and supply than control period.

The unchanged mortality outcomes in high-risk patients during the protocol period, despite reduced deaths in moderate-risk patients, contradict common sense. Even though the proportion of patients with high-risk NEWS requiring ICU transfer was greater in the protocol group than in the pre-protocol group, a systematic process should facilitate appropriate site of care to those, unless presence of ICU bed shortage. However, the median ICU transfer duration was lower in the protocol group than in the pre-protocol group, even no statistical significance, which should lead to better outcome. The main reason to explain this finding is that the high-risk patients during protocol period actually had more severe condition, but the NEWS was not sensitive enough to differentiate this, together with the greater imbalance of ICU bed demand and supply. Consequently, not only the systematic triaging protocol but also the confluence of patient transfers is needed for success.

According to NEWS, the hospital protocol improved patient outcomes, namely, ICU transfer and in-hospital mortality, in patients with moderate risk. In moderate-risk patients, the protocol helped nursing staff identify patients with problems so that they could rapidly provide specific treatment to these patients, which led to improvements in this group.

Limitations

There were three main limitations in this study. First, we did not control enrolled patients in both groups to have the same overall disease severity and prognosis. Consequently, the percentages of moderate- and high-risk patients were higher in the protocol group. Moreover, the ICD-10 diagnoses differed between groups. The fact that the pre-protocol group and the protocol group was observed over different time periods could concern one that perhaps seasonality of some symptoms that could require ICU transfer might have affected the findings. This work was conducted in a real-world setting in which it was not possible to recruit patient in a randomized fashion. A single-center randomized control trial should be conducted in which both the control and protocol groups are selected from patients during the same time period and to which patients are randomly assigned. The cluster multicenter randomized control trial may be another solution. Second, the availability of ICU beds, which we could not control, was different in each period, which could influence outcomes, particularly in the high-risk group. We did not collect data regarding ICU bed availability during the study period. Third, we did not analyze the protocol adherence and compliance of the medical staff, which may have also affected the outcome.

Conclusion

Higher NEWS on admission was associated with increased in-hospital mortality and ICU transfer rates. Implementation of the protocol did not change patient’s outcomes.

Author contributions

Yuda Sutherasan, MD, and Pongdhep Theerawit, MD, contributed to the development of the protocol, planning, data collection, and analyses, and manuscript preparation. Alongkot Suporn, MD, contributed to the study design, planning, data collection, and analyses. Arkom Nongnuch, MD, Pariya Phanachet, MD, and Chomsri Kositchaiwat, MD, contributed to the study design and planning. All authors contributed toward data analysis, drafting and critically revising the paper and agree to be accountable for all aspects of the work. All authors read and approved the final manuscript.

Acknowledgments

We would like to acknowledge the invaluable help and contributions of the medical and nursing staff of the Department of Medicine, Ramathibodi Hospital, Mahidol University. We also thank Dr Taya Kitiyakara for editing the English text of a proof of this manuscript and Analisa Avila and Andrew Stow, ELS, of Edanz Group (www.edanzediting.com/ac), for editing the English text of a draft of this manuscript. This abstract of this paper was presented at the 30th ESICM Annual Congress, September 23–27, 2017 as a poster presentation with interim findings.

Disclosure

The authors report no conflicts of interest in this work.