Head-to-head comparison of 19 prediction models for short-term outcome in medical patients in the emergency department: a retrospective study

Abstract

Introduction

Prediction models for identifying emergency department (ED) patients at high risk of poor outcome are often not externally validated. We aimed to perform a head-to-head comparison of the discriminatory performance of several prediction models in a large cohort of ED patients.

Methods

In this retrospective study, we selected prediction models that aim to predict poor outcome and we included adult medical ED patients. Primary outcome was 31-day mortality, secondary outcomes were 1-day mortality, 7-day mortality, and a composite endpoint of 31-day mortality and admission to intensive care unit (ICU).

The discriminatory performance of the prediction models was assessed using an area under the receiver operating characteristic curve (AUC). Finally, the prediction models with the highest performance to predict 31-day mortality were selected to further examine calibration and appropriate clinical cut-off points.

Results

We included 19 prediction models and applied these to 2185 ED patients. Thirty-one-day mortality was 10.6% (231 patients), 1-day mortality was 1.4%, 7-day mortality was 4.4%, and 331 patients (15.1%) met the composite endpoint. The RISE UP and COPE score showed similar and very good discriminatory performance for 31-day mortality (AUC 0.86), 1-day mortality (AUC 0.87), 7-day mortality (AUC 0.86) and for the composite endpoint (AUC 0.81). Both scores were well calibrated. Almost no patients with RISE UP and COPE scores below 5% had an adverse outcome, while those with scores above 20% were at high risk of adverse outcome. Some of the other prediction models (i.e. APACHE II, NEWS, WPSS, MEWS, EWS and SOFA) showed significantly higher discriminatory performance for 1-day and 7-day mortality than for 31-day mortality.

Conclusions

Head-to-head validation of 19 prediction models in medical ED patients showed that the RISE UP and COPE score outperformed other models regarding 31-day mortality.

Keywords:

• Prediction
• prognosis
• mortality
• emergency department

Acknowledgments

We are grateful for the help from medical students A. van de Koolwijk, C. Hovens, M. Ronner, M. van Roosendaal, S. Cimino, J. Weijers and J. van Welij in collecting the data necessary to perform this study.

Author contributors

PD and PMS collected clinical data with the help of medical students. PD performed the statistical analysis. All authors interpreted data. PD and SL drafted the first version of the manuscript. NZ, WD, SM, JC and PMS critically reviewed the manuscript. All authors have read and approved the final version of the manuscript.

Disclosure statement

The RISE UP score was developed by members of our team. These team members (NZ and PMS) were not involved in data analysis for the current study.

Data availability statement

Additional data are available upon reasonable request.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.