https://orcid.org/0000-0003-3060-4203
To the Editor,
A recent multicentre retrospective cohort study by Martın-Rodrıgueza et al. [Citation1] compared the quick-COVID-19-Severity-Index (qCSI) and National-Early-Warning-Score (NEWS) effectiveness in COVID-19 patients transferred by ambulance to the emergency department (ED), showing that NEWS provides a better prognostic capacity for both early- and long-term mortality [Citation1].
During the pandemic many pre-existing assessment tools, pneumonia or sepsis-specific systems, early warning scores, and new COVID-19 models were designed to optimize clinical management. However, none can detect all patients at high risk of poor outcomes [Citation2–4].
To highlight the scoring systems’ role, we briefly reported the performances of qCSI, NEWS, Systemic-Inflammatory-Response-Syndrome (SIRS), Rapid-Physiology-Score (RAPS), Rapid-Emergency-Medicine-Score (REMS), quick-Sequential Organ-Failure-Assessment (qSOFA), Acute-Physiology-and-Chronic-Health-Evaluation-II (APACHE II), Confusion-Urea nitrogen-Respiratory rate-Blood pressure-age ≥ 65 (CURB-65) and the Pneumonia-Severity-Index (PSI) in 106 consecutive patients with COVID-19 acute respiratory failure admitted to an Acute-Medical-Unit (March/April 2020) in a tertiary care hospital in Lombardy, Italy. At admission 75.5% required non-invasive ventilation, 7.5% were transferred to the intensive care unit (ICU) for mechanical ventilation, and the in-hospital mortality rate was 28.3%.
The more accurate scores for in-hospital mortality were those that included physiological data and age: CURB-65 (area under the receiver operator characteristic, AUROC, 0.73, 95% CI 0.63–0.82), REMS (AUROC 0.77, 95% CI 0.67–0.86), APACHE II (AUROC 0.80, 95%CI 0.71–0.88) and PSI (AUROC 0.83, 95%CI 0.75–0.91), where the last two scores also considered comorbidities. Notably, these four scoring systems for mortality performed the worst for ICU transfers. Except for SIRS (AUROC 0.72, 95%CI 0.56–0.86) and NEWS (AUROC 0.73, 95%CI 0.62–0.83), all the scores performed poorly in predicting ICU transfers ().
Table 1. Area under the receiver operator characteristic.
None of the scores reached acceptable AUROC (>0.7) for both (in-hospital mortality and ICU transfer) outcomes.
Interpretation of COVID-19 studies should consider selection biases, confounders, clinical characteristics, clinical settings, stress on health-care systems and COVID-19 prevalence over time. ICU admission criteria (which may change due to capacity and evolving medical knowledge), non-standard definition of respiratory decompensation, clinical settings (EDs, general-inpatients-wards, Intermediate-Care-Units, or ICUs) and lack of validation (internal/external as for qCSI/NEWS [Citation1]) could explain contradictory results. Moreover, the selection of outcome timeframe is crucial: NEWS, as a track-and-trigger system, should be considered for early mortality [Citation1]. Its performance is not affected by age, sex, or comorbidities up to two days, but all confounding factors relate to longer mortality time [Citation1]. Expanding outcomes would improve the prognostic power of any score, and the change in cut-offs would increase the usability as a screening tool at the cost of specificity, while considering that relatively low event rate from small populations risks incompatibility. All standard indices of accuracy should be considered to limit risk of under- or over-triage, especially in times of limited health-care resources.
A comprehensive acute patient assessment should include evaluation of clinical severity and clinical complexity, optimizing the applicability of the derived models in real life. Scores are useful to share objective data in a common language among practitioners in different settings. However, it is necessary to change the “one-size-fits-all” approach and consider that numbers can support but not replace clinical judgement, emphasizing the importance of skilled evaluation in clinical practice.
Author contributions
S.A: conception and design, acquisition and analysis of the data, drafting and writing the letter;
F.S: interpretation of the data and critical revision of the letter for important intellectual content;
C.C: conception and design, interpretation of the data and critical revision of the letter for important intellectual content;
All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All authors approve the final version to be published.
Ethical approval
The study complied with the Declaration of Helsinki and was approved by the local Institutional Ethics Committee (ATS Val Padana). Accessed data were aggregated and anonymised.
Acknowledgement
This work had no financial support and did not receive any specific grants from funding agencies in the public, commercial, or not-for-profit sectors.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
References
- Martín-Rodríguez F, Sanz-García A, Ortega GJ, et al. One-on-one comparison between qCSI and NEWS scores for mortality risk assessment in patients with COVID-19. Ann Med. 2022;54(1):646–654.
- Haimovich A, Ravindra NG, Stoytchev S, et al. Development and validation of the quick COVID-19 severity index (qCSI): a prognostic tool for early clinical decompensation. Ann Em Med. 2020;76(4):442–453.
- Coughlan D, Rahman S, Honeyford K, et al. Developing useful early warning and prognostic scores for COVID-19. Postgrad Med J. 2021;97(1150):477–480.
- Gupta RK, Marks M, Samuels THA, et al. Systematic evaluation and external validation of 22 prognostic models among hospitalized adults with COVID-19: an observational cohort study. Eur Respir J. 2020;56(6):2003498.