https://orcid.org/0000-0001-9556-0665
Abstract
Introduction
The evaluation of acute poisoning is challenging due to varied toxic substances and clinical presentations. The new-Poisoning Mortality Score was recently developed to assess patients with acute poisoning and showed good performance in predicting in-hospital mortality. The objective of this study is to externally validate the performance of the new-Poisoning Mortality Score and to compare it with the Modified Early Warning Score.
Methods
This retrospective analysis used data from the 2019–2020 Injury Surveillance Cohort, established by the Korea Center for Disease Control and Prevention, to perform external validation of the new-Poisoning Mortality Score. The statistical performances of the new-Poisoning Mortality and Modified Early Warning Scores were assessed and compared in terms of discrimination and calibration. Discrimination analysis involved metrics such as sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve. For calibration analysis, the Hosmer-Lemeshow goodness-of-fit test was utilized and calibration curves for each score were generated to elucidate the relationship between observed and predicted mortalities.
Results
This study analysed 16,570 patients with acute poisoning. Significant differences were observed between survivors and those who died in-hospital, including age, sex, and vital signs. The new-Poisoning Mortality Score showed better performance over the Modified Early Warning Score in predicting in-hospital mortality, in terms of the area under the receiver operating characteristic curve (0.947 versus 0.800), sensitivity (0.863 versus 0.667), specificity (0.912 versus 0.817), and accuracy (0.911 versus 0.814). When evaluated through calibration curves, the new-Poisoning Mortality Score showed better concordance between predicted and observed mortalities. In subgroup analyses, the score system consistently showed strong performance, excelling particularly in substances with high mortality indices and remaining superior in all substances as a group.
Conclusions
Our study has helped to validate the new-Poisoning Mortality Score as an effective tool for predicting in-hospital mortality in patients with acute poisoning in the emergency department. The score system demonstrated superior performance over the Modified Early Warning Score in various metrics. Our findings suggest that the new-Poisoning Mortality Score can contribute to the enhancement of clinical decision-making and patient management.
Introduction
Patients with acute poisoning can be difficult to evaluate due to the wide range of toxic substances and clinical presentations. Risk assessment for these patients is influenced by multiple factors, including patient-specific characteristics such as age and comorbid conditions, as well as various attributes of the poison exposures including the type, route, dosage, timing, and intentionality [Citation1,Citation2]. Predicting the prognosis of patients with acute poisoning is further complicated by the lack of a standardized method for assessing the severity of poisoning. Although the Poisoning Severity Score (PSS) serves as a disease-specific scoring system, its application in toxicology remains limited and, when utilized, has often been either misapplied or modified from its original form [Citation3]. The scoring system, due to reliance on several subjective criteria and its time-consuming nature, may exhibit limited usefulness for specific types of poisonings, thereby restricting its clinical utility [Citation3].
To address these challenges, a novel scoring system, termed the new-Poisoning Mortality Score, has been recently developed to predict mortality among patients with acute poisoning in the emergency department [Citation4]. The new-Poisoning Mortality Score is established based on objective indicators that can be acquired even at the prehospital stage, such as demographic factors, poisoning-related variables, vital signs, and mental status. This facilitates a swift and reliable assessment of mortality risk, enabling early application in clinical settings. The new-Poisoning Mortality Score has demonstrated good performance in predicting in-hospital mortality in patients with acute poisoning, regardless of the toxic substances, route of exposure, age, or sex [Citation4].
Nevertheless, additional validation of the new-Poisoning Mortality Score is essential to establish its clinical utility and generalizability. The objective of this study is to validate the performance of the new-Poisoning Mortality Score in predicting in-hospital mortality among a large cohort of patients presenting with acute poisoning in the emergency department. Moreover, this study aims to compare the performance between the new-Poisoning Mortality Score and the Modified Early Warning Score, an established clinical scoring system designed for rapid assessments in emergency settings. Several studies supported the use of the Modified Early Warning Score as an effective bedside tool for the timely identification of critically ill patients who are at risk of rapid deterioration [Citation5–7]. Comparing the new-Poisoning Mortality Score and the Modified Early Warning will yield further insights into the clinical utility and applicability of these scoring systems for predicting mortality in patients with acute poisoning in the emergency department, thereby informing the development of more effective clinical decision-making tools for this patient population.
Methods
Study design and selection of study patients
This retrospective analysis utilized data collected between 2019 and 2020 from the Injury Surveillance Cohort, a prospective registry established by the Korea Center for Disease Control and Prevention. In this registry, patients are classified based on their mechanism of injury. Specifically, for those presenting with poisoning, additional in-depth investigations were conducted to gather poison-related factors. This detailed information enabled us to accurately identify and select patients with acute poisoning. Baseline characteristics, including demographics, poisoning-related factors, and initial vital signs obtained in the emergency department, were gathered for the selected population. Refer to “Data analysis and the new-Poisoning Mortality Score” for further details. Outcome data, like mortality in the emergency department or post-hospitalization, were also captured in the registry. Patients who were transferred out after initial emergency department management were excluded as their outcomes were uncertain. The reasons for these transfers varied, including the unavailability of intensive care units, transfer to follow-up or local hospitals, and the need for specialized treatments like hyperbaric oxygen therapy. Patients with incomplete data on poisoning-related factors or initial vital signs, and patients who were dead on arrival were also excluded.
Data analysis and the new-Poisoning Mortality Score
The new-Poisoning Mortality Score utilized various factors for assessment, including demographics such as age and sex, poisoning-related factors like intent of poisoning, route of exposure, and category of substances (which is categorized into eight categories and 44 types), and initial vital signs at the emergency department, including systolic blood pressure, heart rate, respiration rate, body temperature, and the “alert, verbal, painful, unresponsive” (AVPU) scale of mental status. The AVPU scale is incorporated into the new-Poisoning Mortality Score as it is a quick and effective tool for assessing a patient’s level of consciousness [Citation8]. The points for each variable in the new-Poisoning Mortality Score are calculated using multivariable logistic regression, with the total score ranging from 0 to 137, as presented in Appendix 1. The Modified Early Warning Score, on the other hand, has a score range of 0–14 and includes variables like systolic blood pressure, heart rate, respiratory rate, body temperature, and the AVPU scale. Detailed descriptions of these scoring systems are available in Supplementary Table 1.
In the foundational study that led to the development of the new-Poisoning Mortality Score [Citation4], substances were classified based on a mortality index. This index was defined as the number of mortalities divided by the number of exposures for each substance within the study cohort. To ensure consistency and clinical relevance in the validation of the new-Poisoning Mortality Score, we utilized this publicly accessible mortality index data for our analysis. In cases of exposure to multiple substances, the primary substance is selected based on the highest mortality index. If the mortality indices are the same, the substance with the largest ingested dose is chosen. The mortality indices of eight categories (A through H) with 44 subtypes of substances are detailed in Supplementary Table 2.
Validation and comparison of the new-Poisoning Mortality Score and the Modified Early Warning Score
The statistical performance of the new-Poisoning Mortality Score and the Modified Early Warning Score was evaluated in terms of discrimination and calibration. For discrimination, the new-Poisoning Mortality Score was compared to the Modified Early Warning Score using metrics such as sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve. The optimal cutoff value was identified using Youden’s index, and the area under the receiver operating characteristic and its 95% confidence interval (CI) were calculated. To assess the performance of the new-Poisoning Mortality Score across different substance categories, subgroup analyses were conducted. Additionally, an analysis of the new-Poisoning Mortality Score was carried out excluding substance data to ascertain the utility of the model in instances where the type of substance involved was unclear or the main toxicant was not clear due to mixed exposures.
Calibration indicates the degree to which a prediction model accurately estimates the absolute risk; poorly calibrated models may either underestimate or overestimate the outcome of interest [Citation9]. In our study, we utilized the Hosmer–Lemeshow goodness of fit test for validating the new-Poisoning Mortality Score due to its established role in assessing the calibration of logistic regression models [Citation10]. The test evaluates whether the predicted probabilities from a model align with the observed outcomes. The test remains widely recognized and accepted in clinical research, providing a familiar and reliable method for assessing the predicting fit and accuracy of the model [Citation11]. Calibration curves for each system were generated to elucidate the relationship between observed and predicted mortalities. The distribution of mortality risk as predicted by our model was assessed using kernel smoothing methods, a set of statistical techniques used for smoothing data points to better understand underlying patterns and trends [Citation12]. A perfectly calibrated model aligns with the identity line. If the curve is below (above) the identity line, the score overestimates (underestimates) the mortality risks. The greater the deviation from the identity line, the more significant the miscalibration. To facilitate interpretation in the clinical setting, we used the Hosmer–Lemeshow goodness-of-fit test to create ten score groups of equal sizes, which were then categorized, according to the mortality rates [Citation4], into four risk groups: very low risk (<0.1%), low risk (0.1–0.9%), intermediate risk (1.0–9.9%), and high risk (≥10.0%).
Statistical analysis
Descriptive statistics for continuous variables were presented as median (interquartile range [IQR]) and differences in the medians were assessed using the Mann–Whitney U test. Categorical variables are presented as counts (percent) and were compared using the chi-square test. Sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve are reported with 95% CIs. A P value of less than 0.05 was considered statistically significant. All statistical analyses were performed using IBM SPSS Statistics version 27.0 (IBM Corp., Armonk, NY, USA) and R version 4.2.3 (R Core Team, Boston, USA).
Results
A total of 484,260 patients were enrolled in the Korea Center for Disease Control and Prevention cohort of emergency department-based injury surveillance between January 2019 and December 2020. Of 20,006 patients presenting with acute poisoning, 3,436 were excluded because of transfer out of the emergency department (n = 779), unknown outcomes (n = 137) or incomplete data on poisoning-related factors or initial vital signs at the emergency department (n = 2,429) or death on arrival (n = 28). A total of 16,570 patients were included in the study. There were 364 in-hospital deaths (mortality rate 2.2%).
outlines the case characteristics of the survivor group (n = 16,206) and in-hospital death group (n = 364). The median [interquartile range] age was significantly older in the in-hospital death group (72 [56–81] versus 41 [24–58] years, P < 0.001), with a larger proportion of patients aged greater than or equal to 70 years (54.9% versus 12.2%, P < 0.001). There was a greater proportion of males in the in-hospital death group (69.0% versus 40.2%, P < 0.001). Intentional exposures were predominant in both groups, with ingestion as the principal route. The survivor group had a greater proportion of exposures to substances with lower mortality indices (such as categories A, B, C and H), while the in-hospital death group had more exposures to substances with greater mortality indices (such as categories D, E, and F). Regarding initial vital signs at the emergency department, the in-hospital death group displayed a greater proportion of abnormal systolic blood pressure (≤69 mmHg; 25.3% versus 4.9%, P < 0.001), heart rate (≥120 beats/min; 18.4% versus 9.5%, P < 0.001), respiratory rate (≤11 breaths/min or ≥25 breaths/min; 40.9% versus 6.6%, P < 0.001), and body temperature (≥39°C; 4.7% versus 0.2%, P < 0.001). Furthermore, this group had a greater proportion of patients with decreased levels of consciousness, as evidenced by being either responsive to pain or unresponsive (61.8% versus 13.0%, P < 0.001). Both the new-Poisoning Mortality Score and the Modified Early Warning Score scores were significantly greater in the in-hospital death group (median [IQR]: 66 [57–77] versus 31 [23–40] for new-Poisoning Mortality Score, P < 0.001; 6 [4–10] versus 3 [2–4] for Modified Early Warning Score, P < 0.001).
Table 1. Comparison of the case characteristics between the survivor group and the in-hospital-death group.
compares the performance of the new-Poisoning Mortality Score and the Modified Early Warning Score for predicting in-hospital mortality in acute poisoning. The new-Poisoning Mortality Score outperformed the Modified Early Warning Score in all aspects, including the area under the receiver operating characteristic curve, sensitivity, specificity, and accuracy. The optimal cutoff value was 52 for the new-Poisoning Mortality Score and five for the Modified Early Warning Score, as determined by Youden’s index. illustrates the area under the receiver operating characteristic curve of both scoring systems for predicting in-hospital mortality.
Figure 1. Area under receiver operating characteristic curve of the new-Poisoning Mortality Score and the Modified Early Warning Score for predicting in-hospital mortality in acute poisoning (0.947 versus 0.800, respectively).
Table 2. The performance of the new-Poisoning Mortality Score and the modified early warning score for predicting in-hospital mortality in acute poisoning.
compares the predicted and observed mortality in different risk groups using the new-Poisoning Mortality Score. The equation for predicting in-hospital mortality was as follows: predicted mortality = 1/(1 + e−z), z = − 9.763 + 0.126 × new-Poisoning Mortality Score. depicts the calibration curve for the new-Poisoning Mortality Score, which showed good calibration, albeit with a tendency to overestimate mortality risk in the high-risk group.
Figure 2. Comparison of the calibration curves for the new-Poisoning Mortality Score and the Modified Early Warning Score for predicting in-hospital mortality in acute poisoning.
Table 3. Risk group stratification by the new-Poisoning Mortality Score for predicting in-hospital mortality in acute poisoning.
presents the performance of the new-Poisoning Mortality Score for predicting in-hospital mortality in each category of substances. The area under the receiver operating characteristic curve values were lower for substances with low mortality indices, such as categories A and C, while its performance was very good for substances with high mortality indices (categories G and H). The overall performance for all categories of substances was good, with an area under the receiver operating characteristic curve value of 0.918.
Table 4. Sub-analysis for the performance of the new-Poisoning Mortality Score for predicting in-hospital mortality in acute poisoning according to the substance category groups.
Discussion
We conducted a comparative analysis between the new-Poisoning Mortality Score and the Modified Early Warning Score, focusing on their capacity to predict in-hospital mortality among patients with acute poisoning. The key finding of our research is the superiority of the new-Poisoning Mortality Score over the Modified Early Warning Score across multiple metrics, including the area under the receiver operating characteristic curve, sensitivity, specificity, and accuracy. This advantage is not limited to specific category of substances, nor is it confined to situations in which the exact substance involved is known. In predictive modelling, calibration refers to the agreement between the predicted probabilities of an outcome and the actual observed outcomes [Citation13]. A well-calibrated model, like the new-Poisoning Mortality Score, ensures that the mortality risk predicted by the model closely aligns with observed mortality rates. This is particularly crucial in the clinical setting, in which accurate risk prediction can guide therapeutic decisions and the allocation of resources [Citation14–16].
The new-Poisoning Mortality Score presents several distinct advantages over other scoring systems, making it a more comprehensive and practical tool for assessing patients with acute poisoning. Notably, it incorporates poisoning-related factors, such as the intent of poisoning, route of exposure, and category of substances, which are not employed in other scoring systems [Citation1,Citation3,Citation17–19]. Intentional poisonings are often more serious than unintentional poisonings since they frequently involve greater doses. Also, it is widely acknowledged that both the type of substance involved, and the route of exposure are significant factors influencing predictive performance [Citation20–22]. By accounting for these poisoning-related factors, the new-Poisoning Mortality Score delivers a more relevant and accurate assessment of a patient’s condition, thus enhancing its predictive performance. A retrospective study of 396 patients designed to predict the mortality rate in patients with acute organophosphate poisoning [Citation22] reported that the area under the receiver operating characteristic curve values of Acute Physiologic and Chronic Health Evaluation II (APACHE II), Simplified Acute Physiology Score II (SAPS II), and Poisoning Severity Score in cases of acute organophosphate poisoning were 0.77, 0.75, and 0.67, respectively. The new-Poisoning Mortality Score exhibited a greater area under the receiver operating characteristic curve of 0.865 in the sub-analysis for the substance categories that includes organophosphates (Category E).
Another strength of the new-Poisoning Mortality Score is its robust performance, especially in situations in which identifying the primary toxic substance was challenging due to the patient’s altered mental status or ingestion of multiple substances. In Korea, toxicological screening and drug quantification are not readily available in many hospitals, necessitating prompt requests for these analyses in cases in which precise identification of the substance or dose is critical. However, requesting these analyses for all patients, including those at very low or low risk, could lead to an inefficient allocation of medical resources, potentially delaying essential analyses for high-risk patients. Furthermore, even in hospitals equipped to provide such services, the process of collecting samples and obtaining results can be time-consuming. An effective prognostic scoring system, like the new-Poisoning Mortality Score, can assist healthcare providers in clinical decision making. The new-Poisoning Mortality Score, particularly in additional subgroup analysis, has demonstrated its robustness even without detailed information about the toxic substance category, reinforcing its role as an effective predictive model in such clinical settings.
Accurate and rapid assessment of patients is crucial for making disposition decisions in the emergency department. In acute poisoning, the clinical development of toxic syndromes (toxidromes) may not be immediate. It is important to have a robust predictive model to screen for high-risk patients and assist in emergency department patient disposition decisions. The new-Poisoning Mortality Score is expected to be useful for objective discrimination between very-low-risk and low-risk patients, potentially reducing unnecessary hospitalizations. Furthermore, patients categorized as high-risk by the new-Poisoning Mortality Score can be considered for transfer to poisoning treatment centres in the early stage and closely monitored for sudden clinical deteriorations necessitating ICU care. Significantly, we observed a marked increase in mortality rates among these patients. These results suggest that specific toxicological treatments and early hemodynamic stabilization in the emergency department could improve clinical outcomes for these patients. Therefore, the new-Poisoning Mortality Score can contribute to clinical decision-making for patients with acute poisoning and improvements in emergency department resource utilization.
This study has several limitations. First, since the data were retrospective, not all patient charts could be retrieved, and discharged patients were not followed up for emergency department readmission and out-of-hospital mortality. Second, the Injury Surveillance Cohort does not provide data on specific causes of death and patient comorbidities. This limitation restricts our ability to conclusively determine the relationship between poison exposure and in-hospital mortality. Also, comorbidities, such as terminal illnesses, were not accounted for in our study; they might influence the aggressiveness of medical care and potentially affect the outcomes [Citation23,Citation24]. Third, the use of data from a single country may limit the generalizability of our findings. Future prospective studies in multiple countries are necessary to validate the applicability of the new-Poisoning Morality Score across different populations. Lastly, the overall mortality in our cohort of 16,570 patients was 2.2%. This presents a potential risk of overestimating or overfitting the predictive performance of the model, especially when the number of predictors significantly exceeds the number of outcome events [Citation25]. Such a discrepancy can give a misleading perception of the effectiveness of the model in predicting outcomes. Instead, it may predominantly reflect random variations or noise within the data. Therefore, it is crucial to exercise caution in interpreting these results.
Despite these limitations, our study provides valuable insights into the utility of the new-Poisoning Mortality Score in predicting in-hospital mortality in patients with acute poisoning. Caution is needed when interpreting our findings, which serve as a stepping stone towards more comprehensive studies in the future.
Conclusions
Our study has helped to validate the new-Poisoning Mortality Score as an effective tool for predicting in-hospital mortality in patients with acute poisoning to the emergency department. The new-Poisoning Mortality Score demonstrated superior performance over the Modified Early Warning Score in various metrics, including the area under the receiver operating characteristic curve. Furthermore, it showed consistent strength in cases in which the primary toxic substance was challenging to identify. Our findings suggest that the new-Poisoning Mortality Score can contribute to enhancement of clinical decision-making and patient management. Further research is needed to confirm its generalizability across different populations.
Ethical approval
The Institutional Review Board of Korea University Hospital approved this study (IRB No. 2023AN0007) but waived the requirement for informed consent because of the retrospective nature of the study.
Supplemental Material
Download MS Word (23.6 KB)Disclosure statement
No potential conflict of interest was reported by the authors.
Additional information
Funding
References
- Valiollahzadeh S, Ostadi A, Rahmani F. Predictive value of national early warning score and modified early warning score in outcome of patients with poisoning. IJMTFM. 2022;12(4):38299. doi: 10.32598/ijmtfm.v12i4.38299.
- Karami Me, Estachri MRA. Principles of toxicotherapy: general & special therapy. Sch Acad J Pharm. 2015;4(3):153–156.
- Schwarz ES, Kopec KT, Wiegand TJ, et al. Should we be using the poisoning severity score? J Med Toxicol. 2017;13(2):135–145. doi: 10.1007/s13181-017-0609-5.
- Han KS, Kim SJ, Lee EJ, et al. Development and validation of new poisoning mortality score system for patients with acute poisoning at the emergency department. Crit Care. 2021;25(1):29. doi: 10.1186/s13054-020-03408-1.
- Kao CC, Chen YC, Huang HH, et al. Prognostic significance of emergency department modified early warning score trend in critical ill elderly patients. Am J Emerg Med. 2021;44:14–19. doi: 10.1016/j.ajem.2021.01.047.
- Cakir E, Esra S, Bindal A, et al. Comparison of the effectiveness of acute physiology and chronic health evaluation II and modified early warning score scoring systems in predicting mortality in patients in the intensive care unit. J Crit Intensive Care. 2019;10(3):75–79.
- El-Sarnagawy GN, Abdelnoor AA, Abuelfadl AA, et al. Comparison between various scoring systems in predicting the need for intensive care unit admission of acute pesticide-poisoned patients. Environ Sci Pollut Res Int. 2022;29(23):33999–34009. doi: 10.1007/s11356-021-17790-y.
- Kelly CA, Upex A, Bateman DN, et al. Comparison of consciousness level assessment in the poisoned patient using the alert/verbal/painful/unresponsive scale and the Glasgow coma scale. Ann Emerg Med. 2004;44(2):108–113. doi: 10.1016/j.annemergmed.2004.03.028.
- Alba AC, Agoritsas T, Walsh M, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017;318(14):1377–1384. doi: 10.1001/jama.2017.12126.
- Hosmer DW, Lemesbow S. Goodness of fit tests for the multiple logistic regression model. Comm Stats. 1980;9(10):1043–1069. doi: 10.1080/03610928008827941.
- Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. Hoboken (NJ): John Wiley; 2013.
- Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference and prediction. 2nd ed. New York (NY): Springer; 2009.
- Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology. 2010;21(1):128–138. doi: 10.1097/EDE.0b013e3181c30fb2.
- Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–1931. doi: 10.1093/eurheartj/ehu207.
- Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14(2):207. doi: 10.1186/cc8204.
- Lee S, Park HJ, Hwang J, et al. Machine learning-based models for prediction of critical illness at community, paramedic, and hospital stages. Emerg Med Int. 2023;2023:1221704. doi: 10.1155/2023/1221704.
- Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a european/North American multicenter study. JAMA. 1993;270(24):2957–2963. doi: 10.1001/jama.1993.03510240069035.
- Peres BD, Melot C, Lopes FF, et al. The multiple organ dysfunction score (MODS) versus the sequential organ failure assessment (SOFA) score in outcome prediction. Intensive Care Med. 2002;28(11):1619–1624. doi: 10.1007/s00134-002-1491-3.
- Olsson T, Lind L. Comparison of the rapid emergency medicine score and APACHE II in nonsurgical emergency department patients. Acad Emerg Med. 2003;10(10):1040–1048. doi: 10.1197/S1069-6563(03)00342-7.
- Daly F, Little M, Murray L. A risk assessment based approach to the management of acute poisoning. Emerg Med J. 2006;23(5):396–399. doi: 10.1136/emj.2005.030312.
- Amirabadizadeh A, Nakhaee S, Jahani F, et al. Prognostic indicators in critically ill poisoned patients: development of a risk-prediction nomogram. Drug Metab Pers Ther. 2020;35(4):20200108.
- Peter JV, Thomas L, Graham P, et al. Performance of clinical scoring systems in acute organophosphate poisoning. Clin Toxicol. 2013;51(9):850–854. doi: 10.3109/15563650.2013.841181.
- Le Conte P, Riochet D, Batard E, et al. Death in emergency departments: a multicenter cross-sectional survey with analysis of withholding and withdrawing life support. Intensive Care Med. 2010;36(5):765–772. doi: 10.1007/s00134-010-1800-1.
- Berglund A, Wigertz A, Adolfsson J, et al. Impact of comorbidity on management and mortality in women diagnosed with breast cancer. Breast Cancer Res Treat. 2012;135(1):281–289. doi: 10.1007/s10549-012-2176-4.
- Debray TP, Damen JA, Snell KI, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356:i6460. doi: 10.1136/bmj.i6460.
Appendix Table 1. Multivariable logistic regression for the calculation of the new-Poisoning Mortality Score in the patients with acute poisoning.