https://orcid.org/0000-0001-7378-7877
Abstract
Objective
To independently validate the negative predictive value of the Tanta University risk model for intensive care requirements in poison center telephone consultations with other physicians.
Methods
This study included 400 consecutive patients with acute poisoning. Clinical and laboratory parameters were recorded during the initial consultation with the poison center. Patients who were already ventilated or on vasopressors at the time of consultation were excluded. The Tanta University risk model score was calculated from the data according to the following equation: Tanta University risk model score = 1.966*Glasgow Coma Scale + 0.329*oxygen saturation (percent) + 0.212*diastolic blood pressure (mmHg) − 0.27*respiratory rate (breaths/minute) + 0.33*standard bicarbonate (mmol/L). Twenty-four hours later, the patients’ courses were followed up by telephone. The Tanta University risk model was then compared to a composite endpoint indicating the requirement for admission to an intensive care unit (vasopressors, need for intubation, or death).
Results
Four hundred patients with acute poisoning were included. Thirty-seven patients had a complicated clinical course as defined by the composite endpoint. Receiver operating characteristic analysis revealed the area under the curve to be 0.87 (95 percent confidence interval 0.83–0.90). An unfavorable Tanta University risk model score was defined as less than 73.46, using a cut-off derived from a previous study of an unrelated series of patients with acute poisoning admitted to our service. Thirty-one of 37 patients with complicated courses had an unfavorable Tanta University risk model score compared to six patients with complicated courses among 306 patients with a favorable Tanta University risk model score (P < 0.0002, Fisher’s exact test). Sixty-three patients had an unfavorable Tanta University risk model score but an uneventful course. The negative predictive value of the Tanta University risk model was 0.98 (95 percent confidence interval 0.96–0.99), sensitivity was 0.84, and specificity 0.83.
Conclusions
In the present study of poison center telephone consultations, the Tanta University risk model was significantly related to the outcomes in patients with acute poisoning. Patients with a favorable Tanta University risk model score (greater than or equal to 73.46) were unlikely to need intensive care unit level of care.
Introduction
Clinicians can rely on poison centers for advice on management and the risk of patients with poisonings. Parameters indicating a higher risk of intensive care unit (ICU) requirement are still being validated.
The Poisoning Severity Score [Citation1] was developed in the late 1990s to grade the severity of patients with poisoning. It shows promise in stratifying risk at initial referrals to poison centers [Citation2]. However, it has several subjective criteria and is time-consuming to calculate [Citation3]. Thus, further collaborative research is needed to develop a poisoning severity score to accurately assess the clinical severity in all patients with poisoning and the risk of deterioration, necessitating invasive procedures that can only be performed in an ICU. Accordingly, a model for patients with acute poisoning to predict the need for ICU care (mechanical ventilation, vasopressors, and/or in-hospital mortality), the intensive care requirement score, was developed in the Netherlands [Citation4]. This intensive care requirement score was shown in a French study [Citation5] to have an excellent negative predictive value for events (mechanical ventilation, vasopressors, and/or kidney replacement therapy, and/or in-hospital death) that indicate the need for ICU care. Our research group in Germany also confirmed this negative predictive value of the intensive care requirement score for ICU care (mechanical ventilation, vasopressors, and/or in-hospital deaths) [Citation6]. Unfortunately, the type of poisoning must be known to calculate the intensive care requirement score.
Recently, a new model, the Tanta University risk model, was introduced. The main advantage of the Tanta University risk model is that it does not require the type of poisoning to be known [Citation7]. Our group has independently validated the negative predictive value of the Tanta University risk model for ICU requirements (mechanical ventilation, vasopressors, and/or death) in poisoned patients who were admitted to our clinical service [Citation8]. We now set out to apply this model to the risk stratification of patients presented to our service as phone consultations.
The specific aims of this study were: a) to evaluate the performance of the Tanta University risk model in poison center phone consultations in predicting the need for intensive care unit level of care; and b) to validate the cut-off value (73.46) derived from our earlier study of hospitalized patients with poisoning [Citation8], for risk stratification.
Methods
IRB approval
The study was approved by the Technical University of Munich’s institutional review board (IRB reference 2023-171-S-KK).
Setting
Our poison center is staffed by experienced medical toxicologists operating a general toxicology ward, an intermediate care unit, and a toxicological intensive care unit.
Our center handles greater than 45,000 calls annually, of which 8,118 in 2023 were consultations from other hospitals. The population served is about 13 million, primarily located in the German state of Bavaria. A quarter of the population lives in eight larger cities (> 100,000 inhabitants), with the remainder divided between medium towns and rural areas.
The German healthcare system is highly regulated. All hospitals in Germany are reimbursed a diagnosis-related flat fee per hospital stay. Relevant to this study, there is no financial incentive to move the patient to a higher level of care.
Inclusion criteria
Telephone contact by a hospital physician asking for advice on managing an adult patient (defined as 18 years or older) with acute poisoning.
Exclusion criteria
We excluded patients who already needed ICU treatment, defined as having needed cardiopulmonary resuscitation, mechanical ventilation, or vasopressors at admission.
Endpoint definition
Similar to previous studies [Citation5,Citation6,Citation8], we defined the need for ICU admission as a composite endpoint (mechanical ventilation during the first 24 hours after admission, the need for vasopressors, and/or death).
Hypothesis to be tested
We tested the hypothesis that the Tanta University risk model, based on the score obtained at phone consultations with the poison center, could identify patients who would need ICU treatment. Even more importantly, we determined the negative predictive value of the Tanta University risk model score in distinguishing patients not requiring ICU care. According to previous studies of the ICU requirement in hospitalized poisoned patients using the intensive care requirement score [Citation6] and the Tanta University risk model [Citation8], we required a negative predictive value of at least 95% for clinical usefulness. We also set out to validate the cut-off point for the Tanta University risk model (73.46) found in our earlier study of in-patients [Citation8].
Classifications used to describe substance exposures
The substance exposures were grouped, using the classification scheme introduced with the intensive care requirement score [Citation4]. This has the advantage that the study population can be directly compared to other published series using the same classification scheme [Citation5,Citation6,Citation8,Citation9] and the not yet published INTOXICATE study [Citation10].
The time from exposure to consultation was grouped as less than 1h, 1–12 h, and >12 h, or unknown.
Tanta university risk model score calculation
The Tanta University risk model score was calculated according to the "general model" described in the original publication [Citation7]. Briefly, the Tanta University risk model score = 1.966* Glasgow Coma Scale score + 0.329*oxygen saturation (%) + 0.212*diastolic blood pressure (mmHg) − 0.27*respiratory rate (breaths/min) + 0.33*standard bicarbonate (mmol/L) in venous blood.
Please note that in this calculation, the plus and minus signs are reversed compared to our earlier publication [Citation8] so that a more positive score, meaning a negative outcome, does not cause confusion.
Statistical methods
We used the same methods described in our earlier studies on risk modeling in severe poisonings [Citation6,Citation8]. In brief, the required sample size was calculated using the method of Arya and colleagues [Citation11]. The calculation yielded a minimal sample size of 384 patients, based on a prevalence of ICU requirement of 50% and a significance level of 0.05. The prevalence was set at 50% to obtain the largest estimate for sample size [Citation11]. Continuous data were presented as median and interquartile range. Comparisons between the two groups were tested for the significance of difference using the Mann-Whitney-U-test. Bonferroni corrections for multiple comparisons were made where appropriate. For categorical data, the significance of the association (contingency) was explored using Fisher’s exact test (for 2 × 2 tables) or the chi-square test (for larger contingency tables), as appropriate. The Youden index was used to determine the optimal cut-off point in the receiver operating characteristic curve of the Tanta University risk model score. The area under the receiver operating characteristic curve and its 95% confidence interval (CI) were calculated. Comparison of the area under the receiver operating characteristic curve with 0.5 (corresponding to random chance) was performed, using the method of DeLong and colleagues [Citation12]. Calculations were carried out using Statview 5.01 and JMP13 software (both from SAS Institute, Cary, NC, USA), and MedCalc 22.014 (MedCalc Software, Ostend, Belgium).
Results
During the study period (10 May 2023 to 21 August 2023), there were 3,027 consultations from physicians in our poison center. Most patients were either excluded for age less than 18 years or missing data, especially venous bicarbonate concentrations. This parameter is not routinely used in many smaller German hospitals, which represent the majority of our callers (see below).
Four hundred consecutive patients with complete data and follow-up could be included in the study. Their demographics, clinical characteristics, the need for ICU care, and the outcomes are listed in . The substances involved, the time interval between exposure and consultation, and the characteristics of calling hospitals are summarized in . The spectrum of exposures in the present study matched those in patients excluded from the study (Supplementary Table 1). There were few single substance poisonings with alcohol or street drugs (). The majority of cases were single substance poisonings with sedatives or poisonings with multiple agents. The majority of consultations were from smaller hospitals (). The type of hospital calling was very similar to those with patients not included in the study (Supplementary Table 1).
Table 1. Clinical characteristics, the need for intensive care unit care, and outcomes of 400 patients with acute poisoning, who had an uneventful course or met the study endpoint.
Table 2. Substances involved, time from exposure to consultation, and location of 400 patients with acute poisoning, who had an uneventful course or met the study endpoint.
Thirty-seven of the 400 patients (9%) met the composite endpoint. The most common endpoint occurrence was mechanical ventilation. Three patients died. The 37 patients meeting the endpoint were compared to the 363 patients with an uneventful course. The patients who met the composite endpoint were significantly older (). There were no significant differences in the type of substance exposure, the time interval from exposure to consultation, the kind of hospital calling, or the size of the town/city where the call originated from. There was a trend towards more patients with an undetermined time interval in the group that reached the composite endpoint. Patients with an unknown time interval between exposure and call had a significantly lower Glasgow Coma Scale than the other patients (data not shown).
There were significant differences in the Glasgow Coma Scale, the diastolic blood pressure, and the serum bicarbonate concentrations between the patients with an uneventful course and those with a complicated course (). We did not find differences between groups in the arterial oxygen saturation and the respiratory rate. All of the parameters above are components of the Tanta University risk model.
The Tanta University risk model score significantly differed in patients who met the study composite endpoint (P < 0.0002). Thirty-one out of 37 patients who met the composite endpoint (including the three patients who died) had a Tanta University risk model score of less than 73.46. In comparison, only 63 of 363 patients with an uneventful course had a Tanta University risk model score of less than 73.46 (P < 0.0002). The six patients misclassified by the Tanta University risk model as low risk had ingested caustics (two patients), venlafaxine, quetiapine (two patients), and a mixture of antifreeze and vodka.
The sensitivity of the Tanta University risk model score using the previously determined cut-off of 73.46 to detect a need for intensive care was 0.84 (95% CI 0.67–0.93), and the specificity was 0.83 (95% CI 0.78–0.86). The negative predictive value of the Tanta University risk model score was 0.98 (95% CI 0.96–0.99). Receiver operating characteristic curve analysis () with the composite endpoint of a complication requiring ICU care showed an area under the receiver operating characteristic curve of 0.87 (95% CI: 0.83–0.90) for the Tanta University risk model. The optimal cut-off was 73.53.
Figure 1. Receiver operating characteristic curve of the Tanta University risk model in predicting intensive care requirement in 400 patients with poisoning. Area under the curve = 0.871, P < 0.001, by the method of DeLong and colleagues [Citation12].
![Figure 1. Receiver operating characteristic curve of the Tanta University risk model in predicting intensive care requirement in 400 patients with poisoning. Area under the curve = 0.871, P < 0.001, by the method of DeLong and colleagues [Citation12].](/cms/asset/8d47cce1-c4b8-4b29-9a4b-e1056158e0cd/ictx_a_2364030_f0001_b.jpg)
Discussion
Our findings show that: (a) a Tanta University risk model score of less than 73.46 was related to a complicated clinical course in a series of poison center consults for patients with acute poisoning; (b) the Tanta University risk model performed well at stratifying the risk of ICU level care in a series of patients with substance exposures vastly different from the exposures in our previous series of inner city in-patients; (c) a Tanta University risk model score greater than or equal to 73.46 largely excluded the need of intensive care with a negative predictive value of 0.98 (95% CI: 0.96–0.99); (d) The almost identical cut-off values between in-patients (earlier study [Citation8]) and poison center consults in this study with a different spectrum of exposures may indicate a limited influence of the type of exposure on risk model performance.
As others have shown [Citation9], a minority (37/400, 9.3%) of the patients in our series needed interventions that usually require an ICU. Despite this, the majority of patients (261/400, 65.3%) were admitted to an ICU. Our data (, ) show that the Tanta University risk model score calculated during poison center consultation was significantly related to a composite endpoint, indicating a need for interventions usually performed in an ICU.
In this study, the negative predictive value of a cut-off derived from an independent series of hospitalized patients [Citation8] indicating a higher risk of ICU interventions was 0.98 (95% CI 0.96–0.99). The negative predictive value exceeds the 0.95 we predetermined to demonstrate clinical usefulness. This may indicate that the Tanta University risk model can be used during the poison center telephone consultation to stratify patients’ risk.
Dylan de Lange recently discussed predicting outcomes in severe intoxications [Citation10]. He presented unpublished data from the INTOXICATE study on the divergent performance of the intensive care requirement score between health care systems in Europe and the developing world. He proposed that while the slope of a risk model (thus, for example, reflecting human physiology) may be the same between different countries, the Y-intercept (and, by extension, the cut-off of a predictive model) may depend on healthcare systems and the type of poisonings.
This hypothesis is in line with the excellent performance of the intensive care requirement score using an identical cut-off (intensive care requirement score greater than 6) in three presumably comparable Western European healthcare systems [Citation4–6]. Using the Tanta University risk model, this study also supports Dylan de Lange’s hypothesis. We found virtually identical optimal cut-off values (73.53 versus 73.46) in the receiver operating characteristic curve analysis of German poison center telephone consultations compared to our previous study on German hospitalized patients [Citation8]. The types of poisoning differ between the poison center consultations () and the inner-city inpatients, who were more often poisoned with street drugs and alcohol [Citation8]. Therefore, the nearly identical optimal cut-off in both studies may indicate that the healthcare system has a greater influence on model performance than the type of poisoning.
The main limitation of our study is a relatively high exclusion rate mainly caused by missing results in serum bicarbonate. This may represent a bias. To exclude a systematic error, we evaluated a subset of 161 patients excluded from the study (Supplementary Table 1).
These patients had very similar exposures and time intervals between exposure and consultation. There was one difference, though. None of the excluded patients had an unknown time interval between exposure and consultation. As the patients in our study with an unknown time interval had a significantly lower Glasgow Coma Scale, we assume that depressed consciousness may trigger blood gas analysis and subsequent eligibility for the study. The fact that risk stratification using this predictive model worked very well in our inpatients (in whom all get bicarbonate concentration measured) also makes a bias due to missing bicarbonate measurements less likely.
Another limitation of the risk model is that it relies on serum bicarbonate concentrations and thus cannot be used in a prehospital (e.g., ambulance) setting. A further limitation is that the indication for invasive procedures, such as intubation, is subjective. For example, in a patient with caustic exposure, one may or may not decide to protect a presently still patent airway.
A general problem with risk modeling is that it must be interpreted in the clinical context. For example, shortly after ingestion in a patient with a low-risk score and an expected severe poisoning (e.g., chloroquine), the expertise of a well-trained medical toxicologist is still required.
Conclusions
In the present study of 400 telephone consultations, 65.3% of patients with poisoning reported to a poison center were treated in an ICU. In contrast, only 9.3% received ventilation or vasopressors or died in the hospital. The Tanta University risk model could stratify poison center consultations according to the risk and the need for interventions normally provided in the ICU. The negative predictive value (98%) of a cut-off of greater than or equal to 73.46 was greater than the 95% value selected for clinical relevance. Patients with poisoning with a score greater than this cut-off had a very low likelihood of requiring ICU care. This may enable clinicians to save resources for patients needing ICU treatment.
Supplemental Material
Download MS Word (17.4 KB)Disclosure statement
No potential conflict of interest was reported by the authors.
Data availability statement
The data supporting this study’s findings are available from the corresponding author, CR, upon reasonable request.
Additional information
Funding
References
- Persson HE, Sjöberg GK, Haines JA, et al. Poisoning severity score. Grading of acute poisoning. J Toxicol Clin Toxicol. 1998;36(3):205–213. doi:10.3109/15563659809028940.
- Casey PB, Dexter EM, Michell J, et al. The prospective value of the IPCS/EC/EAPCCT poisoning severity score in cases of poisoning. J Toxicol Clin Toxicol. 1998;36(3):215–217. doi:10.3109/15563659809028941.
- Schwarz ES, Kopec KT, Wiegand TJ, et al. Should we be using the Poisoning Severity Score? J Med Toxicol. 2017;13(2):135–145.
- Brandenburg R, Brinkman S, de Keizer NF, et al. The need for ICU admission in intoxicated patients: a prediction model. Clin Toxicol. 2017;55(1):4–11. doi:10.1080/15563650.2016.1222616.
- El Gharbi F, El Beze N, Jaffal K, et al. Does the ICU Requirement Score allow the poisoned patient to be safely managed without admission to the intensive care unit? – A validation cohort study. Clin Toxicol. 2022;60(3):298–303.
- Boll R, Romanek K, Schmoll S, et al. Independent validation of the ICU requirement score in a cohort of acutely poisoned adults. Clin Toxicol. 2018;56(7):664–666.
- Elgazzar FM, Afifi AM, Shama MAE, et al. Development of a risk prediction nomogram for disposition of acute toxic exposure patients to intensive care unit. Basic Clin Pharmacol Toxicol. 2021;129(3):256–267. doi:10.1111/bcpt.13619.
- Schmoll S, Heier E-C, Böll R, et al. Independent validation of the Tanta University Risk Model for intensive care requirement in acutely poisoned adults. Clin Toxicol. 2023;61(4):266–269. doi:10.1080/15563650.2023.2188142.
- Brandenburg R, Brinkman S, de Keizer NF, et al. In-hospital mortality and long-term survival of patients with acute intoxication admitted to the ICU. Crit Care Med. 2014;42(6):1471–1479.
- De Lange DW. Predicting outcome in severe intoxications. EAPCCT Fellows’ Webinar; 2023 Nov 2, online.
- Arya R, Antonisamy B, Kumar S. Sample size estimation in prevalence studies. Indian J Pediatr. 2012;79(11):1482–1488. doi:10.1007/s12098-012-0763-3.
- DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845.