The need for ICU admission in intoxicated patients: a prediction model

Abstract

Context: Intoxicated patients are frequently admitted from the emergency room to the ICU for observational reasons. The question is whether these admissions are indeed necessary.

Objective: The aim of this study was to develop a model that predicts the need of ICU treatment (receiving mechanical ventilation and/or vasopressors <24 h of the ICU admission and/or in-hospital mortality).

Materials and methods: We performed a retrospective cohort study from a national ICU-registry, including 86 Dutch ICUs. We aimed to include only observational admissions and therefore excluded admissions with treatment, at the start of the admission that can only be applied on the ICU (mechanical ventilation or CPR before admission). First, a generalized linear mixed-effects model with binominal link function and a random intercept per hospital was developed, based on covariates available in the first hour of ICU admission. Second, the selected covariates were used to develop a prediction model based on a practical point system. To determine the performance of the prediction model, the sensitivity, specificity, positive, and negative predictive value of several cut-off points based on the assigned number of points were assessed.

Results: 9679 admissions between January 2010 until January 2015 were included for analysis. In total, 632 (6.5%) of the patients admitted to the ICU eventually turned out to actually need ICU treatment. The strongest predictors for ICU treatment were respiratory insufficiency, age >55 and a GCS <6. Alcohol and “other poisonings” (e.g., carbonmonoxide, arsenic, cyanide) as intoxication type and a systolic blood pressure ≥130 mmHg were indicators that ICU treatment was likely unnecessary. The prediction model had high sensitivity (93.4%) and a high negative predictive value (98.7%).

Discussion and conclusion: Clinical use of the prediction model, with a high negative predictive value (98.7%), would result in 34.3% less observational admissions.

Keywords:

• Critical care
• outcome
• overdose
• poisoning
• costs

Introduction

Many intoxicated patients are admitted to an Intensive Care Unit (ICU). Although health care systems differ considerably and real figures are lacking up to 40% of the patients visiting an Emergency Department (ED) with an intoxication are admitted to the hospital. A proportion of these patients, varying from 4 to 40%, is admitted to the ICU. On average, the ICU population consists of 1.5–3.7% intoxicated patients.[1–3] However, the in-hospital mortality of intoxicated patients admitted to an ICU is extremely low (2.1%).[1] Apparently, in developed countries, the predominant reason to admit intoxicated patients to the ICU is for observation purposes only and not for immediate treatment. Indeed, severe symptoms might arise if the time to maximum concentration of these xenobiotic substances has not yet been reached.

Clearly, for many patients with severe sequelae of their intoxication the only place to be treated is the ICU. For example, if a patient is mechanically ventilated on the ED or has received cardiopulmonary resuscitation (CPR) then the only place a patient can be treated is the ICU. However, it is necessary to await such symptoms in majority of patients who are not present with these serious sequelae and observations. Since prediction of these adverse symptoms is difficult, many intoxicated patients have to be admitted to the ICU to detect only few with serious symptoms. Although these ICU admissions are justifiable from a safety perspective, it has an economic disadvantage. In most countries a formal cost analysis is lacking, but Irish ICUs estimated the costs at €7717 per intoxicated patient per ICU admission.[4]

A better allocation of intoxicated patients will prevent unnecessary admissions to the ICU, increase the availability of ICU care for those patients that rely on it, and it will reduce costs. However, to create a better allocation it is necessary to identify, from readily available parameters, patients that really need ICU treatment.

Therefore, the aim of this study is to develop a bedside prediction model which predicts the need of ICU treatment for patients with intoxications.

Materials and methods

Setting

We performed a retrospective cohort study using the Dutch “National Intensive Care Evaluation” (NICE) quality assessment registry.[5] This national database prospectively collects parameters recorded in the first 24 h of ICU admission, including several co-morbidities present before hospital admission and/or at the time of ICU admission, and the reason for ICU admission registered according to the Acute Physiology and Chronic Health Evaluation (APACHE) IV classification system.[6] At present, 86 ICUs are participating in the registry (>90% of all ICU admissions in the Netherlands).

Subjects

We included all ICU admissions due to intoxications with at least one APACHE IV intoxication diagnosis (n = 7) originating from the emergency department (ED) during the period 1 January 2010 until 1 January 2015. Predicting the need for ICU treatment of intoxicated patients was not deemed necessary for patients that, in any case, should be admitted to the ICU, therefore these admissions were excluded (defined as receiving mechanical ventilation at ICU admission or having received CPR before ICU admission). We did not exclude patients with an additional medical or surgical diagnosis. If an admission had missing data on one of the final selected covariates we excluded it from the analysis. Baseline statistics of the subjects were recorded.

Definitions

The APACHE IV classification system identifies the following admission diagnoses for intoxication: (1) alcohol, (2) analgesics, (3) antidepressants (cyclic antidepressants, lithium), (4) street drugs (opiates, cocaine, amfetamine), (5) sedatives (hypnotics, antipsychotic, benzodiazepines), (6) toxins not otherwise specified, and (7) poisoning due to carbon monoxide, arsenic or cyanide (the so-called “other poisons”). We added an admission diagnose: (8) combination of two subtypes of intoxication.

The need for ICU treatment was dichotomously defined as mechanical ventilation in the first 24 h after ICU admission and/or vasopressors in the first 24 h after ICU admission and/or death during hospital stay.

Model development

Generalized linear mixed-effects model

A model was developed to predict the need for ICU treatment, based on readily available covariates (registered during the first hour of ICU admission). The covariates were year of ICU admission, age, gender, the presence of a second nonintoxication APACHE IV reason for ICU admission, admission type (medical or surgical), heart rate (HR), systolic blood pressure (SBP), Glasgow Coma Scale (GCS), chronic renal insufficiency, chronic dialysis, chronic obstructive pulmonary disease, chronic respiratory insufficiency, chronic cardiovascular insufficiency, cirrhosis, cerebrovascular accident, neoplasm, hematological malignancy, acquired immune deficiency syndrome, chronic immunological insufficiency, dysrhythmia, gastro-intestinal bleeding, intracranial mass, diabetes, and the type of intoxication. First, we excluded the covariates that occurred less than 10 times in the population that needed and that did not need ICU admission.[7] Second, we tested these covariates on multicollinearity using the variance inflation factor (VIF) and excluded the variables if the VIF was greater than 4. Third, we tested for interaction between the group of intoxication and chronic renal insufficiency, cirrhosis, age, SBP, HR, and GCS. The selected covariates and found interaction terms were used for the development of a generalized linear mixed-effects model with binominal link function and a random intercept per hospital. The covariates that lowered the Akaike information criterion (AIC) significantly with a p value < .01 were identified using backward selection and were included in the final mixed-effects model.[8] During the mixed-effects model development all continuous variables were included as restricted cubic splines.

The performance of the generalized linear mixed-effects model was assessed by measures of discrimination, calibration, and accuracy. The discrimination was expressed as the area under the receiver operating characteristic curve (AUC).[9] The calibration was analyzed by inspecting the calibration plots. The accuracy of the model was assessed by the Brier score.[10] Estimates of the AUC, Brier score and the associated 95% confidence intervals of the models were obtained by bootstrapping with 500 samples.[10–12]

Prediction model based on a point system

The prediction model is based on the covariates included in the final generalized linear mixed-effects model. A practical point system founded on the betas of each covariate was developed using the following six steps according to the Framingham method.[13]

Continuous covariates were categorized and a reference value for each category was set. This reference value was usually the median of the specific category, for instance the reference value for the age category 25–35 year was 30 years.

For each covariate a base category, receiving zero points in the point system, was set. For age the base category was <25 year with a reference value of 20 years. For all dichotomies covariates, the base category was set on 0, indicating that the specific diagnose was absent.

For all nonbase categories, the difference between the reference value of the specific category and the reference value of the base category was calculated. For the age category 25–35 year with a reference value of 30, this difference was 10 years. For all dichotomies covariates this difference was 1.

The risk factor of each category was set by multiplying the calculated differences with the betas of the specific covariates. In the final multivariate regression model the beta of the covariate age was .002, resulting in a risk factor of .02 for the age category 25–35 year.

The constant of the point system was defined by multiplying the beta of the covariate age by 5 and was .011.

The points, rounded to the nearest integer, assigned to each category were calculated by dividing the risk factor by the constant. For the age category 25–35 year this results in 2 points.

To determine the performance of the prediction model, the sensitivity and specificity of several cut-off points based on the assigned number of points were assessed.

The statistical analyses were performed using IBM SPSS statistics 21 (Armonk, NY) and the statistical environment R version 3.3.0. (Vienna, Austria).

Ethics

The data were retrospectively analyzed. The data in the registry is coded and cannot be traced back to individual patients or individual ICUs. The Ethical Institutional Research Board of the Academic Medical Center in Amsterdam concluded that this retrospective analysis is not subject to the Medical Subject Act (in Dutch: “Wet op medisch-wetenschappelijk onderzoek met mensen,” WMO) and therefore informed consent was not deemed necessary (IRB protocol number W15 _239 # 15.0284).

Results

Subjects

During the study period 408,259 admissions were made to the 86 participating ICUs, including 16,434 admissions with at least one diagnosis of intoxication (4.0%), of which 12,404 (3.0%) were directly admitted from the emergency department. The number of admissions that required ICU treatment because of mechanical ventilation and/or CPR prior to ICU admission was, respectively, 1970 (15.9%) and 120 (1.2%) and were excluded. Furthermore, 734 (7.1%) admissions with missing data were excluded. In the supplementary materials the characteristics of the excluded data due to missing data is shown. This table (Appendix of Supplementary Material) shows that the patients with missing data were younger and had less often a combination of an intoxication with another important reason for ICU admission indicating a lower risk for needing an ICU admission. However, these differences are small in clinical perspective. Finally, 9679 admissions were included in the analysis (Figure 1).

Figure 1. Flow chart.

In less than half of these admissions the patient was male (44.3%). The median age was 41.0 years (IQR: 27.0–51.0). The median APACHE IV score was 32.0 (IQR: 22.0–49.0). The five most frequent non-intoxication diagnosis in combinations with an intoxication are coma/change in level of consciousness (10.2%), medical respiratory problems (5.5%), aspiration pneumonia (5.3%), endocrine/metabolic disturbances (4.8%) or drug toxicity (4.5%). A total of 43 patients died at the ICU (ICU mortality 0.4%), and another 22 patients died after ICU discharge while they were still in the hospital (hospital mortality 0.7%). Baseline characteristics of the 9679 included admissions are shown in Table 1. In total, 632 (6.5%) of the patients admitted to the ICU eventually turned out to actually need ICU treatment based on the need of mechanical ventilation (n = 397 (4.1%)), receive vasopressors in the first 24 h after ICU admission (n = 319 (3.3%)), and/or death during hospital stay (n = 65 (0.7%)).

Table 1. Characteristics of the ICU admissions, originating from the emergency department, due to intoxications^a.

	Total	ICU admission not needed^b	Needing ICU admission^c
General characteristics
Number of patients	9679	9047	632
Hospital mortality (n (%))	65 (0.7)	0 (0)	65 (10.3)
Age: (median (25–75%))	41 (27–51)	40 (27–50)	49 (39–58)
Gender, male (n (%))	4292 (44.3)	4005 (44.3)	287 (45.4)
APACHE IV score (median (25–75%))	32 (22–49)	31 (21–45)	66 (48–85)
Type of intoxication (n (%))
Alcohol	1160 (12.0)	1116 (12.3)	44 (7.0)
Analgesic	448 (4.6)	422 (4.7)	26 (4.1)
Antidepressants	1149 (11.9)	1062 (11.7)	87 (13.8)
Street drugs	1107 (11.4)	1017 (11.2)	90 (14.2)
Sedatives	2987 (30.9)	2807 (31.0)	180 (28.5)
Other poisons (e.g., CO, arsenic, cyanide)	34 (0.4)	33 (0.4)	1 (0.2)
Toxins not otherwise specified	1380 (14.3)	1253 (13.8)	127 (20.1)
Combination of two or more intoxication types	1414 (14.6)	1337 (14.8)	77 (12.2)
Combination of intoxication with another important reason for ICU admission	1191 (12.3)	977 (10.8)	214 (33.9)

^a Observational admissions only: admissions that should go to the ICU no matter what (mechanical ventilation or CPR before admission) are excluded from analysis.

^b Not receiving mechanical ventilation and/or vasopressors <24 h of the ICU admission/in-hospital mortality.

^c Receiving mechanical ventilation and/or vasopressors <24 h of the ICU admission/in-hospital mortality.

APACHE: Acute Physiology And Chronic Health Evaluation.

Model development

Generalized linear mixed-effects model

The following covariates were excluded due to a low occurrence: gastro intestinal bleeding, chronic dialysis, neoplasm, hematological malignancy, chronic cardiovascular insufficiency, intracranial mass, AIDS, and cerebrovascular accident. There was no multicollinearity between the included covariates, and no interaction terms between the group of intoxications and the covariates where found. During the generalized linear mixed effects model development, the following covariates where selected based on the AIC: age, type of intoxication, SBP at ICU admission, HR at ICU admission, GCS at ICU admission, the presence of another non-intoxication diagnose, cirrhosis, dysrhythmia, and respiratory insufficiency. All these covariates were included in the final generalized linear mixed-effects model. The results of this final model are presented in Table 2. A GCS <6 (OR 6.62 (5.10–8.27)), age >65 (OR 6.02 (4.10–8.85)) were the strongest predictors of needing an ICU treatment. The discrimination and accuracy of the final generalized linear mixed effects model that predicts the need for ICU treatment was good; the AUC was 0.85 (0.85–0.86) and the Brier score was 0.05 (0.05–0.05). Though patients with a predicted need for ICU treatment lower than ∼80% have a slightly lower observed need for ICU treatment, other patients with a high predicted need for ICU treatment (∼>80%) have a slightly higher observed need for ICU treatment (Figure 2).

Figure 2. Calibration plot of the final multivariate logistic regression model that predicts the need for ICU treatment.

Table 2. Final generalized linear mixed-effects model that predicts the need for ICU treatment.

Covariate	Need for ICU treatment OR (95%CI)
Age
Age <25	Reference
Age >25 and <35	0.99 (0.67–1.47)
Age >35 and <45	2.23 (1.58–3.14)
Age >45 and <55	2.56 (1.83–3.57)
Age >55 and <65	3.51 (2.45–5.05)
Age >65	6.02 (4.1–8.85)
Type of intoxication
Alcohol	Reference
Analgesic	3.18 (1.83–5.54)
Antidepressants	2.82 (1.85–4.29)
Street drug	3.06 (2.03–4.6)
Sedatives	2.03 (1.39–2.94)
Other poisons^a	0.61 (0.04–10.49)
Toxins not otherwise specified	3.49 (2.35–5.18)
Combination of two or more intoxication types	2.52 (1.64–3.86)
SBP at ICU admission
SBP <100	3.6 (2.67–4.85)
SBP >100 and <110	1.43 (1.01–2.01)
SBP >110 and <120	1.07 (0.75–1.52)
SBP >120 and <130	Reference
SBP >130 and <140	0.63 (0.41–0.96)
SBP >140	0.96 (0.69–1.32)
HR at ICU admission
HR <75	Reference
HR >75 and <85	1.04 (0.78-1.37)
HR >85 and <105	1.13 (0.88-1.45)
HR >105	2.15 (1.67-2.77)
GCS at ICU admission
GCS ≥14	Reference
GCS >9 and <14	1.72 (1.35–2.19)
GCS >6 and <9	3.32 (2.57–4.29)
GCS <6	6.62 (5.1–8.57)
Comorbidity
Combination of intoxication with another   important reason for ICU admission	3.79 (3.03–4.75)
Cirrhosis	4.34 (2.03–9.29)
Dysrhythmia	2.6 (1.73–3.91)
Respiratory insufficiency	4.85 (2.84–8.27)

OR: odds ratio; CI: confidence interval; SBP: systolic blood pressure; HR: heart rate; GCS: Glasgow Coma Scale.

^a Examples of “other poisons” are carbon monoxide, arsenic, cyanide.

For simplicity the continuous variables are presented as categories while they were included as restricted cubic splines in the actual model used for prediction of ICU requirement.

Prediction model based on a point system

The prediction model based on a point system (Table 3) shows the parameters that influence the need for ICU treatment, and the points assigned to these covariates. In theory, the minimum and maximum score a patient can obtain is −9 and +54 points, respectively. The minimum and maximum score obtained in the study population was −7 and 36, respectively. Respiratory insufficiency, age >55 year, and a GCS <6 were the most important factors influencing the need for ICU treatment. Alcohol and “other poisons” (e.g., carbon monoxide, cyanide, arsenic) as intoxication type and an SBP ≥130 mmHg are indicators that ICU treatment will probably be unlikely.

Table 3. Prediction model: points assigned to covariates.

Score	Age	SBP	HR	GCS	Type^a	Second diagnose^b	Cirrhosis	Dysrhythmia	Respiratory insufficiency
−6					6
−5					1
−3		≥140
−1		130–140			5
0	<25	120–130	<75	≥14	3	No	No	No	No
1		110–120	75–85		2,4
2	25–35	100–110	85–95		7
3			95–105	9–14
4	35–45	<100	≥105
5								Yes
6	45–55
7				6–9		Yes	Yes
8	55–65								Yes
9				<6
10	≥65
Intoxication type	Intoxication
1	Alcohol
2	Analgesic
3	Antidepressants
4	Street drugs
5	Sedatives
6	Carbon monoxide, arsenic, cyanide poisonings
7	Toxins not otherwise specified
8	Combination of two or more intoxication types

SBP: systolic blood pressure; HR: heart rate; GCS: Glasgow Coma Scale.

^a Legend for the type of intoxication is demonstrated above. ^bA second important reason for ICU admission.

The risk is calculated by adding up the numbers corresponding to the patients’ symptoms and co-morbidities. If a patient scores >6, ICU care is warranted.

Table 4 shows the sensitivity and specificity for various cut-off points based on the assigned number of points of the prediction model. When all patients with a score ≥6 points are admitted to the ICU (65.7% of all intoxicated ED patients), 93.4% of all the patients that actually need ICU treatment are identified (sensitivity). Of the 65.7% eventually only 9.3% will actually need the provided ICU treatment (positive predicted value).

Table 4. Sensitivity and specificity of the cut-off points based on the assigned number of the prediction model.

ICU requirement score	Percentage of patients	Sensitivity	Specificity	Positive predicted value	Negative predictive value
>3	85.3	98.4	15.6	7.6	99.3
>4	79.4	97.1	21.8	8.0	99.1
>5	72.5	96.0	29.2	8.7	99.1
>6	65.7	93.4	36.2	9.3	98.7
>7	58.4	90.7	43.9	10.2	98.5
>8	51.1	88.9	51.6	11.4	98.5
>9	43.7	86.4	59.3	12.9	98.4
>10	37.6	81.8	65.5	14.2	98.1
>11	32.3	77.3	70.8	15.6	97.8
>12	27.0	72.4	76.1	17.5	97.5
>13	22.3	66.7	80.8	19.6	97.2

In Table 5, the characteristics of the patients identified as “ICU treatment required” and “ICU treatment not required” are presented.

Table 5. Demographics of ICU admissions identified by the prediction model as to need or not need ICU admission.

Covariate	ICU admission needed: score >6	ICU admission NOT needed: score ≤6
Number of patients (n (%))	6362 (65.7)	3117 (34.3)
In-hospital mortality (n (%))	63 (1.0)	2 (0.1)
Age (median (25–75%))	46 (35–55)	29 (22–40)
Gender, male (n (%))	2804 (44.1)	1488 (44.9)
APACHE IV score (median (24–75%))	39 (27–57)	23 (16–31)
GCS at admission (n (%))
GCS ≥14	2667 (41.9)	2726 (82.2)
GCS >9 and <14	1718 (27.0)	470 (14.2)
GCS >6 and <9	1144 (18.0)	86 (2.6)
GCS <6	833 (13.1)	35 (1.1)
Type of intoxication (n (%))
Alcohol	533 (8.4)	627 (18.9)
Analgesic	263 (4.1)	185 (5.6)
Antidepressants	739 (11.6)	410 (12.4)
Street drugs	798 (12.5)	309 (9.3)
Sedatives	1959 (30.8)	1028 (31.0)
Other Poisonings (e.g., carbon monoxide, arsenic cyanide)	4 (0.1)	30 (0.9)
Other intoxications	1091 (17.1)	289 (8.7)
Combination of two or more intoxication types	975 (15.3)	439 (13.2)
Combination of intoxication with another important reason for ICU admission	1142 (18.0)	49 (1.5)

APACHE: Acute Physiology And Chronic Health Evaluation; GCS: Glasgow Coma Scale.

Discussion

We developed a model that predicts the course of intoxicated ICU patients. The strongest predictors that an intoxicated ICU patient will need ICU treatment are respiratory insufficiency, age >55 years, and a GCS <6. Alcohol and “other poisonings” as intoxication type and a SBP ≥130 mmHg are indicators that ICU treatment will probably be less likely necessary.

Using the most optimal cut-off point (≥6) the sensitivity of the prediction model is high (93.4%), this stands for a low numbers of false negatives, which is important from a safety perspective. If this prediction model was used, only five patients per year would have been wrongly sent to the general ward instead of sending to the ICU, including one patient (in 5 years) that eventually died. We would like to emphasize that this is merely a theoretical extrapolation and that in reality these patients would have been transferred from the general ward to the ICU.

The low specificity of the prediction model (36.2%) means it still has some problems to discern the patients that actually need ICU treatment from those who do not, resulting in a higher percentage of false positives. Also, our model has a moderate positive predictive value (some people are still being admitted to the ICU who turn out not to need it afterwards) but a very high negative predictive value (98.7%). This means that with our model you can send more people to low-care or intermediate care wards with a minimal risk of missing patients that eventually turned out to need ICU admission.

Application of the prediction model will lead to a large reduction of observational ICU admissions alone. Hypothetically, our prediction model would reduce the amount of admissions by 34.4%, meaning almost 700 admissions per year in the Netherlands. This would be coinciding with major cost savings. Based on Irish statistics (unfortunately there are no Dutch statistics available), the estimations are that over €5.3 million per year could be saved in ICU expenses.[4]

Most current literature on intoxicated ICU patients focuses on risk factors or prediction models that foretell a bad outcome; seldom on factors predicting an uneventful course. Our study demonstrates that alcohol and “other poisonings” as type of intoxication and an SBP ≥130 mmHg as individual characteristics, are indicators that ICU treatment will be likely unnecessary. The SBP of ≥130 mmHg is interesting since it is a separate risk factor from the type of substance used. Thus, for every type of intoxication, an SBP ≥130 mmHg is a predictor for a better outcome compared to patients having an SBP <130 mmHg. These results underline the importance of monitoring intoxicated patients carefully in the ED.

Our study emphasizes the important influence of chronic co-morbidities on the probability for the need of ICU treatment. Cirrhosis and chronic respiratory both predict that a patient will need ICU treatment, which means that these patients, even without clinical signs, have to stay at the ICU regardless of the type and estimated severity of the intoxication. These results are generally in line with other studies analyzing parameters and/or ICU prediction models for intoxicated patients. For example, a Finnish study by Liisanantii et al. also found that respiratory dysfunction is a risk factors for poor outcome.[14] Likewise, old age has proven to be a risk factor for a bad outcome in a large German and English cohort.[15,16] Moreover, a low GCS has been frequently described as a risk factor for mortality, with evidence that only a very low GCS (< 6) predicts mortality which is consistent with our findings.[15–19] We were able to combine these factors into one prediction model based on a point system.

There are several ICU prediction models tested specifically on intoxicated population, with various outcome measures of which mortality is most often investigated. For example, severity of disease models, like the APACHE and SAPS model are frequently analyzed, though the sensitivity range of these models is wide (67–90% and 90-100%, respectively).[20–22]

Furthermore, there has been some modest development in creating new prediction models, of which the Dutch study performed by Ambrosius et al. is most comparable to our study.[23] The authors tested a prediction model, based on literature research and expert opinion, and determined the general need for hospital admission for intoxicated patients at an ED.[24] Obviously, due to a different population and a different sample size, not all parameters are similar. For example, we did not find “respiratory rate” to be an independent predictor, nor did we find an intoxication by an antidepressant only, without other symptoms or co-morbidities, to be a reason for ICU treatment.

This study represents one of the largest multi-center cohorts of ICU admissions caused by intoxication (n = 9679). Also, since we did not exclude admissions with additional diagnoses, our parameters can be used for all intoxicated patients admitted for observation, resulting in high generalizability. However, various limitations should be addressed.

First, our prediction model has not yet been externally validated and is, therefore, not suitable to use in clinical practice at this moment. It does, however, teach us about the parameters that have an influence on the course of the patients. The model has been derived from patients already admitted to the ICU and has been calculated within their first hour of admission. It seems unlikely that calculation of this model on the verge of ED discharge/ICU admission will yield different results. However, before the model can be generalized to other departments, the performance of the model should be further investigated in an Emergency Department. Hereafter, the model can be tested in other countries and the model can be implemented. Another limitation of the external validity of our study might arise from the exclusion of certain patients with missing data. These patients appear to have a lower risk of needing an ICU admission. Including these patients will probably result in a higher percentage of patients with a correct prediction of not needing an ICU admission and result in a higher negative predictive value and specificity of the model.

Second, since we used the APACHE IV categories for intoxication, the specific substances are unknown. This is a limitation because even within a category, some substances have a higher potential to result in ICU treatment. For example, an intoxication with a tricyclic antidepressant is reported to be more likely to result in endotracheal intubation than a selective serotonin reuptake inhibitor.[25] Unfortunately, we were not able to correct for this. It would be interesting to evaluate the specific type of toxins that predict the need for ICU treatment.

Third, we could only include the parameters that were registered in the national database, with the result that we were not able to evaluate all the parameters that were recorded in the first hour of admission. For example, the results of electrocardiography or echocardiograms are not included in our analysis. We would like to emphasize that although this covariate is not in our prediction model, it could be an important clinical marker and therefore should be taken into account, especially with substances known to cause rhythm disturbances.[26,27] Also, the time between ingestion and presentation is not included in the prediction model. Patients that have no symptoms and the time to maximum concentration has past will not need ICU treatment. This information is unavailable in our database and might substantially add to the decision not to admit patients to the ICU. Additionally, it would have been better to correct for the underlying nature of the intoxication (intentional versus accidental) if this information was available in the national database. Also, patients that are not recognized as having an intoxication by the treating physicians were not included in our study. Therefore, some patients who present with, for instance, circulatory shock or changes in consciousness could have been missed.

Fourth, one of our outcome measures was in-hospital mortality, which means that all patients that die in the hospital, even those who die as a consequence of hospital complication (e.g., hospital acquired infection), are identified as “appropriately treated in the ICU”. Hence, this causes the prediction model to be probably less sensitive in clinical practice. Late complications of the initial intoxication, e.g., hepatic encephalopathy some days after a paracetamol poisoning, are disregarded by our model as adequate reasons for late admission to the ICU. It should be emphasized that our model focuses only on the immediate necessity for ICU admission (the first 24 h after intoxication) and does not preclude late complications.

Finally, our prediction model shows some calibration problems; the need for ICU treatment is underestimated for patients with a high probability, and slightly overestimated for patients with a low probability. This underestimation is, however, almost negligible in regard to daily practice, since this would lead to approximately five patients per year being allocated to the general ward that, subsequently, did need mechanical ventilation and/or vasopressors or would have died during hospital admission. Presumably, a great part of these patients at the general ward in need for ICU treatment will be identified in a later phase and can still receive ICU treatment.

An important and essential future step should be to externally validate this prediction model in ED-patients with an intoxication. It would be of great value to have a validated bedside prediction model at the ED, resulting in less unnecessary ICU admissions and a reduction in intoxication associated costs.

Conclusion

For the intoxicated ICU patients, the strongest predictors for needing ICU treatment are respiratory insufficiency, age >55 year, and a GCS <6. Alcohol and poison as intoxication type and a SBP ≥130 mmHg are indicators that ICU treatment will probably be less likely necessary. We developed a prediction model to predict the need for ICU treatment with a high sensitivity and a high negative predictive value. If this prediction model would be used in clinical practice the observational admissions of intoxicated patients would be reduced by 34.3%. The associated reduction in costs could be enormous.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.