ToxNet: an artificial intelligence designed for decision support for toxin prediction

Figures & data

Figure 1. Schematic architecture of ToxNet. The symptom vectors are processed in the graph-based GAT layers and the literature-matching network in parallel. The upper branch is the literature branch, and the lower branch is the GAT branch. GAT: graph attention layer; FC: fully connected layer; GT: ground truth; S: symptom vector; G: graph containing symptoms and metadata; M: number of patients.

Table 1. Included substances in pilot and extended cohort with number of cases with respective F1 score of ToxNet.

	Pilot cohort n (%)	F1 score in pilot cohort	Extended cohort n (%)	F1 score in extended cohort
Included cases out of total PCC calls	8995/781,278 (1.1%/100%)	0.66 ± 0.01	36,033/78,127 (4.6%/100%)	0.27 ± 0.01
ACE-inhibitors/sartans	119/8995 (1.3%)	0.27 ± 0.14	250/36,033 (0.7%)	0.11 ± 0.02
Acetaminophen	1376/8995 (15.3%)	0.23 ± 0.03	1477/36,033 (4.1%)	0.32 ± 0.02
Antidepressants (SSRI)	1073/8995 (11.9%)	0.50 ± 0.04	2107/36,033 (5.9%)	0.11 ± 0.03
Benzodiazepines	577/8995 (6.4%)	0.41 ± 0.05	3942/36,033 (11.0%)	0.25 ± 0.08
Beta blockers	288/8995 (3.2%)	0.54 ± 0.11	434/36,033 (1.2%)	0.22 ± 0.01
Calcium channel antagonists (dihydropyridine type)	75/8995 (0.8%)	0.21 ± 0.16	147/36,033 (0.4%)	0.10 ± 0.03
Cocaine	256/8995 (2.8%)	0.56 ± 0.10	288/36,033 (0.7%)	0.20 ± 0.04
Ethanol	2890/8995 (32.1%)	0.90 ± 0.01	2767/36,033 (7.7%)	0.70 ± 0.02
NSAIDs (excluding ASA and acetaminophen)	1462/8995 (16.3%)	0.52 ± 0.02	2216/36,033 (6.1%)	0.05 ± 0.01
Opiates/opioids (pure µ-agonists)	879/8995 (9.8%)	0.91 ± 0.02	1622/36,033 (4.5%)	0.48 ± 0.02
ASA			664/36,033 (1.8%)	0.36 ± 0.03
Antidepressants (SSNRI)			665/36,033 (1.8%)	0.07 ± 0.01
Antidepressants (tricyclic)			3558/36,033 (9.9%)	0.19 ± 0.03
Calcium channel antagonists (class IV)			85/36,033 (0.2%)	0.07 ± 0.03
Cetirizine/loratadine			161/36,033 (0.4%)	0.07 ± 0.01
Diphenhydramine/doxylamine/dimenhydrinate			2979/36,033 (8.3%)	0.19 ± 0.03
GHB/GBL			979/36,033 (2.8%)	0.45 ± 0.03
Levetiracetam			97/36,033 (0.3%)	0.01 ± 0.01
Lithium			1026/36,033 (2.8%)	0.45 ± 0.05
Methylphenidate			651/36,033 (1.8%)	0.18 ± 0.02
Neuroleptics (atypical)			3435/36,033 (9.5%)	0.27 ± 0.03
Neuroleptics (typical)			1014/36,033 (2.8%)	0.12 ± 0.03
Opiates (buprenorphine)			151/36,033 (0.4%)	0.05 ± 0.00
Pregabalin/gabapentin			396/36,033 (1.1%)	0.04 ± 0.01
Stimulants			914/36,033 (2.5%)	0.44 ± 0.03
THC			467/36,033 (1.3%)	0.48 ± 0.04
Valproic acid			308/36,033 (0.9%)	0.01 ± 0.02
Z-substances			3233/36,033 (9.0%)	0.19 ± 0.10

PCC: Poison Control Center; SSRI: selective serotonin reuptake inhibitors; ASA: acetyl salicylic acid; SSNRI: selective serotonin and norepinephrine reuptake inhibitors; class IV: class IV according to Vaughan Williams; GHB: gamma-hydroxybutyrate; GBL: gamma-butyrolactone; THC: tetra-hydro-cannabinol.

In the pilot cohort, one substance from each class was chosen as representative, in the extended cohort all substances from that class were included.

Figure 2. Comparison of ToxNet in different settings compared with the GAT branch over 10-fold cross-validation in the pilot cohort, 8995 cases. GAT: graph attention network; ToxNet(S): ToxNet in sequential setting; ToxNet: ToxNet in parallel setting.

Figure 3. Comparison of MDs’ performance with ToxNet over 10 different sets evaluated by one MD each set (25 cases per set) in the pilot cohort, overall 250 cases. MDs: medical doctors; GAT: graph attention network; ToxNet(S): ToxNet in sequential setting; ToxNet: ToxNet in parallel setting.

Figure 4. Clinician inter-variability and comparison with ToxNet. Poison classes are ordered alphabetically, each group separated with a white spacing in the pilot cohort, 25 cases.

Figure 5. Comparison of ToxNet with MLP and the GAT branch over 10-fold cross-validation in the extended cohort, 36,033 cases. MLP: multi-layer perceptron; GAT: graph attention network; ToxNet: ToxNet in parallel setting.

Figure 6. Comparison of MDs’ performance with ToxNet over 10 different sets evaluated by one MD each set (50 cases per set) in the extended cohort, overall 500 cases. MDs: medical doctors; GAT: graph attention network; ToxNet: ToxNet in parallel setting.

Figure 7. Clinician inter-variability and comparison with ToxNet. Poison classes are ordered alphabetically, each group separated with a white spacing in the extended cohort, 50 cases.