Development and Implementation of an e-Trigger Tool for Adverse Drug Events in a Swiss University Hospital

Figures & data

Table 1 The IHI trigger tool medication module and the corresponding detectable ADEs

Original IHI trigger	Detectable ADE
M1: C. difficile-positive stool	Antibiotic AE
M2: PTT >100 sec	Bleeding on Heparin
M3: INR >6 ^a	Bleeding under VKA
M4: Glucose <50 mg/dl	Hypoglycemia
M5: Rising BUN or serum creatinine two times (2×) over baseline	Renal toxicity
M6: Vitamin K administration	Bleeding under VKA
M7: Diphenhydramine (benadryl) administration	Allergic reaction
M8: Romazicon (flumazenil) administration	Benzodiazepine AE
M9: Naloxone (narcan) administration	Opioid AE
M10: Anti-emetic administration	Medication-induced nausea
M11: Oversedation/hypotension^b	Overtreatment with hypnotic or antihypertensive agents
M12: Abrupt medication stop	ADE
M13: Other

Notes: ^aGerman Version IHI MMTT used INR >4. ^bUK Version IHI MMTT used systolic blood pressure <80 mmHg.

Abbreviations: BUN, blood urea nitrogen; VKA, vitamin K antagonist.

Figure 1 Study flow chart depicting the tool development phase.

Table 2 The IHI trigger tool medication module triggers and modifications for tools A and B

IHI MMTT trigger	Tool A trigger	Tool B trigger
M1: C. difficile-positive stool	–	M1: C. difficile-positive stool ^a
M2: PTT >100 sec	M2: Thromboplastin time 1 >120 sec and thromboplastin time 2 >13 sec	M2: Thromboplastin time 1 >120 sec and thromboplastin time 2 >13 sec
M3: INR >6	M3: INR > 4^b	M3: INR > 4^b
M4: Glucose <50 mg/dl	M4: Glucagon or 40% and 50% glucose^c	M4: Glucagon or 40% and 50% glucose^c
M5: Rising BUN or serum creatinine two times (2×) over baseline	M5: Rising urea or creatinine 2× above upper limit of normal	M5: Rising urea or creatinine 2× above upper limit of normal
M6: Vitamin K administration	M6: Vitamin K administration	M6: Vitamin K administration
M7: Diphenhydramine (benadryl) administration	M7: Antihistamine administration	M7: Antihistamine administration
M8: Romazicon (flumazenil) administration	M8: Flumazenil administration	M8: Flumazenil administration
M9: Naloxone (Narcan) administration	M9: Naloxone administration	M9: Naloxone administration
M10 Anti-emetic administration	–	–
M11 Oversedation/hypotension	M11.1: Hypotension (SBP <80 mmHg)^c	M11.1: Hypotension (SBP <80 mmHg)^c
M12 Abrupt medication stop	–	–
M13 Other	Rivaroxaban drug-level or anti-factor Xa activity	Rivaroxaban drug-level or anti-factor Xa activity

Notes: ^aNon-digital trigger. ^bAs per German IHI MMTT Version. ^cAs per UK IHI MMTT Version.

Abbreviations: M, medication module triggers; SBP, systolic blood pressure.

Figure 2 Study flow chart depicting the tool implementation phase.

Table 3 Characteristics of included patients

Characteristic	All patients (n = 1008)
Median age (years) [IQR, range]	67 [52, 19–97]
Female (%)	463 (46%)
Male (%)	545 (54%)
Median length of hospital stay (days), [IQR, range]	4 [7, 1–96]

Table 4 Distribution of patients according to number of ADEs they experienced and the number of triggers which were found by Tool B

	Number of patients (%)
Number of triggers
0	805 (80)
1	148 (14.6)
2	44 (4)
3	7 (1)
4	4 (0.4)
Number of ADEs
0	819 (81)
1	119 (12)
2	55 (6)
3	12 (1)
4	3 (0.3)

Table 5 Distribution of 277 ADEs identified in 189 patients, listed according to frequency

Adverse drug event	Number of patients with ADE (% of Pptients with an ADE)
Gastrointestinal symptoms	34 (18.0)
Electrolyte disorders	27 (14.3)
Hypersensitivity reactions	24 (12.7)
Blood disorders	22 (11.6)
Renal impairment	16 (8.5)
Hemodynamic instability	15 (7.9)
Over-anticoagulation	15 (7.9)
Bleeding	14 (7.4)
Infections	14 (7.4)
Hepatopathy	13 (6.9)
Altered level of consciousness	9 (4.8)
Metabolic disorders	9 (4.8)
Dizziness	7 (3.7)
Mucositis	5 (2.6)
Confusion	4 (2.1)
Movement disorders	3 (1.6)
Oedema	3 (1.6)
Respiratory depression	3 (1.6)
Cephalgia	2 (1.1)
Neuropathy	2 (1.1)
Pancreatitis	2 (1.1)
Weight disorders	2 (1.1)
Arrhythmia	1 (0.5)
Visual disturbance	1 (0.5)
Insomnia	1 (0.5)
Muscle disorder	1 (0.5)
Pain at Injection site	1 (0.5)
Pleural effusion	1 (0.5)
Progressive multifocal leukoencephalopathy	1 (0.5)
Pupillary disorder	1 (0.5)
Urinary retention	1 (0.5)
Others	23 (12)

Table 6 Frequency, positive predictive value (PPV), negative predictive value (NPV), sensitivity and specificity of the individual triggers

Trigger	Absolute frequency	PPV	NPV	Sensitivity	Specificity
M1: C. difficile-positive stool	4	1	1	1	1
M2: Thromboplastin time 1 >120 sec and thromboplastin time 2 >13 sec for bleeding under heparin	82	1	0.92	1	0.92
M3: INR > 4 for bleeding under VKA	21	0	1	0	0.99
M3.1 INR > 4 for supratherapeutic INR under VKA	21	0.57	1	1	0.99
M3.2 INR > 4 for bleeding under all medication	21	0.05	1	0.5	0.98
M4: Administration of glucagon or 50% glucose	0	N/A	1	0	1
M5: Rising urea or creatinine 2× above upper limit of normal	41	0.19	0.99	0.47	0.97
M6: Vitamin K administration	56	0.22	1	1	0.96
M7: Antihistamine administration	40	0.33	0.99	0.62	0.97
M8: Flumazenil administration	0	N/A	1	0	1
M9: Naloxone administration for severe opioid ADE	2	1	1	1	1
M9.1: Naloxone administration for all opioid ADEs	2	1	0.99	0.19	1
M11: hypotension (SBP <80 mmHg)	26	0.04	0.99	0.10	0.98
M13: Rivaroxaban drug-level or anti-factor Xa activity	1	0	1	0	0.99

Abbreviation: N/A, not applicable.

Table 7 Sensitivity, specificity and 95% confidence intervals (CI) of Tools A and B

Tools	Sensitivity (95% CI)	Specificity (95% CI)
Chart review	1.00 (0.99–1.00)	1.00 (0.99–1.00)
Tool A	0.41 (0.34–0.48)	0.86 (0.83–0.88)
Tool B	0.43 (0.36–0.50)	0.86 (0.83–0.88)

Table 8 Sensitivity and specificity of the final tool according to month

Month	Sensitivity (95% CI)	Specificity (95% CI)
1	0.63 (0.36–0.84)	0.90 (0.79–0.95)
2	0.22 (0.04–0.60)	0.81 (0.70–0.89)
3	0.46 (0.18–0.76)	0.92 (0.82–0.97)
4	0.44 (0.21–0.70)	0.87 (0.76 −0.93)
5	0.46 (0.20–0.74)	0.90 (0.80–0.96)
6	0.47 (0.22–0.73)	0.86 (0.74–0.92)
7	0.30 (0.13–0.54)	0.88 (0.73–0.94)
8	0.62 (0.39–0.81)	0.75 (0.62–0.84)
9	0.28 (0.11–0.54)	0.86 (0.75–0.93)
10	0.21 (0.07–0.46)	0.88 (0.77–0.92)
11	0.25 (0.07–0.57)	0.81 (0.69–0.89)
12	0.47 (0.25–0.70)	0.85 (0.73–0.91)

Table 9 Monthly fluctuations in triggers and ADEs during the prospective 26-Month implementation phase

Month/Year	Number of records	Total length of stay (days)	Average length of stay per patient (days)	Number of triggers	Number of ADEs	Number of patients with an ADE	Trigger-identified ADE per 1000 patient days	Trigger-identified ADE per 100 admissions	Percent of admission with a trigger-identified ADE
February 2019	20	227	11	7	2	2	9	10	10.5
March 2019	19	136	7	5	1	1	7	5	5.3
April 2019	20	131	7	2	1	1	8	5	5.0
May 2019	20	160	8	3	0	0	0	0	0.0
June 2019	19	151	8	5	1	1	7	5	5.0
July 2019	20	236	12	8	1	1	4	5	5
August 2019	20	112	6	3	0	0	0	0	0.0
September 2019	22	215	10	6	1	1	5	5	5.0
October 2019	22	185	8	3	0	0	0	0	0.0
November 2019	22	121	6	0	0	0	0	0	0.0
December 2019	22	167	8	5	1	1	6	5	4.8
January 2020	22	256	12	6	1	1	4	5	4.5
February 2020	22	193	8	5	2	1	10	9	4.5
March 2020	22	176	8	2	0	0	0	0	0.0
April 2020	22	220	10	3	0	0	0	0	0.0
May 2020	21	180	9	6	1	1	6	5	4.5
June 2020	22	243	11	6	2	2	4	5	9.1
July 2020	22	176	8	3	1	1	6	5	4.5
August 2020	22	170	8	4	1	1	6	5	4.5
September 2020	20	211	11	6	1	1	5	5	4.5
October 2020	22	247	11	4	0	0	0	0	0.0
November 2020	22	236	11	7	3	3	13	14	13.6
December 2020	22	149	7	1	1	1	7	5	4.5
January 2021	21	134	6	1	1	1	8	5	4.5
February 2021	21	147	7	2	1	1	7	5	4.5
March 2021	20	180	9	4	0	0	0	0	0.0
Total	551	4759	N/A	107	23	22	N/A	N/A	N/A
Mean (± SD)	21 ± 1	183 ± 41	9 ± 2	4 ± 2	1	1	5 ± 4	4 ± 3	4 ± 3
Median (IQR)	22 (2)	178 (69.25)	8 (3.5)	4 (3)	1	1	6 (7)	5 (5)	5 (5)

Abbreviations: ADE, adverse drug event; AE, adverse event; ATC, Anatomical Therapeutic Chemical Classification System; BUN, blood urea nitrogen; CI, confidence interval; EHR, electronic health record; EKNZ, ethics committee “Nordwest- und Zentralschweiz”; GTT, global trigger tool; IHI, Institute for Healthcare Improvement; IHI MMTT, Institute for Healthcare Improvement medication module trigger tool; INR, international normalized ratio; N/A, not applicable; NCC MERP, National Coordinating Council for Medication Error Reporting and Prevention; NLP, natural language processing; NPV, negative predictive value; PPV, positive predictive value; PTT, partial thromboplastin time; SAP, Systeme, Anwendungen und Produkte in der Datenverarbeitung-Software; SBP, systolic blood pressure; SQL, structured query language; UK, United Kingdom; USB, University Hospital Basel; VKA, vitamin K antagonist.

Figure 3 Trigger-identified adverse drug events per 1000 patient days during the 26-month prospective study period (“tool implementation phase”).

Figure 4 Trigger-identified adverse drug events per 100 admissions during the 26-month prospective study period (“tool implementation phase”).