Understanding the limitations of retrospective analyses of poison center data

Understanding the limitations of retrospective analyses of poison center data

Poison centers provide poison prevention and safety information, professional education, and assistance with the diagnosis and treatment of poisoning. These invaluable efforts are linked to medically and legally mandated record keeping, which entails continuous and archived collection of data over time. Surrounded by gigabytes of information regarding exposures to a vast array of known and potential toxins, poison center researchers are understandably motivated to study these data. Additionally, there is a general need to participate in academic endeavors, an altruistic desire to change or improve healthcare, reduce unnecessary healthcare expenditures, and a survival advantage to justifying poison centers as valued participants in the public health arena. These powerful stimuli to analyze readily available poison center records have clearly been facilitated by significant advances in computerization that have occurred over the last few decades. Now, in minutes, with little advanced preparation or financial outlay, any poison center researcher can gather and analyze thousands of records.

Reports of poison center data appear frequently in medical journals (1–3), are often incorporated into triage and management guidelines (4–6) and occasionally are used to address consequential public health issues (7). As these studies become more prevalent, it is essential that we identify and acknowledge the strengths, weaknesses, and inherent biases associated with this methodology. A major strength is that the data are already being collected and are stored in a form that is both easy and inexpensive to search. Another strength is the ability to easily gather a very large sample size of cases of a specific exposure. Analysis of the descriptive epidemiology of the cases and trends over time can inform both clinicians and policy makers. Large case numbers lend weight to such analysis, whether or not such confidence is justified.

However, we must accept that a retrospective analysis of data collected for a purpose other than the specific study intent has methodological limitations. The data may be insufficient to answer the study question, incomplete, or frankly inaccurate (8). As an example, the presence or extent of symptoms may not be completely recorded; when no symptoms are listed, either the patient was truly asymptomatic, or the charting was incomplete. Similarly, while doses may be overestimated to create worst case scenarios that help provide conservative medical care, this clearly biases subsequent research questions. Prospective hypothesis-driven studies of poison center data can improve these shortcomings if the data collection instrument is changed to a more complete, detailed and accurate template. Discrepancies between reported and non-reported cases, differences between hospital and poison center records on the same case and some other biases and limitations in poison center data are discussed elsewhere (9–11).

Two articles in the current issue of Clinical Toxicology highlight these concerns. The first article reports on 582 clopidogrel exposures collected over several years (12). Although the authors acknowledge that the exact dose was not collected “in a consistent fashion,” they later state that the dose was “determined in 344”of these exposures and report a mean dose of 249 mg [emphasis added]. Despite only obtaining a final medical outcome in 49% of cases and having a large percentage of patients with exposures to additional substances, they conclude that medical outcomes are generally good and that these data can be used for education and prevention strategies. A poison center could create a triage guideline for clopidogrel exposures based on this type of data. One could argue, however, that with the exception of 7 patients who allegedly ingested clopidogrel alone and developed symptoms, it is unclear that any of the remaining patients actually ingested the drug.

The second relevant article in this issue discusses exactly this concern and introduces us to a term called the “unproven ingestion” (13). Children with clinically worrisome methanol or ethylene glycol exposures were referred by the poison center to hospitals. A unique situation existed where the poison center funded the analysis, which resulted in actual testing in 102 of 115 cases. Only 21 of the 102 children tested had detectable levels. Not all of these 21 children were symptomatic, and some children with symptoms had negative levels (presumably from their other ingestions, or unrelated medical issues). The authors correctly conclude that an unproven ingestion (with or without symptoms) has the potential to introduce a significant bias in the data set and conclude that laboratory confirmation of the exposure is essential.

This is a lesson that most of us have learned clinically at the bedside of a patient who swears that they have ingested an entire bottle of acetaminophen (or anything else) only to have a non-existent or trivial level. Adults, adolescents, and sometimes younger children intentionally deceive us in a poison exposure event, and parents and poison specialists often overestimate ingestions to provide good conservative care. As an example, the current iron guidelines (3) suggest that in the absence of severe symptoms patients who ingest less than 40 mg/kg of elemental iron can be managed at home. Their review of existing case reports and small (subtoxic) human studies is extensive. As is the case above, the actual dose in many of the overdose cases is unknown or unclear. Unfortunately, the details of the presentation of a small abstract were unavailable (14). For many years it was routinely stated that iron doses less than 20 mg/kg were non-toxic. However, when Burkhart and colleagues actually gave 20 mg/kg of elemental iron to human volunteers, four of six subjects required intravenous fluids and one was taken to the hospital in Trendelenburg position. Thus there is a clear difference in the outcomes of patients who actually provide a reliable history with most of the others reported in poison center data and various case reports. Fortunately, iron toxicity produces symptoms and although we can argue about the mg/kg threshold for hospital referral, the presence of clinical symptoms in patients with significant toxicity usually makes this decision easy. So in essence, the guideline is correct, even if the exact mg/kg dose selected is incorrect because patients with consequential ingestions will develop symptoms.

How do we scientifically interpret these clinical assumptions for patients with exposures to toxins with potentially delayed symptoms like sulfonylurea hypoglycemics or long-acting anticoagulant rodenticides? Many of the children reported in poison center data may have had no ingestion at all and certainly most never had the ingestion confirmed in the laboratory. Is it safe and reasonable based on a retrospective analysis of these children to conclude that the absence of reported symptoms implies that this type of ingestion is benign? In actuality, it will be benign for most cases called into the poison center because they probably never ingested the toxin. However, by including non-ingestions as cases in the denominator of our data, we are potentially misled and falsely reassured in our thinking about patient triage.

We can and should do better—not only for the science, but for our patients. The American Association of Poison Control Centers (AAPCC) has adopted a policy that informs researchers that if they use national data they must include the following disclaimer in their manuscript (excerpted in part below):

Case records in this database are from self-reported calls: they reflect only information provided when the public or healthcare professionals report an actual or potential exposure to a substance (e.g., an ingestion, inhalation, or topical exposure, etc.), or request information/educational materials. Exposures do not necessarily represent a poisoning or overdose. The AAPCC is not able to completely verify the accuracy of every report made to member centers.

Yet, poison centers are free to submit their data individually or in collaboration for publication without clearly disclosing to editors, reviewers and readers that an “unproven ingestion” bias alters the results and their interpretation toward an assumption of non-toxicity.

That being said, there is a wealth of information in poison center data, but it should be assessed, analyzed, and written with appropriate attention to methodology and an honest acknowledgment of limitations. Retrospective analyses are acceptable (and encouraged) for rare events. When attempting to study common exposures, investigators should develop a prospective methodology that first posits a testable credible hypothesis and then rigorously collects the data required to answer the scientific question, even if it is outside of routine poison center data collection fields. Additionally, investigators should strive to confirm exposures in the laboratory for at least a subset of cases. Laboratory confirmation should be considered an important factor in the peer review of the publishability of single case reports or claims of lack of toxicity or atypical and unpredictable events. Although desirable, laboratory confirmation is not essential for patients who present with classic signs and symptoms of the reported exposure. However, authors should honestly disclose as a study limitation the possibility of an exposure to a similar toxin such as amphetamine resembling cocaine. Other forms of testing the validity of poison center data—such as the linked comparison of poison center and hospital or emergency department records looking for concordance of data elements—add to the rigor of the methodology.

Since it is inevitable that we will continue to analyze the massive amounts of data that we collect, we should attempt to perform the assessment in a more rigorous fashion. While unfortunately, this implies more effort and some expense, it assures an end-product of greater value.

Notes

12. Hold place for Forrester Clopidogrel exposures in Texas

13. Hold place for the Unproven ingestion