Parallax View

Parallax: The apparent displacement of an object by a change in the position from which it is viewed.

Due to significant variation in clinical practice, multiple societies and commissioned groups have developed and published clinical practice guidelines. Potentially, the guidelines will standardize clinical practice and thereby improve patient outcome, increase cost-effectiveness and serve as an educational tool. Ineffective guidelines may waste resources or worse, harm patients 1&2. But how do we explain the phenomenon when two groups address the same clinical practice question and make different treatment recommendations 3? Did one of the groups draw erroneous conclusions?

Practice guidelines, according to the Institute of Medicine, are defined as "systematically developed statements" in which the recommendations are the result of the stated methodological process. The assumption is that all guidelines are based on the best available evidence 4.

However, there are many methods by which one can develop guidelines and each has its strengths and its weakness.

Consider Evidence Based Medicine (EBM) as a methodology for guideline development. EBM recommendations are based on the best available external clinical evidence. Emphasis is placed on systematic observation, rules of evidence, and critical interpretation of the literature with less value placed on physiologic principles and clinical experience. EBM emphasizes empirical knowledge (studies) over rational thinking 5. The Randomized Controlled Trial (RCT) is considered the gold standard as RCTs demonstrate efficacy of treatment for the "average" randomized patient chosen from a carefully selected group of study patients. If no randomized trial has been performed, the next best external evidence is used. There may be huge gaps in the evidence. The reliance on published studies means that the EBM methodology is susceptible to the publication bias against "negative" studies.

Another method by which to develop a guideline is consensus 2. Consensus methods attempt to assess the extent of agreement and resolve disagreement. In contrast to EBM methodology, expert opinion without explicit critical appraisal, as well as conclusions based on physiology or bench research are included as part of the basis for recommendations. Although some guidelines are not commissioned by a professional organization, to most people, the term "consensus guidelines" implies that the work has arisen from a committee of experts selected by the leaders of a society, and that the document has been thoroughly vetted to assure that conclusions are unbiased and appropriate. There is an assumption that a specialty society oversaw the development of the guidelines to ensure that the recommendations reflected the consensus of more than just the few authors 6.

Two different groups of clinical toxicologists have developed a practice guideline regarding the indications for administration of Ipecac. The first Ipecac guideline (Position Statement) was developed by the American Academy of Clinical Toxicology (AACT) and European Association of Poison Centres and Clinical Toxicologists (EAPCCT), published in 1997 and updated in 2004 7&8. The second Ipecac guideline, developed by a Consensus Panel, is published in this issue of the Journal 9. The two guidelines differ in recommended indications for the administration of Ipecac.

The two guidelines originated as a result of different perceived needs, although both guidelines were developed in response to an observed variation in clinical practice. In 1993, the Presidents of the AACT and EAPCCT noted the variation in the clinical practice of GI decontamination and developed an EBM methodology to write Position Statements. All relevant scientific literature was reviewed. Clinical and experimental studies were given precedence over case reports. Abstracts and expert opinion were not considered. A draft was then produced and peer-reviewed by an international group of clinical toxicologists chosen by the two societies. The guidelines were then discussed at open forums at the NACCT and the EAPCCT International Congress. Peer review and approval by both Boards of Trustees were required. These guidelines were evidence based and did not rely on consensus opinion. These guidelines were intended for the inhospital treatment of the poisoned patient 7.

In 2001, following institution of a toll-free 800 number for all US Poison Centers, Public Health Analysts noted the wide variation in prehospital recommendations between Poison Centers. Due to the potential for Poison Centers to answer each others' calls, as well as the potential to confuse the public, a seven member committee was appointed and supported by the federal government grants to develop consensus guidelines. The American Association of Poison Centers (AAPCC), AACT, and American College of Medical Toxicology (ACMT) nominated members. Additional members were subsequently added by the committee itself.

The Consensus Panel used a modified Delphi process. In the Delphi process, opinions are grouped or ranked,rankings are summarized and participants then rerank. The rankings are then assessed for degree of consensus. In this Consensus Panel, an individual (separate from the Panel) performed a structured search of the literature, reviewed every article and assigned a level of evidence. The panel then reviewed the summary with option to request additional information. Full articles were available for review by panel members. Assigned members then wrote a draft opinion that was reviewed by panel members using a modified Delphi consensus process. Comments on the draft were submitted to the Panel Chair who collated them, blinded them and presented them to the entire Panel for comment (strongly agree, agree, disagree, strongly disagree). Panel voting was also blinded. Consensus was defined by agreement of 6/8 of the panel members (A SPI liason had been added by the panel). Revised guidelines and comments continued to recirculate until consensus was reached. If any member objected strongly to any component, that component was revised. All comments from member of contributing organizations were addressed by the group. When a draft reached consensus, a secondary review panel of representatives from public health, emergency services and consumer groups provided additional feedback 9. Board of Trustees' approval was not obtained from AACT, AAPCC, or ACMT. In fact, the AACT Board of Trustees voted not to support these Consensus Guidelines (only one Trustee voted to approve). These guidelines were intended for prehospital care of patients with certain ingestions.

BUT, there is little scientific evidence in the toxicology literature on which to base clinical practice guidelines. No matter what the methodology used to draft clinical practice guidelines, there is just not much evidence. Therefore, other factors must have impact on the conclusions drawn by the groups writing clinical practice guidelines.

Firstly, practice guidelines are not based on data alone and depend on interpretation of the evidence and on clinicians' experience 10. Interpreting the literature involves value judgments. The scientific basis of the judgments can be understood if the process is clearly stated.

Secondly, many factors influence the working process of groups. Not surprisingly, there is "evidence" that group processes are influenced by the chairperson. Composition of the guideline development panel and methodology of guideline construction play a role in determining the end result 11&12. Sometimes, groups take on a life of their own and do not adhere to either the agreed methodology or the purpose of the group. Many practice guidelines developed by specialty societies do not include a description of the professionals undertaking the guidelines, the type of searches performed or the grading of recommendations 13. Perhaps most concerning, less than half of the guidelines developed do not adhere to the methodologic standards and format that had previously been agreed 1. We can conclude that no matter what the methodology, writing guidelines is far from standardized.

There are inherent problems with all methods used to write guidelines. Some of the problems with the conclusions reached by the process of consensus and EBM guidelines are as follows:

The reaching of "consensus" does not mean the "correct" answer has been found-there is the danger of deriving collective ignorance rather than wisdom. Consensus results should be matched to observable events. Unless the findings can be tested against observed data, we can not be sure that the methods have produced the best answer 14. In some consensus groups, there is no evidence of collection of adverse events as a result of the recommended treatment; there is no risk:benefit analysis. In clinical toxicology, this is of even more concern as we know that current available data does not include the total number of adverse events or even the total number of poisonings.

Conversely, conclusions based solely on EBM do not guarantee the best method of practice has been elucidated for a subgroup of patients or for the patient for which the clinician is caring. The applicability of the evidence to a particular patient depends on the individual being conformable to the group for which the practice guideline was developed in all relevant aspects. Knowledge, judgment and clinical experience are required in situations where the patient does not meet the attributes of those studied 15&16.

Therefore, it should not be surprising that two groups with different methods and different people drew different conclusions. Note that one set of guidelines was intended for inpatient care and the other set of guidelines was intended for prehospital care of poisoned patients. The following are some of the differences between the two processes.

Table

Consensus Guidelines	Position Papers
Modified Delphi	EBM
TESS data used	TESS data not used
Membership comments	Membership discussion at symposium
Included abstracts, case reports and expert opinion	Did not include abstracts or expert opinion
Secondary review by panel of agencies	Board approval by EAPCCT and AACT
Participants paid	Participants not paid

What will be the effect on clinical practice when conclusions from two groups of toxicologists are somewhat at odds? Different recommendations on the same topic may cause some confusion and cause selective adoption of practices that reflect preferences and prejudices rather thanevidence-based judgment. Are there enough similarities in the recommendations to prevent confusion and selective adoption of practices? Both guidelines recommended limited use of Ipecac. It was the limits placed on the indications for administration that were different. The AACT Position Statement made a general statement that Ipecac had not been shown to change outcome and should not be routinely administered to the poisoned patient 7. The Consensus Paper stated that there may be certain times when ipecac syrup may have an acceptable benefit:risk ratio 8, but then listed so many caveats, that in reality, it is unlikely that there are many (if any) situations where one would consider administration. The recent recommendation by the American Academy of Pediatrics to discontinue the use of Ipecac makes it even more unlikely that Ipecac will be administered.

Guidelines are guidelines, not dictums. That the recommendations differ must make us realize the limitations of the recommendations. These guidelines are not "standard of care" and have no place in a court of law. In addition to the guidelines, the clinical toxicologist has the knowledge, judgment, and experience on which to make decisions regarding patient care.

Two groups of people using different methodologies made different practice recommendations based on a significant lack of evidence. It is not surprising that the recommendations changed when the position changed from which they were viewed.

Donna Seger, M.D.

Medical Director, Middle Tennessee Poison Center

Assistant Professor of Medicine and Emergency Medicine

Vanderbilt University Medical Center

Immediate Past President

American Academy of Clinical Toxicology