The ODAC Chronicles: Part 6b. ODAC’s structure and function

ODAC’s origin & structure

In a previous editorial, I described the Oncologic Drugs Advisory Committee (ODAC)’s origin and structure, and compared an ODAC presentation with a trial by jury where the judge is the US Food and Drug Administration (FDA), the district attorneys are the FDA’s medical and statistical reviewers, the jury is the ODAC committee and the accused is the anticancer agent being presented [1]. The accused is defended by the pharmaceutical company staff (e.g., Chief Medical Officer, project clinician or statistician). In the US Judicial System, the accused is innocent until proven otherwise. At ODAC, the accused is guilty unless proven otherwise with a high degree of statistical significance. Contrary to the usual trial by jury, the judge has the final word and the jury is just advisory. In concluding, I stated that ‘a committee of expert advisors such as ODAC should be appropriately constituted, should include representation from true peer groups, should be utilized more effectively, should consider every anticancer agent that is a candidate for approval and its advice should be followed each and every time’. Some additional issues, as well as how the committee functions, are discussed below.

Training for ODAC members

The FDA must provide education and training to each new member of an advisory committee before the member participates in a committee meeting [101].

Advisory committee members may be trained in a variety of ways, including the review of written materials and videotapes, attendance at advisory committee meetings prior to appointment to a committee, one-on-one discussions with FDA staff, attendance at new advisory committee member orientation, and attendance at new reviewer training sessions. New members should be given the opportunity to attend at least one advisory committee meeting prior to their appointment as members.

All of the above is critical to a committee member’s participation in decisions and recommendations that require knowledge of the FDA’s laws, regulations and guidance documents. None of the above (with limited exceptions) is happening. The average committee member has only superficial knowledge of key issues, such as accelerated approval, regular approval, fast track, expedited review and expanded access. However, the FDA will frequently ask the committee to decide on whether a new agent should receive an accelerated or regular approval. During ODAC meetings, I have requested clarification by the FDA on the implications of accelerated approval versus regular approval and received less than adequate responses [102].

Not knowing the requirements or implications of accelerated or regular approval, the average committee member is ill equipped to make any recommendation. Committee members are experts in their fields (e.g., medicine or statistics) but they are not regulatory experts. They can evaluate a new agent based on its merits as a cancer therapy, based on an assessment of risks and benefits to the patient, based on clinical data; and, as experts, come to well-thought out medical recommendations. Their medical expertise does not enable them to evaluate from a regulatory viewpoint and much less make regulatory recommendations. If the training proposed above were truly implemented then committee members may be better equipped to respond to regulatory questions and make recommendations on regulatory issues. There is substantial information on the FDA website about the FDA’s laws, regulations, guidance documents and ODAC. Also, the FDA will occasionally discuss these matters in their introductory presentations at ODAC meetings. However, in the absence of a (well-designed, implemented and required) formal training program, this information exists out there in cyberspace and is close to useless.

Agenda for ODAC meetings

The agenda is prepared by the FDA. It is made public a few weeks before the meeting. Committee members have no input on the agenda and learn about it at the same time as the general public. The committee has no knowledge of the reasons why a specific agent is chosen for presentation or why other agents are approved or turned down without the ODAC’s input. The FDA does not discuss these important matters with the committee. Would the ODAC not be in a better position to advise the FDA if the committee were informed about these matters?

Data available to the committee

The information supplied to committee members consists of the ‘Briefing Document’ prepared by the pharmaceutical company and the FDA’s clinical and statistical review documents. The ‘Briefing Document’ has to be submitted to the FDA by the pharmaceutical company (the sponsor) several weeks prior to the meeting. The FDA reviews and circulates it, under confidentiality, to ODAC members. The FDA’s review documents are simultaneously posted on the website. The sponsor usually does not receive a copy of the FDA review documents prior to the meeting. The documents are not posted on the website until the day before the meeting. The general public has access to these documents on the same day as the ODAC members.

The ‘Briefing Document’ summarizes, in approximately 200 pages (including tables and references), an average of 12 years of research and development, and focuses on the last 7 or more years that represent the average clinical development time. Clearly, the document is just a summary and therefore incomplete. ODAC needs to act and make recommendations based on this and presentations during the meeting. The FDA is privy to this document when preparing their own review, arguments and presentation slides. There is no opportunity for the sponsor to prepare a defense to any criticism the FDA may present. A key segment of information, usually not supplied, is a listing of the agreements between the sponsor and FDA during development. This information (e.g., regarding choice of end points, study design and conduct and interim analyses) would give the committee important insights as to why the studies were performed in certain ways that may otherwise not be obvious. It is not unusual for the sponsor to be called to task by the committee for perceived flaws in the data or study design when these occurred by agreement with, or were mandated by, the FDA.

FDA’s questions for ODAC

On the day of the meeting, committee members will find, among the handouts provided to them by the FDA, the slides that will be used in the presentation and a list of questions that they are to discuss and on which they are to vote. The questions are many times worded in negative and leading terms. At times it seems that the FDA is asking the committee ‘How should this guilty party be sentenced?’ The committee should not be led to a certain recommendation. It should be impartial and its actions should be based on the available data. The questions, as mentioned above, may require a regulatory recommendation (e.g., is accelerated or regular approval warranted). The lists of questions may focus on the technical details of the data but frequently fail to include the most important question – should this new agent be approved and made available to patients who may benefit from it?

For example, at the ODAC meeting on Genasense® (Genta Inc.), the first question on the FDA’s list was ‘…does the committee believe that the small differences in response rates (<5%) and in PFS (difference in median days between arms – 13 days, p = 0.006) represent real effects of Genasense, when added to DTIC?’ I requested that – ‘The question needs to be worded differently since the way it is worded it is biased towards the analysis of the data by the FDA. I think we need to consider as a committee both the FDA’s analysis as well as the sponsor’s analysis. So, I would say that the qualification of the differences as small should be taken out and the 13 days, which comes from the FDA analysis, should be taken out’ [103]. The FDA (Robert Temple) agreed and the ODAC chairperson (Donna Przepiorka) stated, ‘Thank you for accommodating this need’ and that ‘…a more unbiased question perhaps would be…does the committee believe that the observed differences in response rate and progression-free survival represent real effects of Genasense when added to DTIC?’ The FDA need to be careful in drafting these questions. They must be written so that they are not leading or introduce any bias.

At that same meeting, noting that none of the four questions asked whether Genasense should be approved, I requested that ‘…at the end of the session today we really have to address question number five…should Genasense be approved and made available to the patients who need it? We need to give a recommendation on whether or not there is an effect and if that effect is important enough to merit approval of this agent, and that question is not asked so I would ask that we add that as question number five’ [104]. Why is this most important question not asked routinely? There is always the possibility that response to technical questions may be negative and yet the overall assessment of medical benefit may favor approval. In fact, specific technical questions may be interpreted to invite a negative response. The FDA should be careful to ensure that there is not even the appearance that the questions are being written in this fashion. There should be a clear focus on patient benefit. The technical details of clinical trials are important but should not be interpreted in isolation when making this overarching decision. Clinical research is not as perfect, precise or flawless as we would like it to be. The p-values are not the only consideration either. In the end, one must decide on the basis of good clinical judgment and experience if a new agent (warts and all) will benefit patients, and one must then act accordingly.

Temporary voting members & consultants

The FDA may choose to invite other experts (members of other advisory committees) to an ODAC meeting and to grant them the right to vote when expertise is required that is not available among current voting standing members of the committee or to comprise a quorum. Up to ten such temporary members may be added to the ODAC. Additionally, the FDA may choose to invite expert consultants and also grant them the right to vote. Why would up to ten additional members be needed? ODAC members represent a broad expertise encompassing a variety of different cancers. When even more specific expertise is required, would it not suffice to invite two or three other experts? If more than that are invited, the number of temporary members and consultants could be sufficient to control the committee’s vote and recommendations. The FDA should refrain from this as it will give the impression that they are trying to sway the committee’s vote. However, why should these other experts and consultants have a vote? They may be participating in an ODAC meeting for the first time. They are not regulatory experts and receive no training prior to their participation. They are not knowledgeable about ODAC’s charter, responsibilities or procedures. They should have a voice but not a vote.

Conducting an ODAC meeting

In the course of a meeting, the sponsor will present the appropriate clinical data and the FDA will present their clinical and statistical assessment. This is followed by discussion where committee members ask questions of both the sponsor and FDA. Subsequently, the questions submitted by the FDA are discussed and put to a vote.

ODAC meetings are conducted by the committee’s chairperson. Members are usually asked to raise their hands or signal the chair when they wish to speak. The executive secretary keeps a list and advises the chair on who has the next turn. Generally, the meetings are orderly in that members will not interrupt others and will wait their turn to speak. These rules do not seem to apply to the FDA staff. They will speak out of turn, interrupt when others are speaking and immediately comment (without requesting a turn) when a member has made remarks not to their liking. The committee’s chair allows this to happen. These interruptions and comments are particularly unfortunate when they occur during a discussion around a vote or during the voting itself.

My own experience has been as follows: 19 of the last 30 times I have spoken at ODAC meetings my statements have been immediately followed by a statement from the FDA. On five of these occasions, two members of the FDA staff spoke up. On four of these occasions, I was interrupted before I could complete my statement.

At the meeting on prostate cancer end points, I was interrupted by Robert Temple [105]. I was asking the question ‘Why is there not more interest in developing new agents for prostate cancer?’ And stating that ‘In part, it may be the hurdles that have to be overcome in getting those agents approved’. Robert Temple objected to my statements since ‘there may just not be any drugs around’ and that ‘we don’t know whether it is the difficulty’. Following Robert Temple’s interruption I raised two points. First, that he had interrupted without asking for a turn to speak. The chairperson had specifically requested that those wishing to speak should raise their hands to request a turn. Second, since he had said that I had ‘no evidence to support’ what I was saying, I needed to reply to that statement. The chairperson, Maha Hussain, allowed turns to Robert Temple, Grant Williams and Otis Brawley before allowing me to speak. My statement was as follows ‘I am glad to see the FDA jumping in and commenting every time I say anything, which means that what I am saying is important enough and/or controversial enough to merit a response from them even if they don’t raise their hands and ask for a turn’. The facts are that there are many drugs available that could be studied for prostate cancer but most pharmaceutical companies are not considering this indication due to the regulatory hurdles, particularly the issues around end points. Due to this, they choose to focus on other indications with less difficult paths to approval.

At the meeting on Genasense, I was interrupted in the middle of a statement once by Richard Pazdur and on another occasion by Robert Temple. During my next turn as I was speaking, I noticed that Richard Pazdur had turned on his microphone and was about to interrupt me (third interruption that day) and I stated ‘…don’t interrupt me; I am not finished, Dr. Pazdur. Please turn off your microphone. Let me talk. You interrupted me once before and that is enough. Okay?’ Well, that got their attention and I was allowed to speak uninterrupted [106].

The real question is, why all these interruptions by the FDA? Are they providing clarification? Are they answering questions? Or, are there ulterior motivations? One might think they are trying to lead the discussion in some predetermined direction for their own purposes. In any case, it is only good manners to allow a person to finish what he is saying without interruptions. This is particularly important when that person has an advisory role and you have invited him to provide opinions, guidance and recommendations. Should you not politely listen to what is being said? If you disagree you can always ask for a turn at the microphone. If the meeting is being conducted by a chairperson and all around the table are expected to request a turn at the microphone, should the FDA not abide by those rules?

Meeting minutes & transcripts

ODAC meetings are generally open to the public. They are web cast in real time and are also filmed and later made available in CD or VHS format. The proceedings are recorded and later transcribed. The transcripts (∼300 pages per half day of a meeting) are made available on the FDA’s website. However, these are difficult to navigate as they are not well indexed. Fortunately, in PDF or MS Word format, one can do a search for keywords and find the information one is seeking. My impression is that very few will read these transcripts. The minutes of the meetings are expected to provide an executive summary of the proceedings and some detail on the voting and recommendations. Unfortunately there have been no minutes posted on the website since the meeting of December 2003. Minutes are supposed to be generated by the executive secretary and circulated to the committee members for review and comment. This has worked well in the past and could continue to be a really valuable record of these meetings. Why is it not happening?

Conclusions

ODAC members should have formal training on those regulatory matters where their opinion is sought. This is a simple issue, you train and inform them or you do not ask them to opine on regulatory issues, one or the other. If ODAC members are not trained, then the questions posed to them should be limited to their areas of expertise – does the data on this new agent show clinical benefit for the patient? Not ‘should this new agent receive accelerated approval?’ Personally, I am strongly for their being formally trained so that they can better advise the FDA. They should evaluate each and every new agent filed with the FDA for approval. Their advice and recommendations should always be followed by the FDA. Their participation in the review and approval process should start at least at the time of the end of Phase II meeting and optimally as early as the first meeting where the sponsor discusses clinical development with the FDA. They must have input on meeting agendas and minutes. Meeting transcripts should be made user friendly through appropriate formatting, indexing and book marking. Their access to data should occur earlier. ODAC members should be allowed to meet as a group, without FDA oversight. The number of nonmember experts and consultants invited by the FDA to ODAC meetings should be kept to a minimum. These invitees should not have a vote. The ODAC chairperson should be empowered to effectively chair each meeting without FDA intervention. Meetings should be conducted following the rules of order. Interruptions should not be tolerated. The FDA should refrain from any action that could, even remotely, give the impression that they are manipulating the ODAC. All of these recommendations (and others mentioned in prior editorials) can be enacted at the FDA’s discretion under the existing regulations [1–3]. The responsibility for bringing about these important changes lies with the FDA’s leadership. If they want ODAC to be truly effective, they can make it happen.

‘Transformation begins with a sense of crisis and urgency. No institution will go through a fundamental change unless it believes it is in deep trouble and needs to do something different to survive.’

Lou Gerstner – Chairman, IBM

Information resources

Additional information and background is available at the FDA’s website www.FDA.gov (Accessed September 2005)