Trial registration and results disclosure: impact of US legislation on sponsors, investigators, and medical journal editors

ABSTRACT

Background: Recently enacted federal legislation, the Food and Drug Administration Amendments Act of 2007 (FDAAA), mandates public registration and disclosure of results of “applicable” clinical trials of drugs, biologics and devices on www.clinicaltrials.gov. The law calls for registering more information about more trials than has been the policy of many medical journal editors to date. Beginning December 2007, results disclosure will occur in three stages initially with links to information from the FDA and NIH about FDA-approved products, as well as to Medline citations. A Basic Results Database will appear in September 2008, and an Expanded Results Database two years later. Results for trials of FDA-approved products must be posted within 12 months of trial completion. Such postings will not be peer reviewed or contain explanatory text or discussion. Sponsors who file regulatory submissions (IND, NDA, 510(k), PMA, etc.) to the FDA must certify compliance with FDAAA's registration and disclosure provisions. The FDA's determination of such compliance will be made public, and if non-compliance is not cured within 30 days, sponsors may be fined up to $10,000/day.

Conclusions: FDAAA requirements differ in a number of ways from those of the International Committee of Medical Journal Editors (ICMJE) and other journal editors, creating potential conflicts for sponsors and investigators who must comply with the law, and a need for better alignment of editors' policies with the law. The public will have access to massive amounts of clinical trial data as a result of FDAAA but it is unclear this will be useful to patients and prescribing physicians.

Key words: :

• Clinical research
• www.clinicaltrials.gov
• FDAAA
• Food and Drug Administration Amendments Act of 2007
• Title VIII
• ISMPP

Introduction

“May you live in interesting times.” – Chinese curse.

This aphorism applies to those involved in clinical research, particularly study design and the planning/development of manuscripts intended for peer-reviewed publication, as well as the editors of medical journals who shape the content read by physicians (and patients) globally. The recently-enacted Food and Drug Administration Amendments Act of 2007 (FDAAA) or Public Law 110–851, has implications for clinical research – whether conducted in the U.S. or elsewhere and regardless of funding source – and especially for reporting of clinical trials. What have been issues of technical/procedural interest to medical journal editors, academic and industry scientists and clinical investigators are now regulated with the force of federal law. This legislation is complex, generating a veritable cottage industry of conferences, webcasts and seminars to explain the law to its many stakeholders. This commentary will review passage of FDAAA and some of its mandates; contrast FDAAA with policies now in place under the International Committee of Medical Journal Editors (ICMJE); discuss a number of implementation-related issues as-yet undecided in the early months since the law's enactment September 27, 2007; and raise questions about the benefits of FDAAA to various audiences.

FDAAA: factors leading to its passage

A recent editorial by Dr. Trish Groves in British Medical Journal describes results disclosure under FDAAA from the perspective of a major medical journal2. Groves's commentary appears to have been stimulated by her participation in the January 17, 2008 conference on FDAAA hosted by the International Society for Medical Publication Professionals (ISMPP) in Washington, DC, USA. Presentations and discussions at this conference (information available at www.ismpp.org) among Congressional staff who developed the legislation, editors at two ICMJE journals, spokespersons for the FDA and NIH-National Library of Medicine, legal counsel and industry representatives identified numerous challenges for all parties involved in clinical trials and their publication.

The law has far-reaching components, including renewal of user fees from the pharmaceutical and device industries under Prescription Drug User Fee Act (PDUFA) and Medical Device User Fee Act (MDUFA), as well as important new authority given the FDA to improve post-marketing safety surveillance of marketed products. The FDA now has explicit power to require post-approval studies or trials by drug manufacturers and to mandate changes to drug labels when it deems necessary1. However, the major interest in FDAAA from a clinical research perspective relates to Title VIII – Clinical Trial Databases.

The events leading to passage of Title VIII are debatable, but Congress’ main intention was to improve transparency of clinical research. Senator Charles Grassley (Iowa) testified in hearings in the U.S. House of Representatives in February 2007, prior to passage of the legislation:

“Last month, Senator Christopher Dodd and I introduced two reform bills … to fix the safety shortcomings at FDA. Our first bill would elevate and empower the office with[in] the FDA that is responsible for monitoring FDA-approved drugs after they are on the market … . The second bill … would expand an existing public data base by mandating the registry of all clinical trials and the results of those trials. This reform is key to establishing greater transparency regarding clinical trials, the good ones and the bad ones, and to hold drug makers and drug regulators accountable and to give doctors all the information they can to their patients3.”

Those remarks reflect the common perception that “industry” routinely acts to suppress or delay publication, or partially reports trial results that do not favor a company's intervention 4, 5. Indeed, Groves's lead statement referred to reports of apparently biased publication of anti-depressant trials6 and delayed disclosure of a surrogate-marker study of a cholesterol-lowering medication7 as “more of the same.”1 Readers should consider that the authors of the anti-depressant publication study stated themselves that they do not know whether the publication bias is due to failure to submit negative studies or medical journals’ refusal to publish them6. The FDA statement on the Vytorin-study criticisms describes the lack of medical significance and of general applicability of the findings8. In any case, I believe such events are exceptions to the thousands, if not tens of thousands, of industry-sponsored clinical trials that have been conducted and reported in routine fashion in the peer-reviewed literature over the last 1–2 decades – but agree that any attempt to interfere with disclosure of valid clinical trial results is a violation of public trust and ethical obligations to the study participants, and cannot be condoned.

How FDAAA compares to ICMJE policies

While the legal requirements of FDAAA differ from the policies of the ICMJE today (see Table 1), academics and editors of the ICMJE advocated passage of FDAAA, and some have commented enthusiastically about its passage, saying, “With this legislation, clinical trials in the United States will be played out in the public arena9.” One can't help but wonder though if this isn't an example of, “Be careful what you ask for–you just might get it.” FDAAA makes little distinction between commercial, academic and government sponsorship of clinical research in terms of the requirements to both register trials and publicly disclose trial results (and it appears to apply to trials conducted outside the US as well, given that they meet the definition of an “applicable clinical trial,” explained below).

Table 1. Comparison of FDAAA and ICMJE policies

Issue	FDAAA	ICMJE
Which trials must be registered	“Applicable” trials of drugs and devices subject to FDA regulation – excludes drug Phase 1 and device feasibility studies	All prospective, interventional human trials
When to register	Within 21 days of start of enrollment	Prior to start of enrollment
Where to register	www.clinicaltrials.gov	Primary register per WHO ICTRP* criteria, including www.clinicaltrials.gov
What to register	51 elements; enrollment status to be updated within 30 days of change, at least every 12 months	20 elements from WHO ICTRP originally; since expanded
Public disclosure of trial registration	Within 30 days, except for pre-market investigational device trials – delayed until FDA approval	Immediate, after quality control
Results disclosure	Links to FDA and NIH information by 12–26-2007 Basic results by 09–27-2008 Expanded results by 09–27-2010 Required by 12 mos after LPV^†	Not stipulated, but posting results as a brief ( < 500 word) abstract or table is not considered prior publication
Peer review of data	No	Yes
Certification of Compliance	Required for sponsors of regulatory submissions to FDA, and applicants for research grants from NIH, FDA	Trial registration per ICMJE criteria required for manuscript to be considered for publication

* WHO ICTRP = World Health Organization International Clinical Trials Registry Platform

† LPV = Last Patient Visit for 1° outcome data collection in trials of FDA-approved products

At a basic level, it would be regrettable if a high-quality, multi-year, multi-center, inter-national clinical outcome trial for a promising new drug (or new indication) was rejected for publication because it had been registered 8 or 15 days after the first patient enrolled. While the ICMJE has successfully developed policies to encourage clinical trial registration since 200510, only in mid-2007 did it mention results publication and indicate its willingness to consider trials that had limited results previously disclosed as one table or abstract ≤ 500 words posted on the same registry where initial registration occurred. ICMJE also indicated that future events might require changes in its policies on trial registration – reflecting the legislative efforts (prior to passage of FDAAA) underway in Congress at the time¹¹. FDAAA goes far beyond ICMJE- and indeed, peer-review norms by stipulating requirements for trial registration and three stages of public disclosure of clinical trial results – both content and timing – on www.clinicaltrials.gov

First, the NIH and FDA have begun providing links to publicly available information about pivotal and post-marketing trials of FDA-approved drugs and devices on www.clinicaltrials.gov. These include FDA reviews, Advisory Committee meeting materials, Medline citations, summary bases of approval for drugs and summaries of safety and effectiveness for devices.

Secondly, starting in September 2008, a Basic Results database must show results of applicable trials initiated since September 2007 of FDA-approved products including baseline and demographic characteristics, participant flow, primary and secondary outcomes including statistical significance, a point of contact and whether any agreements restrict the ability of the principal investigator to present or publish the results. Industry sponsors now wonder if a fairly standard agreement that a sponsor must receive a manuscript or abstract for review purposes prior to submission for publication or presentation – not for sponsor approval – constitutes such an agreement? If investigators in a multi-center trial are prohibited from publishing their center's results prior to the complete study data being published, is this a restrictive agreement? Results are to be posted within 12 months of the estimated or actual trial completion date, whichever is earlier [sic]. Trial completion is defined as the date of the last patient visit for collecting data on the trial's primary outcome. Since most trials do not complete prior to the date originally anticipated, this stipulation may create logistical dilemmas for investigators; at a minimum anticipated trial date entries on www.clinicaltrials.gov will need to be updated regularly. Moreover, collection of longer-term, blinded secondary outcome measures may be impaired once primary outcome results are publicly disclosed.

Regulations for reporting adverse events must be developed by March 2009, or they will be posted as now prescribed in the law. Extensions of the timeline for results posting will occur for products not yet approved for any use to 30 days after initial FDA approval, and are also possible “for good cause.” Seeking peer-reviewed publication is not listed as “good cause.”

Thirdly, regulations will be developed for the Expanded Results databank by September 2010, with a public meeting to obtain stakeholder input in March 2009. Some questions that will be considered include possible development of lay summaries of trials, whether to post results of studies of unapproved drugs and devices, updating posting requirements, quality control, etc. Importantly, information posted on www.clinicaltrials.gov is neither peer-reviewed nor reviewed in advance by FDA, and will have no interpretive discussion. A comprehensive review of trial registration and of the challenges of results disclosure on www.clinicaltrials.gov was recently published12.

Implementing the legislation – who will benefit?

Several questions remain in the aftermath of Title VIII: Who will ensure the validity of posted data and who will ultimately benefit from this massive data reporting? Meta-analysts will probably find the information posted of some value. Although unintended, plaintiffs’ attorneys and perhaps State Attorney Generals are another group likely to scour results posted; opinions will doubtless differ as to whether the latter is “beneficial,” and if so, to whom.

It is hard to imagine that patients or practicing physicians – both the audiences intended by Congress as beneficiaries of Title VIII – will read through tables of data without accompanying discussion or commentary. Even assuming they do so, will such information be useful to these audiences? At the ISMPP conference, Dr. Donald Lindberg, Director of the National Library of Medicine said, “We know how to register clinical trials. We don't know how (or how best) to post results – and we don't know how to do so for the public.” Theresa Toigo of the FDA's Office of Special Health Issues raised a similar concern, saying, “Transparency is a worthwhile goal – but we want patients and healthcare professionals to be able to use and rely on the posted information.”

Will a patient or physician assume that study results posted on www.clinicaltrials.gov were approved by the government? Might there be adverse public health consequences of posting non-peer-reviewed study results per FDAAA (by way of the law of unintended consequences)? This is not alarmist speculation; peer-reviewed publication of major clinical trials can be followed by inappropriate prescribing decisions by practicing physicians with resulting harms 13, 14s. Posting of trial results in a format that “… intentionally omit[s] any place for interpretation of the trial's results” may have greater potential for such effects9.

New challenges for peer-reviewed medical publication

For these reasons, peer-reviewed publication must remain the standard for communication of clinical research findings. Yet, the differences between FDAAA and ICMJE policies have consequences for editors of peer-reviewed medical journals. A major question after FDAAA is whether posting a trial's results in accord with the law will cause editors to invoke the Ingelfinger Rule on prior publication (embargoing dissemination or public discussion of a trial's results prior to publication in the peer-reviewed medical journal).

It is unusual to have a clinical trial published less than 12 months from last patient visit – data queries and database lock, data analysis, interpretation and manuscript preparation can take months; peer-review and revisions or reanalysis likewise, not to mention the time from acceptance to actual publication. Most trials are simply not sufficiently novel, “… important and robust …” to achieve fast track review in a top-tier journal, Groves's seeming answer to concerns about timeliness of peer-reviewed publication versus results posting1. Practically, consider that acceptance rates at the top medical journals today are in the range of 10 % or less – and to receive rapid-track or fast-track publication, a manuscript has to be extraordinarily unique and/or impactful. This probably only happens to 10 % of accepted papers – i.e. roughly 1 % of submitted manuscripts.

It is encouraging that two senior editors at ICMJE journals (Annals of Internal Medicine and New England Journal of Medicine) have publicly stated at recent meetings that they would not consider compliance with FDAAA to violate prior publication restrictions (as did Groves for the BMJ1) – but this is not current ICMJE policy.

Further, there have been instances of other journal editors rejecting manuscripts solely because the trial sponsor posted a summary of results on either a company website or in the public database administered by PhRMA - www.clinicalstudyresults.org – in an effort to demonstrate transparency15. Frank Rockhold of GSK also expressed these notions at the ISMPP FDAAA conference. These cases may become commonplace unless ICMJE editors adopt policies more consistent with the law, add them to the Uniform Requirements at www.icmje.org, and communicate such changes to the editorial community at large. Other editors will need to clearly convey their approaches in Instructions to Authors. It is disappointing that an early version of FDAAA passed by the U.S. Senate specifically allowed for delayed results disclosure on www.clinicaltrials.gov while awaiting peer-reviewed publication; the enacted law does not.

Many persons in “industry” are seeking guidance from NIH and FDA on details necessary for compliance with the legislation. Meanwhile, given the speed with which FDAAA was enacted following its passage, the NIH and FDA are themselves working to understand and implement the law. For instance, Congress recognized there are real differences between development of drugs and devices, and FDAAA specifies delayed public disclosure of registration of applicable device trials until an [investigative] device is approved or cleared for marketing by the FDA. It is entirely possible that a manuscript describing the pivotal trial(s) for a new device could be submitted for publication prior to FDA clearance. The study would have been registered, but not yet posted publicly on www.clinicaltrials.gov, because the NIH and www.clinicaltrials.gov may interpret the law as prohibiting public disclosure of the trial prior to FDA clearance – even if the trial sponsor requests public posting. Will the manuscript then not be considered by the journal, because its registration was not publicly accessible?

FDAAA and academia: boon or boondoggle?

Individual investigators, including many at academic institutions within and/or outside the U.S., may have little inkling of the extensive changes wrought by FDAAA. At the ISMPP conference, academic trialist and professor, Dr Ken Jamerson of the University of Michigan School of Medicine indicated general support for FDAAA, but expressed strong concerns that the legal requirements and short timelines for posting trial results could have the unintended and negative consequences of eroding the academic mission and potentially interfering with peer-reviewed publication, including having the time to teach residents and fellows about the full peer-reviewed publication process. He was also uncertain about the implied re-analysis of posted study results by others.

Consider the perspective of an investigator who may have spent years of his/her career developing preliminary information about a disease or condition, and/or a medical intervention or approach to treatment. He or she succeeds in obtaining funding and conduct a large clinical outcome trial, which is published in a major scientific journal. The results are posted per FDAAA, perhaps even prior to normal publication. Other researchers, some of whom may have never received funding for study in the field, may then combine data summaries for analysis of tangential measures, or reanalyze results by a completely different analytical method. A whole industry of “second-guessing” expertise may be created. How will the investigators who spent years of work leading to the development, conduct, and reporting of such trials react to this? How will such efforts be perceived within academia?

Certainly, meta-analysis of relatively homogeneous clinical trials with similar designs and endpoints is a powerful tool, often the best way to quantify treatment effects – both favorable and adverse. However, it is a two-edged sword. The recent controversy over the cardiovascular-risk profile of the thiazolidinedione rosiglitazone for type 2 diabetes is a case in point. Information about many small clinical trials was publicly available and combined with a few large studies to perform a meta-analysis of myocardial infarction and fatal cardiovascular disease (CVD) outcomes: The conclusion was that rosiglitazone increased the risk of myocardial infarction (MI) and fatal CVD outcomes (although the latter was not statistically significant)16. When published, there was tremendous public reaction followed by condemnations of the manufacturer and the FDA. Questions were immediately raised about the methodology used, including the relative weighting of trials of varying size and treatment duration, and the exclusion of trials with zero events in both treatment and control groups, with new analyses published. The most recent meta-analysis re-analyzed the same 42 trials as in the first one in multiple ways using continuity corrections, and showed lower odds ratios that were not statistically significant17. These authors concluded that the risk of MI and fatal CVD from taking the drug for diabetes is uncertain, and in any case, did not justify “ … what the authors of the original meta-analysis (as well as the media, the U.S. Congress, and worried patient groups) decried as an ‘urgent need for comprehensive evaluations.’”17 The title of an accompanying editorial says it all, “Rosigilitazone: A thunderstorm from scarce and fragile data.”18 The point here is not whether the drug does or does not have an adverse cardiovascular profile. Rather, this is a prime example of the type of public (and political) consternation that can follow re-analysis of data collected by others, often in trials performed for very different purposes, by researchers who were not involved in the design or conduct of the underlying studies.

Beyond this, it has always been possible for investigators not involved in the performance of a clinical outcome trial to request and receive access to a trial's complete electronic database for post hoc analytical purposes. However, without close interaction with the study sponsor and/or principal investigators, secondary investigators may misunderstand data coding or database formatting and produce analyses that result in published articles that are simply erroneous19. One such case required retraction of an article post-publication. The likelihood of such events would only seem to be increased if based on summary data posted online.

Conclusions

The primary role of peer-reviewed publication for medical research communication should not be diminished by the coming availability of clinical trial outcomes data on the internet, but there will be other consequences, as discussed above. Analogous to how Winston Churchill described democracy, peer review is far from perfect, but it is the best system we have. There are now important differences between the peer-reviewed medical publication process and the legal requirements for disclosure of trial results on www.clinicaltrials.gov. In order to avoid the inevitable conflicts that will otherwise occur between compliance with the new law and attempts to achieve peer-reviewed publication, many stakeholders including ISMPP hope that ICMJE will consider modifying its policies on both trial registration and prior disclosure of results. Education of non-ICMJE editors will be especially important for ensuring consistency in handling manuscripts for publication as well. FDAAA has irrevocably changed the process of clinical trial reporting.

“Like it or not, we live in interesting times.” – Robert F. Kennedy, Capetown, S.A., 1966.

Acknowledgment

Declaration of interest: LH moderated the International Society for Medical Publication Professionals' Conference on Food and Drug Administration Amendments Act of 2007 on January 17, 2008, which stimulated this commentary. He takes responsibility for its content. LH is currently VP of Medical Affairs for a unit of a medical device company, and previously spent > 15 years in clinical drug development at a major pharmaceutical company, including several years managing registration and publication of clinical trials. These experiences have influenced his perspectives, but he declares no conflict of interest. No one at his current or former place of work has reviewed, approved, or edited this Commentary.