Poor compliance with reporting research results – we know it’s a problem … how do we fix it?

Summary

If patients knew that most results from clinical research were not shared as completely or as quickly as they should be, how many patients would volunteer for research studies?

If patients knew this problem was worse for government-funded research than pharmaceutical industry-funded research, how many would want their tax dollars allocated to government-funded research?

We may live in an era of evidence-based medicine, but when the accessible evidence base is being built so incompletely and slowly, patients and the medical research community clearly have a problem. Indeed, despite the ethical, scientific, and potentially, legal, responsibilities researchers have to report results in a timely manner… they are struggling to do so. And, although much attention has focused on results reporting for industry-funded research, it appears that the situation for non-industry-funded research is similar or far worse. Nobody appears to occupy the moral high ground when it comes to reporting results, but all funders and researchers have a moral duty to do so. Quite simply, results don’t report themselves. We propose that professional medical writers (not ghostwriters) could help ensure results are reported in a complete, timely, and ethical manner. Readers are encouraged to evaluate the evidence summarized below and to suggest other potential solutions to the results reporting problem. The ultimate beneficiary of timely communication of research results will be patients.

Keywords::

• MeSH: Disclosure, mandatory reporting
• Peer review, research
• Clinical trials
• Academic medical centers
• Drug industry. Non-MeSH: Medical writer
• Medical writing

Low results posting rates

Even when posting results in a public access database is a legal requirement, with the threat of $10,000 a day fines, researchers are still having difficulty meeting their obligations. In a study of results posting on ClinicalTrials.gov, Prayle et al. purported that only 22% (163/738) of clinical trials subject to FDAAA reporting requirements had results posted within the legally required 12-month timeframe1. Further, results posting for non-industry-funded trials (9%; 37/421) was 4-fold worse than that for industry-funded trials (40%; 126/317)1. A similar difference was reported by Giel et al. who noted that compliance with FDAAA results posting requirements was 4-fold worse for government-funded trials and 6-fold worse for academic-funded trials than for industry-funded trials2. These differences are not trivial. Compliance is not perfect in industry, but noncompliance seems even worse in government and academia. No wonder a number of US Congress Representatives fired off ‘please explain’ letters to the Commissioner of the FDA3 and the Director of the NIH⁴. These Representatives reinforced the critical importance of timely and accurate posting of clinical trial results and questioned whether the FDAAA law and financial penalties were working as intended. Disturbingly, the problems with results posting could intensify. The Trial and Experimental Studies Transparency (TEST) Act5, introduced into US Congress on August 2, 2012, requires posting results from far more trials than FDAAA and also places considerable emphasis on posting results from trials conducted outside the US6. The legislation does not mention how researchers might hope to comply with TEST, despite failures with FDAAA.

Low publication rates

Poor results posting on ClinicalTrials.gov is not the only cause of concern. Poor publication of clinical trial results in peer-reviewed journals is also a major issue. In a study of NIH-funded clinical trials, Ross et al. showed that only 46% (294/635) of trials had results published in Medline-indexed journals 30 months after trial completion7. Consistent with the findings from Prayle’s study on results posting1, the findings from Ross’ study suggest that nonpublication problems may be worse in academia than industry.

Unfortunately, Ross et al. did not provide a breakdown of publication rates by funding source, but his sample did include NIH studies that had external funding. Interestingly, the highest publication rates were for Phase 2/3 or Phase 3 studies (60%) and for IND/IDE studies (61%)7. As these studies are primarily conducted by industry, the higher publication rates may reflect a positive influence from industry. Indeed, as of August 11, 2012, of all the Phase 2 and 3 studies registered in ClinicalTrials.gov, 51% (26,573/51,702) were funded by industry and only 17% (8800/51,702) were funded by the NIH. Bourgeois et al. also found that the overall publication rate of industry-funded trials (230/346, 66%) was higher than that of government-funded trials (41/74, 55%)8. Initially, however, industry was slower to publish8. The publication rate 2 years after trial completion was lower for industry-funded trials than government-funded trials; however, industry-funded trials were more likely to have larger sample sizes and more centers8. These factors could contribute to the time required to publish. A systematic review of the literature by Schott et al. up to December 2009 found that 25–50% of drug licensing studies remain unpublished9. Schott et al. found mixed evidence on whether studies funded by industry or independent of industry were more likely to be published, but most of the evidence suggested that industry-funded trials had higher rates of publication. The need to report all results in a timely manner was one of the ten key recommendations recently made by editors and industry leaders to enhance the credibility of industry-sponsored research10. Arguably, this recommendation would hold for non-industry sponsored research as well.

Collectively, enough studies now indicate that trial results are not being published as they should, with recent evidence reinforcing that this problem is not unique to industry-funded trials. Indeed, with the low and slow publication rate for government-funded trials, patients would be right to question whether their tax dollars are being used appropriately. Patients would also be right to question whether the failure to report clinical trial results violates informed consent agreements and the ethical principles in the Declaration of Helskinki11. Based on the analysis by Ross et al., up to 60,000 patients may have participated in clinical trials where results were never published7. One wonders whether anyone has informed these 60,000 patients that the results from their voluntary efforts have not been shared as they should have been.

Medical writing support

Given the shameful state of results posting and publication rates, it would be easy to blame researchers… doing so, however, would be ineffectual. If researchers can’t meet their results reporting requirements, even with the threat of severe legal and financial penalties, further threats and finger-pointing are unlikely to improve the situation. Rather than blame researchers, let’s support them. We advocate a ‘less tsk tsk’ and ‘more fix fix’ approach. We propose that providing researchers with medical writing support would help researchers meet their reporting responsibilities.

Those who may be quick to blame researchers for not meeting their results posting and publication commitments may not fully appreciate the time it takes to complete the associated tasks. Even the NIH, which is responsible for ClinicalTrials.gov, the world’s largest results posting database, underestimated the time it should take researchers to post results. For example, the NIH now suggests that it may take 25 hours to complete their posting requirements (up from their original 10-hour estimate) and an additional 8 hours (up from their original 5-hour estimate) to update results, with two updates expected per trial12. A survey of biopharmaceutical companies indicated that it could take 22 hours to collate results and 38 hours to prepare and post them13. Thus, researchers would need to allocate an extra 25–60 hours for every study just to complete their clinical trial results posting requirements. Within the pharmaceutical industry, entire departments have been created just to help prepare results for posting and, even then, industry has struggled to keep up with the workload. The extra resources that industry has allocated to results posting, however, may explain why industry compliance is 4 - to 6-fold better than non-industry compliance1,2. Considerable time is also required to prepare a manuscript for publication; the average time for manuscript preparation, including review cycles, is estimated to be 7 months14. This is time that many researchers struggle to find during their working week. Indeed, a meta-analysis of 21 studies showed that lack of time was one of the leading factors associated with failure to publish15. If there was a sudden and dramatic change in research culture and funding, perhaps researchers would be given the extra days, weeks, or months needed to complete their overdue results reporting requirements. Patients could then be reassured that results from the research they helped to complete, if not fund, with their tax dollars, would now be shared with others.

We can’t see this happening.

What we have seen happening in industry and what we think should be happening in other research settings is to provide researchers with professional medical writing support. Professional medical writers, not to be confused with ghostwriters16, help researchers complete many of the time-intensive tasks associated with reporting results. These highly trained professionals, many of whom have advanced science-based degrees, comply with the ethical guidelines issues by medical journal editors17 and medical writing associations18–20. Whether these writers work on industry-funded or non-industry funded research does not matter; the same ethical principles and practices apply. Professional medical writers are legitimate and valuable contributors to results reporting21. For example, when professional medical writers help authors prepare manuscripts, these manuscripts are less likely to be retracted for misconduct22, are more compliant with best-practice reporting guidelines23, and are accepted more quickly for publication24.

Funding a medical writing solution

But where is the money going to come from to purchase medical writing services? To enhance future results reporting perhaps grant application forms should include a line item for medical writing services, unless applicants can prove they always meet results reporting requirements without such services. Given the current state of results reporting, whether the NIH should issue grants to generate more results when existing results have not been reported is not a moot point. In practical terms, in the same way that researchers request grant money for laboratory technicians, who help them generate data, and for statisticians, who help them analyze data, researchers should request grant money for professional medical writers, who will help them report data. Requesting medical writing services should not be seen as shirking a responsibility. Instead, requesting medical writing services should be seen as a sign that researchers are well aware of the deficiencies in results reporting and that they are committed to gaining and allocating the services required to report results appropriately. Notably, the need for, and the value of, medical writers have already been recognized within the NIH, with the National Cancer Institute recommending the use of ‘dedicated medical writers’ to accelerate the preparation of high quality trial documents (e.g., protocols)25. Seeing a request for medical writing support should not come as a surprise to public or private funding agencies. Given that the NIH spends $12 billion on clinical research ($3.5 billion on clinical trials alone)7, reserving some of this funding for the critical step of results reporting would seem justified.

Although including funds for medical writing services may enhance future results reporting practices, what could be done to address the backlog of missing results? Professional medical writers could certainly help correct this situation, but where would the money come from? Theoretically, funds could come from two sources. One suggestion is for granting agencies to quarantine a minor proportion of the funds they have set aside for future research. Agencies could then use these funds to provide their researchers with dedicated medical writing support and clear the backlog of overdue results. Patients, philanthropists, and taxpayers may well prefer that slightly fewer results were generated in order for overdue results to be reported. Another suggestion is for funds to come from the FDA. If the FDA actually enforced FDAAA, it could expect to gain between $5.75 and $17.25 million in funds (and that assumes that the 575 trials currently facing a $10,000 a day fine1 would have results reported within 1–3 days; any further delays would add substantially to the FDA’s windfall). With this level of funding, the FDA could provide researchers from fined organizations with access to between 70 and 210 full-time medical writers (median salary for contract medical writers in 2011 = $82,000)26. As current and past leaders of not-for-profit medical writing associations (International Society for Medical Publication Professionals, American Medical Writers Association, European Medical Writers Association, Association of Regulatory and Clinical Scientists), we believe these associations would work with the FDA and researchers to help them find a readily available pool of highly trained, if not internationally certified, medical writers. Having medical writers available could solve the results reporting problem… and solve it quickly. Assuming a 35-hour work week, the current backlog of 575 trials requiring results disclosure1 could, theoretically, be cleared within 6 weeks. If it took 25 hours for each results disclosure project, 70 writers could have the 14,375-hour job finished in 5.9 weeks; if it took 60 hours, 210 writers could have the 34,500-hour job finished in 4.7 weeks. Perhaps the US Congress Representatives should have altered their ‘please explain’ letters to ‘please fund’ letters.

In conclusion, we believe there is sufficient evidence that results reporting is a problem in both industry- and non-industry-funded research. We also believe there is insufficient focus on how to solve this problem. We have described how professional medical writers (not ghostwriters) could help researchers meet their overdue and future results reporting requirements and, importantly, how funds for medical writing services could be sourced. We welcome practical suggestions from others as to how to enhance timely, complete, and ethical results reporting. If results are not shared as they should be, what rights do researchers or funders have to tell patients that their results may be used to help others?

Transparency

Declaration of funding

No external sponsors were involved in this editorial and no external funding was used.

Declaration of financial/other relationships

All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that: (1) all authors have or do provide ethical medical writing services to academic, biotechnology, or pharmaceutical clients; (2) all authors’ spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and (3) all authors are active in national and international not-for-profit associations that encourage ethical medical writing practices.