Clinical trial transparency update: an assessment of the disclosure of results of company-sponsored trials associated with new medicines approved in Europe in 2014

Abstract

Background: The objective of this study was to assess the timely disclosure of results of company-sponsored clinical trials related to all new medicines approved by the European Medicines Agency (EMA) during 2014. This is the final extension of three previously reported studies of trials related to all new medicines approved in Europe in 2009, 2010 and 2011, and in 2012 and 2013. The original study found that over a three-year period over three-quarters of all trials were disclosed within 12 months and almost 90% were disclosed by the end of the study (31 January 2013). The extension studies (2012 and 2013 approvals) both showed an improvement in results disclosure within 12 months to 90%, and an overall disclosure rate of 92% and 93% respectively by the end of the studies.

Methods: The methodology used was exactly as previously reported. Various publicly available information sources were searched for both clinical trial registration and disclosure of results. All completed company-sponsored trials related to each new medicine approved for marketing by the EMA in 2014, carried out in patients and recorded on a clinical trials registry and/or included in an EMA European Public Assessment Report (EPAR), were included. Information sources were searched between 1 May and 31 July 2016.

Outcome measures and results: The main outcome measure was the proportion of trials for which results had been disclosed on a registry or in the scientific literature either within 12 months of the later of either first regulatory approval or trial completion, or by 31 July 2016 (end of survey). Of the completed trials associated with 32 new medicines licensed to 22 different companies in 2014, results of 93% (505/542) had been disclosed within 12 months, and results of 96% (518/542) had been disclosed by 31 July 2016.

Conclusions: The disclosure rate within 12 months of 93% suggests that industry is continuing to achieve disclosure in a timely manner. The overall disclosure rate at study end of 96% indicates that the improvement in transparency amongst company-sponsored trials has been maintained in the trials associated with new medicines approved in 2014.

Keywords:

• Clinical trials
• publication bias
• results disclosure
• transparency
• trial registration
• trial registries
• trial publication

Introduction

The ethical and scientific importance of disclosing clinical trial results is widely recognised^1,2. In recent years global and regional requirements that will reduce the risk of publication bias have been refined and expanded^3–7.

In December 2012, in response to variation in reported clinical trial transparency rates, and to evaluate the situation from an industry perspective, the Association of the British Pharmaceutical Industry (ABPI) initiated a study designed to assess the timely disclosure of results of company-sponsored trials related to all medicines recently approved in Europe over a continuous three-year period (2009, 2010 and 2011)⁸. The study was continued for a fourth and fifth year for trials related to all new medicines approved in 2012⁹ and 2013¹⁰. A trend towards increasing rates of results disclosure over the five-year period was confirmed¹⁰.

The objective of the current study was to extend the assessment for a sixth and final year (for all company-sponsored trials related to medicines approved in 2014) and determine whether observed improvements in transparency are being maintained.

Methods

In 2014, 32 new medicines (licensed to 22 different companies) containing new active substances (NASs), excluding vaccines, were approved for marketing by the European Medicines Agency (EMA). The study methodology, information sources searched and data extraction procedures were identical to those used in our previous studies^8–10. As in both the original and follow-up studies, there was no sampling involved as all completed company-sponsored trials related to each new medicine approved by the EMA in 2014, carried out in patients and recorded on a clinical trials registry and/or included in an European Public Assessment Report (EPAR), were included in the assessment.

Sources

The most comprehensive source of information was the US National Institutes of Health (NIH) national registry, ClinicalTrials.gov, which identified 2032 registered trials (irrespective of sponsor and trial status) related to the 32 medicines assessed. The European registry (EudraCT, clinicaltrialsregister.eu) included 654 associated trials, the majority of which were also registered on ClinicalTrials.gov. Some of the company registries provided additional information (15 of the medicines were associated with companies which had registries). The WHO International Clinical Trials Registry Platform (ICTRP), which provides access to 17 national and regional primary registries, was also searched.

The study assessed trial results disclosure using the earliest date of either posting in a registry or publication in the scientific literature, and disclosure was assessed firstly within 12 months (of either the date of first regulatory approval either by the EMA or by the US Food and Drug Administration [FDA], or the date of completion of the trial if after the date of first approval) and secondly at 31 July 2016, the end of the study.

After the initial data extraction, removal of duplicates and a preliminary assessment, responsible staff at each of the European Marketing Authorisation Holders (MAHs) were consulted to clarify specific questions. Enquiries included the provision of missing trial start or completion dates; clarification of trial registration; and evidence of results disclosure that may not have been readily identifiable through the search protocol. Where additional information that had clearly been in the public domain prior to the cut-off date for data collection (31 July 2016) was provided through this consultation, the assessment was amended. However, if results had been disclosed (by posting in a registry or publication in a journal) after 31 July 2016, the assessment was not changed. The final rates of clinical trial results disclosure for each medicine were captured in summary spreadsheets (accessible as supplementary information).

The chi-square test was used to examine whether there was a trend in the percentage of trials with results disclosed (at 12 months) over time during the continuous six-year period of EMA approvals (2009 to 2014) assessed in this current and three previously reported studies^8–10. The same test was used to examine whether there was a consistent trend over time, or if the trend deviated from a linear relationship.

Results

From the various sources, after removing duplicates, we identified 622 completed company-sponsored clinical trials in patients related to the 32 new medicines approved in Europe in 2014. Of these, 80 were unevaluable, both at the 12 month time period and at 31 July 2016, 78 due to having been completed within the 12 months prior to 31 July 2016 with results not yet required to be disclosed (Figure 1). Two trials were unevaluable due to missing or conflicting information; a phase I trial had results posted in EudraCT but was not publicly accessible and a phase III trial had an unknown completion date (the trial may have been terminated prior to enrolment, but the company failed to provide further information). Of the evaluable trials, 505/542 (93%) had been disclosed within the 12 month target and 518/542 (96%) were disclosed at 31 July 2016 (Table 1).

Figure 1. Disposition chart showing breakdown of trial assessment at 12 months.

*Trials completing within the 12 months prior to 31 July 2016 were not required to have reported by 31 July 2016 (the study end date).

Table 1. Number of completed company-sponsored clinical trials relating to 32 new medicines approved in 2014 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months^a	Results disclosed within 12 months^b		Unevaluable at 31 July 2016^a	Results disclosed by 31 July 2016
I/II	358	35	297/323	92%	35	308/323	95%
III	237	32	194/205	95%	32	196/205	96%
IV	12	7	5/5	100%	7	5/5	100%
other	15	6	9/9	100%	6	9/9	100%
Total	622	80	505/542	93%	80	518/542	96%

^a Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

^b Twelve months measured from the later of either the date of first regulatory approval (Europe or US) or trial completion date.

The disclosure rate for the smaller, earlier phase I/II trials was slightly lower than that for the larger phase III trials, which reached 95% (194/205) within 12 months and 96% (196/205) at 31 July 2016 (Table 1). As the approval date for the new medicines in this study was relatively recent, very few phase IV trials had been completed. Of the 24 trials for which results remained undisclosed at the end of the study, 15 related to the smaller, earlier phase I and II trials; five of these were phase I and 10 were phase II. Of the nine phase III (or II/III) trials that remained undisclosed at the end of the study, two were sponsored by a company which failed to provide any additional information during our consultation process, and seven were carried out in Asia by the Japanese parent company of the European MAH; results of three of these seven trials will be posted on ClinicalTrials.gov, while the others were not required to register and reporting of results was not mandatory under Japanese regulations (company communication).

Trend analysis suggested a highly significant trend towards increasing rates of results disclosure at 12 months over the continuous six-year period of EMA approvals assessed (Figure 2, chi-square test p < .001), with some evidence of a departure from a linear trend over time (p = .02).

Figure 2. Percentage of trials with results disclosed.

Sensitivity analyses

There were two unevaluable trials where the key dates were missing or information was conflicting (Figure 1). If these two trials had been assessed as undisclosed at 31 July 2016, the overall disclosure rate would have fallen from 96% to 95% (516/542). The remaining 78 unevaluable trials had completed within the last 12 months and were within the required results disclosure timeframe.

Occasionally, use of “completion date” rather than “primary completion date” might have led to a different assessment at 12 months, but this was not quantified and would not have affected the final assessment at 31 July 2016. Around 4% (18/505) of trials relied solely upon conference abstracts for assessment of disclosure at the 12 month time period. If all of these trials had been assessed as undisclosed, the disclosure rate at 12 months would fall from 93% to 90% (487/542).

Discussion

This study was the final extension of three previously reported studies^8–10 that have now assessed the timely disclosure of results of a large and comprehensive cohort of all company-sponsored clinical trials in patients related to all 142 new medicines approved by the EMA over a continuous six-year period (2009 to 2014); we did not select a sample of either new medicines or companies. The current study, together with the original⁸ and two extension studies^9,10 included trials related to new medicines from 64 different companies (MAHs) and a wide range of therapeutic areas. Due to mergers, acquisitions and licensing agreements affecting more than half (81/142) of these new medicines, many more companies were likely to have been involved in the development process.

In the current study of 32 new medicines approved in 2014, 93% (505/542) of trials had results disclosed within 12 months, continuing the significant trend towards increasing disclosure rates observed previously (p < .001)^9,10. There was also some evidence of departure from a linear trend (p = .02) suggesting that the rate of increase in disclosure has tailed off in later years as the figure approaches 100%.

The overall disclosure rate of 96% at study end for the results of trials associated with new medicines approved in 2014 is higher than our published results for trials associated with new medicines approved in the previous five years (range 86% to 93%)^8–10. The combined end of study disclosure rate over the six years of this study (2009 to 2014), including all completed company-sponsored trials conducted in patients related to all of the 142 approved new medicines, was 92% (2114/2303).

The majority of the European MAHs had staff with specific responsibility for ensuring that transparency commitments are fulfilled, particularly the large multinational companies, and responses to our enquiries continued to be generally timely and thorough. However, where a medicine had been affected by a licensing deal, merger or acquisition, we occasionally noted delays, sometimes due to the current European MAH not having immediate access to the relevant information. Therefore, to ensure that transparency commitments are fulfilled during licensing or acquisitions, an assessment of compliance with clinical trial disclosure requirements should be considered a standard component of the due diligence process.

ClinicalTrials.gov continued to be the most useful data source. As it contains two thirds of global trial registrations¹¹, this was expected. We found that duplication with other registries is increasing, particularly with the European registry EudraCT, and observed that some trials were routinely registered multiple times, and regularly on ClinicalTrials.gov, EudraCT and a company’s own registry when available. If all corresponding trial identifiers are cross referenced then duplicate registry records are easily identified, although it has been estimated that approximately 45% of all duplicate registrations on the WHO ICTRP currently go undetected, corresponding to a reduction in the number of unique records on the portal by approximately 5%¹². Duplicates could affect the assessment of bias by inflating the number of trials that do not publish results and policy makers, trial sponsors and registries are encouraged to enact policies and quality assurance processes to improve the quality of published records¹². While authors and trial sponsors must provide accurate data associated with each trial’s unique identifier, registries and journals share responsibility for ensuring the accuracy of the public information they provide, including any automatic links.

As we have discussed previously¹³, studies that set out to measure the success of a single registry do not measure overall clinical trial transparency or fully assess the potential for publication bias associated with any individual medicine. This study continues to illustrate that a number of sources need to be searched to have confidence that as much relevant information as possible on a medicine and its associated clinical trials will be identified, and a comprehensive measure of transparency achieved. Manual searching or matching publications to the correct clinical trial registry record is resource intensive and open to error. However, a systematic review of the processes used to link clinical trial registrations to their published results found that the linkage of trial registries to their corresponding publications continues to require extensive manual processes (such as inference or contacting of trial investigators or authors) and relying on automatic links alone to draw conclusions about the rate of non-publication will likely over-estimate the rate of non-publication¹⁴.

For example, an initiative to automate the tracking of results disclosure reported rates of disclosure for industry-sponsored studies of around 73% in 2016¹⁵. An algorithm identified all completed phase II to IV trials since 2006 on ClinicalTrials.gov and searched for results posted on the registry or linked to a published journal article on PubMed by National Clinical Trial (NCT) identifier (present in PubMed’s secondary source ID field, title or abstract). By focusing on results posted on only one registry, and relying on links between publications on PubMed and trial registries which are not always routinely completed, this approach, as noted by Bashir et al.¹⁴, is likely to underestimate results disclosure.

Analyses focusing solely on publication rates do not take into account unsuccessful efforts to publish. For example, 85% (65/76) of Pfizer-sponsored clinical trials for approved products that completed in 2010 were published within 52 months of study completion. However more than 50% required submission to more than one journal to achieve publication, and 21% required three or more attempts¹⁶.

A retrospective review of publication status by study outcome for all human drug research studies conducted by GlaxoSmithKline and completed between 1 January 2009 and 30 June 2014, of which 98% (1041/1064) had results posted on one or more public registries, reported that over 10% of all studies and 13% of those with negative outcomes required three or more submission attempts before they were accepted for publication in peer reviewed medical journals¹⁷. The authors suggest that sponsors and journal editors should share similar information to contribute to better understanding of issues and barriers to full transparency. Over the period studied there was no evidence of submission or reporting bias¹⁷.

Zarin et al.¹¹ have proposed a number of actions for various stakeholder groups for improving the trial reporting system over the next decade. These include the suggestions that journal editors and peer reviewers should check the denominator by searching registries for the relevant registered trial; that registries and results databases should coordinate with other registries to improve the ability to identify a unique list of trials; and that trial sponsors and investigators should use a unique trial registry number whenever communicating about a trial, and keep registry records up to date.

In addition, a report by the Academy of Medical Sciences (AMS) provides fresh impetus for the industry to better communicate research findings and sets out a number of recommendations to improve the process, including publication of rigorous results regardless of outcome, reporting of findings in more accessible formats, trial registration, and those who fund research providing incentives to support the communication of results for funded projects by requiring in applications an effective communication plan and “intelligent openness” of results¹⁸.

Our study confirms that the results of more than 90% of completed, company-sponsored clinical trials in patients related to new medicines recently approved in Europe have been disclosed. Disclosure rates have improved over time and, for medicines approved by the EMA in 2014, only 4% of trials remained undisclosed at the end of our study. For company-sponsored trials in patients related to newly approved medicines, disclosure is approaching 100%. Although this assessment is now complete, the ABPI continues to work with companies to ensure that observed improvements in clinical trial transparency are maintained.

Looking to the future, it is clear that implementation of existing laws and commitments to routine registration of all clinical trials followed by disclosure of results (summaries on registries and publication in the literature linked through trial identifiers), provides the framework to avoid publication bias in clinical development. The industry has also implemented principles for responsible trial data sharing³, ensuring that where more detailed information (such as patient-level data) is required to support systematic reviews and/or further medical research, it can be provided. However, as clinical research adapts to embrace the use of “big data”, real world data and personalized data, transparency will need to be maintained.

Limitations

The limitations associated with this study have been detailed previously^8–10. Firstly, limitations relate to the availability of information in the public domain, including the potential for double-counting and/or conflicting information due to duplication across multiple sources, as well as the difficulty of matching journal publications to registered trials if trial identifiers are not included in the publication abstract or the journal citation is absent from the registry record. Secondly, this is a quantitative study; we counted the number of trials for which results have been disclosed in a variety of formats, but did not assess whether the planned primary and secondary endpoints had been fully reported. Finally, we did not assess trial registration, and would not have been able to identify a trial if it had not been included in an EPAR or a registry.

Conclusion

In this final follow-up study, results disclosure within 12 months of 93% and overall disclosure rate at study end of 96% for company-sponsored trials associated with new medicines approved in 2014 was higher than recorded in our previous studies, confirming that the significant improvement in timely disclosure previously observed over five continuous years of European approvals has been maintained.

Transparency

Declaration of funding

Publication support for this study was funded by the Association of the British Pharmaceutical Industry (ABPI). The ABPI represents the UK-based biopharmaceutical industry.

Author contributions: S.P. critically revised the manuscript for important intellectual content and contributed to the interpretation of the data. The study was carried out by B.R.D. and two medical information specialists from Livewire Communications.

Declaration of financial/other relationships

S.P. has disclosed that she is Interim Executive Director of Research, Medical & Innovation at the ABPI, and Director of Actaros Consultancy Ltd and the MedicoMarketing Partnership. B.R.D. has disclosed that he is a freelance consultant in Pharmaceutical Marketing and Communications.

A CMRO reviewer on this manuscript is a full-time employee of Johnson & Johnson. All other CMRO reviewers have no relevant financial or other relationships to disclose.

Acknowledgements

The authors thank the Board of Management of the ABPI for supporting the conduct of the study. Belen Granell Villen (ABPI) assisted with the identification of contacts and communication with MAHs. Paul Basset advised on the applicability of statistical tests, and performed the time trend analysis. B.R.D. managed the project on behalf of Livewire Communications. Ros Lea (R.L.) and Alex Morrison from Livewire Communications assisted with the research. R.L. provided editorial support for the manuscript.