Clinical trial transparency update: an assessment of the disclosure of results of company-sponsored trials associated with new medicines approved in Europe in 2013

Abstract

Background: The objective of this study was to assess the timely disclosure of results of company-sponsored clinical trials related to all new medicines approved by the European Medicines Agency (EMA) during 2013. This is an extension of two previously reported studies of trials related to all new medicines approved in Europe in 2009, 2010 and 2011, and in 2012. The original study found that over a three year period over three-quarters of all trials were disclosed within 12 months and almost 90% were disclosed by the end of the study. The extension study (2012 approvals) showed an improvement in results disclosure within 12 months to 90%, and an overall disclosure rate of 92% by the end of the study.

Methods: The methodology used was exactly as previously reported. Various publicly available information sources were searched for both clinical trial registration and disclosure of results. All completed company-sponsored trials related to each new medicine approved for marketing by the EMA in 2013, carried out in patients and recorded on a clinical trials registry and/or included in an EMA European Public Assessment Report (EPAR), were included. Information sources were searched between 1 May and 31 July 2015.

Outcome measures and results: The main outcome measure was the proportion of trials for which results had been disclosed on a registry or in the scientific literature either within 12 months of the later of either first regulatory approval or trial completion, or by 31 July 2015 (end of survey). Of the completed trials associated with 34 new medicines licensed to 24 different companies in 2013, results of 90% (484/539) had been disclosed within 12 months, and results of 93% (500/539) had been disclosed by 31 July 2015.

Conclusions: The disclosure rate within 12 months of 90% suggests that industry is continuing to achieve disclosure in a timely manner. The overall disclosure rate at study end of 93% indicates that the improvement in transparency amongst company-sponsored trials has been maintained in the trials associated with new medicines approved in 2013.

Keywords:

• Clinical trials
• publication bias
• results disclosure
• transparency
• trial publication
• trial registration
• trial registries

Introduction

Measures to enhance clinical trial transparency and reduce the risk of publication bias continue to be updated and expanded^1–6. The new EU Clinical Trials Regulation³, due to apply by October 2018⁷ at the latest, will harmonize the conduct of clinical trials in the EU and contains new transparency requirements for the disclosure of clinical trial information. In the US, the Department of Health and Human Services (HHS) and National Institutes of Health (NIH) have issued new policies to expand the scope of clinical trial registration and results reporting requirements². In 2015, the World Health Organization (WHO) published a statement which updates and expands their 2005 position on clinical trials registration and reaffirms the ethical imperative to report the results of all clinical trials⁵.

Despite recent initiatives, existing regulatory requirements^8,9 and industry commitments^1,10, studies on clinical trial transparency rates continue to report that not all clinical trial results are published or that a proportion fail to report results within required timelines^11–14.

To evaluate the situation from an industry perspective, the original study, initiated in December 2012 by the Association of the British Pharmaceutical Industry (ABPI), was designed to assess the timely disclosure of results of company-sponsored trials related to all medicines recently approved in Europe over a continuous three year period (2009, 2010 and 2011)¹⁵. The study was continued for a fourth year for trials related to all new medicines approved in 2012¹⁶.

The objective of the current study was to extend the assessment for a fifth year (for trials related to medicines approved in 2013) and determine if observed improvements in transparency are being maintained.

Methods

In 2013, 34 new medicines (licensed to 24 different companies) containing new active substances (NASs), excluding vaccines, were approved for marketing by the European Medicines Agency (EMA). The study methodology, information sources searched and data extraction procedures were identical to those used in our previous studies^15,16. As in both the original and follow-up study, there was no sampling involved as all completed company-sponsored trials related to each new medicine approved by the EMA in 2013, carried out in patients and recorded on a clinical trials registry and/or included in an European Public Assessment Report (EPAR), were included in the assessment.

Sources

The most comprehensive source of information was the US National Institutes of Health (NIH) national registry, ClinicalTrials.gov, which identified 2471 registered trials (irrespective of sponsor and trial status) related to the 34 medicines assessed. The European registry (EudraCT, clinicaltrialsregister.eu) included 632 associated trials, the majority of which were also registered on ClinicalTrials.gov. Some of the company registries provided additional information (17 of the medicines were associated with companies which had registries). The WHO International Clinical Trials Registry Platform (ICTRP), which provides access to 16 national and regional primary registries, was also searched. The Japanese Pharmaceutical Information Centre (JAPIC) clinical trial registry (clinicaltrials.jp) continued to be an important additional source for trials associated with some new medicines.

The study assessed trial results disclosure using the earliest date of either posting in a registry or publication in the scientific literature, and disclosure was assessed firstly within 12 months (of either the date of first regulatory approval either by the EMA or by the US Food and Drug Administration [FDA], or the date of completion of the trial if after the date of first approval) and secondly at 31 July 2015, the end of the study.

After the initial data extraction, removal of duplicates and a preliminary assessment, responsible staff at each of the European Marketing Authorisation Holders (MAHs) were consulted to clarify specific questions. Enquiries included the provision of missing trial start or completion dates; clarification of trial registration; and evidence of results disclosure that may not have been readily identifiable through the search protocol. Where additional information that had clearly been in the public domain prior to the cut-off date for data collection (31 July 2015) was provided through this consultation, the assessment was amended. However, if the company amended results information or a trial was published after 31 July 2015, the assessment was not changed. The final rates of clinical trial results disclosure for each medicine were captured in summary spreadsheets (accessible as supplementary information).

The chi-square test was used to examine whether there was a trend in the percentage of trials with results disclosed (at 12 months) over time during the continuous five year period of EMA approvals (2009 to 2013) assessed in this current and two previously reported studies^15,16. The same test was used to examine whether there was a consistent trend over time, or if the trend deviated from a linear relationship.

Results

From the various sources, we identified 606 completed company-sponsored clinical trials related to the 34 new medicines approved in Europe in 2013. Of these, 67 were unevaluable, both at the 12 month time period and at 31 July 2015, all due to having been completed within the 12 months prior to 31 July 2015 with results not yet required to be disclosed (Figure 1). Of the evaluable trials, 484/539 (90%) had been disclosed within the 12 month target and 500/539 (93%) were disclosed at 31 July 2015 (Table 1).

Figure 1. Disposition chart at 12 months.

Chart showing breakdown of trials assessment at 12 months.

*Trials completing within the 12 months prior to 31 July 2015 were not required to have reported by 31 July 2015 (the study end date).

Table 1. Number of completed company-sponsored clinical trials relating to 34 new medicines approved in 2013 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months*	Results disclosed within 12 months#		Unevaluable at 31 July 2015*	Results disclosed by 31 July 2015
I/II	339	32	270/307	88%	32	279/307	91%
III	253	28	210/225	93%	28	214/225	95%
IV	10	5	3/5	60%	5	5/5	100%
other	4	2	1/2	50%	2	2/2	100%
Total	606	67	484/539	90%	67	500/539	93%

* Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

# Twelve months measured from the later of either the date of first regulatory approval (Europe or US) or trial completion date.

The disclosure rate for the smaller, earlier phase I/II trials was lower than that for the larger phase III trials, which reached 93% (210/225) within 12 months and 95% (214/225) at 31 July 2015 (Table 1). As the approval date for the new medicines in this study was relatively recent, very few phase IV trials had been completed. Of the 39 trials for which results remained undisclosed at the end of the study, 28 relate to the smaller, earlier phase I and II trials.

Trend analysis suggested a significant trend towards increasing rates of results disclosure at 12 months over the continuous five year period of EMA approvals assessed in this ongoing study (Figure 2, chi-square test p < .001), with some evidence of a departure from a linear trend over time (p = .02).

Figure 2. Percentage of trials disclosed at 12 months by year of EMA approval.

Sensitivity analyses

There were no unevaluable trials where the key dates were missing. All the unevaluable trials had completed within the last 12 months and were within the required results disclosure timeframe. Occasionally, use of “completion date” rather than “primary completion date” might have led to a different assessment at 12 months, but this was not quantified and would not have affected the final assessment at 31 July 2015. Around 7% (36/484) of trials relied solely upon conference abstracts for assessment of disclosure at the 12 month time period. If all of these trials were excluded, the disclosure rate at 12 months would fall from 90% to 83% (448/539).

Of the 39 trials assessed as undisclosed, 22 related to two new medicines which, although approved by the EMA in 2013, were not approved by the FDA until September 2015. During the consultation process, it was confirmed that results of all 22 trials were posted on ClinicalTrials.gov and/or the company’s own registry, in full compliance with the FDA Amendment Act (FDAAA) of 2007⁸. Had these trials been disclosed before 31 July 2015, the disclosure rate would have reached 97% (522/539). As this was a post-hoc observation, it is mentioned only as part of the sensitivity analysis.

Discussion

This study was a further extension of two previously reported studies^15,16 that assessed the timely disclosure of results of the large and comprehensive cohort of company-sponsored clinical trials related to all new medicines approved by the EMA over a continuous four year period (2009 to 2012).

The disclosure of results at 12 months had previously shown a significant increase year on year over a continuous four year period – 71% (317/447) in 2009, 81% (116/144) in 2010, 86% (186/216) in 2011¹⁵, and 90% (307/340) in 2012 (p < .001)¹⁶. In the current study of 34 new medicines approved in 2013 (a larger group of medicines than approved and assessed in any of the previous four years, range 12 to 23), 90% (484/539) of trials had results disclosed within 12 months, similar to the disclosure rate observed for 2012 approvals¹⁶.

A trend towards increasing rates of results disclosure over the five year period was confirmed (p < .001). There was also some evidence of departure from a linear trend (p = .02) suggesting that the observed increase in disclosure rates has tailed off or plateaued in more recent years. Each year, the number of new medicines approved, as well as the date of initiation and number of associated clinical trials, vary considerably, therefore we did not expect the trend to be smooth and linear from year to year.

The overall disclosure rate of 93% at study end for the results of trials associated with new medicines approved in 2013 is similar to that for trials associated with new medicines approved in 2010 (93%), 2011 (91%)¹⁵ and 2012 (92%)¹⁶. The combined end of study disclosure rate over five continuous years (2009 to 2013), including trials related to a total of 110 approved new medicines marketed by 52 different European MAHs (at the time of assessment), was 91% (1596/1761). In fact, due to mergers, acquisitions and licensing agreements affecting more than half (65/110) of these new medicines, many more companies were involved in the development process.

A significant proportion (28/39, 72%) of the trials which remained undisclosed were early phase I or II trials. Many of these were initiated over ten years ago, prior to the publication of industry commitments (through the International Federation of Pharmaceutical Manufacturers and Associations [IFPMA] Joint Position Paper of 2005, updated in 2008 and 2009¹⁰) and the International Committee of Medical Journal Editors (ICMJE) principles applied in 2005¹⁷, before any regulatory requirements⁸ and/or prior to the availability of clinical trial registries. In fact 26 of the 28 undisclosed phase I/II trials either pre-dated or were out of scope of disclosure commitments or requirements. In addition, during the consultation period following the end of the study, subsequent posting or publication of the results of at least 23 of the 39 undisclosed trials were brought to our attention by the companies concerned. As long as medicines whose development was initiated before 2005 are still reaching approval, a proportion of early trials are likely to pre-date disclosure commitments and the retrospective assessment of disclosure remains unlikely to reach 100%.

As observed previously¹⁶ many of the European MAHs had staff with specific responsibility for ensuring that transparency commitments are fulfilled, particularly the large multinational companies, and responses to our enquiries were generally timely and thorough. There was sometimes a delay in results disclosure where a medicine had been affected by a licensing deal, merger or acquisition, due largely to uncertainty around which company was responsible and the current European MAH not always having direct access to the relevant information.

The ethical and scientific importance of disclosing clinical trial results is not in question¹⁸ and requirements continue to be updated and expanded on a global^1,4,5 and regional level^2,3,6.

An increasing number of studies have investigated the disclosure of trials results against various commitments, initiatives and regulations. However, comparing results between studies that may have assessed disclosure at varying time points and on different sub-sets of trials, used different means of disclosure, or assessed against defined requirements or posting results on a specific registry (ClinicalTrials.gov), is subject to limitations.

This point is illustrated by a recent study by Miller et al.¹⁴ which took a similar approach to this current and two previously reported studies^15,16 by focusing on clinical trial results disclosure related to recently approved medicines. However, in contrast to our study which assessed all new medicines approved by the EMA during specified years, Miller et al. assessed trials from only 15 new molecular entities (sponsored by 10 large companies) approved by the FDA in 2012, out of a possible 39, and found approximately two-thirds (65%) of the 318 associated trials were publicly disclosed, either through results reporting in ClinicalTrials.gov or published in the medical literature.

Variance in disclosure rates between the two studies is a direct consequence of differences in methodology and the subset of trials assessed. For example, Miller et al.¹⁴ assessed only trials included in the FDA approval (excluding any trials which were either specific to other geographies or which were ongoing at the time of the regulatory submission), and included all phase I trials (in both patients and healthy volunteers), which were, and still are, outside the scope of FDAAA reporting requirements. Our studies have assessed all completed company-sponsored trials conducted in patients. It is important to highlight that, as part of the complex drug development process, early phase I trials involve relatively small numbers of individuals, were sequentially designed to maximize safety considerations and inform decisions to progress to further stages of development, and at the time of completion, were unlikely to be published individually. Now that clinical trial registries and databases are available, the routine disclosure of summary results of phase I trials is straightforward, but applying an “ethical” criterion retrospectively to these early developmental trials is misleading. Putting this further into perspective, it is the larger randomized, controlled phase III trials (together with post-marketing, outcomes and real world usage trials) that are critical for accurate evaluation of the value of a medicine through systematic reviews (and which potentially contribute to publication bias if selectively published).

Of the drugs assessed by Miller et al.¹⁴, 7/15 were in our current 2013 cohort of EMA approved new medicines. Of these, five were associated with a company with its own clinical trial registry (an information source not included in the methodology used by Miller et al.¹⁴) which provided additional information to results posted on ClinicalTrials.gov or EudraCT.

Interestingly, Miller et al.¹⁴ note that mergers, acquisitions, collaborations and licensing agreements may complicate compliance with disclosure requirements, which we also found in this and in previous studies^15,16.

This observation prompts the suggestion that, in future, an assessment of compliance with clinical trial disclosure requirements should perhaps be a standard component of the due diligence process when licensing or acquisition deals are in negotiation.

Although ClinicalTrials.gov continues to be the most useful data source, we found that duplication with other registries is increasing; a finding confirmed by a recent study on trends in global clinical trial registration on the national and regional registries searchable via the WHO’s ICTRP¹⁹. Posting of summary results is now mandatory for interventional trials registered with the EU Clinical Trials Database (EudraCT) and completed on or after 21 July 2014²⁰, and required retrospectively for trials completed before that date in accordance with specific timeframes²¹. Consequently an increase in posting of results on EudraCT was noted compared to our previous studies, although phase I information is not publicly accessible on EudraCT.

Recent disclosure rates for company-sponsored trials are encouraging but there is still scope for improvement. We continued to find publications not clearly linked (by inclusion of the trial registration numbers in publication abstracts or PubMed indexing) to the trials they report. Once this link is routinely included by all authors and journals, in line with ICJME⁴ and WHO recommendations⁵, it will be much easier to determine precise disclosure rates. Older early development trials, initiated prior to the availability of registries or completed prior to industry or regulatory reporting requirements, continue to be under-reported.

As drug development is a global and complex process, this study illustrates that a number of sources need to be searched to have confidence that as much relevant information as possible on a medicine and its associated clinical trials will be identified, and a comprehensive measure of transparency achieved.

Our two previously reported^15,16 studies, and this current study, have now collected five years’ worth of comprehensive data, assessing all completed company-sponsored trials in patients related to all new medicines approved by the EMA from January 2009 to December 2013. The ABPI continues to work with companies concerned to monitor transparency and to ensure that observed improvements are maintained.

Limitations

The limitations associated with this study have been detailed previously^15,16. Firstly, limitations relate to the availability of information in the public domain, including the potential for double counting and/or conflicting information due to duplication across multiple sources, as well as the difficulty of matching journal publications to registered trials if trial identifiers are not included in the publication abstract or the journal citation is absent from the registry record. Secondly, this is a quantitative study; we counted the number of trials for which results have been disclosed in a variety of formats, but did not assess whether the planned primary and secondary endpoints had been fully reported. Finally, we did not assess trial registration, and would not have been able to identify a trial if it had not been included in an EPAR or a registry.

Conclusion

In this follow-up study, results disclosure within 12 months of 90% and overall disclosure rate at study end of 93% was similar to that recorded in our previous studies, indicating that the improvement in disclosure previously observed over four continuous years of European approvals has been maintained for company-sponsored trials associated with new medicines approved in 2013.

Transparency

Declaration of funding

Publication support for this study was funded by the Association of the British Pharmaceutical Industry (ABPI). J.S. critically revised the manuscript for important intellectual content and contributed to the interpretation of the data. The study was carried out by B.R.D. and two medical information specialists from Livewire Communications. The ABPI represents the UK-based biopharmaceutical industry.

Declaration of financial/other relationships

J.S. has disclosed that she is Head of Medical Affairs at the ABPI. B.R.D. has disclosed that he is a freelance consultant in Pharmaceutical Marketing and Communications.

The CMRO peer reviewer on this manuscript has no relevant financial or other relationships to disclose.

Acknowledgments

The authors thank the Board of Management of the ABPI for supporting the conduct of the study. Achenyo Ochuma (ABPI) assisted with the identification of contacts and communication with MAHs. Paul Basset advised on the applicability of statistical tests, and performed the time trend analysis. B.R.D. managed the project on behalf of Livewire Communications. Ros Lea (R.L.) and Alex Morrison from Livewire Communications assisted with the research. R.L. provided editorial support for the manuscript.