Clinical trial transparency update: an assessment of the disclosure of results of company-sponsored trials associated with new medicines approved in Europe in 2012

Abstract

Background:

The objective of this study was to assess the timely disclosure of results of company-sponsored clinical trials related to all new medicines approved by the European Medicines Agency (EMA) during 2012. This is an extension of the previously reported study of trials related to all new medicines approved in Europe in 2009, 2010 and 2011, which found that over three-quarters of all these trials were disclosed within 12 months and almost 90% were disclosed by the end of the study.

Methods:

The methodology used was exactly as previously reported. Various publicly available information sources were searched for both clinical trial registration and disclosure of results. All completed company-sponsored trials related to each new medicine approved for marketing by the EMA in 2012, carried out in patients and recorded on a clinical trials registry and/or included in an EMA European Public Assessment Report (EPAR), were included. Information sources were searched between 1 May and 31 July 2014.

Outcome measures and results:

The main outcome measure was the proportion of trials for which results had been disclosed on a registry or in the scientific literature either within 12 months of the later of either first regulatory approval or trial completion, or by 31 July 2014 (end of survey). Of the completed trials associated with 23 new medicines licensed to 17 different companies in 2012, results of 90% (307/340) had been disclosed within 12 months, and results of 92% (312/340) had been disclosed by 31 July 2014.

Conclusions:

The disclosure rate within 12 months of 90% suggests the industry is now achieving disclosure in a timely manner more consistently than before. The overall disclosure rate at study end of 92% indicates that the improvement in transparency amongst company-sponsored trials has been maintained in the trials associated with new medicines approved in 2012.

Keywords: :

• Clinical trials
• Publication bias
• Results disclosure
• Transparency
• Trial publication
• Trial registration
• Trial registries

Introduction

The risks associated with publication bias, and consequently the desirability of registration and results disclosure of all clinical trials, are now well documented1. Over the last twenty years, a variety of measures have been designed to improve transparency around clinical trial information, and hence ensure that the risk of publication bias is reduced.

At the same time, various studies and surveys have been undertaken to assess whether commitments, registries, policies or laws have been successfully implemented or have improved transparency. As we discussed in a recent editorial2, different studies have investigated a variety of subsets of trials, including those conducted during a specific time period, or posted on a single registry, or with results disclosed only through publication in the literature or disclosure through a single registry. Not surprisingly, these studies have given rise to a range of disclosure figures and comparing results is subject to limitation.

In light of the variation in reported clinical trial transparency rates, the original study3, initiated by the Association of the British Pharmaceutical Industry (ABPI), was designed to assess the timely disclosure of results of company-sponsored trials related to medicines recently approved (during 2009, 2010 and 2011) for use in patients in Europe. Therefore, the objective of the current study was to extend the assessment for a further year (for trials related to medicines approved in 2012) and determine whether observed improvements in transparency are being maintained.

Methods

In 2012, 23 new medicines (licensed to 17 different companies) containing new active substances (NASs), excluding vaccines, were approved for marketing by the EMA. The study methodology, information sources searched and data extraction procedures were identical to those used in our previous study3. As in the original study, there was no sampling involved as all completed company-sponsored trials related to each new medicine approved by the EMA in 2012, carried out in patients and recorded on a clinical trials registry and/or included in an EPAR, were included in the assessment.

Sources

The most comprehensive source of information was the US National Institutes of Health (NIH) national registry, ClinicalTrials.gov, which showed 1313 registered trials (irrespective of sponsor and trial status) related to the 23 medicines assessed. The European registry (EudraCT, clinicaltrialsregister.eu) included 352 associated trials, the majority of which were also registered on ClinicalTrials.gov. Some of the company registries provided additional information (13 of the medicines were associated with companies which had registries). The International Federation of Pharmaceutical Manufacturers and Associations (IFPMA) portal, a source of information in our previous study, was closed during 2014 (while the current study was ongoing), with users being redirected to the WHO portal (which also links to a range of primary registries). We also noted that the Japanese Pharmaceutical Information Centre (JAPIC) clinical trial registry (clinicaltrials.jp) was an important additional source for trials associated with some new medicines.

The study assessed trial results disclosure using the earliest date of either posting in a registry or publication in the scientific literature, and disclosure was assessed firstly within 12 months (of either the date of first regulatory approval either by the EMA or by the US FDA, or the date of completion of the trial if after the date of first approval) and secondly at 31 July 2014, the end of the study.

After the initial data extraction, removal of duplicates and a preliminary assessment, responsible staff at each of the European Marketing Authorisation Holders (MAHs) were consulted to clarify specific questions. Enquiries included the provision of missing trial start or completion dates; clarification of registration; and evidence of results disclosure that may not have been readily identifiable through the search protocol. Where additional information that had clearly been in the public domain prior to the cut-off date for data collection (31 July 2014) was provided through this consultation, the assessment was amended. However, if the company amended results information or a trial was published after 31 July 2014, the assessment was not changed. The final rates of clinical trial results disclosure for each medicine were captured in summary spreadsheets (accessible as supplementary information).

Results

From the various sources, we identified 392 completed company-sponsored clinical trials related to the 23 new medicines approved in Europe in 2012. Of these, 52 were unevaluable, both at the 12 month time period and at 31 July 2014, all due to having been completed within the 12 months prior to 31 July 2014 with results not yet required to be disclosed (Figure 1). Of the evaluable trials, 307/340 (90%) had been disclosed within the 12 month target and 312/340 (92%) were disclosed at 31 July 2014 (Table 1).

Figure 1. Disposition chart at 12 months. Chart showing breakdown of trials assessment at 12 months. Trials completing within the 12 months prior to 31 July 2013 were not required to have reported by 31 July 2014 (the study end date).

Table 1. Number of completed company-sponsored clinical trials relating to 23 new medicines approved in 2012 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months*	Results disclosed within 12 months#	Unevaluable at 31 July 14*		Results disclosed by 31 July 14
I/II	235	22	187/213	88%	22	191/213	90%
III	143	24	114/119	96%	24	115/119	97%
IV	9	4	4/5	80%	4	4/5	80%
other	5	2	2/3	67%	2	2/3	67%
Total	392	52	307/340	90%	52	312/340	92%

*Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

#Twelve months measured from the later of either the date of first regulatory approval (Europe or US) or trial completion date.

The disclosure rate for the smaller, earlier phase I/II trials was lower than that for the larger phase III trials, which reached 96% within 12 months and 97% at 31 July 2014 (Table 1). As the approval date for the new medicines in this study was relatively recent, very few phase IV trials had been completed. Of the 28 trials for which results remained undisclosed at the end of this study, 22 relate to the smaller, earlier phase I and II trials and the majority (20/22) either pre-dated or were out of scope of specific disclosure requirements.

Sensitivity analyses

There were no unevaluable trials where the key dates were missing. All the unevaluable trials had completed within the last 12 months and were within the required results disclosure timeframe. Occasionally, use of ‘completion date’ rather than ‘primary completion date’ might have led to a different assessment at 12 months, but this was not quantified and would not have affected the final assessment at 31 July 2014. Around 10% (31/307) of trials relied solely upon conference abstracts for assessment of disclosure at the 12 month time period. If all of these trials were excluded, the disclosure rate at 12 months would fall from 90% to 81% (276/340). A number of these trials were published in full or results were posted outside the 12 month timeframe.

Discussion

The overall disclosure rate of 92% at study end for the results of completed company-sponsored clinical trials associated with new medicines approved in 2012 is similar to that for trials associated with new medicines approved in 2010 (93%) and 2011 (91%)3. The combined end of study disclosure rate for the four years (2009 to 2012) is 90% (1096/1222). The disclosure rate of results at 12 months has shown a steady increase year on year – 71% (317/447) in 2009, 81% (116/144) in 2010, 86% (186/216) in 20113 and 90% (307/340) in 2012, suggesting that timely disclosure is now more consistently achieved. This increase in disclosure over time is supported statistically by trend analysis (p < 0.001).

It is interesting to note that many of the European MAHs had staff with specific responsibility for ensuring that transparency commitments are fulfilled, indicating the increasing commitment of the pharmaceutical industry to transparency initiatives. We also found that the responses from the companies were on the whole prompt and comprehensive. However, where a medicine had been affected by one or more licensing deals and/or mergers or acquisitions, disclosure of trial results was sometimes delayed, and the current European MAH may not always have direct access to the relevant information.

Although the trials in this study all related to new medicines approved in 2012, some of the early phase trials for several of the medicines still dated back more than ten years, preceding both the earliest collective industry commitments to disclosure of clinical trial information (through the IFPMA Joint Position Paper of 2005, updated in 2008 and 20094) and the publication of the International Committee of Medical Journal Editors (ICMJE) principles5, at a time when the first functioning public registry (ClinicalTrials.gov) was beginning to be routinely used. Therefore, it is not surprising that we still found that some of these older trials, which may have pre-dated registration commitments, have not been registered. For these trials, results disclosure relied solely on publication in the literature, which, at that time, was often not routinely undertaken for individual early phase trials.

With reference to the sensitivity analysis, it is noteworthy that none of the trials were unevaluable due to missing or conflicting information, for example trial completion dates. A larger proportion of trials relied upon conference abstracts for initial disclosure within 12 months than previously but, even excluding all of these, the disclosure rate remains above 80%.

A number of studies have investigated the disclosure of results of trials in the light of the various commitments, initiatives and regulations. As we noted before2^,3, since these have assessed different trial sub-sets, different means of disclosure, defined requirements or a specific registry, it is not surprising that the results are variable and difficult to compare.

For example, a recent White Paper6 found that clinical trial transparency is in a healthier state than shown in previous studies, when multiple information sources are reviewed. The analysis focused on nearly 7500 phase II and III industry-sponsored trials completed between 1 January 2008 and 1 June 2012 and found that 78% of trials had results either published in a peer-reviewed journal or as a conference abstract, or posted on ClinicalTrials.gov.

In contrast, a recent study that focused on the US FDA requirement to post summary results on ClinicalTrials.gov reported that only 13.4% of 13,327 trials (completed or terminated between 1 January 2008 and 31 August 2012) had disclosed summary results on the registry within 12 months of completion7. The population examined included a mix of trials of different phases, interventions, sizes and durations supported by a range of sponsors (although the largest funding source was industry at 65.6%). These findings differ markedly from the disclosure rates for industry-sponsored trials associated with recently approved medicines in our previous3 and current study. In part this variation may be explained due to the recognition by the authors, following a detailed sample review of approval dates and labeling information, that up to 45% of industry-funded trials may not have been required to report results. Based on this review, the authors estimated that during the five-year study period, approximately 79 to 80% of industry-funded trials reported summary results or had a legally acceptable reason for delay. In addition, the study focused only on disclosure through posting of results in a single registry; the inclusion of disclosure through journal publication and additional registries would likely increase result reporting rates for all groups.

In addition, a recent analysis of a random sample of 400 phase II–IV trials registered in ClinicalTrials.gov and completed in 2008 found that only 70% had results disclosed, either in a peer-reviewed journal or posted on the registry, within four years of completion8. The authors note that transparency rates for studies completed in later years might behave differently. An earlier study by Bourgeois et al.9 pointed out that although 66.5% of industry sponsored trials (registered on ClinicalTrials.gov between 2000 and 2006) had been published, when those for which results were posted on a public site were included, the overall disclosure rate reached 88%.

Although recent disclosure rates for company-sponsored trials are encouraging, there is still room for improvement in a number of areas. Due to the multiplicity of registries, there is still considerable duplication of effort; a number of sources have to be searched to have confidence that as much relevant information as possible on a medicine and its associated clinical trials will be found.

It is important that all sponsors and responsible parties (researchers, institutes and pharmaceutical companies) work to the same standards of information disclosure (ideally through a single registry or at least to the same format). The variation between registries in terms of scope, format and content means that there remains some inconsistency, such as whether phase I trials (particularly in healthy volunteers) should be included or not and if whole-trial information can be easily extracted where study centers are in multiple countries. In addition, there is still a historical problem with publications not being clearly linked (by inclusion of the trial registration numbers in publication abstracts or PubMed indexing) to the trials they report. Once this link is routinely included by all authors and journals, it will be much easier to determine precise disclosure rates.

Prospectively, registration of all clinical trials is now a requirement, and measures to enhance transparency continue to be expanded. The EU, as well as the US, now requires summary results to be disclosed, and over the last year, further progress has been made with the new EU Clinical Trials Regulation10 being accompanied by a new EMA policy on proactive publication of clinical study reports11. This will complement the ‘joint principles for responsible clinical trial data sharing’ issued by the industry associations of North America and Europe12, enhancing the availability of detailed data to support further research. In the US, new proposals have been published by the Department of Health and Human Services (HHS) and National Institutes of Health (NIH) with the intent of widening the scope of both trial registration and summary results reporting in ClinicalTrials.gov13.

Limitations

The limitations associated with this study have been detailed previously3. Firstly, limitations relate to the availability of information in the public domain, including the potential for double-counting and/or conflicting information due to duplication across multiple sources, as well as the difficulty of matching journal publications to registered trials if trial identifiers are not included in the publication abstract or the journal citation is absent from the registry record. Secondly, this is a quantitative study; we counted the number of trials for which results have been disclosed in a variety of formats, but did not assess whether the planned primary and secondary endpoints had been fully reported. Finally, we did not assess trial registration, and would not have been able to identify a trial if it had not been included in an EPAR or a registry.

Conclusion

In this follow-up study, the increase in results disclosure within 12 months to 90%, (up from 71% for trials associated with 2009 approvals), suggests the industry is now achieving disclosure in a timely manner more consistently than before. The overall disclosure rate at study end of 92% was similar to that recorded in our previous study, indicating that the improvement in disclosure observed over three years of European approvals has been maintained for company-sponsored trials associated with new medicines approved in 2012.

Transparency

Declaration of funding

This study was funded by the Association of the British Pharmaceutical Industry (ABPI). The study was designed by B.R.D. and B.R., and the research carried out by B.R.D. and a medical information specialist from Livewire Communications. The ABPI represents the UK-based biopharmaceutical industry.

Declaration of financial/other relationships

B.R. has disclosed that at the time of the study he was a full time Medical, Innovation and Research Director at the ABPI. B.R.D. has disclosed that he is a freelance consultant in Pharmaceutical Marketing and Communications.

CMRO peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

The authors thank the Board of Management of the ABPI for supporting the conduct of the study. Professor Stephen Send of the Luxembourg Institute of Health advised on the applicability of statistical tests, and performed the time trend analysis. B.R.D. managed the project on behalf of Livewire Communications. Ros Lea from Livewire Communications assisted with the research.