Clinical trial transparency: an assessment of the disclosure of results of company-sponsored trials associated with new medicines approved recently in Europe

Abstract

Background:

Previous studies have raised concerns around the transparency and disclosure rates of clinical trial results on clinical trial registries and in the scientific literature. The objective of this study was to assess the timely disclosure in the public domain of results of company-sponsored clinical trials related to all new medicines approved by the European Medicines Agency (EMA) over a recent 3 year period.

Methods:

The study surveyed various publicly available information sources for both clinical trial registration and disclosure of results (including clinical trial registries, the International Federation of Pharmaceutical Manufacturers and Associations [IFPMA] Clinical Trials Portal, EMA European Public Assessment Reports and PubMed), searched from 27 December 2012 to 31 January 2013. The study covered all 53 new medicines (except vaccines and fixed-dose combinations) approved for marketing by 34 pharmaceutical companies by the EMA in 2009, 2010 and 2011. It included all completed company-sponsored clinical trials conducted in patients and recorded on a clinical trial registry and/or included in an EPAR.

Outcome measure and results:

The main outcome measure was the proportion of trials for which results had been disclosed on a registry or in the scientific literature either within 12 months of the later of either first regulatory approval or trial completion, or by 31 January 2013 (end of survey). Of the completed clinical trials associated with all 53 new medicines approved by the EMA between 2009 and 2011, 77% had results disclosed within 12 months. By 31 January 2013, this had increased to 89%. Rates of results disclosure within 12 months were 71%, 81% and 86% for new medicines approved in 2009, 2010 and 2011 respectively. Disclosure increased to 86%, 93% and 91% respectively by 31 January 2013. Although this was a purely quantitative study which did not aim to assess the content of disclosure against any specific requirements, limitations relating to a number of difficulties in finding all relevant data from multiple sources in the public domain were captured.

Conclusions:

Results of over three-quarters of all company-sponsored clinical trials related to new medicines recently approved by the EMA were disclosed within a year of completion or regulatory approval, and almost 90% were disclosed by 31 January 2013, suggesting transparency is now better than has sometimes been reported previously.

Key words::

• Clinical trials
• Publication bias
• Results disclosure
• Transparency
• Trial registration
• Trial registries

Introduction

It is widely accepted that, for a number of compelling reasons, all clinical trials involving patients should be registered in advance and the results published. Firstly, there is an ethical duty to the patients involved that knowledge gained as a result of their participation in an experiment should be publicly available for the benefit of future patients1,2. Secondly, there is an ongoing requirement for the value of medical interventions to be assessed through systematic reviews of the available clinical data3,4, and to do this reliably, independent researchers need to be able to access the protocols and results of all clinical trials pertaining to a medical intervention. In addition, investigators need all the available evidence before designing and initiating further trials to prevent avoidable risks to patients from unnecessary duplication1. In view of the above, under-reporting of clinical trial results has been described as ‘scientific misconduct’5. The risks associated with publication bias have been well documented since the 1980s; a study by Simes6 is often cited as the first demonstration of the value of a comprehensive clinical trial registry. For medicines approved before 2003, between 22% and 57% of supporting trials submitted in new drug applications to the Food and Drug Administration (FDA) remained unpublished up to 5 years after approval7,8. For many years, a research culture existed where clinical studies reporting significant or positive results were more likely to be submitted and published than those with non-significant or negative results9,10 and two recent editorials11,12 once again drew attention to the issue of publication bias.

Today, it is a legal requirement in the US that all applicable clinical trials should be registered in advance13 and the reporting of results of these trials on the national registry ClinicalTrials.gov was made mandatory in the FDA Amendment Act (FDAAA) of 200714. This defines ‘applicable clinical trials’ as “controlled clinical investigations, other than phase 1 clinical investigations, of drugs or biological products subject to Food and Drug Administration (FDA) regulation” and further states that these generally include:

“… interventional studies (with one or more arms) of FDA-regulated drugs, biological products or devices that meet one of the following conditions:

• The trial has one or more sites in the United States

• The trial is conducted under an FDA investigational new drug application or investigational device exemption

• The trial involves a drug, biologic or device that is manufactured in the United States or its territories and is exported for research”

In Europe, trial registration in the European Register (EudraCT) within the scope of the EU Clinical Trials Directive15 has been implemented since 2004. The EudraCT database is currently being upgraded to make summary results of all clinical trials (since 1 May 2004) publicly accessible16, with the recent release of EudraCT v9 making this achievable17. Furthermore, as the EU Clinical Trials Directive is being revised as a Clinical Trials Regulation, discussions are ongoing on the introduction of new clinical trial transparency measures18. Meanwhile, the European Medicines Agency (EMA) is currently consulting on a new policy for proactive publication of clinical trial data with a view to introducing this from 201419. The principle of transparency is further supported by the International Committee of Medical Journal Editors (ICMJE) and its policy of mandatory registration of clinical trials as a condition for manuscript submission20.

In 2005, the International Federation of Pharmaceutical Manufacturers and Associations (IFPMA), representing the global pharmaceutical industry, prepared a Joint Position Paper21 which called for the routine disclosure of clinical trial information (including results) through registries and databases, and this was updated in 200922. While member companies from the Pharmaceutical Research and Manufacturers of America (PhRMA), the European Federation of Pharmaceutical Industries and Associations (EFPIA) and the Japan Pharmaceutical Manufacturers Association (JPMA) signed up to these principles, implementation was left to the individual companies and has been somewhat variable in terms of both completeness and timing. A similar industry Joint Position Paper was also produced to cover the publication of results of clinical trials in the scientific literature23.

In spite of all these initiatives, studies have shown that not all published clinical trials have been adequately registered24, that a significant proportion of registered trials remain unpublished25–27, and that some trials which are registered on ClinicalTrials.gov have failed to post results within the requirements of the FDAAA28–30. Prayle et al.30 found that, of the trials on ClinicalTrials.gov which completed in 2009 and met the criteria for mandatory registration and summary results posting, 78% had failed to post summary results in the required time frame. However the FDA has since contested this figure, giving several reasons why the methodology overestimates non-compliance with data reporting laws31,32. One point not contested was that reporting rates were higher for industry-sponsored trials than for those sponsored by governmental or academic institutions31,32.

In December 2012, amid continuing concerns and publicity around the transparency and disclosure of clinical trial information, and a year after the Health Research Authority was created in the UK to protect and promote the interests of patients and the public in health research, the House of Commons Select Committee on Science and Technology announced an inquiry into clinical trials and disclosure of data33. In light of the variation in reported disclosure rates, and in response to the call for evidence by the House of Commons Select Committee, the Association of the British Pharmaceutical Industry (ABPI), in its role representing innovative research-based biopharmaceutical companies in the UK, initiated this study. To comply with the principle of transparency, currently clinical trials can be registered on any one (or more) of a number of free, publicly accessible, internet-based registries which may depend on the location of the clinical trial or the registering company.

We set out to systematically research the clinical evidence freely available in the public domain for company-sponsored clinical trials related to new medicines recently approved for use in patients by the EMA. We did not limit this assessment to a single registry or to prevailing laws or requirements in specific territories, and we counted either posting of summary results in a clinical trial registry or publication in the scientific literature as evidence of disclosure. To our knowledge this is the first study to assess the proportion of company-sponsored clinical trials for which results were publicly disclosed by searching a range of clinical trial registries and databases, and/or by publication in the scientific literature.

The objective of the study was to assess the timely disclosure in the public domain of company-sponsored clinical trials conducted in patients and relating to all the new medicines approved by the EMA over a recent 3 year period.

Methods

We identified 53 new medicines (New Active Substances, NASs) approved by the EMA (excluding vaccines and combination products) in 2009, 2010 and 2011 licensed to 34 different companies. Dates of approval in both Europe and the US (where applicable) were identified from the EMA (www.ema.europa.eu/ema) and FDA (www.fda.gov) websites.

For many years the principle of clinical trial registration did not specify a particular public registry; therefore in this study, when attempting to identify all company sponsored trials associated with any given medicine, several sources for both registration and results disclosure were explored. For each new medicine, four main clinical trial information sources were searched: the company registry of the marketing authorisation holder (if available), the established US national registry (ClinicalTrials.gov), the European Clinical Trials Register (EudraCT, clinicaltrialsregister.eu), and the International Federation of Pharmaceutical Manufacturers and Associations’ (IFPMA) Clinical Trials Portal (clinicaltrials.ifpma.org). Additional sources not routinely searched, but which helped to identify duplicate trials, included the WHO International Clinical Trials Registry Platform (ICTRP http://apps.who.int/trialsearch), which, as with the IFPMA portal, links to other primary sources; the International Standard Randomised Controlled Trial Number Register (ISRCTN controlledtrials.com); and the Japan Pharmaceutical Information Centre (JAPIC clinicaltrials.jp). We were also aware of the discontinued PhRMA site (clinicalstudyresults.org), where a number of companies had previously posted results but which was unavailable at the time of our study.

Trials were also identified from the European Public Assessment Report (EPAR) published by the EMA for every medicine granted a central marketing authorisation by the European Commission.

PubMed was searched for all published clinical trials using a search strategy including all drug synonyms for each new medicine, limiting the search to ‘clinical trial’ then viewing the abstract field or indexing for a clinical trial registry identifier to enable the trial to be linked accurately to a publication.

Data extraction

For each of the 53 new medicines, a list of completed company-sponsored trials was compiled on an Excel spreadsheet by one of a team of experienced researchers during the study period 27 December 2012 to 31 January 2013 inclusive. The primary source of information was either the company registry of the European marketing authorisation holder or ClinicalTrials.gov, with additional trials identified from other sources including the European Register and the IFPMA portal (Figure 1).

Figure 1. Systematic research methodology to identify company sponsored clinical trials relating to each new medicine and to assess the disclosure of results. IFPMA = International Federation of Pharmaceutical Manufacturers and Associations; EPAR = European Public Assessment Report.

For each trial the following information was recorded (where available): registering company, trial registry, registry identifier and any other trial identifiers, date of entry, trial title or description, phase, indication, number of patients, date of last update, date of trial completion (both primary and actual completion date if given), whether summary results were posted and date of posting, journal citation details (where a publication was referenced), and earliest publication date.

When identifying a study completion date, it was found that some sources listed a single date while ClinicalTrials.gov sometimes provided two dates, a primary completion date (date of last collection of data on the last patient’s last visit) and study completion date. Depending on the study type or therapeutic area the two dates could be found to be the same or different, or only one date was given. For consistency, we took the ‘study completion date’ as listed in each source.

All completed trials on patients as designated in the source as phase I, II, III or IV, regardless of where they were conducted, were included. There were some trials where the phase was not given or not clear; these were classed as ‘others’ and included long term extensions (which were distinct from the initial study), and post-marketing studies, including observational and chart-review studies, company meta-analyses and assessments of laboratory parameters across a number of other clinical studies.

Duplicate trials were removed as far as possible, with multiple registry identifiers listed together to aid identification. Trials in healthy volunteers and pharmacokinetic trials conducted in patients with renal or hepatic impairment (but without the target disease) were excluded (as these were not patients who might have benefited from the intervention). Compassionate use or expanded access protocols were also excluded as these generally did not yield data which can be analysed for publications. In addition, trials conducted by third party institutes (or different licensees for territories outside Europe) were excluded, except where they appeared in the EPAR (as these trials were considered central to the development programme). Interim results from long-term follow-up studies were included in our assessment (as separate trials, adding to the evidence base) if they were specifically cited in the EPAR and/or registered under a separate registry identifier. Where a medicine had been in-licensed by a company for the European market or where the originator company had been acquired, trials conducted by both the originator and the licensee or new owner were included.

Publication details of trial results identified from the registry records were compiled. Additional publications, recorded on the registry record but which preceded the trial, provided only background information or described only the protocol were excluded. Clinical trial publications identified in PubMed were matched to trials using the trial identifier (most often the ClinicalTrials.gov NCT number) when present in the abstract or indexing. Manual matching of publications with trials was only attempted if the identity of the trial was obvious from information in the published abstract e.g. protocol code, trial design, patient numbers, primary outcome measures. The earliest date of either summary results posted on a registry or publication in the scientific literature for each trial was noted, using the ePublication date for journal articles where available.

Assessment of disclosure of results

The 2005 IFPMA Joint Position on the disclosure of clinical trial information via clinical trial registries and databases21 (as well as the 2009 update22) supported the principle that all clinical trials in patients should be registered in advance on a public registry and that results should be made publicly available on clinical trials databases “no later than one year after the medicinal product is first approved and commercially available in one country” or, if completed after the first approval, “no later than one year after trial completion”22. The IFPMA Joint Position on publication of clinical trial results states that industry-sponsored clinical trials should be considered for publication in the scientific literature regardless of outcome and submission should be “wherever possible within 12 months and no later than 18 months” of either the regulatory approval or trial completion23.

This study assessed clinical trial results disclosure for each completed company-sponsored trial using the earliest date of either posting of results on a publicly available registry or database or publication in the scientific literature. Disclosure was assessed and recorded for two time points: firstly, within 12 months (of either the first regulatory approval by either the EMA or FDA [if applicable], or the date of completion of the trial if after the first approval); and secondly, at 31 January 2013, the end of the study period.

While presentations at international conferences often represent the first public disclosure of results, there are no comprehensive and publicly available sources for reliably identifying all conference abstracts. Therefore we made no additional attempt to locate conference abstracts other than the routine search of PubMed, but if their existence was brought to our attention by the European marketing authorisation holder, abstracts published in journal supplements were accepted as valid evidence of disclosure of the trial and its results for the purely quantitative purpose of this study.

For some trials, it was impossible to reach an assessment for disclosure in the 12 month timeframe: if either the date of trial completion or the date of posting of results was unclear or not stated in the source, the trial was marked as unevaluable in that timeframe. If a trial completed after 31 January 2012, so that less than 12 months had elapsed by 31 January 2013, and results had not been disclosed by then (the endpoint of our study), it was marked as unevaluable. Unevaluable trials were not entered into the assessment. In a small number of cases, the information on a trial was either so limited or unclear that it was classed unevaluable even though we felt it should be included in the overall assessment.

Repeat searches of sources were undertaken and complete listings for all 53 medicines were checked and validated by one of three senior researchers to ensure consistency of assessment.

Following the initial assessment of disclosure, a listing of queries was compiled for each medicine, where publications or summary results could not be identified, or where key dates were missing. Companies responsible for the European marketing authorisation were then consulted on these queries, to allow them the opportunity to provide additional information or to draw attention to disclosure of results which the researchers may have been unable to locate or link to a specific trial through the described methodology; if such disclosure was shown to be in the public domain, either within the 12 month target period or by 31 January 2013, the assessment was amended. However, if the company acted on the information to achieve disclosure after 31 January 2013, the assessment was not amended. Overall, this well-defined consultation process served to improve the accuracy of our assessment but did not change the extent of disclosure within the study period. Finally, the overall rates of clinical trial results disclosure for each medicine were captured in summary spreadsheets (accessible as supplementary information).

Results

The most comprehensive source of information on clinical trials conducted on the 53 new medicines assessed in this study was ClinicalTrials.gov, with a total of 2852 related trials (irrespective of sponsor) registered at the time the study was conducted (Table 1). Of these, 1179 (41%) were marked as completed. Of the 34 different companies marketing the 53 new medicines, only 13 had their own registry, so correspondingly the total number of trials identified from these registries was lower than for other sources.

Table 1. Total number of clinical trials posted on each registry relating to the 53 new medicines approved by the EMA in 2009, 2010 and 2011.

Registry	Total posted	Marked as complete
Company’s own	526	323
ClinicalTrials.gov	2852	1179
European Clinical Trials Register (clinicaltrialsregister.eu)	903	247
IFPMA portal (clinicaltrials.ifpma.org/clinicaltrials/index)	3,642	252

Number of trials includes those registered by industry and other sponsors, and ongoing and completed trials. There is considerable overlap and duplication of trial records within these four sources.

After adding completed company-sponsored trials from all sources and removing duplicates and exclusions, a total of 997 trials relating to the 53 new medicines approved over the 3 year period entered our assessment (Table 2). Of these 997 trials, 190 were unevaluable for assessment within the 12 month timeframe for one of the reasons previously described (Figure 2). Of these, 96 had been completed after 31 January 2012, that is less than 12 months before our study completion; 30 were missing a trial completion date and 64 had summary results posted but the date of posting was unknown. Out of the 807 evaluable trials, 619 had results disclosed within the 12 month timeframe, an overall disclosure rate of 77%.

Figure 2. Chart showing breakdown of trial assessment at 12 months.

Table 2. Number of completed company-sponsored clinical trials relating to 53 new medicines approved in 2009, 2010 and 2011 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months*	Results disclosed within 12 months^≠		Unevaluable at 31/01/13*	Results disclosed by 31/01/13
I/II	503	83	306/420	73%	38	400/465	86%
III	434	67	301/367	82%	56	347/378	92%
IV	41	23	11/18	61%	18	21/23	91%
Other	19	17	1/2	50%	3	16/16	100%
Total	997	190	619/807	77%	115	784/882	89%

*Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

^≠Twelve months measured from date of first regulatory approval (Europe or US), or trial completion date if later than first approval.

When additional results disclosed outside the 12 month period but before 31 January 2013 were taken into account, the number of unevaluable trials decreased to 115 (Figure 3) and the disclosure rate increased to 89% (784/882 evaluable at that date). To accurately assess a trial as disclosed at the 12 month time point, the date of trial completion and the date of posting on a registry or publication was needed. The decrease in number of unevaluable trials at 31 January 2013 was due to the fact that more trial results were available at this time point, including the 64 trials with summary results on registries classed as unevaluable at 12 months and 11 trials for which the study completion date was missing. This left 19 trials classified as unevaluable at 31 January 2013 for which a study completion date was missing. Most of these were in EPARs but had not been found in registries, did not have results, and/or were associated with either conflicting or insufficient information to make a definite assessment of undisclosed.

Figure 3. Chart showing breakdown of trial assessment at 31 January 2013.

It is interesting that the proportion of phase III trials for which results were disclosed within 12 months was higher than the proportion for phases I/II or IV (Table 2).

Clinical trials for 21, 12 and 20 new medicines approved by the EMA in 2009 (Table 3), 2010 (Table 4) and 2011 (Table 5) respectively were assessed in our analysis. Not surprisingly, there are more completed trials associated with the 2009 approvals than with the more recent approvals. For new medicines approved in 2011, 86% of trials had results disclosed in the 12 month timeframe, a slight increase compared with 81% for 2010 and 71% for 2009 approvals. However, when assessed at the 31 January 2013 time point, the disclosure rate for the trials related to new medicines approved in 2009 had increased to 86%, and, for the two more recent years, to at least 90%.

Table 3. Number of completed company-sponsored clinical trials relating to 21 new medicines approved in 2009 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months*	Results disclosed within 12 months^≠		Unevaluable at 31/01/13*	Results disclosed by 31/01/13
I/II	274	26	167/248	67%	16	213/258	83%
III	216	33	141/183	77%	28	169/188	90%
IV	25	10	9/15	60%	10	13/15	87%
Other	5	4	0/1	0%	2	3/3	100%
Total	520	73	317/447	71%	56	398/464	86%

*Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

^≠Twelve months measured from date of first regulatory approval (Europe or US), or trial completion date if later than first approval.

Table 4. Number of completed company-sponsored clinical trials relating to 12 new medicines approved in 2010 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months*	Results disclosed within 12 months^≠		Unevaluable at 31/01/13*	Results disclosed by 31/01/13
I/II	99	41	51/58	88%	9	88/90	98%
III	94	11	63/83	76%	8	75/86	87%
IV	9	7	1/2	50%	2	7/7	100%
Other	12	11	1/1	100%	0	12/12	100%
Total	214	70	116/144	81%	19	182/195	93%

*Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

^≠Twelve months measured from date of first regulatory approval (Europe or US), or trial completion date if later than first approval.

Table 5. Number of completed company-sponsored clinical trials relating to 20 new medicines approved in 2011 which had disclosed results, grouped by phase of study.

Phase	Total trials	Unevaluable at 12 months*	Results disclosed within 12 months^≠		Unevaluable at 31/01/13*	Results disclosed by 31/01/13
I/II	130	16	88/114	77%	13	99/117	85%
III	124	23	97/101	96%	20	103/104	99%
IV	7	6	1/1	100%	6	1/1	100%
Other	2	2	0/0		1	1/1	100%
Total	263	47	186/216	86%	40	204/223	91%

*Unevaluable if a key date was missing or unclear, or 12 months had not elapsed since trial completion.

^≠Twelve months measured from date of first regulatory approval (Europe or US), or trial completion date if later than first approval.

It is also noteworthy that of the total of 98 trials whose results remained undisclosed at 31 January 2013 (Table 2), nearly half (45) were early phase (I or II) trials related to new medicines approved in 2009 (Table 3).

Sensitivity analyses

Unevaluable trials

The disposition charts (Figures 2 and 3) show the sub-classification of unevaluable trials. Both at the 12 month assessment, and at 31 January 2013, 96 trials could not be evaluated as the trial had completed within the previous 12 months and therefore could not be assessed. At the 12 month assessment, 64 trials had results postings (usually on a company site) but without a validated date of posting. If these had all been assessed as disclosed within the timeframe, the disclosure rate would have increased from 77% to 78% (683/871). At the same time, 30 trials were missing a study completion date, or there was conflicting information from different sources. If these had all been assessed as undisclosed, the disclosure rate would have fallen from 77% to 74% (619/837). If both these groups were included on this basis, the overall assessment would have fallen from 77% to 76% (683/901).

There were 19 studies for which the study completion date was missing and we had insufficient or conflicting information at the time of this study, and these remained assessed as unevaluable at 31 January 2013. Many of these appeared in EPAR documents, but were not found in registries. If all these 19 had been assessed as undisclosed at 31 January 2013, the overall disclosure rate would have fallen from 89% to 87% (784/901).

Study completion dates

As stated in the methods, we used the study completion date provided by the source. In ClinicalTrials.gov, there was a separate field for primary completion date and study completion date, but often these dates were the same. The majority (503, 62%) of trials assessed at 12 months were assessed from the approval date, not the study completion date. For the remaining 304 (38%), the source often only gave a single date. Occasionally, use of the ‘primary completion date’ (where it differed from the ‘study completion date’) might have led to a different assessment at 12 months (i.e. undisclosed rather than disclosed). However, this was not quantified within the protocol of this study. This would not have affected the final figure at 31 January 2013.

Abstracts

Few of the trials (41/807 or 5%) relied solely on abstracts for assessment of disclosure at the 12 month time period. If all these were excluded, and the trials assessed as undisclosed, the overall disclosure rate at 12 months would fall from 77% to 72% (578/807). Several of these were followed up with full publications or posting of results outside the 12 month timeframe, so the effect of removing the abstracts at 31 January 2013 would have been smaller.

Discussion

This study assessed a large and comprehensive cohort of company-sponsored clinical trials related to all 53 new medicines approved by the EMA over a 3 year period. The finding that 77% of trials had disclosed results within 12 months was higher than7,25–30 or similar to8 disclosure rates reported by other studies which range from 12% to 78% up to 5 years from approval. Our intention was to examine the actual situation regarding the extent to which results of trials relating to a group of recently approved medicines were made publicly available, without reference to specific prevailing requirements. Given the wide range of years over which trials included in our assessment were actually conducted, some more than 10 years ago, together with the broad scope of inclusion (all trials carried out in patients), a disclosure rate of 100% was not expected. The overall disclosure rates of 77% within 12 months and 89% at 31 January 2013 are therefore encouraging. The increasing disclosure rate from 71% for approvals in 2009 to 86% for approvals in 2011 within the 12 month timeframe along with at least 90% disclosure of results by 31 January 2013 for trials associated with medicines approved in 2010 and 2011 shows a positive trend which may reflect a steady improvement in processes within companies to ensure timely reporting of trial results. The variations in available information across the sources resulted in some trials being classified as unevaluable, but the sensitivity analysis presented in the results suggests that this did not substantially affect the overall results.

In exploring the degree of transparency around information on clinical trials, previous studies have investigated the proportion of trials published in the scientific literature submitted in support of new drug applications7,8, trials which had been adequately registered and subsequently published24, publication rates of trial results within a single registry (ClinicalTrials.gov)25–27, and the proportion of trials in ClinicalTrials.gov with summary results posted in compliance with the FDAAA28–30. We searched a range of sources in an attempt to identify all company-sponsored clinical trials for each medicine and, for the purposes of this study, we considered either the posting of results on public registries or publication in the scientific literature as disclosure of results. In some ways, our assessment was more demanding than others, in the inclusion of a wider range of trials than may be found in a single source, in looking for disclosure of results regardless of whether the trial pre-dated the commitments to disclosure described in the FDAAA14 and the IFPMA Joint Positions21–23, and in assessing disclosure strictly within 12 months regardless of whether this was through posting on a registry or publication in a journal (where in practice the publication process may extend the actual publication time beyond 12 months). However, for some trials registered on ClinicalTrials.gov where the primary completion date might have been earlier than the ‘study completion date’ (used in the interests of consistency across all the sources) our assessment may have been less demanding for the assessment at 12 months. In practice, a large proportion of the trials assessed in our study were associated with marketing authorisation applications, so the majority of the disclosure assessments were measured from the date of first regulatory approval (rather than from the trial completion date).

The variation in reported disclosure rates either through publication or posting of summary results may be due to several factors, including study design, proportion of included trials which predate transparency initiatives, and mix of clinical trial sponsors. Historically, the results of trials supporting regulatory submissions were more likely to be published than to be posted on registries, so studies focusing only on registry postings might be expected to show lower disclosure rates for older trials. Studies have found that industry-funded trials were more likely to report results on ClinicalTrials.gov than trials funded by the National Institutes of Health or other funders28,30, although there may not be a difference in the rate of publication in the scientific literature25,26, except that industry-funded trials may be slower to reach publication25. It is noteworthy that Bourgeois et al.25 included data both for publication and for the availability of results electronically where trials were not published; combined, this would have given a figure of 88% for overall disclosure rate for industry-funded trials, similar to our figure of 89%.

Notably, the UK National Institute for Health Research Health Technology Assessment Programme achieved a 98% publication rate for all their funded studies commissioned since 2002, demonstrating the advantage of funding organisations requiring publication, irrespective of results, as part of the contractual arrangement at the time of funding34.

Our focus on all trials relating to new medicines approved by the EMA includes trials completed over a number of years rather than concentrating on a narrow time period. We found that where the ownership of a medicine had changed during development as a result of licensing deals, mergers and/or acquisitions, disclosure of trial results was often delayed or less likely to have been completed by the final owner and/or the EMA license holder (especially within a target time limit) if disclosure had not already been undertaken by the originating company who sponsored the trial. This often reflected a delay in the development of the medicine itself, so that some trials submitted to the regulatory authority were relatively old and pre-dated the initiatives around transparency of information and subsequently contributed to lower disclosure rates. In several cases our correspondence with the companies alerted them to the situation, with the result that action has since been taken to ensure that such trials are now disclosed; so that if this study was repeated today, the disclosure rates to date would be higher.

The trials associated with the 53 new medicines in this study were conducted over a period of transition in relation to the publication of trial results. Traditionally, pharmaceutical companies would have completed a medicine’s development programme, submitted the collected results of all relevant clinical trials to the regulatory authorities for marketing approval, and in parallel sought publication in peer reviewed journals. Therefore, while some of the earlier trials in our cohort achieved disclosure through journal publication, results may not have been posted on trial registries; more recently, results are more likely to have been posted on registries in advance of journal publication. When deciding to assess all trials of approved medicines, we were aware that the study would include trials conducted at various phases throughout the development period and over a wide time frame. We did not split the cohort of trials according to year of trial completion; this might have given interesting comparisons from year to year, although the numbers of trials per year would have been small.

This study evaluated the overall extent to which company-sponsored trial results had been made publicly available, rather than compliance with specific commitments to publish (whether legal or otherwise). We did not evaluate the disclosure of trials conducted by institutes or individual researchers unrelated to the originator and/or the European license holder, and, as the new medicines selected were approved in recent years, this study included relatively few trials completed after the European marketing authorisations were granted.

The framework for ensuring clinical trial transparency has been in place for some years and is expected to be further strengthened internationally with the European requirement for disclosure of results16. While recognising that the process of developing new medicines is a global undertaking, the ABPI is committed to supporting its member companies in efforts to improve clinical trial transparency, and has developed a toolkit35 to help integrate trial registration, results posting and publication (in line with the IFPMA Joint Positions22,23) into the medicines development process. The ABPI is also developing plans to continue to monitor disclosure of company-sponsored trials over the coming years, and it is hoped that this will document further improvement in the disclosure rates through consistent implementation of best practice in trial transparency.

Limitations

Firstly, this study has a number of limitations relating to the availability of information in the public domain.

1. One of the current issues with trial registration is that trials may be registered on any one or more of a number of registries. There are now a number of such registries of varying utility and covering different geographies and no single source is comprehensive. Many trials are now registered on multiple registries, indicating unnecessary duplication of effort, and also adding to the potential for double-counting. It would be considerably easier to measure clinical trial disclosure rates if there was a single global registry for all trials and their results. While the setting up of registries by individual pharmaceutical companies demonstrates a positive attitude to transparency, this was another source of potential duplication in our study. For some medicines the trial identifiers used in referencing trials in the EPAR did not match identifiers used in any registry, again increasing the risk of double-counting. The most comprehensive and user-friendly registry in terms of searching and extracting information was ClinicalTrials.gov. By contrast, although undergoing review and redevelopment, the European Register currently has fewer records, data displayed for some clinical trials is incomplete (particularly between May 2004 and March 2011)36, the trial completion status and completion date is often unclear and this registry currently carries no results. Where present, a company registry usually had the most clinical trial results, but if the date of posting was not recorded it was not possible to validate the date that results were made publicly available.

2. It was frequently difficult to match publications to registered clinical trials, either because the clinical trial identifier was not included in the PubMed abstract, or because the journal citation was absent from the registry record. By searching PubMed, Huser and Cimino37 identified associated publications for an additional 44% (22/50) of trials on ClinicalTrials.gov which did not have publication links (NCT number in the abstract or PubMed article ID in the registry record). Existing trial registry and journal policy would benefit from more rigorous implementation37. As major clinical trials often give rise to multiple publications this was a further complication when attempting to identify the earliest publication of primary outcome data. This lack of direct linkage is a particular problem with older publications, and, while the consultation process established some of the linkages, we cannot be absolutely certain that none of the trials listed as undisclosed have an associated publication.

3. The necessity of searching multiple sources also occasionally gave rise to conflicting information. The definition of ‘study completion’ is sometimes not clear, and in a small number of cases the date given for study completion differed from registry to registry. We used the stated ‘study completion date’ provided by each source, but recognise the potential for some inconsistency.

4. The trials classified as unevaluable represent a further shortcoming in the completeness of the information in the public domain (see the sensitivity analysis in the Results section).

Secondly, this study is a purely quantitative study – we counted the numbers of trials where the results have been disclosed in a variety of formats, but we made no attempt to assess whether this disclosure was complete in terms of reporting the results of all primary and secondary endpoints to the degree required for inclusion in a full systematic review.

Finally, in focusing on all trials relating to all NASs approved in a 3 year period, we covered all new medicines from that period which were approved for use in patients. However, we did not set out to assess trial registration, and we would not have been able to identify a trial if it had been conducted without being registered and then not included in the regulatory application for marketing authorisation. Therefore, we cannot be certain that we found absolutely all the trials relating to each of these medicines. In addition, we did not assess medicines which failed in development or for which approval was declined (information which would potentially be useful to researchers in the future) and we did not assess approvals related only to new indications, or new formulations (including combinations) which would have followed on from NASs approved in years before 2009.

Conclusion

The timely disclosure of clinical trial results, in accordance with the latest principles of transparency, is now being implemented to a greater extent by the global pharmaceutical industry than previously reported. The finding that results of nearly 90% of all company-sponsored trials related to medicines approved in 2009, 2010 and 2011 are now disclosed (either on a registry or in a scientific publication) is already encouraging, and the increasing commitment of the pharmaceutical industry to transparency in clinical trial reporting should lead to continued improvement in the future.

Transparency

Declaration of funding

Publication support for this study was funded by the Association of the British Pharmaceutical Industry (ABPI). The study was designed by B.R.D. and B.R. and the research was carried out by a team of medical information specialists from Livewire Communications led by B.R.D. The ABPI represents the UK-based biopharmaceutical industry.

Declaration of financial/other relationships

B.R. is full time Medical, Innovation and Research Director at the ABPI. B.R.D. is a freelance consultant in Pharmaceutical Marketing & Communications.

CMRO peer reviewers may have received honoraria for their review work. The peer reviewers on this manuscript have disclosed that they have no relevant financial relationships.

Acknowledgements

The authors thank the Board of Management of the ABPI for supporting the conduct of this study. B.R.D. managed the team provided by Livewire Communications with the expertise to carry out the searching during the study period. Ros Lea from this team also provided editorial support and advice on the draft manuscript.