Reporting of results of interventional studies by the information service of the National Institutes of Health

Abstract

The Food and Drug Administration Amendments Act of 2007 mandated that sponsors of applicable studies must provide results within one year of study completion. We aimed to analyze the factors associated with reporting of results from interventional studies registered on ClinicalTrials.gov. On May 20, 2010, we retrieved 20 available fields from 57,233 closed studies on the website and identified 31,161 interventional studies that were required to post results. We compared the proportion of studies with results versus studies without results by age, gender, and disease status of participants, by interventions, sponsors, phase of clinical trials, and completion dates. The results of studies were reported for 4.7% of applicable studies, 8% of industry-sponsored studies, 7.5% of Phase II and 6.5% of Phase IV clinical trials, 4.9% of drug studies, and 0% of genetic studies. Withdrawn (n = 486) and suspended (n = 414) interventions did not provide results. The percentage of studies with results varied from 0% to 21% among different sponsors. The first studies with results were completed in 1992. The proportion of studies with results increased over time. Completion dates were not available for 7446 studies. The database does not have fields available to facilitate routine analysis of the rate of compliance with federal law for posting results. The analysis of accuracy of the protocols in relation to the results and publications is not possible without time-consuming evaluation of individual postings and individual publications.

Keywords:

• registries
• research standards
• access to information
• clinical trials
• disclosure

Introduction

Transparency in designing, conducting, and reporting of human experiments and observational studies is essential to guarantee the integrity of clinical research.¹ Registration of clinical trials with information about study sponsors and protocols makes clinical research more transparent. In 2000, the National Institutes of Health (NIH) requested that clinical trials assessing pharmacologic treatments for serious or life-threatening diseases be registered in the ClinicalTrials.gov online database. This website was developed by the National Library of Medicine as an information service for the NIH and the US Department of Health and Human Services.² However, at that time, NIH policy did not require mandatory registration of all human studies. In 2005, the World Association of Medical Editors started to require mandatory registration of all clinical studies as a condition of publication.³ Sponsors must now provide the World Health Organization (WHO) with a minimum dataset of information, including details of study design, recruitment activities, ethics review of research, target sample size, conditions of eligibility for subjects to participate in the study, and primary and secondary outcomes.⁴ Stakeholders can find detailed information about study protocols on ClinicalTrials.gov, but study results are not yet consistently available online.

Selective publication of positive results and outcomes^5–7 led to further scrutiny and called for public disclosure of study results. The Food and Drug Administration Amendments Act (FDAAA) of 2007 mandated that sponsors of applicable studies must provide study flow, baseline subject characteristics, and outcomes after active and control interventions within one year of study completion.⁸ The FDAAA regulations defined the following clinical trials as needing to adhere to the requirements: Phase II–IV interventional studies, studies involving any drugs, biological products, or medical devices regulated by the FDA, studies having at least one site in the US or which are conducted under an investigational new drug application or investigational device exemption; and studies initiated or ongoing as of September 27, 2007, or later.^9–12 Results can be posted within three years of completion of the study for trials investigating off-label use of previously approved drugs.

Registration of clinical trials on ClinicalTrials.gov improved the transparency of clinical research tremendously. ^13–15 Stakeholders can find the WHO minimum dataset for the design of 92,385 studies.¹⁶ Harvard University has recognized the achievement of this website with their Innovations in American Government Award.¹⁷ The degree of sponsor compliance with federal law in providing results of studies on ClinicalTrials.gov has not been examined as yet. We aimed to examine the completeness of the posted study designs and factors associated with reporting study results.

Methods

We retrieved all closed studies from ClinicalTrials.gov as of May 20, 2010. We retrieved all 20 available fields, including the ClinicalTrials.gov identifier, age group, gender, disease status of the eligible subjects, and examined interventions, recruitment status, study sponsors, study type and design, phase of clinical trials, start and completion dates, and posting of study results. Field locations were as described online at http://prsinfo.clinicaltrials.gov/definitions.html. We used the exact data provided by the sponsors. We further categorized interventions as behavioral, biological, device, dietary supplements, disease management, drug, education, genetic, procedural, and exercise. We also redefined conditions into larger diagnostic categories with the first disease stated. For example, when the condition was defined as “arthralgia, pain assessment”, we analyzed it under the category of “arthralgia”. The available data do not have a single field to define the studies that are applicable to the US Public Law 110-85 (FDAAA), Title VIII, Section 801 for mandatory reporting of results. Therefore, we defined closed not-recruiting interventional trials, excluding Phase 0–I trials, as applicable to comply with FDAAA regulations.

We calculated descriptive frequency statistics without formal hypothesis testing because we did not sample the data but rather analyzed all closed studies available. We then compared the proportions of studies having results with the proportions of studies without results in categories of age, gender, disease status, interventions, study sponsorship, types, and completion dates. All calculations were performed with frequency procedure using SAS 9.1 software (SAS Institute Inc., Cary, NC).

Results

We retrieved 57,299 records but eliminated 66 records with misplaced fields, leaving a total of 57,233 records for analysis. We analyzed the completeness of the minimum dataset and found that 393 studies did not provide the gender for eligible subjects, 287 studies did not specify the type of study, and 5908 did not specify intervention or exposure (Table 1). We noticed a marked inconsistency in the classification and reporting of patient conditions, which made statistical analysis difficult. For example, “non small cell lung cancer” was reported variously as “non-small cell lung cancer”, “non small cell lung carcinoma”, or “NSCLC”. More than 50% of the studies included adult subjects, and 85% of all studies recruited both genders. Children and seniors were included in a very small proportion of studies. More than 60% of closed studies were completed, and more than 65% of closed studies examined the effects of pharmacologic treatments. A total of 39% of all studies were sponsored by industry. Most of the studies (97%) did not have their results posted on the website.

Table 1 Distribution of the studies closed in www.clinicaltrials.gov on 20 May 2010

Category	Frequency	Percentage
Age
Adult	10,254	17.92
Adult/senior	32,678	57.1
Child	3428	5.99
Child/adult	2544	4.44
Child/adult/senior	8158	14.25
Senior	171	0.3
Gender	Frequency missing (n = 393)
Both	48,224	84.84
Female	5408	9.51
Male	3208	5.64
Funding sources
Industry	22,288	38.94
Industry with other funding sources	4835	8.45
Recruitment
Active, not recruiting	13,751	24.03
Approved for marketing	16	0.03
Completed	36,992	64.63
Enrolling by invitation	1330	2.32
No longer available	24	0.04
Suspended	587	1.03
Temporarily not available	7	0.01
Terminated	3554	6.21
Withdrawn	685	1.2
Withheld	287	0.5
Study types	Frequency missing (n = 287)
Expanded access	47	0.08
Interventional	48,859	85.8
Observational	8040	14.12
Recruitment
Active, not recruiting	13,751	24.03
Approved for marketing	16	0.03
Completed	36,992	64.63
Enrolling by invitation	1330	2.32
No longer available	24	0.04
Suspended	587	1.03
Temporarily not available	7	0.01
Terminated	3554	6.21
Withdrawn	685	1.2
Withheld	287	0.5
Study results
Has results	1793	3.13
No results available	55,440	96.87
Phases	Frequency missing (n = 16,589)
Phase 0	176	0.43
Phase I	7163	17.62
Phase II	12,775	31.43
Phase II	10,657	26.22
Phase II–II	1182	2.91
Phase IV	6488	15.96
Phase I–II	2203	5.42
Interventions	Frequency missing (n = 5908)
Behavioral	3297	6.42
Biological	3947	7.69
Device	3303	6.44
Diet	112	0.22
Dietary supplement	912	1.78
Disease management	8	0.02
Drug	33,605	65.47
Education	66	0.13
Genetic	250	0.49
Other	1815	3.54
Procedure	3688	7.19
Radiation	207	0.4
Exercise	115	0.22

Of 31,161 applicable closed interventional studies, 4.5% had results available. Proportions of studies with results contained similar age and gender groups (Table 2). Studies with children as subjects tended to report results more frequently. Industry-sponsored studies reported results more often (8.1%) than nonindustry-funded studies (Table 3). Phase III and IV clinical trials reported results more often (total 14%) compared with Phase II trials (3%, Table 3). Suspended interventions (n = 414) and withdrawn interventions (n = 486) did not provide results or reasons for the cause of suspension or withdrawal (Table 3). The first studies containing results (n = 3) were sponsored by Merck and added to the database in 2009. These three studies are listed as NCT00882440, NCT00886600, and NCT00887250, and were completed in 1992. Among the applicable closed interventional studies, 7446 did not provide a completion date. The proportion of the studies with results increased over time (see Figure 1).

Figure 1 Time trend in percentage of interventional studies with results among all applicable closed studies.

Table 2 Age and gender distribution of the interventional, active, not recruiting studies applicable to reporting the results (Phase 0–I excluded)

Age	Gender	Has results	No results available	n	Percentage with results
Adult	Both	151	3807	3958	3.82
	Female	39	876	915	4.26
	Male	6	410	416	1.44
	Total	196	5093	5289	3.71
	Frequency missing (n = 10)
Adult/senior	Both	787	15,719	16,506	4.77
	Female	46	1348	1394	3.30
	Male	28	708	736	3.80
	Total	861	17,775	18,636	4.62
	Frequency missing (n = 16)
Child	Both	157	1951	2108	7.45
	Female	6	34	40	15.00
	Male	0	31	31	0.00
	Total	163	2016	2179	7.48
Child/adult	Both	30	1066	1096	2.74
	Female	10	151	161	6.21
	Male	1	23	24	4.17
	Total	41	1240	1281	3.20
	Frequency missing (n = 1)
Child/adult/senior	Both	113	2987	3100	3.65
	Female	12	385	397	3.02
	Male	4	119	123	3.25
	Total	129	3491	3620	3.56
	Frequency missing (n = 35)
Senior	Both	4	82	86	4.65
	Female	0	8	8	0.00
	Male	0	0	0
	Total	4	90	94	4.26

Table 3 Funding distribution, phases, and completion status of the interventional, active, not recruiting studies applicable to reporting of results (Phase 0–I excluded)

Funding source	Has results	No results available	n	Percentage with results
Total	1394	29,767	31,161	4.47
Industry	1126	12,789	13,915	8.09
Industry with other sources	72	2644	2716	2.65
National Institutes of Health	23	2214	2237	1.03
Other/National Institutes of Health	19	1367	1386	1.37
Other/unknown	15	840	855	1.75
Network/National Institutes of Health	2	665	667	0.30
US Federal	5	444	449	1.11
National Institutes of Health/other	0	366	366	0.00
Phases
Total	1268	23,268	24,536	5.17
Phase I–II	26	1444	1470	1.77
Phase II	283	8788	9071	3.12
Phase II–II	19	798	817	2.33
Phase II	613	7555	8168	7.50
Phase IV	327	4683	5010	6.53
Frequency missing (n = 6625)
Completion status
Completed	1243	25,390	26,633	4.67
Enrolling by invitation	0	768	768	0.00
Suspended	0	414	414	0.00
Terminated	151	2709	2860	5.28
Total	1394	29,767	31,161	4.47
Withdrawn	0	486	486	0.00

Among the applicable 31,161 closed interventional studies, the studies of hypertension and influenza reported results more often (7.5% and 9.7%, respectively, Table 4). The most commonly reported examined disease states included breast cancer, schizophrenia, and pain. Most of the closed studies of these conditions did not report the results (Table 4). The proportion of studies with results varied substantially between different sponsors (Table 5). Several pharmaceutical firms, including GlaxoSmithKline, Pfizer, Merck, and Eli Lilly and Company, sponsored more than 50 studies each and provided the results for more than 10% of the total sponsored studies. Two pharmaceutical firms, ie, Alcon Research and Eli Lilly and Company, provided results for more than 20% of applicable sponsored studies. Several sponsors did not provide results for funded studies. For instance, the National Cancer Institute sponsored 208 closed interventions without results and Memorial Sloan-Kettering Cancer Center sponsored 190 closed interventions without results. Genetic treatments were examined in 48 closed studies. None of the genetic studies provided results (Table 6). The studies of several drug, including zidovudine, risperidone, and vildagliptin, did not report their results.

Table 4 Distribution of conditions of subjects in closed interventional, active, not recruiting studies applicable to reporting of results (Phase 0–I excluded)

Conditions	Has results	No results available	n	Percentage with results
Total	1394	29,763	31,157	4.47
Largest number with results
Diabetes	99	1378	1477	6.70
Hypertension	35	435	470	7.45
Human immunodeficiency virus	33	1165	1198	2.75
Asthma	30	513	543	5.52
Largest number closed studies
Breast cancer	16	490	506	3.16
Schizophrenia	18	431	449	4.01
Pain	13	342	355	3.66
Obesity	4	351	355	1.13
Leukemia	4	319	323	1.24
Lymphoma	2	320	322	0.62
Prostate cancer	12	301	313	3.83
Osteoarthritis	13	245	258	5.04
Influenza	25	232	257	9.73
Cardiovascular disease	3	233	236	1.27
Colorectal cancer	2	233	235	0.85
Rheumatoid arthritis	12	219	231	5.19
Major depression	13	206	219	5.94
Lung cancer	3	216	219	1.37
Depression	3	210	213	1.41
Anemia	12	192	204	5.88
Frequency missing (n = 4)

Table 5 Sponsors of the closed interventional, active, not recruiting studies applicable to reporting of results (Phase 0–I excluded)

Sponsors	Has results	No results available	n	Percentage with results
Total	1394	29,767	31,161	4.47
Largest number with the results
GlaxoSmithKline	115	728	843	13.64
Pfizer	99	604	703	14.08
Merck	93	412	505	18.42
Eli Lilly and Company	76	283	359	21.17
Schering-Plough	43	220	263	16.35
Boehringer Ingelheim Pharmaceuticals	37	181	218	16.97
Sanofi-aventis	36	590	626	5.75
Alcon Research	35	131	166	21.08
UCB Inc.	33	135	168	19.64
Abbott	25	163	188	13.30
Bayer	23	206	229	10.04
Wyeth	23	241	264	8.71
Takeda Global Research and Development Center Inc.	21	85	106	19.81
Results number of closed applicable studies
Novartis Pharmaceuticals	5	694	699	0.72
AstraZeneca	17	422	439	3.87
National Institute of Allergy and Infectious Diseases	8	386	394	2.03
National Heart, Lung, and Blood Institute	6	337	343	1.75
Department of Veterans Affairs	1	324	325	0.31
National Institute of Mental Health	4	239	243	1.65
National Cancer Institute	0	208	208	0.00
Memorial Sloan-Kettering Cancer Center/National Cancer Institute	0	190	190	0.00
Hoffmann-La Roche	4	183	187	2.14
Bristol-Myers-Squibb	17	168	185	9.19
Johnson and Johnson Pharmaceutical Research and Development, LLC	7	163	170	4.12
Novo Nordisk	12	155	167	7.19
National Institute of Diabetes and Digestive and Kidney Diseases	2	159	161	1.24
Amgen	3	150	153	1.96
National Center for Complementary and Alternative Medicine	2	132	134	1.49
Astellas Pharma Inc	1	121	122	0.82
North Central Cancer Treatment Group/National Cancer Institute	0	114	114	0.00
National Institute on Drug Abuse	1	109	110	0.91

Table 6 Treatments that were examined in the closed interventional, active, not recruiting studies applicable to reporting of results (Phase 0–I excluded)

Interventions	Has results	No results available	n	Percentage with results
Total (n = 4 missing)	1394	29,763	31,157	4.47
Biological	165	1740	1905	8.66
Device	84	1914	1998	4.20
Drug	1045	20,342	21387	4.89
Genetic	0	48	48	0.00
Interventions tested in largest number of studies
Topiramate	0	66	66	0.00
Epoetin-alfa	0	71	71	0.00
Risperidone	0	64	64	0.00
Buprenorphine	0	39	39	0.00
Zidovudine	0	33	33	0.00
Levetiracetam	1	30	31	3.23
Aripiprazole	1	28	29	3.45
Bortezomib	0	28	28	0.00
Etanercept	1	27	28	3.57
Vildagliptin	0	28	28	0.00
Thalidomide	0	25	25	0.00
Atorvastatin	1	23	24	4.17
Esomeprazole	1	23	24	4.17
Levitra/placebo	1	23	24	4.17
Pregabalin	3	20	23	13.04
Procedure: acupuncture	0	23	23	0.00
Rituximab	1	22	23	4.35
Nitrous oxide	0	21	21	0.00
Arsenic trioxide	0	20	20	0.00
Interventions with largest number of reported results
Rotigotine	9	7	16	56.25
Dexlansoprazole MR/dexlansoprazole MR/placebo	6	0	6	100.00
Pemetrexed	5	8	13	38.46
Pemetrexed/cisplatin	5	4	9	55.56
Biological: Engerix™-B	4	0	4	100.00
Telavancin/vancomycin	4	0	4	100.00
Atomoxetine	3	15	18	16.67
Duloxetine/placebo	3	7	10	30.00

Abbreviation: MR, modified release.

Of 31,161 applicable closed interventional studies, 2860 were terminated and 5.3% of 2860 reported results (Table 7). In terminated studies, the most common diseases involved were diabetes and human immunodeficiency virus. Of the terminated studies in prostate cancer, anemia, asthma, and rheumatoid arthritis, more than 10% reported results. The terminated studies of subjects with lymphoma, pain, obesity, heart failure, and colorectal cancer did not provide results. The majority of interventions that were suspended or withdrawn examined the effects of drugs (Table 8). Breast cancer and prostate cancer were among the most common conditions for trials which were suspended or had interventions withdrawn.

Table 7 Terminated interventional, active, not recruiting studies applicable to reporting of results (Phase 0–I excluded) by type of condition (shown for ≥20 total studies)

Conditions	With results	No results	n	Percentage with results
Total	151	2709	2860	5.28
Diabetes	13	135	148	8.78
Human immunodeficiency virus	5	94	99	5.05
Breast cancer	2	52	54	3.70
Lymphoma	0	53	53	0.00
Schizophrenia	2	41	43	4.65
Prostate cancer	5	35	40	12.50
Anemia	5	30	35	14.29
Pain	0	33	33	0.00
Leukemia	3	29	32	9.38
Non small cell lung cancer	7	47	54	12.96
Obesity	0	31	31	0.00
Asthma	4	24	28	14.29
Heart failure	0	27	27	0.00
Colorectal cancer	0	26	26	0.00
Osteoarthritis	1	22	23	4.35
Myeloma	0	22	22	0.00
Ovarian cancer	1	20	21	4.76
Rheumatoid arthritis	4	17	21	19.05
Crohn’s disease	1	19	20	5.00
Hypertension	1	19	20	5.00

Table 8 Patient conditions in withdrawn and suspended interventions applicable to reporting of results by type of treatment (shown if ≥10 interventions). The results are not available for all studies

Withdrawn interventions	Biological	Device	Drug	Procedure	Radiation	Total
Total	20	47	341	45	2	486
Breast cancer	3	0	12	0	0	16
Diabetes	1	0	9	2	0	12
Asthma	1	1	7	2	0	11
Human immunodeficiency virus	4	0	5	0	0	11
Suspended interventions
Total	50	43	254	41	1	414
Prostate cancer	1	0	8	1	0	11
Brain and central nervous system tumors	2	0	8	0	0	10
Human immunodeficiency virus	1	0	8	1	0	10
Leukemia	3	0	6	1	0	10
Melanoma	6	0	4	0	0	10

Discussion

We found that a statistical analysis of compliance with mandatory reporting of results was difficult to perform. Missing or inconsistently reported study details, including patient diseases, study completion dates, and reported interventions may lead to wrong conclusions about a sponsor’s compliance with federal law regarding study registration and reporting of outcomes. We could not identify a single well-defined field that has applicability status of individual studies to provide results. One variable provides information about the posting of the results. The changes in protocols and deviations from the planned presentation of the primary outcomes and safety outcomes were not easy to analyze without time-consuming evaluation of each study. Reporting of studies completed before September 2007 was available in a small proportion of the interventions, when the sponsors decided to comply.

Both clinicians and the general public need complete and accurate information about study protocols and results.⁹ Stakeholders should be able to find a clear description of interventions, including prior FDA approvals, off-label evaluations, investigational new drug applications and numbers, and investigational device exemptions, as well as the applicability of reported results.^18–20 Critical appraisal of the protocols and reported results on a regular basis by clinical epidemiologists may be worthwhile to ensure integrity of the clinical research reported on ClinicalTrials.gov. Clinicians and patients need independent access to protocols and market approval status of the tested interventions. Finally, our analysis found that none of the suspended or withdrawn studies reported either the baseline characteristics of enrolled subjects or the exact reasons for terminating the study. Posting the results of a study should be mandatory for all trials, regardless of prior FDA approval.^9,11

Our study had several limitations. We defined the applicability of studies without evaluation of the market status of individual studies. We did not analyze deviations from the protocols when reporting the results. We did not analyze whether the sponsors posted the study results in a timely manner according to the expected date. Future research should analyze time intervals between completion of the study, posting of results, and publication of the results in peer-reviewed journals.

We conclude that compliance with the requirements to post results of closed studies is low for both industry- and nonindustry-sponsored studies. The need for studies to report the results should be identified in the database during the registration of the studies.

Acknowledgment

The authors would like to thank Marilyn Eells and Michele Rockne for editing and formatting the manuscript.

Disclosure

The author reports no conflict of interest in this work.