Abstract
Retrospective outcomes studies for orphan diseases are scarce in the literature because of the significant methodological and data challenges that researchers have to overcome. Over the last two decades, many opportunities have emerged to mitigate these challenges. In this editorial we examined the common challenges in retrospective outcomes studies as well as the emerging opportunities that researchers can utilize to overcome those challenges. With the introduction of two retrospective orphan disease studies in this issue, we encourage more researchers to take advantage of the emerging opportunities to conduct and publish more retrospective outcomes studies for orphan diseases. This can provide important real-world insights into orphan diseases, including how they are being treated.
Introduction
Orphan diseases are rare clinical conditions that individually affect a small portion of the population, but collectively affect a large share of the total population. The number of affected patients for a disease to be classified as an orphan disease varies from country to country, and ranges from 1 to 8 per 10,000 patients. Yet, according to the National Institute of Health and the European Organization for Rare Diseases, about 5000–7000 orphan diseases exist, affecting 30 million AmericansCitation1,Citation2 and 6–8% of EuropeansCitation3,Citation4.
Not surprisingly, parallel to the rarity of orphan diseases is the scarcity of retrospective studies of clinical, health, and economic outcomes for orphan diseases. For example, on December 1, 2011, among all publications recorded in the PubMed database containing the phrase ‘claims database’ in their abstracts, there were 102 publications containing ‘diabetes’ in their abstracts, but no publications containing the name of any of the 100 most common orphan diseases, as listed by Rare Diseases Europe (Eurodis) and OrphanetCitation5.
Also, as reported in a forthcoming systematic literature reviewCitation6, Cheng et al. found that ‘oncology orphan drugs marketed in the U.S. have varying levels and quality of clinical evidence, and a paucity of evidence demonstrating their economic value.’ The Food and Drug Administration clearly recognizes the challenges represented in developing evidence on benefit–risk in orphan drugs, and in practice sometimes applies a different evidentiary standard in these cases. In the past, researchers have proposed that the new federal institute for ‘comparative effectiveness research’ should take a flexible and balanced approach to evidentiary standards. This would apply particularly for orphan diseases, where retrospective studies are often the best source for both comparative and effectiveness informationCitation7.
Compared with prospective studies, retrospective studies have the advantages of needing a relatively smaller study budget, having a quick turn-around time, and better real-world representation. These advantages are particularly important for studies of orphan diseases, but have not yet led to the wide use of retrospective outcomes studies for orphan diseases to support related policy and reimbursement decisions. This situation is likely related to the daunting challenges rooted in research funding, disease characteristics, data sources and methodological limitations.
Challenges
As we all know, most clinical studies are funded by industry and/or governmentCitation8: therefore, the production of information based on clinical studies is mainly driven by potential patent-protected economic returns or the public’s willingness to subsidize clinical studies. With low prevalence rates and low public awareness, orphan disease studies would be expected to produce a lower economic return or attract a smaller public demand. Therefore, they are less likely to receive research funding than more common diseases. Although the Orphan Drug Act of 1983, the Rare Disease Act of 2002 and other government legislation attempt to counteract this funding challengeCitation2,Citation9–11, most orphan disease studies would not be expected to receive the same level of financial support as that for studies of more common diseases, especially when the studies are not related to an orphan drug, or are not associated with ‘rare disease regional centers of excellence’.
Orphan diseases themselves also impose clinical challenges to retrospective studies. They often do not have commonly agreed diagnostic criteria or classification systems like more common diseases have, nor is there adequate clinical epidemiological information. As a result, when planning a retrospective study for an orphan disease, researchers often encounter difficulties in identifying patients with the orphan disease, or in adjusting for relevant co-morbid conditions.
Other challenges include: (1) under-diagnosis or delayed diagnosis of a rare disease, (2) data acquisition difficulties, (3) inadequate disease classification systems, (4) difficulties in identifying appropriate control or comparative groups, (5) difficulties in identifying and controlling for confounding factors, and (6) analytical difficulties dictated by small sample size, missing observations, outliers, measurement errors, and dependence between the compared groups.
Opportunities
Often, however, challenges come with opportunities. Over the last decade, many opportunities for retrospective outcomes studies of orphan disease have emerged. For example, remarkable collaborations between patient advocacy groups, government officials and scientists who study orphan diseases have played a significant role over the last decade in the creation and adoption of public policies and regulationsCitation12,Citation13. This collaboration will likely draw more public attention and a greater level of research funding for retrospective outcomes studies for orphan diseases.
Advances in biomedical research have also provided some opportunities. Over the last decade biomarker tests have been made available for more orphan diseases that can enable researchers to identify patients with orphan diseases more easily and more preciselyCitation2. Applications of computerized informatics in clinical practice and healthcare management have also created tremendous opportunities for retrospective orphan disease studies. For example, widely used electronic medical record systems and administrative health claims processing systems have generated many richer and more systematic data sources that reflect the natural history of orphan diseases. In addition, more and more orphan disease institutes and alliances have started to develop patient registry databases, allowing researchers to collect data and conduct studies more easilyCitation12. In recent years, we have even seen a movement in the creation of global patient registries for orphan diseasesCitation14.
From a research methodology perspective, applying new or more sophisticated approaches in retrospective orphan disease studies can help to overcome some of these challenges. For instance, within-subject study design and time-series study design can help researchers to overcome the difficulties in selecting control or comparative groups; combining multiple existing databases can help researchers to identify sufficient numbers of patients for their retrospective orphan disease studies; missing data techniques and outlier detection methods can help researchers to minimize the impact of missing data and outlier distortion; and bootstrapping re-sampling and Monte Carlo Markov chain methods can help researchers to deal with the impact of small sample size and distribution distortion.
One of us has previously argued that both regulators and payers should have flexible evidentiary standards and be willing to consider real-world dataCitation7,Citation15,Citation16. Sir Michael Rawlins, the head of the National Institute for Health and Clinical Excellence in the UK, has eloquently expressed a similar view in this Harveian Oration: ‘Experiment, observation, and mathematics, individually and collectively, have a crucial role in providing the evidential basis for modern therapeutics. Arguments about the relative importance of each are an unnecessary distraction. Hierarchies of evidence should be replaced by accepting – indeed embracing – a diversity of approaches.’
In this issue
Two retrospective orphan disease studies published in this issueCitation17,Citation18 illustrate our belief that retrospective outcomes studies for orphan diseases are difficult but achievable, and can shed important light on real-world outcomes of a treatment for an orphan disease, especially if we take advantage of the emerging opportunities described above.
The two studies combined multiple healthcare claims databases to identify as many patients as possible with subependymal giant cell astrocytoma (SEGA), and used a within-subject study design to overcome the difficulties in identifying a control group. Though using the same data source, these two papers differ in their study objectives, study period definitions, analytical methods, and outcome measures. The first study examined the prevalence rates of post-surgery SEGA diagnosis, repeated SEGA surgery and surgical complications among patients with SEGA surgery, and found that SEGA surgery was associated with statistically significant risks of having post-surgery SEGA diagnosis, requiring repeated SEGA surgery and developing surgical complications. The second study used a repeated measure analysis and bootstrapping re-sampling methods to examine the changes in the prevalence rates of SEGA-related conditions following SEGA surgery, and found that SEGA surgery was associated with increased prevalence rates of seizures, hydrocephalus, vision disorders, headaches, autism, and stroke or hemiparesis during the second through twelfth post-surgery months. Although these findings should be validated by future studies using different data sources, such as patient registries, they do shed some light on the real-world effectiveness of SEGA surgery.
To conclude, we would encourage more researchers to conduct and publish retrospective database studies for orphan diseases, helping to provide an important real-world insight into the effectiveness of treatment options.
Transparency
Declaration of funding
This editorial was not funded.
Declaration of financial/other interests
P.S. has disclosed that he is employed by Kailo Research Group, a company that has received unrestricted research grants from Novartis. L.P.G. had disclosed that he has no significant relationships with or financial interests in any commercial companies related to this article.
The CMRO peer reviewer on the editorial has disclosed that he has no relevant financial relationships.
References
- NORD. Rare disease information. Available at: http://www.rarediseases.org/rare-disease-information [Last accessed 15 November 2011]
- Griggs RC, Batshaw M, Dunkle M, et al. Clinical research for rare disease: opportunities, challenges, and solutions. Mol Genet Metab 2009;96:20-6
- Taruscio D, Capozzoli F, Frank C. Rare diseases and orphan drugs. Ann Ist Super Sanita 2011;47:83-93
- EURORDIS. Rare Diseases: Understanding This Public Health Priority. Available at: http://www.eurordis.org/IMG/pdf/princeps_document-EN.pdf [Last accessed 2 December 2011]
- Eurodis, Orphanet. Rare disease in numbers: preliminary report from an on going bibliographic study initiated by Eurordis in partnership with Orphanet. Available at: http://ec.europa.eu/health/archive/ph_threats/non_com/docs/rdnumbers.pdf [Last accessed 15 November 2011]
- Cheng MM, Ramsey SD, Devine EB, et al. Systematic review of comparative effectiveness data for oncology orphan drugs. Am J Manag Care 2012;18:47-62
- Garrison LP Jr, Neumann PJ, Radensky P, et al. A flexible approach to evidentiary standards for comparative effectiveness research. Health Aff (Millwood) 2010;29:1812-17
- Dorsey ER, de Roulet J, Thompson JP, et al. Funding of US biomedical research, 2003–2008. JAMA 2010;303:137-43
- Braun MM, Farag-El-Massah S, Xu K, et al. Emergence of orphan drugs in the United States: a quantitative assessment of the first 25 years. Nat Rev Drug Discov 2010;9:519-22
- Holtzclaw Williams P. Policy framework for rare disease health disparities. Policy Polit Nurs Pract 2010;12:114-18
- Wellman-Labadie O, Zhou Y. The US Orphan Drug Act: rare disease research stimulator or commercial opportunity? Health Policy 2009;95:216-28
- Dunkle M, Pines W, Saltonstall PL. Advocacy groups and their role in rare diseases research. Adv Exp Med Biol 2010;686:515-25
- Tambuyzer E. Rare diseases, orphan drugs and their regulation: questions and misconceptions. Nat Rev Drug Discov 2011;9:921-9
- Rubinstein YR, Groft SC, Bartek R, et al. Creating a global rare disease patient registry linked to a rare diseases biorepository database: Rare Disease-HUB (RD-HUB). Contemp Clin Trials 2010;31:394-404
- Takemoto SK, Arns W, Bunnapradist S, et al. Expanding the evidence base in transplantation: the complementary roles of randomized controlled trials and outcomes research. Transplantation 2008;86:18-25
- Garrison LP Jr, Neumann PJ, Erickson P, et al. Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value Health 2007;10:326-35
- Sun P, Kohrman M, Liu Z, et al. Outcomes of resecting subependymal giant cell astrocytoma (SEGA) among patients with SEGA-related tuberous sclerosis complex: a national claims database analysis. Curr Med Res Opin 2012;28(4)
- Sun P, Krueger D, Liu Z, et al. Surgical resection of subependymal giant cell astrocytomas (SEGA) and changes in SEGA-related conditions: a US national claims database study. Curr Med Res Opin 2012;28(4)