Role of 5 alpha-reductase inhibitors in the management of prostate cancer

Abstract

Prostate cancer is one of the most complex and enigmatic oncologic problems in medicine. It is highly prevalent, particularly in elderly males. Unfortunately, its generally protracted and variable clinical course and high association with treatment-related morbidity raise serious questions about the ideal treatment strategy for the individual patient. 5 alpha-reductase (5AR) inhibitors have a dramatic effect on benign prostatic disease with low toxicity. Thus, there is much interest in the potential role of 5AR inhibitors in the prevention and treatment of prostate cancer. Finasteride is the only agent that has been shown in a randomized clinical trial to decrease the risk of prostate cancer with a reduction of almost 25%. Additionally, a recent analysis of the Prostate Cancer Prevention Trial (PCPT) has found that finasteride improves the performance characteristics of prostate-specific antigen (PSA) blood test as a screening tool for prostate cancer, for both cancer detection as well as for detection of high risk disease. Finally, 5AR inhibitors have been studied as a component of multimodal therapy for all stages of prostate cancer, with the goal of improving oncologic outcomes while avoiding the toxicity of medical and surgical castration.

Keywords:

• prostate cancer
• 5-α-reductase inhibitors
• finasteride
• dutasteride
• chemoprevention

Introduction

Prostate cancer is the most common noncutaneous malignancy in males. The American Cancer Society estimates that in the year 2006, a total of 234 460 men will be diagnosed with prostate cancer and 27 350 will die of this disease (Jemal et al 2006). At the start of the year 2003, nearly 2 million American men were living with prostate cancer. Most commonly a disease of elderly men, the average age at diagnosis is 68 and 27.3% of all new diagnoses are in men 75 years of age or older (Ries et al 2006).

The characteristics of prostate cancer have changed dramatically since the introduction of prostate-specific antigen (PSA)-based screening in 1986. PSA screening has led to a drastic increase in the detection rate of prostate cancer along with an associated downward stage migration. While there are no prospective, randomized studies that prove earlier detection of prostate cancer leads to decreased mortality, the reality is that widespread PSA-based screening for prostate cancer has continued. Taking into consideration that many tumors currently detected may be of an indolent nature, a current challenge is identifying the patient who may benefit from treatment for whom side effects of treatment may be acceptable.

The high prevalence and considerably lower mortality of prostate cancer, coupled with the significant potential morbidity of therapy for prostate cancer, have sparked much interest in alternative approaches against prostate cancer such as prevention (Thompson et al 2003), minimally invasive surgical therapies (Ahmed et al 2005; Tooher et al 2006) and active surveillance strategies (Carter et al 2002). Well designed randomized clinical trials addressing questions in these areas will hopefully lead to more efficacious and appropriate treatment of prostate cancer with lower disease specific mortality, while minimizing treatment related morbidity. Medical therapy with 5 alpha-reductase (5AR) inhibitors may play a role in accomplishing some of these goals.

The results from the Prostate Cancer Prevention Trial (PCPT) have sparked interest in the various roles of 5AR inhibitors in the management of prostate cancer. In this review, we examine the pharmacology of 5AR inhibitors and how this class of agents relate to the pathophysiology of prostate cancer as well as their role in prevention and treatment of prostate cancer.

Androgens and prostate cancer

The physiologic functions and pathologic conditions of the prostate, like all other endocrine glands, are regulated by numerous endogenous hormones and growth factors. Testosterone is the predominant circulating androgen in males. It is a steroid hormone, synthesized from cholesterol in Leydig cells within the interstitium of the testis. Its production is stimulated by luteinizing hormone (LH), secreted by the anterior pituitary gland in response to the cyclic release of luteinizing hormone releasing hormone (LHRH) by the hypothalamus. LHRH release, in a negative feedback fashion, is inhibited by testosterone. Greater than 95% of endogenous androgen is produced by the testis, with the remainder produced as androstendione by the adrenal cortex. This small amount of nontesticular androgen has a minimal impact on prostate function in physiologically normal males (Partin and Rodriguez 2002).

Testosterone is taken up from the systemic circulation by the prostatic glandular and stromal cells. Once within the prostate, testosterone is rapidly and irreversibly converted to dihydrotestosterone (DHT) by the enzyme 5AR. This leads to a five-fold higher concentration of DHT versus testosterone within the intracellular prostate, versus an eleven-fold higher concentration of testosterone within the circulation. DHT then binds to the androgen receptor within the cytosol, is actively transported into the nucleus, and serves as a transcription factor for prostatic gene expression and thus prostatic cellular function. The higher concentration of intracellular DHT, in addition to its higher affinity for the androgen receptor, support the importance of 5AR in normal and pathologic prostate physiology (Partin and Rodriguez 2002).

In 1974, both Walsh and Imperato-McGinley described the deficiency of 5AR as an inheritable form of pseudohermaphroditism (Imperato-McGinley et al 1974; Walsh et al 1974). Affected individuals known locally as “guevedoces”, literally “penis at 12 years of age”, were originally discovered in a remote village in the Dominican Republic where this enzyme deficiency was highly prevalent. Clinical manifestations of this disorder include the presence of ambiguous genitalia, undescended testicles, a blind vaginal pouch, a small clitoris-like phallus and absence of all internal female reproductive structures. At puberty, the phenotype of these individuals changes to a more normal male pattern manifested by development of a functional penis, deepening of the voice, muscle development, etc. However, adult males with this condition have a diminutive prostate and lack male pattern baldness, facial hair, and acne (Imperato-McGinley et al 1974). Subsequent clinical evaluation of affected individuals with magnetic resonance imaging, trans-rectal ultrasound and prostate biopsy confirmed presence of a small, atrophic prostate. PSA was undetectable and affected individuals developed neither benign prostatic hyperplasia (BPH) nor prostate cancer (Imperato-McGinley et al 1992).

Identification of individuals with congenital deficiency of 5AR has served as a naturally occurring model ultimately leading to the development of a pharmacologic inhibitor of 5AR. Finasteride is an orally active, competitive inhibitor of the nicotinamide adenine dinucleotide phosphate, reduced form (NADPH)-dependent 5AR enzyme’s type 2 isozyme. Despite its steroid structure, it has no affinity for any steroid receptors, including androgens, estrogens, and progesterone (Stoner 1992), allowing the inhibition of 5AR without concomitant binding at other physiologic locales. Guided by early animal models (Brooks et al 1981, 1982; Wenderoth et al 1983), several investigators described the ability of finasteride to suppress serum DHT to approximately 70% of baseline levels in male humans (Rittmaster et al 1989; Stoner 1990, 1992; Gormley et al 1992). Since that time, several large scale clinical trials have demonstrated the efficacy of finasteride in the medical management of BPH (Rittmaster et al 1989; Gormley et al 1992; Stoner 1992; McConnell et al 1998, 2003) and in the prevention of prostate cancer (Thompson et al 2003).

In the 1990s, dutasteride was identified as an inhibitor of both isozymes (type 1 and type 2) of 5AR. Nonselective inhibition of both 5AR isozymes produces more than a 90% reduction in serum DHT (Bramson et al 1997). While type 2 5AR is the predominant isozyme within the prostate, its inhibition by finasteride may lead to up regulation of type 1 5AR at extraprostatic sites (ie, liver and skin) with resultant paracrine effects (elevated serum DHT) on the prostate (Bramson et al 1997). Additionally, dutasteride has a significantly longer half life (~180 hours) than finasteride (5–8 hours) (Steiner 1996; Bramson et al 1997). The clinical significance of the distinctions between finasteride and dutasteride on malignant prostate pathophysiology has been postulated (Andriole et al 2004), but has yet to be confirmed in clinical trials.

In the absence of prostate cancer, 5AR inhibition has a dramatic effect on serum PSA. An approximate 50% reduction in the PSA from baseline is to be expected after 6 months of continuous therapy with either finasteride or dutasteride (Gormley et al 1992; Roehrborn et al 2002). When patients on therapy with 5AR inhibitors for 6 months or greater undergo PSA based screening for prostate cancer, the measured PSA should be doubled before comparing it with reference values in consideration for prostate needle biopsy (Andriole et al 1998). After several years of therapy with 5AR inhibitors, the measured PSA must be increased by a factor of 2.5, to account for continued PSA decrease in treated men (Etzioni et al 2005).

Prevention of prostate cancer with 5-alpha reductase inhibitors

Men with congenital deficiency of 5AR have low levels of DHT, a diminutive prostate, and complete lack of prostatic glandular epithelium (Imperato-McGinley et al 1974Imperato-McGinley et al 1992) and are notable for a lack of reported cases of adenocarcinoma of the prostate. These findings, along with the well known androgen sensitive nature of prostate cancer prompted two large scale clinical trials investigating the possible role of 5AR inhibitors in the prevention of prostate cancer: the Prostate Cancer Prevention Trial (PCPT) and the Reduction by Dutasteride of Prostate Cancer Events (REDUCE) (Thompson et al 2003; Andriole et al 2004).

The PCPT was a multicenter, randomized, double blind, placebo-controlled clinical trial comparing finasteride with placebo in the prevention of adenocarcinoma of the prostate. 18 882 men over the age of 55, with a PSA of 3.0 ng/ml or lower and a normal digital rectal examination (DRE) were randomized to receive 7 years of finasteride (5 mg) or placebo daily. The study participants were monitored with annual PSA (blindly adjusted in the finasteride group) and DRE. Transrectal ultrasound guided biopsies were performed for PSA greater than or equal to 4 ng/ml or for an abnormal DRE. Additionally, all men were recommended to undergo an end of study biopsy after the 7 years of treatment had elapsed (Thompson et al 2003). Of the 9060 men included in the final statistical analysis, 18.4% of men in the finasteride group were diagnosed with adenocarcinoma of the prostate, compared with 24.4% in the placebo group; a 24.8% relative risk reduction (p<0.001). A significant difference persisted across all subgroups analyzed and approximately 98% of cancers in both groups were organ confined.

Despite the impressive 24.8% relative risk reduction in the diagnosis of prostate cancer in the PCPT study population, several issues have prevented the widespread acceptance of finasteride as preventative therapy for prostate cancer. First and foremost, finasteride therapy was associated with an increased diagnosis rate of poorly differentiated prostate cancer compared with placebo (37.0% vs 22.2% of cancers diagnosed; 6.4% vs 5.1% overall) (Thompson et al 2003). Additionally, prostate cancer diagnosis rates were far higher than expected in both study groups (18.4% and 24.8% in this relatively low-risk group followed for 7 years vs lifetime prevalence of approximately 17%). Finally, nearly half of all positive cancer diagnoses were from end-of-study biopsies, tumors whose clinical significance is unknown (Grover et al 2006).

The PCPT was met with immediate skepticism by some investigators and clinicians, fueled by concerns that finasteride had selected for and accelerated the growth of high grade tumors (Scardino 2003). Subsequent analysis of PCPT data provide an alternative hypothesis to explain the slightly higher percentage of high grade tumors detected in the treatment group (Etzioni et al 2005; Thompson et al 2005, 2006). First, the greatest difference in the proportion of high versus low-grade cancers detected between the treatment and placebo groups was seen in those study subjects undergoing for-cause biopsies as opposed to end-of-study biopsies. The increased hazard ratio for detecting high grade prostate cancer in the treatment group did not increase over the course of the clinical trial. In fact, the number of men diagnosed with high grade prostate cancer during the end-of-study biopsies were similar across both study groups. A subsequent analysis found that finasteride influenced the characteristics of PSA as a screening test, increasing its sensitivity for all grades of prostate cancer in general, and high grade prostate cancer in particular (Thompson et al 2006). Additionally, PSA is more sensitive for high grade prostate cancer at various PSA cutoff levels ranging from 1.1 ng/ml to 10.1 ng/ml (Thompson et al 2005), decreasing the false negative rate and increasing the rate of diagnosis of high grade tumors. Therefore, the combination of cancer chemoprevention, PSA reduction (in benign disease), and increased sensitivity of PSA for high grade prostate cancer may have led to the increased rate of diagnosis of high grade prostate cancer in the finasteride group despite an overall reduction in the incidence of prostate cancer diagnosis. Other analyses from the PCPT regarding the effect of finasteride on tumor grade and the interaction of the reduction in gland volume with detection of cancer and high grade cancer are expected in the near future.

Expression of type 1 5AR is greater in some cell lines of neoplastic prostate tissue (Iehle et al 1999; Andriole et al 2004). As discussed previously, dutasteride inhibits both type-1 and type-2 5AR. Secondary analysis of a large scale clinical trial evaluating the efficacy of dutasteride for BPH noted a 1.1% rate of prostate cancer diagnosis in the treatment group compared with 1.9% in the placebo group (Roehrborn et al 2002). A clinical trial is underway to prospectively evaluate the role of dutasteride in the prevention of prostate cancer. The REDUCE trial is a multicenter, randomized, international, placebo-controlled, double blind clinical trial initiated in 2003 which is designed to evaluate the ability of dutasteride to decrease the risk of biopsy detectable prostate cancer in men with moderately elevated PSA levels (Andriole et al 2004).The relative efficacy of finasteride versus dutasteride for prevention must await the conclusion of this trial but the substantially different characteristics of subjects in the two studies will probably make such conclusions challenging regardless of the study’s outcomes.

Treatment of prostate cancer with 5-alpha reductase inhibitors

The heterogeneous behavior of prostate cancer as well as multiple factors limiting patient accrual are some of the inherent challenges in the development and completion of clinical trials comparing various therapeutic modalities (active surveillance, surgical extirpation, surgical ablation, radiation therapy, hormonal therapy, and chemotherapy). As a result, studies are typically small, poorly controlled, and fraught with a number of biases (Cookson 2006). However, the search for the optimal method of prostate cancer management (greatest cancer control rate with lowest risk of complications and side effects) continues. In spite of these limitations, a number of studies have addressed the potential role of 5AR inhibitors in the management of prostate cancer.

Murine prostatic carcinoma has been used for the investigation of various hormonal therapy regimens. Various cell lines of murine prostate cancer (Dunning R3327, Fisher 344) share several important characteristics with human prostate cancer, including basic histology, slow rate of growth, hormonal sensitivity, and expression of 5AR (Zaccheo et al 1997). Murine models have demonstrated the ability of various 5AR inhibitors to inhibit macroscopic rat prostate carcinogenesis (Tsukamoto et al 1998) and decrease the rate of rat prostate cancer growth (Zaccheo et al 1997, 1998). However, when compared with medical and surgical castration, 5AR inhibitors produce a lesser reduction in rat prostate cancer growth. In summary, results from animal model studies do not support the use of 5AR inhibitors as monotherapy for the treatment of prostate cancer.

While surgical extirpation or radiation therapy are commonly recommended for the treatment of localized prostate cancer, hormonal therapy is most frequently used for locally advanced disease, biochemical recurrence after localized therapy, and metastatic disease. Medical or surgical castration is the most effective form of hormonal therapy. However, it is poorly tolerated, with a high percentage of patients experiencing erectile dysfunction, loss of libido, “hot flashes”, osteoporosis, fatigue, and muscle wasting. While monotherapy with 5AR inhibitors has a minimal impact on prostate cancer (Presti et al 1992), there is much interest with combining the well tolerated hormonal effects of 5AR inhibitors with other therapeutic modalities.

To demonstrate the effect of 5AR inhibitors prior to localized therapy, Andriole and colleagues (2004) randomized 46 men with clinically localized prostate cancer to either dutasteride or placebo for 6 to 10 weeks prior to radical prostatectomy (Andriole et al 2004). While this double blind, prospective, randomized clinical trial was not powered for specific clinical end points, important effects of 5AR inhibition were observed. Serum and intraprostatic DHT was decreased by over 95% in the dutasteride group, versus no change in the placebo group. Cellular apoptosis indices were increased in the dutasteride group while measurements of vascular density were decreased. Finally, tumor volume and prostatic intraepithelial neoplasia (PIN) volume were lower in the dutasteride group. There were no clinical differences between the two groups (ie, pathologic grade/stage or rate of positive surgical margins) (Andriole et al 2004). The clinical significance of the observed effects of dutasteride is unknown.

The morbidity associated with medical and surgical castration make them less attractive regimens for routine use in the prevention of recurrent disease or disease progression following localized therapy for prostate cancer (radical prostatectomy or radiation). Given its excellent tolerability, investigators have evaluated the role of finasteride as a potential agent for the prevention of disease progression. In a randomized, multicenter, placebo controlled clinical trial, Andriole and colleagues (1995) randomized 120 men to either placebo or 10 mg finasteride following radical prostatectomy with residual PSA levels. Finasteride did not prevent biochemical recurrence, but it delayed the onset of PSA progression by 6 months and the rate of PSA progression by 14 months after 2 years of therapy. This study, however, did not demonstrate a survival benefit for the finasteride group compared with placebo.

Nonsteroidal antiandrogens (flutamide, biclutamide, nilutamide) are competitive inhibitors for the androgen receptor. They do not suppress serum testosterone and thus, do not produce the side effects associated with castration (erectile dysfunction, loss of libido, “hot flashes”, osteoporosis, fatigue, and muscle wasting). While adjuvant monotherapy with nonsteroidal antiandrogens after radiation therapy or radical prostatectomy is not indicated for localized prostate cancer (McLeod et al 2006), high dose bicalutamide (150 mg daily) may play a role in the adjuvant management of locally advanced (Iversen et al 2000) or metastatic prostate cancer (Tyrrell et al 1998).

Combination therapy with nonsteroidal antiandrogens and 5AR inhibitors may provide better cancer control than antiandrogens alone, while continuing to avoid the toxicity of castration. In addition to decreasing intraprostatic DHT, 5AR inhibitors down-regulate androgen receptor expression and may decrease the rate of androgen receptor mutation and subsequent androgen-independent prostate cancer (Wang et al 2004). Combination therapy with bicalutamide and finasteride inhibit the in vitro proliferation of prostate cancer cells; although to a lesser degree than castration. The combination of finasteride and flutamide has been shown to decrease the total prostatic weight in an equivalent manner to medical castration in a rat model (Fleshner and Trachtenberg 1992).

Several phase II trials have examined the effect of antiandrogens and finasteride as combination therapy for various stages of prostate cancer. Barqawi and colleagues (2003) noted a 58% rate of PSA regression to undetectable levels in 71 men with PSA recurrence after surgery or radiation therapy. Potency was maintained in all those evaluated. Combination therapy has also been used in advanced prostate cancer. Brufsky and colleagues (1997) treated 20 such men with flutamide until PSA levels reached nadir (mean 9.1 weeks). The addition of finasteride further reduced PSA levels in 19 men. PSA levels continued to decline in 18 patients, declined to less than 1 ng/ml in 13 patients, and reached undetectable levels in 7 patients. Only 3 patients reported libido as “poor” and of the 11 patients potent at baseline, 8 were at least partially potent at last follow-up. Follow-up data demonstrates good durability of the PSA reduction, with a median time to failure of 30 months. At 7 years, 25% of the original group had remained castration-free. Five year overall survival was 65% (Oh et al 2003). A similar phase II trial with biclutamide and finasteride produced comparable results. Thirty of 36 patients with advanced prostate cancer reached a second PSA nadir after the addition of finasteride to biclutamide. Eleven were free of progression at last follow-up (mean 3.9 years). Of the 19 who progressed, 12 reached a third PSA nadir after transition to standard medical androgen deprivation. The average time to failure of the treatment protocol was 21.3 months. All potent patients at baseline remained potent at second PSA nadir (Tay et al 2004).

The above studies have similar time to failure rates compared with castration (Potosky et al 2001). Unfortunately, the above studies are limited by their observational nature and small number of patients. While the combination of nonsteroidal antiandrogens and 5AR inhibitors seems like a viable option to avoid the toxicity of castration, long term, blinded, randomized, large scale clinical trials with survival data are necessary before definitive conclusions can be made.

Intermittent androgen deprivation allows for transient recovery of testosterone levels and may improve quality of life compared with permanent castration (Spry et al 2006). In a retrospective review of 101 individuals undergoing intermittent androgen deprivation, the “time off period” (when androgen deprivation was ceased) was prolonged from 15 to 31 months in individuals taking finasteride 5 mg daily (Scholz et al 2006). Improved quality of life was inferred, but not objectively quantified. Additionally, there was no difference in disease specific survival (Scholz et al 2006). At the time of writing there have been no randomized clinical trials validating the treatment of prostate cancer with intermittent androgen deprivation therapy and 5AR inhibitors.

Safety and tolerability of 5-alpha reductase inhibitors

There are extensive data from randomized clinical trials demonstrating the tolerability and safety of finasteride (McConnell et al 1998, 2003; Thompson et al 2003). The PLESS (Proscar Long-Term Efficacy and Safety) trial randomized 3040 men with BPH to 4 years of daily therapy with either finasteride (5 mg) or placebo. Symptoms and side effects were assessed every 4 months throughout the duration of the study. The drop out rate was higher with placebo (42%, p<0.001) versus finasteride (34%). Men in the placebo group were more likely to drop out due to lack of improvement or worsening of disease or to receive medical or surgical therapy for BPH. Statistically significant drug-related adverse effects in the finasteride group included sexual dysfunction, breast enlargement/tenderness, and rash. However, the risk of these adverse effects was relatively low compared with placebo. Rates of decreased libido and impotence were different only during the first year. There was no difference in the rates of serious adverse events (McConnell et al 1998).

During seven years of finasteride therapy versus placebo in the 18 882 men enrolled in the PCPT, similar tolerability rates were seen. In the placebo group, 28.9% of men temporarily discontinued therapy sometime during the 7 year study, compared with 36.8% in the finasteride group (p<0.001). Reduced ejaculate volume, erectile dysfunction, loss of libido, and gynecomastia were more common in the finasteride group (p<0.001 for each comparison) but the overall risk for these adverse effects remained low compared with placebo. There was one case of breast cancer in each arm of the trial. The death rate (all-cause and prostate cancer) was not significantly different between the two groups (Thompson et al 2003).

Despite the dual isotype blockade of 5AR and subsequent lower serum and prostatic levels of DHT, the tolerability of dutasteride is similar to that of finasteride. A 2 year study of dutasteride for the treatment of BPH revealed rates of impotence, decreased libido, ejaculation disorder, and gynecomastia similar to that of finasteride. The drug-related side-effects were transient with no statistically significant difference noted at 2 years. (Roehrborn et al 2002).

Summary and conclusions

5AR inhibitors are effective for the medical management of BPH and their impact on the hormonal physiology of the prostate with relatively-low toxicity has led to studies of their use for the prevention and treatment of adenocarcinoma of the prostate. In spite of the fact that finasteride is the only agent shown to prevent prostate cancer in a randomized clinical trial, there is not widespread acceptance of its use for the prevention of this malignancy. More recent analyses of the improved performance of PSA for prostate cancer detection with finasteride as well as follow-up analyses may help resolve the issue of increased tumor grade with finasteride. The role of dutasteride in the prevention of prostate cancer remains to be proven. Finally, while preliminary data indicate a possible role of 5AR inhibitors in various forms of multimodal therapy for prostate cancer, there have been no definitive studies demonstrating improved survival with its use in any regimen. The role of 5AR inhibitors in the management of prostate cancer therefore remains unknown.

Notes

Note: The views expressed in this article are those of the authors and do not reflect the official policy of the Department of Defense or other Departments of the US Government.