https://orcid.org/0000-0002-3003-8048
ABSTRACT
Objectives: To systematically review and discuss the current evidence from placebo-controlled clinical trials that investigated the use of alternative medicines and herbal remedies in the management of erectile dysfunction (ED).
Methods: A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-based systematic review using specific keyword combinations was conducted on the PubMed and Scopus databases. Randomised controlled trials investigating herbal medicine in at least one group and using the International Index of Erectile Function (IIEF) as an outcome in patients primarily diagnosed with ED were included for review.
Results: Following the literature search, screening and eligibility analysis, a total of 42 articles were included. The 42 articles were categorised as single herb extractions (n = 14), combination herbal formula (n = 5), combination of herbal formula and non-herbal nutraceuticals (n = 7), non-herbal nutraceuticals (n = 5), acupuncture and moxibustion (n = 2), diet and nutrition (n = 3), exercise (n = 5), and topical treatments (n = 1). Based on the results, Korean ginseng, Pygnogenol and Prelox, Tribulus terrestris, Lepidium meyenii, L-arginine, acupuncture and lifestyle interventions were the more predominantly investigated treatments interventions for ED.
Conclusions: Panax ginseng, Pygnogenol, Prelox and Tribulus terrestris have promising evidence as herbal products, alongside L-arginine as a nutritional supplement, for ED based on IIEF outcomes, and warrant further clinical investigation. The mechanisms of action remain unclear, but each of these appears to in part increase nitric oxide synthesis. Importantly, improved diet and exercise should be considered, particularly in patients with obesity or diabetes mellitus.
Introduction
Erectile dysfunction (ED) is defined as the inability to achieve and/or maintain an appropriate penile erection that is sufficient for sexual intercourse, and clinically classified as psychogenic (such as relationship dissatisfaction, societal pressures, anxiety or depression) or organic (underlying causes or comorbidities) [Citation1,Citation2]. This is a common and increasing male sexual health concern, with a prevalence of up to 31% [Citation2] and projected to affect up to 322 million men by 2025 [Citation3]. The prevalence of ED is reported as 1–10% of males aged <40 years (mostly psychogenic), >40% of men aged >40 years, and 50–100% of men aged >70 years (mostly organic) [Citation3].
Importantly, ED has a significant negative impact on a man’s quality of life, including loss of self-esteem, avoidance of intimacy, anxiety and depression, which in turn exacerbate sexual dysfunction. This also impacts the relationship between sexual partners, including negatively affecting sexual desire and satisfaction of partners [Citation4]. Furthermore, ED is considered an independent risk factor for the development of metabolic syndrome, cardiovascular disease and type 2 diabetes mellitus, sharing common underlying mediators, and therefore ED is considered an increased mortality risk [Citation5–7]. As psychological, endocrine, immune and metabolic interactions underlie the pathogenesis [Citation1], ED is considered a multidimensional and complex pathology [Citation4].
The penis is comprised of arterioles and capillaries, blood filled sinuses, and smooth muscles. Sympathetic activation maintains smooth muscle contraction (vasoconstriction) and penile flaccidity [Citation8]. Normal erection is initiated through external stimuli via somatic and autonomic pathways, where parasympathetic activation leads to nitric oxide (NO) production by the NO synthase (NOS) enzyme in endothelial cells and non-adrenergic non-cholinergic nerves in the penis [Citation1,Citation8]. Through molecular cascades, NO reduces cytosolic calcium, leading to smooth muscle relaxation and penile vasodilation (erection) [Citation1,Citation8].
The clinical assessment of ED requires a detailed history and clinical examination relevant to psychological and organic causes [Citation1]. For diagnostic and research purposes, the diagnosis and severity of ED is classified based on the International Index of Erectile Function (IIEF) [Citation9]. The IIEF is a validated subjective score with high sensitivity and specificity for ED [Citation1], and is used as a global standard in the clinical investigation of conventional and surgical interventions for ED [Citation9,Citation10]. ED can be classified as severe (IIEF score ≤7), moderate (8–11), mild-to-moderate (12–16) and mild (17–21), while a IIEF score of ≥22 indicates no ED [Citation9].
Pharmaceutical treatments for ED are typically phosphodiesterase-5 inhibitors that inhibit the outflow of blood from the penis to induce erection. Common examples include sildenafil citrate (Viagra), tadalafil (Cialis) and vardenafil (Levitra) [Citation2,Citation11]. Although generally well tolerated, adverse effects include headaches, dyspepsia, nasal congestion, flushing, syncope, vision loss, priapism, and myocardial infarction [Citation12]. More invasive options for ED management include intracavernosal injection therapy of vaso-active substances, vacuum erection devices, and penile prosthesis implants [Citation2,Citation11]. However, not all patients respond to the treatments available, and adverse effects and costs may further limit pharmacological intervention [Citation13]. These treatments also only deal with the symptoms of ED and do not address any underlying pathogenesis [Citation14].
Complementary and alternative medicines, including dietary supplements and herbal remedies, are increasingly being used for treatment of ED, particularly through over-the-counter and internet sources [Citation6,Citation13,Citation15,Citation16]. These can be described as ‘a group of diverse medical and healthcare systems, practices, and products that are not presently considered to be part of conventional medicine’ [Citation17,Citation18]. This is a broad definition, incorporating traditional medicine systems and modalities. Examples include herbal medicines (remedies) and dietary supplements, nutritional and lifestyle therapies, acupuncture and traditional Chinese medicine, body therapies (such as massage, cupping and acupressure), homeopathy, mind–body techniques (such as meditation and yoga), energy medicine (such as reiki), and other traditional medicine disciplines such as Ayurveda, Unani and Naturopathy [Citation18].
Up to 718 plant species are used in traditional medicine as aphrodisiacs, compounds that increase sexual arousal, libido, potency (erection) and/or sexual pleasure [Citation13,Citation19]. Top selling products on the market include items containing the herbs Panax ginseng (Korean ginseng), Tribulus terrestris (Tribulus), Epimedium gradiflorum (Horny goat weed), Lepidium meyenii (Maca), Ginkgo biloba (Ginkgo), Eurycoma longifolia Jack (Tongkat ali), and Pausinystalia johimbe (yohimbine), and nutraceuticals such as B complex vitamins, zinc, trace minerals, L-arginine, aspartate and dehydroepiandrosterone (DHEA) [Citation6,Citation15,Citation20,Citation21].
The increased use of alternative medicines and herbal remedies for ED with limited scientific investigations on extractions, efficacy, safety and dosage is a challenge to clinicians, whereas a comprehensive overview of clinically relevant research is these disciplines is currently lacking. Therefore, the present study aimed to systematically review and discuss the current evidence from placebo-controlled clinical trials that investigated the use of traditional medicine and herbal remedies in the management of ED as assessed by the IIEF.
Methods
A systematic review of the literature for placebo-controlled clinical trials investigating traditional and herbal medicine was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [Citation22]. Studies of interest were placebo-controlled clinical trials investigating an alternative and/or herbal medicine in the treatment of ED that reported the IIEF as a primary outcome.
The keyword search was conducted on the PubMed and Scopus databases on 16 January 2021, using the following keyword string and combination of Boolean operators: (‘Phyto*’ OR ‘plant medicine*’ OR ‘polyherb*’ OR ‘herb*’ OR ‘natural medicine*’ OR ‘traditional medicine*’ OR ‘complementary medicine*’ OR ‘alternative medicine*’ OR ‘CAM’ OR ‘Chinese medicine’ OR ‘acupuncture’ OR ‘acupressure’ OR ‘moxibustion’ OR ‘tuina’ OR ‘qigong’ OR ‘hydrotherapy’ OR ‘homoeopathy’ OR ‘aromatherapy’ OR ‘aromatic oil*’ OR ‘nutraceutical*’ OR ‘supplement*’ OR ‘nutrition*’ OR ‘diet*’ OR ‘vitamin*’ OR ‘mineral*’ OR ‘reflexology’ OR ‘massage’ OR ‘exercise’ OR ‘unani’ OR ‘ayurveda’ OR ‘Asian medicine’ OR ‘eastern medicine’ OR ‘cupping’ OR ‘naturopathy’ OR ‘yoga’ OR ‘reiki’ OR ‘energy medicine’ OR ‘spiritual medicine’ OR ‘meditation’ OR ‘chiropractic’ OR ‘osteopathy’ OR ‘thermal therapy’ OR ‘light therapy’ OR ‘music therapy’ OR ‘Pycnogenol’ OR ‘Arginine’ OR ‘Aspartate’ OR ‘DHEA’ OR ‘Lepidium’ OR ‘Maca’ OR ‘Yohimbine’ OR ‘Epimedium’ OR ‘Ginseng’ OR ‘Ginkgo’ OR ‘Tongkat ali’ OR ‘Eurycoma’ OR ‘Tribulus’) AND (‘erectile dysfunction’ OR ‘IIEF’) AND (‘clinical’ OR ‘trial*’ OR ‘placebo’). The search was limited to English original articles only.
Articles retrieved were analysed for the removal of duplicates, and then screened based on the title and abstract. Subsequently, the full text was reviewed for eligibility based on the inclusion and exclusion criteria (). The retrieval, screening, and eligibility of articles for inclusion were conducted by two independent researchers (R.F., K.L.), and any disagreement was settled through discussion.
Table 1. Inclusion and exclusion criteria for eligibility
Results
The literature search retrieved a total of 1220 articles, of which 299 duplicates were removed. The remaining 921 articles were screened for suitability based on title and abstract, with 777 articles being excluded. A total of 144 full-text articles were then reviewed for eligibility based on the inclusion criteria (), and 102 articles were removed for not having ED as a primary participant diagnosis (n = 57) or the IIEF as a primary outcome (n = 14), for being observational studies (n = 5) or not placebo-controlled clinical trials (n = 11), for not sole use of traditional or herbal intervention in at least one group (n = 14), while one article was removed for non-English language. A total of 42 articles were included for the analysis and discussion ().
Figure 1. Flow diagram illustrating the methodology and results of the search strategy
summarises the included studies [Citation23–64]. These are categorised into single herb extractions (n = 14), combination herbal formula (n = 5), combination herbal formula and non-herbal nutraceuticals (n = 7), non-herbal nutraceuticals, (n = 5), acupuncture and moxibustion (n = 2), diet and nutrition (n = 3), exercise (n = 5), and topical treatments (n = 1).
Table 2. Alternative medicines and herbal remedies investigated in ED
Discussion
The results of the systematic review reveal a paucity of studies for many alternative medicines and herbal remedies used for ED. A total of 42 studies were retrieved, which generally varied significantly on the interventions used to investigate ED in controlled trials. This included single and combined herbal interventions, non-herbal nutraceuticals, combined herbal and non-herbal nutraceuticals, acupuncture, lifestyle intervention, and topical herbal applications. However, based on the results and related literature, Korean ginseng, Pygnogenol and Prelox, Tribulus terrestris, Lepidium meyenii, L-arginine, acupuncture and lifestyle interventions are further discussed in detail.
Panax ginseng
In traditional Chinese medicine practice, Panax ginseng (Korean ginseng), particularly the steamed aged root called red ginseng, has been used as an aphrodisiac to improve sexual performance for thousands of years [Citation65,Citation66]. Indeed, Panax ginseng has become a popular global herbal supplement for male reproductive disorders, including sexual performance and ED [Citation67]. The present review included four studies investigating Panax ginseng based on IIEF outcomes, where three reported benefit [Citation23–25] and one reported no change [Citation26]. An additional study combined ginseng with vitamin E, which also improved erectile function [Citation27]. Ginseng was further used in a combination herbal formula including KBMSI-2 [Citation28] and VigRX Plus (VXP) [Citation29], both improving erectile function. Furthermore, one study was found to investigate the related species Panax notoginseng, finding an improvement in Intercourse Satisfaction only, although there was an improvement in the Androgen Deficiency in the Aging Male questionnaire, pre- and post-sleep penile circumference [Citation30].
These results agree with previously published Korean language placebo-controlled trials, where ginseng significantly improved erectile function and sexual satisfaction reports by patient and partner [Citation68–70], as well as penile blood flow [Citation71]. However, Kim et al. [Citation72] reported no benefit of Panax ginseng in a placebo-controlled trial that was based on the Watts Sexual Function questionnaire. Based on a meta-analysis of five clinical studies, Borrelli et al. [Citation73] also concluded that Panax ginseng significantly improved erectile function.
Ginsenosides (steroid-like saponins) are considered the major active isolates that are unique to ginseng species, particularly Panax ginseng [Citation74]. These are a heterogeneous group of triterpenoidal glycosides that vary based on the number of sugars and the bonding positions on the aglycone skeleton [Citation75]. The plant also contains polysaccharides, alkaloids, and phenolic compounds [Citation76]. Importantly, the bioactivity of ginseng requires both saponin and non-saponin metabolites working synergistically [Citation76]. Ginseng extractions and ginsenosides have been shown in animal studies to induce NO synthesis in the endothelium, inducing vasodilation of the corpus cavernosum and subsequent erection [Citation65]. Ginseng is also reported to increase testosterone concentrations, improving erection and increasing copulatory behaviour [Citation65]. However, few studies included in the results of the present systematic review reported a positive impact of ginseng on testosterone [Citation24–26], neither did the combination formulae containing ginseng [Citation28,Citation29]. Borelli et al. [Citation73] also reported no pooled increase in testosterone with ginseng supplementation.
Panax ginseng is further considered safe, without significant herb–drug interaction at dosages of 0.5–3 g/day [Citation74]. However, significantly more research on the potential mechanisms and active isolates for ginseng to improve ED is warranted.
Pygnogenol and Prelox
Pinus pinaster Ait. subsp. Atlantica (Maritime pine) is a tree native to Southwestern France. Pygnogenol is a proprietary patented extraction of the powdered Pinus pinaster bark, standardised to 70% ± 5% procyanidins [Citation77,Citation78]. Procyanidins are biopolymers of catechin and epicatechin subunits, with significant antioxidant properties that have numerous health promoting and disease management benefits [Citation79]. Pygnogenol has significant antioxidant, anti-inflammatory and immune regulating properties [Citation80–85]. Importantly, evidence further suggests increased vascular NO synthesis and vasodilation, and catecholamine antagonists action, which can improve erection [Citation78,Citation86].
Pygnogenol was reported in two studies as a sole herbal intervention for ED, with both studies reporting improvement in erectile function [Citation31,Citation32]. Prelox is a combination of Pygnogenol®, L-arginine, L-citrulline and roburins, included as a combination therapy in four studies that all reported a benefit in erectile function and other domains of the IIEF across the studies [Citation33–36]. Prelox also reported improved testosterone levels [Citation33,Citation34,Citation36]. However, the impact of arginine with Pygnogenol is not clear. In an uncontrolled trial, L-arginine caused a non-significant improvement in erectile function over 1 month. Following the introduction of Pygnogenol alongside L-arginine in the same cohort caused a significant improvement of erectile function over the next month. In the third month, doubling the dose of Pygnogenol further improved the normal erection in the cohort [Citation87]. This suggests that Pygnogenol may be effective without the inclusion of arginine in the form of Prelox. Pygnogenol therefore appears to be supported by these clinical trials for use in ED. However, more research is warranted to establish efficacy, dosage and mechanisms of action of Pygnogenol in ED.
Tribulus terrestris
Tribulus terrestris is a medicinal plant used for thousands of years in India and China, and well documented in the Ayurvedic, Unani, Siddha and Chinese Traditional Medicine systems [Citation88]. Alongside ginseng, it is a common herb used in male sexual health products [Citation67]. Steroidal saponins spirostanol and furostanol are characteristic secondary metabolites of Tribulus terrestris, along with flavonoids. Alkaloids include those related to β-carboline and amide alkaloids [Citation89].
Tribulus terrestris was investigated as a single herb in two of the included studies, both using standardised extractions for saponins and reporting a positive effect on the IIEF and various domains [Citation37,Citation38]. Furthermore, with Tribulus terrestris as the major constituent, the combination formulae Tradamix TX1000, VXP, and NRL/MW/201,901 were also shown to increase IIEF and all domains [Citation29,Citation39,Citation40]. This generally agrees with the review by Borrelli et al. [Citation73], although a non-English study excluded from this review found no difference to placebo in IIEF or testosterone [Citation90].
Animal studies suggest a pro-erectile effect of Tribulus terrestris on the corpus cavernosum. This is mediated through increased NO in the endothelium and nitrergic nerve endings, inducing vasodilation. Animals also show a dose-dependent increase in sexual behaviour [Citation88,Citation89]. Furthermore, Tribulus terrestris is suggested to increase testosterone levels [Citation88,Citation89]. However, there was no increase reported by the three studies in the present review that included testosterone analysis [Citation29,Citation38,Citation39]. Further studies are needed to determine efficacy and potential mechanisms of Tribulus terrestris for ED.
Lepidium meyenii
Lepidium meyenii (Maca or Peruvian bark), a member of the Brassicaceae family, is native to the Central Andes Mountains of Peru at altitudes of 4000–4500 m. As a food supplement and for medicinal purposes, this plant has been used traditionally for >2000 years [Citation91,Citation92]. Important and unique secondary metabolites identified include imidazole alkaloids, hydantoins, meyeniins A − C, alkamides, glucosinolates, and phytosterols [Citation92].
Only one study included in the present review investigated Lepidium meyenii as a single herbal intervention, improving IIEF with no significant impact on testosterone [Citation41]. This agrees with the review by Borrelli et al. [Citation73]. In a systematic review investigating Lepidium meyenii on male and female sexual function [Citation93], an additional two studies were further reported in healthy male cohorts that both showed no significant effect on sexual function compared to placebo [Citation94]. In the Shin et al. [Citation93] review, it was concluded that Lepidium meyenii improves sexual desire over a period of at least 6 weeks.
Animal studies investigating sexual behaviour in rats using pulverised Lepidium meyenii hypocotyls have yielded mixed results [Citation91]. Here, glucosinolates are proposed as being active in male sexual function [Citation92]. Although sexual function is regulated partly by testosterone, there is little evidence that Lepidium meyenii can increase serum or testicular testosterone [Citation91,Citation94,Citation95]. However, animal studies have further suggested improvement of ED through increase in the intracavernosal pressure to mean arterial pressure ratio [Citation92].
Although reported as well tolerated in the included study, Lepidium meyenii may cause psychological adverse events, including anxiety, mood swings, hallucinations, and addictive behaviour [Citation15]. Significantly more clinical and laboratory research is warranted to determine efficacy, dosage, and tolerability.
L-arginine
L-arginine is an amino acid obtained through dietary sources, particularly meats and nuts, or produced from L-citrulline [Citation96]. Importantly, L-arginine is the only substrate for NOS to produce NO [Citation97]. Through vasodilation mediated by increased NO, L-arginine has been found to be effective in hypertension [Citation97]. This vasodilation also mediates increased cavernosal blood flow [Citation67]. The excellent bioavailability profile of L-arginine further demonstrates a positive effect on NO production [Citation96].
L-arginine was found to be used as a sole intervention in one study, where a significant improvement was reported [Citation42]. There were four studies combined with Pycnogenol as Prelox that all reported positive effect on the IIEF [Citation33–36]. L-arginine was combined with yohimbine hydrochloride in two studies, also both reporting a positive impact on the IIEF [Citation43,Citation44]. Lastly, L-arginine was also found to improve the IIEF in combination with adenosine monophosphate [Citation45].
In a systematic review, Chang Rhim et al. [Citation98] included a total of 10 studies, with four studies using L-arginine as a sole therapy. Of those 10 studies, six were included in the results of the present systematic review [Citation33,Citation34,Citation42–45]. In two excluded studies that used the Cologne Erectile Inventory (KEED) as an outcome, L-arginine improved sexual dysfunction, alongside ED [Citation99] and O’Learys Questionnaires [Citation100]. In healthy Japanese male cohorts without ED, L-arginine improved the IIEF when combined with citrulline [Citation101] and ornithine [Citation102]. Pygnogenol, yohimbine, adenosine monophosphate and ornithine may be used in combination as they are all proposed to improve NO, and therefore may offer synergistic effects [Citation96]. L-arginine may further be more effective in patients with ED with low NO, where patients with low urinary nitrites and nitrates appear to have greater benefit [Citation100]. This dietary supplement was reported as well tolerated by all studies. As a natural dietary supplement with good absorption and bioavailability, this can allow for long duration of treatment for ED [Citation96].
Acupuncture
Acupuncture is an ancient and traditional Chinese medicine practice established for over 2500 years that has become popular globally over recent decades [Citation103]. It is defined as the insertion of needles into specific points of the body that can be further manipulated using various techniques. The proposed mechanism is to manipulate meridians, which can simply be described as energy channels, to restore energy (qi) balance between yin (female, moon, dark, concealed) and yang (masculine, bright, open) in the body to restore health [Citation103,Citation104]. Acupuncture research can be difficult to bridge ancient and modern principles, where there are some barriers to research that include basic terminology, the difficulties in standardised treatment methods, and effective measures for assessing treatment [Citation105]. However, acupuncture has been investigated in various forms of reproductive medicine [Citation18].
Acupuncture treatment was reported to be beneficial in one included study in the results [Citation46]. With the addition of warm needling moxibustion, it was also shown to be superior to standard acupuncture [Citation47]. Both studies are relatively small and not clearly reported. This broadly agrees with the systematic review by Cui et al. [Citation106], where too few studies are available for any conclusions for the efficacy of acupuncture in ED [Citation2].
Lifestyle changes
Lifestyle changes are part of almost all traditional systems of medicine, although these may vary on the traditional basis of the advice provided [Citation18]. There were a total of eight studies that investigated nutritional and/or exercise in male cohorts with ED. Two studies by the same group investigated specific nutritional advice and exercise in obese [Citation48] and non-obese male patients with ED over a 2-year duration [Citation49]. This improved the IIEF compared to controls who just received general information on healthy lifestyles, alongside improvement of numerous metabolic parameters.
This is supported by the Mediterranean diet showing clinical benefit to reduce the deterioration of sexual function in type 2 diabetics in both males and females compared to those on a low-fat diet, where systemic inflammation predicted ED and severity [Citation107]. The Mediterranean diet further reduced prevalence of ED in men with metabolic syndrome, alongside inflammatory markers and endothelial function scores [Citation108]. This diet is rich in seeds and nuts, olive oil, seafood, whole grains and fruits and vegetables [Citation108]. In this context, the intervention of nuts (raw walnuts, almonds, and hazelnuts) over 2 weeks improved the Orgasmic Function and Sexual Desire domains of the IIEF compared to a standard Western diet devoid of nuts [Citation50]. Furthermore, dietary based weight loss in obese and diabetic men also improved sexual and endothelial function [Citation109,Citation110], where a high protein, moderate carbohydrate and low-fat diet further improved systemic inflammation [Citation110]. Inflammation, particularly through tumour necrosis factor-alpha, reduces NOS expression, which inhibits vasodilation. Importantly, this is mediated through intracellular reactive oxygen species generation in the endothelial cells [Citation111]. The improvement in inflammatory markers may partly explain the mechanism of appropriate nutrition on erectile function, particularly in obese and diabetic men. Here, a healthy nutritional approach may improve NO production with a correlated reduction in pro-inflammatory cytokines, which can improve ED in men with metabolic syndrome [Citation112].
The association between ED and cardiovascular diseases is mediated in part by endothelial dysfunction, in which poor nutrition and sedentary lifestyle are contributing factors to the shared risk [Citation113]. Regular physical activity can improve sexual dysfunction and ED through multiple proposed mechanisms that are relevant to ED and cardiovascular disease, including endocrine modulation, increased NO, improved arterial blood pressure, lipoprotein and glucose regulation [Citation114]. Specific aerobic training [Citation51] and interval exercise programmes [Citation52] were shown to be effective in two of the included studies. These interventions further improved inflammation [Citation52] and vascular function [Citation51] in these patients. This is supported by Khoo et al. [Citation114], reporting that moderate intensity and high volume exercise of >200 min/week improve weight, waist circumference, sexual function and testosterone compared to small exercise volume in abdominally obese and sedentary males. Furthermore, specific pelvic floor exercises were found to be effective in improving the IIEF compared to lifestyle changes in two studies [Citation53,Citation54]. Pelvic floor exercises combined with physical activity and psychoeducation for sexual rehabilitation also reported an improvement in erectile function in males with ischaemic heart disease or implantable cardioverter defibrillator [Citation55].
Nutritional and exercise-based approaches appear to be beneficial for ED. This is more apparent in obese, metabolic syndrome and diabetic men. Therefore, the integration of appropriate nutrition and exercise programmes should be considered as part of a holistic approach to ED.
Limitations and strengths
The participants in the present included studies were varied, across different forms of ED including psychogenic or organic, as well as comorbidities such as obesity, hypertension, dyslipidaemia and diabetes, which limits any comparison between studies. In many studies, the underlying pathogenesis or mechanisms of action were not investigated. Although standardised and validated, the IIEF remains a subjective scoresheet that has some limitations. Furthermore, the present review did not consider a single intervention, and numerous different forms of alternative medicines and herbal remedies are reported. Lastly, only 42 studies were included, which is a small representation of the alternative medicines and herbal remedies used for ED, making any conclusions relatively unclear even for the more prominent interventions investigated.
These limitations were partially addressed by using a keywords search, which was expansive for various common modalities of alternative medicines and herbal remedies. This was supported by the inclusion of common alternative medicines and herbal remedies used on the market to capture further studies. The PRISMA guidelines were used as a framework for the systematic review, while the inclusion of studies only with placebo/no treatment comparison groups and standardised and validated clinical outcome (IIEF) provided some consistency in the review.
Conclusion
The reported use of alternative medicines and herbal remedies for ED is extensive, particularly through dietary supplements available on the market. However, there is limited research into specific treatments in the improvement of erectile function. There is also a significant amount of heterogeneity in the interventions investigated and the dosage and duration. Based on the present results, Panax ginseng, Pygnogenol or Prelox and Tribulus terrestris have some promising evidence as herbal products, alongside L-arginine as a nutritional supplement. The mechanisms of action remain unclear, but each of these appears to increase NO synthesis and NO production to induce erection. Importantly, improved diet and exercise should be considered, particularly in patients with obesity or diabetes mellitus.
Disclosure of potential conflicts of interest
No conflict of interest is reported by the author(s).
Abbreviations
: DHEA: dehydroepiandrosterone; ED: erectile dysfunction; IIEF, International Index of Erectile Function; NO(S): nitric oxide (synthase); PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; VXP: VigRX Plus
Acknowledgments
None.
References
- Pastuszak AW. Current diagnosis and management of erectile dysfunction. Curr Sex Heal Rep. 2014;6(3):164–176.
- Ciocanel O, Power K, Eriksen A. Interventions to treat erectile dysfunction and premature ejaculation: an overview of systematic reviews. Sex Med. 2019;7(3):251–269.
- Ma M, Yu B, Qin F, et al. Current approaches to the diagnosis of vascular erectile dysfunction. Transl Androl Urol. 2020;9(2):709–721.
- Yafi FA, Jenkins L, Albersen M, et al. Erectile dysfunction. Nat Rev Dis Prim. 2016;2(1):16003.
- Loprinzi PD, Nooe A. Erectile dysfunction and mortality in a national prospective cohort study. J Sex Med. 2015;12(11):2130–2133.
- Cui T, Kovell RC, Brooks DC, et al. guide to ingredients found in top-selling nutraceuticals for men’s sexual health. J Sex Med. 2015;12(11):2105–2117.
- Leisegang K, Henkel R, Agarwal A. Obesity and metabolic syndrome associated with systemic inflammation and the impact on the male reproductive system. Am J Reprod Immunol. 2019;82(5):e13178.
- Cartledge J, Minhas S, Eardley I. The role of nitric oxide in penile erection. Expert Opin Pharmacother. 2001;2(1):95–107.
- Rosen RC, Cappelleri JC, Gendrano N. The International Index of Erectile Function (IIEF): a state-of-the-science review. Int J Impot Res. 2002;14(4):226–244.
- Bilal A, Abbasi NUH. Abbasi N ul H. International Index of Erectile Functioning-5: a critical appraisal. J Psychosexual Health. 2020;2(3–4):270–272.
- Kim S, Cho MC, Cho SY, et al. Novel emerging therapies for erectile dysfunction. World J Mens Health. 2020;38(1):1–17.
- Anand Ganapathy A, Hari Priya VM, Kumaran A. Medicinal plants as a potential source of phosphodiesterase-5 inhibitors: a review. J Ethnopharmacol. 2021;267:113536.
- Sin VJ, Anand GS, Koh HL. Botanical medicine and natural products used for erectile dysfunction. Sex Med Rev. 2020. Online ahead of print. DOI:https://doi.org/10.1016/j.sxmr.2020.10.005
- Chung E. A review of current and emerging therapeutic options for erectile dysfunction. Med Sci. 2019;7:91.
- Corazza O, Martinotti G, Santacroce R, et al. Sexual enhancement products for sale online: raising awareness of the psychoactive effects of yohimbine, Maca, horny goat weed, and ginkgo biloba. Biomed Res Int. 2014;2014:841798.
- Srivatsav A, Balasubramanian A, Pathak UI, et al. Efficacy and safety of common ingredients in aphrodisiacs used for erectile dysfunction: a review. Sex Med Rev. 2020;8(3):431–442.
- Herman PM, Craig BM, Caspi O. Is complementary and alternative medicine (CAM) cost-effective? A systematic review. BMC Complement Altern Med. 2005;5(1):11.
- Miner SA, Robins S, Zhu YJ, et al. Evidence for the use of complementary and alternative medicines during fertility treatment: a scoping review. BMC Complement Altern Med. 2018;18(1):158.
- Malviya N, Jain S, Gupta VB, et al. Recent studies on aphrodisiac herbs for the management of male sexual dysfunction - a review. Acta Pol Pharm. 2011;68(1):3–8.
- Balasubramanian A, Thirumavalavan N, Srivatsav A, et al. An Analysis of Popular Online Erectile Dysfunction Supplements. J Sex Med. 2019;16(6):843–852.
- Kuchakulla M, Narasimman M, Soni Y, et al. A systematic review and evidence-based analysis of ingredients in popular male testosterone and erectile dysfunction supplements. Int J Impot Res. 2021;33(3):311–317. Online ahead of print.
- Moher D, Liberati A, Tetzlaff J, et al. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
- Hong B, Ji YH, Hong JH, et al. A double-blind crossover study evaluating the efficacy of korean red ginseng in patients with erectile dysfunction: a preliminary report. J Urol. 2002;168(5):2070–2073.
- Kim TH, Jeon SH, Hahn EJ, et al. Effects of tissue-cultured mountain ginseng (Panax ginseng CA Meyer) extract on male patients with erectile dysfunction. Asian J Androl. 2009;11(3):356–361.
- De Andrade E, De Mesquita AA, De Almeida CJ, et al. Study of the efficacy of Korean red ginseng in the treatment of erectile dysfunction. Asian J Androl. 2007;9(2):241–244.
- Choi Y, Park C, Jang J, et al. Effects of Korean ginseng berry extract on sexual function in men with erectile dysfunction: a multicenter, placebo-controlled, double-blind clinical study. Int J Impot Res. 2013;25(2):45–50.
- Najafabadi BT, Jafarinia M, Ghamari K, et al. Vitamin E and ginseng combined supplement for treatment of male erectile dysfunction: a double-blind, placebo-controlled, randomized, clinical trial. Adv Integr Med. 2021;8(1):44–49.
- Park NC, Kim SW, Hwang SY, et al. Efficacy and safety of an herbal formula (KBMSI-2) in the treatment of erectile dysfunction: a preliminary clinical study. Investig Clin Urol. 2019;60(4):275–284.
- Shah GR, Chaudhari MV, Patankar SB, et al. Evaluation of a multi-herb supplement for erectile dysfunction: a randomized double-blind, placebo-controlled study. BMC Complement Altern Med. 2012;12(1):155.
- Yamashita SI, Ono H, Takara T, et al. Effects of Sanchi ginseng extract on the sexual function in Japanese men - a randomized, double-blind, placebo-controlled, parallel-group trial. Japanese Pharmacol Ther. 2018;46:561–580.
- Ďuračková Z, Trebatický B, Novotný V, et al. Lipid metabolism and erectile function improvement by Pycnogenol®, extract from the bark of Pinus pinaster in patients suffering from erectile dysfunction - a pilot study. Nutr Res. 2003;23(9):1189–1198.
- Trebaticky B, Muchova J, Ziaran S, et al. Natural polyphenols improve erectile function and lipid profile in patients suffering from erectile dysfunction. Bratislava Med J. 2019;120(12):941–944.
- Stanislavov R, Nikolova V, Rohdewald P. Improvement of erectile function with Prelox: a randomized, double-blind, placebo-controlled, crossover trial. Int J Impot Res. 2008;20(2):173–180.
- Ledda A, Belcaro G, Cesarone MR, et al. Investigation of a complex plant extract for mild to moderate erectile dysfunction in a randomized, double-blind, placebo-controlled, parallel-arm study. BJU Int. 2010;106(7):1030–1033.
- Stanislavov R, Rohdewald P. Improvement of erectile function by a combination of French maritime pine bark and roburins with aminoacids. Minerva Urol Nefrol. 2015;67(1):27–32.
- Aoki H, Nagao J, Ueda T, et al. Clinical Assessment of a Supplement of Pycnogenol® and l-arginine in Japanese Patients with Mild to Moderate Erectile Dysfunction. Phytother Res. 2012;26(2):204–207.
- GamalEl Din SF, Abdel Salam MA, Mohamed MS, et al. Tribulus terrestris versus placebo in the treatment of erectile dysfunction and lower urinary tract symptoms in patients with late-onset hypogonadism: a placebo-controlled study. Urologia. 2019;86(2):74–78.
- Kamenov Z, Fileva S, Kalinov K, et al. Evaluation of the efficacy and safety of Tribulus terrestris in male sexual dysfunction—A prospective, randomized, double-blind, placebo-controlled clinical trial. Maturitas. 2017;99:20–26.
- Sansalone S, Leonardi R, Antonini G, et al. Alga Ecklonia bicyclis, Tribulus terrestris, and glucosamine oligosaccharide improve erectile function, sexual quality of life, and ejaculation function in patients with moderate mild-moderate erectile dysfunction: a prospective, randomized, placebo-contr. Biomed Res Int. 2014;2014:12139.
- Pune B, Healthcare M, Plaza N, et al. Randomized, double blind, placebo controlled clinical study to assess efficacy and safety of NRL/MW/201901 in subjects suffering from erectile dysfunction. Int J Res Pharm Sci. 2020;11(3):2790–2801.
- Zenico T, Cicero AF, Valmorri L, et al. Subjective effects of Lepidium meyenii (Maca) extract on well-being and sexual performances in patients with mild erectile dysfunction: a randomised, double-blind clinical trial. Andrologia. 2009;41(2):95–99.
- Mozaffari-Khosravi H, Fallahi M, Afkhami-Ardekani M. Effect of oral supplementation of L-arginine on sexual function in men with type 2 diabetes: a double-blind clinical trial. J Nutr Food Secur. 2017;2:165–172.
- Lebret T, Hervé JM, Gorny P, et al. Efficacy and safety of a novel combination of L-arginine glutamate and yohimbine hydrochloride: a new oral therapy for erectile dysfunction. Eur Urol. 2002;41(6):608–613.
- Akhondzadeh S, Amiri A, Bagheri AH. Efficacy and safety of oral combination of yohimbine and L-arginine (SX) for the treatment of erectile dysfunction: a multicenter, randomized, double blind, placebo-controlled clinical trial. Iran J Psychiatry. 2010;5(1):1–3.
- Neuzillet Y, Hupertan V, Cour F, et al. A randomized, double-blind, crossover, placebo-controlled comparative clinical trial of arginine aspartate plus adenosine monophosphate for the intermittent treatment of male erectile dysfunction. Andrology. 2013;1(2):223–228.
- Engelhardt PF, Daha LK, Zils T, et al. Acupuncture in the treatment of psychogenic erectile dysfunction: first results of a prospective randomized placebo-controlled study. Int J Impot Res. 2003;15(5):343–346.
- Liu P, Liu SJ, Xu SW, et al. Clinical observation on warm needling moxibustion for erectile dysfunction. J Acupunct Tuina Sci. 2017;15(1):59–62.
- Esposito K, Giugliano F, Di Palo C, et al. Effect of lifestyle changes on erectile dysfunction in obese men: a randomized controlled trial. J Am Med Assoc. 2004;291(24):2978–2984.
- Esposito K, Ciotola M, Giugliano F, et al. Effects of intensive lifestyle changes on erectile dysfunction in men. J Sex Med. 2009;6(1):243–250.
- Huetos AS, Muralidharan J, Galiè S, et al. Effect of nut consumption on erectile and sexual function in healthy males: a Secondary outcome analysis of the fertinuts randomized controlled trial. Nutrients. 2019;11(6):1372.
- La Vignera S, Condorelli R, Vicari E, et al. Aerobic physical activity improves endothelial function in the middle-aged patients with erectile dysfunction. Aging Male. 2011;14(4):265–272.
- Lamina S, Okoye CG, Dagogo TT. Therapeutic effect of an interval exercise training program in the management of erectile dysfunction in hypertensive patients. J Clin Hypertens. 2009;11(3):125–129.
- Dorey G, Speakman M, Feneley R, et al. Randomised controlled trial of pelvic floor muscle exercises and manometric biofeedback for erectile dysfunction. Br J Gen Pract. 2004;54(508):819–825.
- Dorey G, Speakman MJ, Feneley RC, et al. Pelvic floor exercises for erectile dysfunction. BJU Int. 2005;96(4):595–597.
- Palm P, Zwisler AD, Svendsen JH, et al. Sexual rehabilitation for cardiac patients with erectile dysfunction: a randomised clinical trial. Heart. 2019;105(10):775–782.
- Forest CP, Padma-Nathan H, Liker HR, Forest CP, Padma-Nathan H, Liker HR. Efficacy and safety of pomegranate juice on improvement of erectile dysfunction in male patients with mild to moderate erectile dysfunction: a randomized, placebo-controlled, double-blind, crossover study. Int J Impot Res. 2007;19(6):564–567.
- Virani N, Chandola H, Vyas S, et al. Clinical study on erectile dysfunction in diabetic and non-diabetic subjects and its management with Ficus relegiosa Linn. AYU (An Int Q J Res Ayurveda). 2010;31(3):272.
- Cherdshewasart W, Nimsakul N. Clinical trial of Buten superba, an alternative herbal treatment for erectile dysfunction. Asian J Androl. 2003;5(3):243–246.
- Mamidi P, Thakar A. Efficacy of Ashwagandha (Withania somnifera Dunal. Linn.) in the management of psychogenic erectile dysfunction. AYU (An Int Q J Res Ayurveda). 2011;32(3):322.
- Punyawudho B, Puttilerpong C, Wirotsaengthong S, et al. A randomized, double-blind, placebo-controlled crossover study of Cappra® for the treatment of mild or mild to moderate erectile dysfunction in Thai male. Afr J Tradit Complement Altern Med. 2013;10(2):310–315.
- Agostini R, Rossi F, Pajalich R. Myoinositol/folic acid combination for the treatment of erectile dysfunction in type 2 diabetes men: a double-blind, randomized, placebo-controlled study. Eur Rev Med Pharmacol Sci. 2006;10(5):247–250.
- Ng CF, Lee CP, Ho AL, et al. Effect of niacin on erectile function in men suffering erectile dysfunction and dyslipidemia. J Sex Med. 2011;8(10):2883–2893.
- Reiter WJ, Pycha A, Schatzl G, et al. Dehydroepiandrosterone in the treatment of erectile dysfunction: a prospective, double-blind, randomized, placebo-controlled study. Urology. 1999;53(3):590–594.
- Mohammadzadeh-Moghadam H, Nazari SM, Shamsa A, et al. Effects of a Topical Saffron (Crocus sativus L) Gel on Erectile Dysfunction in Diabetics. J Evidence-Based Complement Altern Med. 2015;20(4):283–286.
- Leung KW, Wong AS. Ginseng and male reproductive function. Spermatogenesis. 2013;3(3):e26391.
- Jang DJ, Lee MS, Shin BC, et al. Red ginseng for treating erectile dysfunction: a systematic review. Br J Clin Pharmacol. 2008;66(4):444–450.
- Patel DP, Pastuszak AW, Hotaling JM. Emerging treatments for erectile dysfunction: a review of novel, non-surgical options. Curr Urol Rep. 2019;20(8):44.
- Choi HK, Choi Y. Evaluation of clinical efficacy of Korea red ginseng for erectile dysfunction by International Index of Erectile Function (IIEF). J Ginseng Res. 2001;25:112–117.
- Choi HK, Seong DH, Rha KH. Clinical efficacy of Korean red ginseng for erectile dysfunction. Int J Impot Res. 1995;7(3):181–186.
- Choi HK, Choi YD, Adaikan PG, et al. Effectiveness of Korea red ginseng in erectile dysfunction - Multi-national approach. J Ginseng Res. 1999;23:247–256.
- Choi H, Choi Y. Penile blood change after oral medication of Korean red ginseng in erectile dysfunction patients. J Ginseng Res. 2003;27:165–170.
- Kim SW, Paick J. Clinical efficacy of Korean red ginseng on vasculogenic impotent patients. Korean J Androl. 1999;17:23–28.
- Borrelli F, Colalto C, Delfino DV, et al. Herbal dietary supplements for erectile dysfunction: a systematic review and meta-analysis. Drugs. 2018;78(6):643–673.
- Choi MK, Song IS. Interactions of ginseng with therapeutic drugs. Arch Pharm Res. 2019;42(10):862–878.
- He M, Huang X, Liu S, et al. The difference between White and Red Ginseng: variations in ginsenosides and immunomodulation. Planta Med. 2018;84(12/13):845–854.
- Hyun SH, Kim SW, Seo HW, et al. Physiological and pharmacological features of the non-saponin components in Korean Red Ginseng. J Ginseng Res. 2020;44(4):527–537.
- Gulati OP. Pycnogenol(R) in metabolic syndrome and related disorders. Phytother Res. 2015;105(7):949–968.
- Rohdewald P. A review of the French maritime pine bark extract (Pycnogenol), a herbal medication with a diverse clinical pharmacology. Int J Clin Pharmacol Ther. 2002;40(4):158–168.
- Schoonees A, Visser J, Musekiwa A, et al. Pycnogenol ® (extract of French maritime pine bark) for the treatment of chronic disorders. Cochrane Database Syst Rev. 2012;4:CD008294.
- Noda Y, Anzai K, Mori A, et al. Hydroxyl and superoxide anion radical scavenging activities of natural source antioxidants using the computerized JES-FR30 ESR spectrometer system. Biochem Mol Biol Int. 1997;42(1):35–44.
- Cossins E, Lee R, Packer L. ESR studies of vitamin C regeneration, order of reactivity of natural source phytochemical preparations. Biochem Mol Biol Int. 1998;45(3):583–597.
- Bayeta E, Lau BH. Pycnogenol inhibits generation of inflammatory mediators in macrophages. Nutr Res. 2000;20(2):249–259.
- Cho KJ, Yun CH, Yoon DY, et al. Effect of bioflavonoids extracted from the bark of Pinus maritima on proinflammatory cytokine interleukin-1 production in lipopolysaccharide-stimulated RAW 264.7. Toxicol Appl Pharmacol. 2000;168(1):64–71.
- Comhaire FH, Mahmoud A. The role of food supplements in the treatment of the infertile man. Reprod Biomed Online. 2003;7(4):385–391.
- Peng Q, Wei Z, Lau BH. Pycnogenol inhibits tumor necrosis factor-alpha-induced nuclear factor kappa B activation and adhesion molecule expression in human vascular endothelial cells. Cell Mol Life Sci. 2000;57(5):834–841.
- American Botanical Council. Scientific and Clinical Monograph for PYCNOGENOL®, 2019. Accessed Apr 2021. Available at: https://www.pycnogenol.com/fileadmin/pdf/ABC_Pycnogenol_Monograph_2019.pdf
- Stanislavov R, Nikolova V. Treatment of erectile dysfunction with pycnogenol and L-arginine. J Sex Marital Ther. 2003;29(3):207–213.
- Chhatre S, Nesari T, Somani G, et al. Phytopharmacological overview of Tribulus terrestris. Pharmacogn Rev. 2014;8(15):45–51.
- Zhu W, Du Y, Meng H, et al. A review of traditional pharmacological uses, phytochemistry, and pharmacological activities of Tribulus terrestris. Chem Cent J. 2017;11(1):60.
- Santos CA, Reis LO, Destro-Saade R, et al. Tribulus terrestris versus placebo in the treatment of erectile dysfunction: a prospective, randomized, double blind study. Actas Urol Esp. 2014;38(4):244–248.
- Gonzales GF. Ethnobiology and ethnopharmacology of Lepidium meyenii (Maca), a plant from the peruvian highlands. Evid Based Complement Alternat Med. 2012;2012:193496.
- Beharry S, Heinrich M. Is the hype around the reproductive health claims of maca (Lepidium meyenii Walp.) justified? J Ethnopharmacol. 2018;211:126–170.
- Shin BC, Lee MS, Yang EJ, et al. Maca (L. meyenii) for improving sexual function: a systematic review. BMC Complement Altern Med. 2010;10(1):44.
- Gonzales GF, Córdova A, Vega K, et al. Effect of Lepidium meyenii (MACA) on sexual desire and its absent relationship with serum testosterone levels in adult healthy men. Andrologia. 2002;34(6):367–372.
- Zhang Y, Zhou F, Ge F. Effects of combined extracts of Lepidium meyenii and Allium tuberosum Rottl. on erectile dysfunction. BMC Complement Altern Med. 2019;19(1):135.
- Koolwal A, Manohar JS, Rao TS, et al. l-arginine and Erectile Dysfunction. J Psychosexual Health. 2019;1(1):37–43.
- McRae MP. Therapeutic benefits of L-Aarginine: an umbrella review of meta-analyses. J Chiropr Med. 2016;15(3):184–189.
- Chang Rhim H, Kim MS, Park YJ, et al. The potential role of arginine supplements on erectile dysfunction: a systemic review and meta-analysis. J Sex Med. 2019;16(2):223–234.
- Klotz T, Mathers MJ, Braun M, et al. Effectiveness of Oral L-Arginine in First-Line Treatment of Erectile Dysfunction in a Controlled Crossover Study. Urol Int. 1999;63(4):220–223.
- Chen J, Wollman Y, Chernichovsky T, et al. Effect of oral administration of high-dose nitric oxide donor L-arginine in men with organic erectile dysfunction: results of a double-blind, randomized, placebo-controlled study. BJU Int. 1999;83(3):269–273.
- Najima M, Munekata M, Kohiyama R. Efficacy of supplement containing arginine and citrulline on male sexual function in healthy Japanese. Healthc New Drugs. 2016;12:50–62.
- Najima M, Munekata M, Kohiyama R. Efficacy of Supplement containing arginine and ornithine on male sexual function in healthy Japanese. Medl Consult New Remed. 2016;53:81–92.
- Patil S, Sen S, Bral M, et al. The Role of Acupuncture in Pain Management. Curr Pain Headache Rep. 2016;20(4):1–8.
- Linde K, Vickers A, Hondras M, et al. Systematic reviews of complementary therapies - An annotated bibliography. Part 1: acupuncture. BMC Complement Altern Med. 2001;1(1):3.
- Priebe T, Stumpf SH, Zalunardo R. Can a science-based definition of acupuncture improve clinical outcomes? J Integr Med. 2017;15(3):165–171.
- Cui X, Zhou J, Qin Z, et al. Acupuncture for erectile dysfunction: a systematic review. Biomed Res Int. 2016;2016:2171923.
- Maiorino MI, Bellastella G, Caputo M, et al. Effects of Mediterranean diet on sexual function in people with newly diagnosed type 2 diabetes: the MÈDITA trial. J Diabetes Complications. 2016;30(8):1519–1524.
- Esposito K, Ciotola M, Giugliano F, et al. Mediterranean diet improves erectile function in subjects with the metabolic syndrome. Int J Impot Res. 2006;18(4):405–410.
- Collins CE, Jensen ME, Young MD, et al. Improvement in erectile function following weight loss in obese men: the SHED-IT randomized controlled trial. Obes Res Clin Pract. 2013;7(6):e450–e454.
- Khoo J, Piantadosi C, Duncan R, et al. Comparing effects of a low-energy diet and a high-protein low-fat diet on sexual and endothelial function, urinary tract symptoms, and inflammation in obese diabetic men. J Sex Med. 2011;8(10):2868–2875.
- Leisegang K, Henkel R, Agarwal A. Redox regulation of fertility in aging male and the role of antioxidants: a savior or stressor. Curr Pharm Des. 2016;23:2017–2018.
- Maiorino MI, Bellastella G, Esposito K. Lifestyle modifications and erectile dysfunction: what can be expected? Asian J Androl. 2015;17(1):5–10.
- Duca Y, Calogero AE, Cannarella R, et al. Erectile dysfunction, physical activity and physical exercise: recommendations for clinical practice. Andrologia. 2019;51(5):e13264.
- Khoo J, Tian HH, Tan B, et al. Comparing effects of low- and high-volume moderate-intensity exercise on sexual function and testosterone in obese men. J Sex Med. 2013;10(7):1823–1832.