Testosterone and cardiovascular risk: world’s experts take unprecedented action to correct misinformation

In November, 2013, an article entitled “Association of Testosterone Therapy with Mortality, Myocardial Infarction, and Stroke in Men with Low Testosterone Levels” by Vigen et al. was published in the Journal of the American Medical Association (JAMA) [1]. This was the first study designed to investigate cardiovascular risks with testosterone (T) therapy to report a negative association with this treatment. The story was carried prominently on major media throughout the world, and medical groups declared caution regarding testosterone therapy. In January, 2014, a second study reporting an association with increased rates of non-fatal MIs with testosterone prescriptions by Finkle et al. was published in the journal Plos One [2].

In response to these stories, a number of articles were written in the lay press that appeared to accept these results as definitive, and labeled T therapy as dangerous. This included several high-profile editorials in the US, including the New York Times, with an editorial entitled, Overselling Testosterone, Dangerously [3], in which it asserted that men were being treated because they could not face the prospect of aging, and that they were embarking on a “perilous journey,” referring to the MI risks reported by Finkle et al. Anti-pharma advocacy groups called for restrictions on the use of testosterone therapy, including black box labeling, reserved for major risks. Seemingly in response to this, the US Food and Drug Agency issued a safety bulletin announcing it would review cardiovascular risks with testosterone products. Across the US, plaintiff attorneys advertised for men who had suffered a stroke or heart attack while being treated with testosterone.

Given this remarkable reaction, one would expect that the studies indicating increased cardiovascular risks with testosterone were based on solid, experimental data with clear results. Nothing could be further from the truth. Yet these weak, retrospective, highly statistical studies were hailed as providing new, solid information of risk, trumping a rich literature developed over 30 years demonstrating risks with low levels of testosterone, and benefits of higher endogenous serum T, or T therapy.

The study by Finkle et al. published in Plos One was a retrospective analysis of 55,593 men based entirely on insurance claims [2]. No clinical information was available other than diagnosis codes and prescription utilization. The authors compared non-fatal MIs rates in the 12 months prior to men receiving a testosterone prescription with up to 90 days following the prescription (until first prescription refill). They compared these results with men who received a prescription for phosphodiesterase 5 inhibitors (PDE5i), indicated for erectile dysfunction. Reported results were that non-fatal MI’s were increased following a testosterone prescription, but not following PDE5i prescription [2].

It is astonishing how much attention this article received despite major limitations that render the study essentially non-informative. First and foremost, there was no control group. Since the risk of MI increases as men age, there should be no surprise that each year of observation (or portion of a subsequent year, as in this case) will be associated with an increased rate of MI. Without a control group it is unknown whether testosterone prescriptions were associated with an MI rate that was higher, lower, or unchanged compared with a similar group of untreated men. Second, this was a retrospective dataset and not a planned experiment, so conclusions must be regarded extremely cautiously as these types of studies are prone to error and bias.

Third, the use of PDE5i as a comparison is a classic example of apples and oranges. These were dissimilar groups (one with T deficiency, one with ED) treated with dissimilar medications (testosterone, PDE5i). This type of comparison provides no information. Fourth, MI rates were determined by diagnosis code only, and were unverified. Error rates in determining true MI by chart review have been reported as high as 12%, meaning there is considerable uncertainty regarding the primary endpoint [4]. Fifth, there was no clinical information available other than diagnosis codes. This means fundamental information was absent, including testosterone levels pre or post-treatment, smoking history or obesity.

Finally, and most disturbingly, several aspects of the study raise concerns this was a data-mining exercise with the express goal of obtaining a statistically significant result worthy of publication. The lead author is the owner of Consolidated Research, Inc, which owns the dataset used in the study, and several co-authors are his employees. This commercially proprietary relationship to study data raises important concerns regarding motivation and rewards for publication. In addition, a Medline search reveals that only one of the eight authors has ever published an article on testosterone, and that was more than a decade ago. The study design of a full year of observation prior to prescription but only up to 90 d following prescription (much shorter for many men whose refill will occur at 30 d) is strangely unbalanced and without reasonable explanation. Moreover, the short exposure time following receipt of a prescription corresponds to a period when a medication effect would be least likely to be seen, and any observed effect is most likely due to the underlying condition, eg, testosterone deficiency. Finally, it must be assumed that the authors investigated longer time periods post-prescription, such as one year, since those data were available to them and a greater number of results over a longer time course would have greatly strengthened the reliability and credibility of the study. Their failure to report results for longer follow-up leads inevitably to the unfortunate conclusion that no significant increase in MI rates were seen over longer observation periods.

Given the major problems with the study by Finkle et al. it can be generously characterized as “non-informative.” Indeed, one may speculate that it may not even have passed peer review for publication if it were not for the fact that the JAMA paper by Vigen et al. had several months earlier already “established” a new concern for testosterone with regard to CV risk. Once we believe something, new information that appears to confirm that belief tends to be scrutinized less carefully. For groups who already exhibited “hormonophobia” (with acknowledgment to Drs. Bruno Lunenfeld and Svetlana Kalinchenko), and were thus primed against the use of T therapy due to concerns of pharmaceutical industry profits, or aggressive marketing, or anger at anti-aging claims, or anti-sex attitudes, the reporting of CV risks with testosterone was received with a response akin to glee, allowing these groups and individuals to assume a position of, “See, we always knew testosterone therapy was bad.”

It is thus of critical importance to look carefully at the original article suggesting CV risk, by Vigen et al. in JAMA [1]. In this study, a retrospective analysis was performed of 8709 men with serum T <300 ng/dl in the Veterans Administration healthcare system in the US who had undergone coronary angiography. Rates of heart attacks, strokes, and deaths were calculated for men who subsequently received a testosterone prescription and compared to untreated men. The initial reported results were that the absolute rate of events was 19.9% in the group who did not receive testosterone therapy versus 25.7% in the testosterone therapy group at 3 years following angiography. However, that statement was incorrect, and misleading, as “absolute rates” refer to raw data. Shockingly, the actual percentages of individuals who suffered an event was half as great in the T-group as in the no-T group, at 10.1% versus 21.2%, respectively. The authors do not acknowledge those raw numbers, and came to an opposite conclusion by applying complex statistics involving adjustment for more than 50 variables. To correct for the misuse of the term “absolute rate of events,” a revised version of the pdf was made available online post-publication on 12 November 2013, but appeared without a correction until 15 January 2014 [5]. In the corrected version, the term “absolute rate of events’ was replaced with “Kaplan- Meier estimated cumulative percentages with events,” which more accurately reflects the highly statistical nature of the results.

On 5 March 2014 JAMA published several letters regarding this article [6], the authors’ reply [7], and also a new correction [8]. Challenged by a letter that the authors had improperly excluded a group of 1132 men, the authors replied they had revisited this value and had made “an incorrect notation” in the original manuscript, in the text and a figure. They asserted the numbers of men excluded for this reason was only 128, not 1132. This is an 89% error rate, involving >1000 individuals. The numbers for a second group excluded for a different reason were then increased by more than 900 individuals, from 397 to 1301, a three-fold increase. Finally, and most astonishingly, 100 women were now identified among the original group of 1132 individuals, meaning that one out of eleven “men” were actually women.

These revelations in a second correction raise substantial concerns regarding the reliability of the remainder of the data, and their interpretation. A letter recommending retraction of the article based on “gross data mismanagement and contamination” rendering the study “no longer credible” was submitted to the Editor-in-Chief of JAMA, Dr. Howard Blauchner, on 25 March 2014. A subsequent list was submitted two weeks later, signed by 20 medical societies and more than 150 of the most distinguished medical scholars, researchers, and clinicians from 28 countries, including 8 emeritus professors, 9 journal editors, and 14 society presidents. The only continent not represented was Antarctica. ISSAM and several regional Aging Male societies were signers. As of the time of this writing, JAMA has not indicated whether it plans to retract the article.

This action by the world’s experts in testosterone and men’s health is unprecedented. A call for retraction of an article is not taken lightly, particularly by the very organizations and individuals dedicated to medical education and research. This remarkable initiative occurred, in my opinion, not only because the quality of the study by Vigen et al. failed to meet minimum standards of accuracy and reliability, but also because experts in the field have already witnessed the damage this article has caused to men’s health around the world through its publication of misinformation.

For whatever reason, testosterone has always been controversial. Yet those of us who treat men with testosterone deficiency have seen for ourselves the important benefits of this treatment. Those of us who have researched testosterone are familiar with an extensive literature spanning more than 30 years in which low levels of testosterone have been repeatedly associated with increased mortality and cardiovascular risk factors, and treatment with testosterone appears to improve both outcomes and risk factors [9]. In particular, two high-quality recent studies demonstrated reduced mortality, by half, in testosterone-deficient men who received testosterone compared with men who did not [10,11]. Those studies garnered minimal or no attention from medical community or the public. However, the publication of two weak studies with questionable methodology and interpretation appear, and suddenly there is a media frenzy that alters medical thought and practice regarding men with testosterone deficiency.

Science is messy, and there will always be articles published that yield unexpected or awkward results. Every once in a while, though, it is important for the research and clinical medical community to stand up and say, “No! This is wrong!” We do this for science, for the integrity of medical literature, and most importantly, we do it for the health and welfare of our patients.

Note: This issue will be discussed at the next meeting of ISSAM in Almaty, 9–11 September 2014. For more information and updates go to www.issam.pro.

Declaration of interest

Abraham Morgentaler lists the following disclosures- Research grants: Lilly, Auxilium, Antares; Consulting: Auxilium, AbbVie; Scientific advisory boards: Auxilium; Lecture honoraria: Bayer, Merck.