Absolute risk reduction may depend on the duration of the follow-up

Abstract

Response to: Diamond DA and Ravnskov U. How statistical deception created the appearance that statins are safe and effective in primary and secondary prevention of cardiovascular disease. Expert Rev Clin Pharm 2015;8(2):201–210

I have read the paper titled “How statistical deception created the appearance that statins are safe and effective in primary and secondary prevention of cardiovascular disease” in your journal [1] with great interest.

As a biostatistician, I don’t want (and cannot) to comment on the pharmacological aspects raised in the paper. However, one of the most emphasized points raised by the authors is – statistically – unfounded in my opinion.

The authors argue that absolute risk reduction (ARR) (or its reciprocal, the number needed to treat (NNT)) should have been used instead of relative risk in presenting the results of the trials they have reviewed. They even state that this was an intentional decision to deceive the readers. While I can’t speak about the intentions of those presenting the results of these studies, the authors are methodologically wrong in their first claim. The application of relative risk is correct, and the application of ARR would have been incorrect.

The problem with absolute risk reduction – and therefore with NNT too – is that it depends on the length of the follow-up,[2–4] should the risk of the disease accumulate over time (which is clearly the case for cardiovascular diseases). The relative risk is constant in this sense (apart from the possible time-varying treatment effectiveness), but how it translates to risk difference depends on the risk of the control group, that is, the baseline (background) risk, which in turn depends on how long we observe it.

Let us examine a concrete example to illustrate this. Assuming an incidence of 1%/year (just to provide a numerically simple example) without treatment, the risk in the control group will be 1% in 1 year. Assuming a treatment effectiveness of 50%, this translates to an absolute risk reduction of 0.5%points (NNT = 200). In 2 years, with the effectiveness remaining completely constant, the incidence raises to 1.99% in the control group, and 0.995% in the treatment group (ARR = 0.995% points, NNT = 100.5). In 5 years, we have a risk of 4.9% and 2.45% (ARR = 2.45% points, NNT = 40.8). In 10 years, we have an NNT of 20.9, in 20 years, 11. Thus, the metrics suggested by the authors change over time, in other words, depend on the length of the follow-up – while the actual effectiveness of the drug is constant!

This is actually the very reason why the usage of relative risk makes sense: we can extrapolate, at least when assuming a time-invariant treatment effect, even if the study had a duration that is shorter than the clinically realistic duration of the treatment with the drug.

Therefore, the application of ARR and NNT is only valid if the length of the follow-up is comparable to the risk period. However, quite the opposite is true in our case: the relevant time horizon for the onset of cardiovascular diseases can be measured in decades, while the cited trials had a follow-up of years. The application of ARR or NNT in this case is clearly inappropriate.

This issue has been described decades ago in the literature (see, e.g., the discussion of [5]).

To put my remarks into a broader context, the authors are actually right that for many purposes, ARR is the useful indicator (to evaluate the public health impact of the intervention, to judge side effects etc.),[6,7] and the application of relative risk may indeed be unfortunate or even misleading in these cases.[8] We nevertheless have to keep in mind that ARR is a derived indicator: it is the product of relative risk (a property of the drug) and the baseline risk (property of the disease). A drug trial should – of course – measure what is characteristic for the drug. It would be not only unnecessary, but downright harmful to measure only a “composite” indicator (it would, for instance, prevent us from assessing the drug’s effectiveness in another country, with different disease risk); we should rather measure the “components”. The authors are right that users of these results should place them into context (e.g., start with calculating ARR for their own country), and due to this exact reason, the original publications should also report ARRs. But if the authors consider the presented ARRs to be “miniscule”, even that means that the risk of the disease, and not the drug’s effectiveness is exaggerated. (However, like I have warned beforehand, it is quite questionable in my opinion to judge the risk of a cardiovascular disease based on trials having only a few years of follow-up.)

To sum up: what the authors label as “statistical deception” is in fact the statistically sound approach, and what they suggest as statistically correct approach would be – ironically – deceptive.

Financial & competing interests disclosure

The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.