Validity and selection

Per Wandell and coworkers in their paper “Effect of cardiovascular drugs on mortality in atrial fibrillation and chronic heart failure” in the current issue of this journal conclude that: … “life may be prolonged in patients with AF and CHF in primary care prescribed anticoagulants, calcium channel blockers and statins.’ (1). This is not only a carefully worded but also a comforting conclusion, indicating that general practitioners are doing something good while treating patients with atrial fibrillation (AF) and heart failure (HF) according to the current guidelines.

But is it true?

What the study is about

The authors base their conclusions on a large study of the mortality of patients with AF and chronic heart failure (CHF), who were in the custody of 75 primary care centres in the Stockholm County. The authors had access to individual-level data and assessed the risk of dying due to different treatment modalities such as the use of anticoagulants, antiplatelet drugs, calcium channel blockers and statins. The patient data were registered from 1 January 2001 until 31 December 2007. Follow-up time was from the first registration until 31 December 2007. A database was constructed using the Total Population Register, the Inpatient Register and the Swedish Cause of Death Register, containing data on age, gender, education and hospital admissions.

A lot of work and impressive registry linkages

The study represents a large burden of work. Information on 1159 men and 1155 women diagnosed with AF and CHF was available for the analysis. The main objective was to assess the effect of anticoagulant treatment and antiplatelet drugs on mortality, taking into account a number of known confounders such as age, co-morbidity, education, marital status and other pharmacotherapy. The database enabled the authors to construct regression models with mortality as the outcome, adjusting for the potential confounders listed above.

The statistical models are appropriate, and the authors convincingly show that active treatment was associated with prolonged life.

Are their conclusions valid?

But again, is it true? Can one on the basis of the present available data draw the conclusion quoted above? We shall start by briefly looking at some of their results and then discuss the potential pitfalls associated with the study design.

Patients treated with anticoagulants gained almost 2 years compared with those not treated. This fits with older studies, but the readers do not know whether patients with increased risk of haemorrhage or death were excluded from this treatment (2). The indication for such treatment should be based upon the CHA2DS2-VASc score (3). It may be that elements of the score are included in the ‘propensity score’ used for adjusting the main effects, but this is unknown to the reader. The “propensity score” is an attempt to come around the possible biases in non-randomized assessments, but unobserved factors affecting choice of treatment cannot be accounted for in a matching procedure.

The “lack of effect” of anticoagulants in patients above 80 years (adjusted HR, 0.73) may be a question of low power.

Patients treated with anticoagulants gained 1.3 years compared to those treated with antiplatelet drugs. This is also consistent with previous studies, but again the selection of high-risk patients may explain the finding.

Calcium channel antagonists are contraindicated in systolic HF patients, but patients on such treatment gained 1.2 years in this study. It is likely that high-risk patients were excluded, whereby an increased survival was observed among those treated.

The use of statins in patients with AF and HF is not an established indication. A few uncontrolled trials have suggested that statins may be beneficial in heart failure, but also that statins may have harmful effects in such patients (4). Thus, the gain of 1.5 years in statin-treated HF patients suggests the selection of low-risk subjects for this treatment modality.

Study design

We have raised the questions on whether the results shown in Wandell et al.'s paper are valid. Validity can be general or internal. General validity implies that the results are representative for the population they are meant to represent. Internal validity refers to the study design and to what extent variables have been independently ascertained, and that no biases have been introduced during the data collection and affected the effect estimates.

General validity

Any study sample can be looked upon as a sample of subjects from a theoretically defined population (the underlying population at risk). In this case, the population will consist of all subjects under the custody of the 75 primary care centres. An unknown number of individuals in this population will suffer from AF/CHF during the study period. A fraction of these are included in the study sample. How large this fraction is when compared with the total disease frequency is unknown to us. And we do not know how these unrecognized cases fare with regard to fatality and survival, making it impossible to generalize from the present data. The right arm in the figure represents these subjects.

We may assume that the fraction of unrecognized patients with AF/CHF is small and unlikely to influence the conclusion; but again, the external validity is reduced.

Internal validity

The internal validity of the study depends upon an unbiased independent ascertainment of the exposure variables (here: choice of drug treatment), known confounders possibly affecting the outcome and the endpoints (deaths). There is no reason to believe that an independent ascertainment of exposure and outcome has been violated in this study, and one is left with the crucial question of whether all confounding variables have been recognized and taken care of. In any observational study, it is likely that unknown confounders are unevenly distributed in different exposure categories, wherefore an extremely careful attitude regarding causal inferences is warranted.

From the short summary of the results presented above, we have argued that selection is a determinant factor for ending up in the different treatment categories. This is where the observational studies differ from the randomized controlled trial. The main purpose of the randomization process is to nullify all the uncontrollable variables known to influence the effects of the exposure. In this case, the exposure is actually the clinician's final decision with regard to which treatment to choose for the particular patient. The arguments or factors leading to that decision are not fully accounted for by the listed confounding variables, simply because they are unknown to us. We do not know what made the clinician determine why this particular treatment was chosen. This is represented in the left arm in Figure 1 that branches into treatment and non-treatment.

Figure 1. Schematic display of non-random distribution of patients with AF and CHF from the population at risk.

The authors argued for their study by saying that they wanted to ‘…compare the effects of antiplatelets and anticoagulants on mortality, also by sex and age groups in concomitant AF and CHF’ and to ‘…understand the effects on mortality of cardiovascular drugs that are commonly prescribed to these patients’. With regard to the first objective the authors found that anticoagulation (mainly warfarin) was better than anti-platelet drugs, a result that is in accordance with Andersen et al (5). A recent Cochrane study (6) assessed the same issue in patients with heart failure and sinus rhythm and showed no difference in effect, underlining only the problem in using observational data where unknown underlying risk of death is likely to differ.

To this it should be added that the two treatment modalities may have different effects on different causes of death. This brings us to the second objective – to understand the effects of cardiovascular drugs on (total) mortality. Acetylic salicylic acid is associated with lower risk of cancer deaths (7), an increasingly frequent cause of death in the elderly population, whereas warfarin is not known to have any such effect. Total mortality is a highly heterogeneous group endpoint, and it is difficult to see how the present study can contribute to our understanding of the effects of different cardiovascular drugs on mortality.

The study is a good example of how different well-established data files on single individuals can be linked to assess possible associations. The data can be used for creating further hypotheses, but the inferences drawn are hampered by the influence of unknown confounders and cannot be relied upon as causal associations.

It is a comfort to the authors of the present papers (and to the readers) that many of their findings are consistent with previous results already established by randomized controlled trials (RCT). Discrepant findings from observational trials compared with those from RCT cannot be used to argue against the effectiveness of any particular treatment. The propensity score is a way to match for confounding, but it is up to the authors to convince us that this matching was sufficient to avoid a selection based on unknown variables.

Declaration of interest: The author reports no declarations of interest. The author alone is responsible for the content and writing of the paper.