Secondary tests for stratification of risk for atherosclerosis

Keywords::

• Biomarker
• B-type natriuretic peptide
• Cardiovascular risk
• Coronary calcium score
• C-reactive protein
• Reclassification
• Stratification
• Troponin

Atherosclerosis, like Alzheimer’s disease, is an hidden process that only becomes clinically apparent when significant damage has already occurred. Like Alzheimer’s it is diagnosed histopathologically or on post mortem studies. It is now becoming possible to image large vessel atherosclerosis with magnetic resonance imaging (MRI) or computerised tomography (CT) but this is still not sensitive enough to identify smaller vessel disease until significant stenosis has occurred. However these techniques are not practical to use in a large scale clinical screening as opposed to a research laboratory.

This lack of a true diagnostic test means that other approaches have to be applied to try to identify individuals with the disease. The most commonly used version is to rely on mathematical estimation of a future risk of a definitive event from cardiovascular disease (CVD) risk factors identified from populations recruited to epidemiological studies or from primary care databases1,2. The most commonly used of these systems are the Framingham risk calculator3, the European SCORE calculator and the QRISK-2 calculator (UK)4 but others including additional CVD risk factors exist including the Munster Heart Study (PROCAM) and the Reynolds risk score. Applying population-derived equations to individuals is fraught with problems2,5. Users ignore the fact that the predictive power of these calculators is to exclude disease and not to identify people at risk (i.e. high negative predictive values). Some significant risk factors pertaining to sub-sets of the general population are ignored as they add little to prediction in the general case – e.g. family history of CVD, lipoprotein(a), ethnicity – and have to be added back as secondary adjustments1. They also ignore intra-individual variation in risk factor levels which leads to considerable inaccuracy in risk estimation as the confidence intervals on any estimate can be substantial, i.e. 25% of the calculated value6,7. Given these limitations it is surprising that these calculation methods give a quite respectable C-statistic (receiver-operator characteristic area-under-curve) of 0.70 in most validation studies2.

The quest to better identify individuals at risk has led to recent interest in methods of re-classifying individuals at intermediate risk as those with risk for CVD (UK) or coronary heart disease (USA) >20% per decade (equivalent to a CVD mortality 5%; Europe) are classified as high risk and suggested as candidates for automatic pharmaceutical treatment of their CVD risk factors1. Intermediate risk is subject to a variety of cut-offs ranging from 5–10% at the lower bound to 15–20% at the upper boundary though 25% might be statistically more plausible given the errors involved. It is recognised that wherever a cut-off is set there will be misclassification on either side of the threshold6,7. By using two separate thresholds, the opportunity for misclassification is increased. There is disagreement about how to identify the best secondary tests as added changes in C-statistics tend to be small and thus newer statistical approaches based on net re-classification indices (NRIs) and clinically significant NRIs are often used8,9 but there is, as yet, no consensus on how they should be applied10,11. These data can also be cited as weighted comparison analyses if data exist about relative prevalences and population weightings of individual conditions as opposed to confounders12.

In the field of atherosclerosis a number of studies have recently investigated different modalities for re-classification. The oldest and best established is the use of high sensitivity assays for C-reactive protein13 which form the added component in the Reynolds Risk score14. More recently studies have suggested that B-type natriuretic peptide and high sensitivity measures of troponins15 especially troponin-T16 may add to Framingham risk score (FRS). In two recently published papers a further approach is described, based on a multi-analyte immunoassay panel of inflammation, angiogenesis growth factor and apoptosis markers associated with atherosclerotic plaques.

The first paper describes the analytical validation of a seven-marker panel derived from the logistic regression analyses of biomarkers in the Marshfield Clinic Personalised Medicine database of 1084 individuals17. The markers assessed comprised Cutaneous Tcell Attracting Chemokine (CTACK), Eotaxin, Factor Activating ExoS Ligand (Fas ligand), Hepatocyte Growth Factor (HGF), Interleukin-16 (IL-16), MCP-3 (Monocyte Chemoattractant Protein-3 (MCP-3), and Soluble Factor Activating ExoS (sFAS). Other biomarker panels for reclassification of CVD risk have also been described based on more commonly measured plasma analytes15,18. The analysis of the risk stratification panel was based on two categories of people: the ‘No event’ were not known to have suffered a CVD event but had not been systematically investigated for the presence of subclinical atheroma; and the CVD event group had a myocardial infarction in the 3 months prior to sample collection. This immediately poses a problem because it is possible that marker positivity is due to the response to severely damaged myocardium, rather than a marker of future risk. This criterion excludes other potential components relevant to CVD including markers of stroke and peripheral arterial disease. A further problem is the very wide limits allowed for pre-analytical variation (±30%) and the low precision of the assay methods (inter-operator coefficient of variation [CV] up to 23 ± 5%). Finally, the control group was only sampled once, so natural intra-individual variation in marker concentrations was not evaluated. These factors will result in unacceptably large misclassification errors if used in screening populations.

We have previously evaluated the effect of analytical imprecision on misclassification rates in a screening population, using traditional risk factors (blood pressure, lipids, age). With analytical CVs 2.5–5% and intra-individual CVs 6.5–7.5% the 95% confidence interval of the estimated risk for a single sample at a 30% cut off was ±6.9% (i.e. for a patient whose ‘true’ risk is 30%, the impact of intra-individual variation in marker levels means that their calculated risk will be in the range 23.1–36.9%)6. This leads to significant misclassification at the 30% risk threshold with 30.3% of ‘true’ positives being given a false negative result and 20.4% of those given a positive result which should have received a negative result. Similar error rates applied at other risk threshold and have been validated in other analyses7. Absolutely correct classification for the Framingham risk equation would have required an infinite number of tests, but in this scenario it was necessary to repeat testing nine times before the incremental improvement on classification was acceptable. With the much wider CVs reported by Nolan et al.17, the number of misclassifications occurring and therefore repetitions to minimise error needed will be unacceptably high.

Furthermore analyses and panels used to determine risk should rely on truly independent risk factors/biomarkers. Thus in the Framingham equation close co-correlates of principal risk factors did not add significantly to the basic model and were excluded. These included triglycerides (anti-correlate of HDL-C), obesity (co-correlate of blood pressure). This pruning can be performed by test of C-statistics, but this is relatively crude and in the past it has been routinely performed using co-correlation matrices. As simple matrix data is best suited for the analysis of two-way correlations, principal component or factor analysis is commonly used in more complex co-correlation scenarios with Eigen value (>1) or preferably parallel analysis to increase specificity of associated components and identify independent risk factors or biomarkers. Such data analyses for any biomarker panel help clarify its utility in a panel and allow substitution if analytical limitations are significant. Such analyses are rarely reported for biomarker panel datasets. Further work to improve the assays would be essential before they could be considered in practice. Furthermore, the pre-analytical variation of up to 30% for some analytes would either mean very precise collection protocols would be essential, which would render the test useless for primary care, or new inhibitors of the pre-analytical change would need to be developed.

The second paper describes the clinical validation of the marker panel in the Multi-Ethnic Study of Atherosclerosis (MESA) cohort study using a selected random sample of 11.1% of the cohort – 623 patients with 21 CVD events – and a general sample of 823 patients including all 222 with CVD events19. The MESA sample improves on many studies which are limited to Caucasian cohorts and then are subject to ethnicity-based error13,20. This heterogeneity may account for the relatively low C-statistic of 0.65 found for the Framingham risk score in this study. The samples used in this study had been collected some years previously and were therefore analysed retrospectively with knowledge of CVD events that had occurred, but possible differences in pre-analytical collection procedures would not have been known, and although the sample size was large enough, hopefully to remove this influence, it cannot be rejected as a possible confounder. The small cohort was used to derive a Cox proportional hazard model which was then applied to the full dataset. The sampling method used in case–control selection from larger cohorts can significantly affect the results of any analysis21. Use of a single pass assessment of the model will give a false impression of the performance: a non-parametric boot-strapping technique as described for other screening populations where multiple data sets are hard to acquire should have been used to improve assessment, in particular to identify confidence intervals for the effectiveness of the test22. Further work therefore needs to be carried out to assess whether this risk stratification panel would be clinically useful.

Though many individual comparisons with the Framingham Risk scores have been performed in a wide variety of cohort studies, only a few studies have investigated multiple techniques in a single cohort23. The current consensus from inter-study comparisons favours the use of N-terminal B-type natriuretic peptide (BNP)24–26 and/or coronary artery calcium scoring (CACS)23 as the best methods of re-classification for atherosclerosis in the general population. These methods seem to show similar results in randomised control trials as in epidemiological cohort studies but with reduced effect sizes while many other proposed biomarkers have failed to demonstrate any added benefit when applied in the more rigorous RCT design of trials with hard CVD endpoints27. The utility of other screening panels and methods remains to be clearly established in both epidemiological and randomised control trial populations.