Predictive Ability of Direct-to-Consumer Pharmacogenetic Testing: When is Lack of Evidence Really lack of Evidence?

KEYWORDS::

• direct-to-consumer
• personal genome test
• pharmacogenetic test
• predictive ability

For several years personal genome tests have been offered directly to consumers via the internet. Based on single genome scans, these direct-to-consumer (DTC) tests predict susceptibility to common multifactorial diseases, such as Type 2 diabetes, coronary heart disease and nonfamilial cancer, inform about predisposition to drug response, report carrier status for monogenic diseases, or provide all of the above. The market is served by a few major players, such as 23andMe and deCODEme, and numerous lesser-known companies such as YouScript, GenePlanet and Theranostics [101–105].

The market for DTC personal genome testing is steadily increasing, even though the evidence on the predictive ability and clinical utility of these tests is limited. The few available studies have shown that predicted risks of multifactorial diseases differed between companies and were sometimes even contradictory for the same individual [1,106], but large-scale studies on the predictive ability are lacking. From prediction studies that investigated genetic risk models based on different but comparable selections of SNPs, we know that the predictive ability of genetic testing for multifactorial diseases is low to moderate, except when one or more SNPs had substantial impact on disease risk [2]. From this indirect evidence it is concluded that the DTC offer of genetic testing for multifactorial diseases is premature.

To date, most of the discussion about DTC personal genome tests has focused on the prediction of these multifactorial diseases and less attention has been given to the predictive ability and clinical utility of pharmacogenetic testing. Yet, many companies offer pharmacogenetic testing to inform about the genetic susceptibility to drug response and side effects of treatment [3], such as the efficacy of clopidogrel and the risk of side effects from abacavir treatment. A recent comparison of eight DTC companies showed that genetic testing was clearly recommended by the US FDA for only four out of 30 drugs reviewed [3]. It may be that many of the remaining drugs have not been approved either because empirical studies demonstrated that their predictive ability is insufficient, or because they have not yet been investigated in empirical studies.

The predictive ability of DTC pharmacogenetic testing is indicated by the test‘s sensitivity and specificity, which are calculated from the percentages of drug response or treatment side effects in genotype subgroups. An example that has been intensively investigated is abacavir, a drug that reduces the amount of virus in the blood of HIV patients [4], but approximately 5% of the patients have hypersensitivity reactions [5]. Prediction studies have shown that these hypersensitivity reactions are well predicted by the presence of the HLA-B*5701 allele as demonstrated by the high sensitivity and specificity for the HLA test. The two largest studies so far showed that the sensitivity was 100% and that the specificity was over 96% for predicting abacavir-induced hypersensitivity reactions [6,7]. This high predictive ability makes genetic testing for the HLA allele essential for decision-making of abacavir treatment in HIV patients, irrespective of whether the test is prescribed in clinical care – as recommended by the FDA [107] – or ordered by patients via commercial tests.

An example of a pharmacogenetic test with a lower predictive ability is CYP2C19 testing for clopidogrel response, which is offered by various companies including 23andMe, YouScript and GenePlanet [101,103,104]. Clopidogrel is a platelet inhibitor that reduces the risk of adverse cardiovascular events in patients who underwent coronary stenting [8]. The efficacy of clopidogrel differs between patients and is associated to CYP2C19 genotype status, but the sensitivity and specificity in a recent study indicated that the predictive ability, despite FDA recommendation [108], was only slightly better than random (33 and 71%, respectively; prediction is random when sensitivity = 100 - specificity) [9].

Unlike abacavir, clopidogrel and others, the predictive ability of many gene–drug associations has not been investigated in empirical prediction studies. When no prediction studies are available, it is generally concluded that evidence on the predictive ability is lacking [10]. This conclusion is unsatisfactory in a time when genetic and therapeutic discoveries are increasing, and budgets for translational research are shrinking. There is an urgent need to find alternative sources of evidence to expedite translational research and advance the introduction of new tests and therapies in healthcare practice or in commercial applications.

The high predictive ability of the HLA test for abacavir hypersensitivity is not a lucky draw, but a direct inference from the strong association between the HLA allele and abacavir-induced hypersensitivity reactions. In the aforementioned two studies, the odds ratios for this association were over 1400 [6,7]. Also, the low predictive ability of the CYP2C19 test for clopidogrel efficacy is in line with expectations, as the largest meta-analysis so far reported an odds ratio of ‘only‘ 1.34 [11]. These examples show that the odds ratios from gene–drug association studies are informative for the predictive ability of pharmacogenetic tests.

Epidemiological odds ratios can be used as an indication of the predictive ability [12], because odds ratios, sensitivity and specificity all are calculated from a 2 × 2 contingency table that reports the drug response or treatment side effects by genotype subgroups. A simple formula shows that the odds ratio can be calculated from the sensitivity and specificity, namely:

Using this formula, we calculate that the odds ratios for the abovementioned examples of abacavir and clopidogrel are >1400 (sensitivity: 100%; specificity: >96%) and 1.21 (sensitivity: 33%; specificity: 71%). Stronger gene–drug associations yield pharmacogenetic tests with higher predictive ability.

When prediction studies are not available, an indication of the predictive ability of pharmacogenetic tests can thus be obtained from epidemiological studies. The sensitivity and specificity, and also the positive and negative predictive value, can be calculated from a 2 × 2 table when basic epidemiological parameters as the odds ratio, the variant frequency and the frequency of the outcome (e.g., drug response or adverse side effects) are known [13]. For example, in a recent study, the hazard ratio for the association between the CYP2D6 gene and tamoxifen response was 0.86 [14]. Based on the genotype and outcome frequencies from the same study and interpreting the hazard ratio as an odds ratio, we calculate that the sensitivity and specificity of CYP2D6 testing, which is offered by YouScript [103], would be 35 and 62%, respectively (see Supplementary Material), which again suggests a predictive ability that is only slightly better than random. Another example is the association between the SLCO1B1 gene and the risk of statin-induced myopathy, which is offered as a test by GenePlanet and Theranostics [104,105], and for which the odds ratio was estimated 6.8 in one study [15] and 1.5 in a more recent study [16]. Using genotype and outcome frequencies from the first study, we calculate that the specificity would be 76% in both scenarios, but that the sensitivity would decrease from 69 to 33% when the odds ratio is lower. In both scenarios, the genetic test is unlikely of interest to individual consumers as the risk of myopathy, the positive predictive value, increases from an average of 1.6 to 4.4% and from 1.6 to 2.1%, respectively (see Supplementary Material).

When evidence from prediction studies is lacking, gene–drug association studies can provide useful insights in the expected predictive ability of pharmacogenetic tests and indicate whether further research is warranted. When simple calculations from a 2 × 2 table suggest appreciable sensitivity and specificity, a formal prediction study can be pursued to investigate whether the epidemiological parameters were representative for the population in which genetic testing is intended. Several meta-analyses have reported considerable between-study heterogeneity in the odds ratios [11,17,18], which implies that the association might be stronger and the test might be more predictive in certain (sub)populations, but also vice versa. The predictive ability might also be different when gene–drug associations are investigated in populations that did not represent the full clinical spectrum that is observed in healthcare practice or in studies that used intermediate rather than clinical end points. Yet, it should not be expected that the results of prediction studies will be surprisingly different than those obtained from simple calculations when there is limited variation in the epidemiological parameters.

When no prediction studies are available, simple calculations could be considered to preapprove DTC pharmacogenetic testing if epidemiological parameters can be obtained from robust gene–drug association studies that investigated clinically relevant outcomes in representative populations. These parameters and the expected predictive ability could then be publicly challenged by experts, and be empirically validated in postmarketing studies. The expected positive and negative predictive value and their departure from the pretest risk could, or even should, be communicated to consumers so they can make informed decisions about the benefits of DTC pharmacogenetic testing.

Financial & competing interests disclosure

This work was supported by the National Cancer Institute at the NIH (grant number HHSN261201200425P), the Centre for Medical Systems Biology and the Vidi grant from The Netherlands Organisation for Scientific Research. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Additional information

Funding

This work was supported by the National Cancer Institute at the NIH (grant number HHSN261201200425P), the Centre for Medical Systems Biology and the Vidi grant from The Netherlands Organisation for Scientific Research. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript.