The Influence of a Gene-Expression Signature on the Treatment of Diagnostically Challenging Melanocytic Lesions

Abstract

Aim: The effect of a gene-expression-based test on treatment of melanocytic neoplasms by dermatologists was evaluated. Patients & methods: Pathologists submitted diagnostically challenging melanocytic neoplasms to a clinical laboratory for testing accompanied by pretest surveys documenting the intended treatment recommendations. The actual treatment rendered by dermatologists was then documented after testing. Changes between the pretest recommendations and actual treatment were analyzed. Results: In 71.4% (55/77) of cases, there was a change from pretest recommendations to actual treatment. The majority of changes were consistent with the test result. There was an 80.5% (33/41) reduction in the number of biopsy site re-excisions performed for cases with a benign test result. Conclusion: The actual treatment of diagnostically challenging melanocytic neoplasms is influenced by the test.

Keywords::

• clinical utility
• gene-expression signature
• melanocytic nevi
• melanoma
• molecular diagnostic techniques
• RT-PCR

As the seventh most common cancer among adults, melanoma will account for an estimated 10,130 deaths in 2016 [1]. Although late-stage melanoma is among the most fatal forms of skin cancer, 10-year survival rates for early stage disease are as high as 95% [2]. Therefore, early and accurate diagnoses of melanocytic neoplasms are vital for improved patient outcomes.

The current standard for melanoma diagnosis is histopathologic examination by a pathologist, and the majority of melanocytic neoplasms can be accurately classified by this approach. In some instances, however, confidently differentiating benign melanocytic nevi from malignant melanoma can be extremely difficult or impossible [3–5]. Numerous studies have demonstrated that pathologists arrive at different diagnoses in some cases when evaluating the same melanocytic neoplasm, and even a consensus diagnosis among multiple expert pathologists does not always align with the ultimate clinical outcome [3,4,6,7].

Given the limitations of traditional histopathologic diagnosis, ancillary techniques have been sought by pathologists confronting diagnostically challenging melanocytic neoplasms. These techniques include FISH, array comparative genomic hybridization and gene-expression profiling, all of which are classified by the US FDA as laboratory developed tests (LDTs) [8–11]. These tests are not (yet) subject to FDA oversight, but rather have been designed and validated according to regulations of the Clinical Laboratory Improvement Amendments [12].

In an effort to establish a systematic, evidence-based process for assessing molecular technologies currently regulated as LDTs, the US CDC and Prevention has led various initiatives such as the ACCE model (analytical validity, clinical validity, clinical utility and ethical/legal/social issues) and the EGAPP initiative (evaluation of genomic applications in practice and prevention) [13,14].

Recently, a novel LDT based on a 23-gene-expression signature has been developed to aid in the diagnosis of histologically ambiguous melanocytic neoplasms. This LDT uses quantitative reverse transcription-PCR to measure the differential expression of 14 tumor marker genes and the average expression of nine housekeeping genes [9]. The development of this test was designed to closely align with the evidence structure proposed through these government-led initiatives. To this end, the gene-expression signature has been shown to differentiate melanoma from nevi with a sensitivity and specificity >90% in large cohorts of diverse lesions [9,15] with robust analytical performance and reproducibility [16]. In addition, a previous study showed that the use of the gene-expression signature would likely reduce healthcare costs by 8.3% over a 10-year period, primarily by reducing the number of missed melanomas [17].

A recent study demonstrated that pathologists using the test for challenging melanocytic neoplasms made more definitive diagnoses and treatment recommendations [18]. However, that study did not quantify the test’s impact on the actual treatment of the neoplasm, which is ultimately provided by a dermatologist. In an effort to more thoroughly understand the effect of the test on the full spectrum of patient care, the current study aimed to measure the change between proposed treatment recommendations made by pathologists prior to testing (i.e., the treatment the pathologist was planning to recommend before obtaining the test result) and the actual care provided by the treating dermatologists after receiving the pathologist’s final diagnosis and treatment recommendations (which were now informed by the test result).

Methods

Representative sections of diagnostically challenging melanocytic neoplasms were submitted by US-based dermatopathologists to Myriad Genetic Laboratories, Inc. (UT, USA) between October 2013 and May 2014 for gene-expression testing. Samples were submitted as part of a prospective clinical experience program aimed at assessing the influence of the test on diagnostic and medical management decisions among physicians [18]. In this program, dermatopathologists submitted for testing any melanocytic lesions encountered in their clinical practice for which uncertainty existed regarding the histopathologic diagnosis, and for which they felt ancillary testing could prove useful. Samples were submitted prospectively as part of the normal diagnostic work-up of the lesion.

The test was carried out on formalin-fixed paraffin-embedded tissue sections according to previously described methods [16]. Briefly, a pathologist identified representative areas of the lesion using a single hematoxylin and eosin stained section and macro-dissected the area from unstained tissue sections that were pooled into a single tube for RNA extraction. A quantitative reverse transcription-PCR assay was run to measure the expression of each of 23 genes (14 biomarkers and nine housekeepers, Box 1) and a weighted algorithm was applied to the expression values to produce a single numeric score plotted on a scale ranging from -16.7 to +11.1. During the course of the clinical experience program, an indeterminate reporting range was introduced to minimize false negatives. It was found that scores between -2.0 and -0.1 comprised approximately half of the malignant lesions that appeared to be misclassified by the signature in the initial validation study of the test, and scores within this range were therefore reported as ‘indeterminate’ after 7 May 2017 [9]. Any score <0 calculated prior to this date was reported as likely benign. Scores between -2.0 and -0.1 that were calculated after 6 May 2014 were reported as indeterminate, while scores <-2 that were calculated after this date were reported as likely benign [9,15]. Scores ≥0 were reported as likely malignant throughout the study period [18].

Pre-test surveys were completed by participating pathologists at the time of sample submission (the full survey may be viewed in the Supplemental Material). These surveys documented the pathologist’s favored diagnosis and intended treatment recommendations at that time (prior to obtaining the test result). The pre-test diagnoses were recorded on the surveys as ‘favor benign’, ‘favor malignant’ or ‘indeterminate’. Options for pre-test treatment recommendations were: ‘no further treatment necessary’, ‘no further treatment necessary if lesion is completely excised’, ‘close clinical surveillance of the biopsy site for possible recurrence’, ‘re-excision with a margin of normal skin’, ‘wide local excision’, ‘sentinel lymph node biopsy and/or other evaluation for evidence of metastasis’ and ‘other’.

The test result was reported to the submitting pathologist, who considered it within the context of all other relevant data and then issued a final pathology report. The pathology report was then considered by the dermatologist, who made the final treatment decision. Post-test surveys utilizing the same treatment options included on the pre-test survey were completed by the dermatologist between 3 and 6 months after testing to document the actual treatment carried out for each patient.

All patient identifiers were removed prior to data analysis. The study protocol satisfied criteria for waiver of consent and waiver of authorization as established by Quorum Review IRB (WA, USA), based on determination of minimal risk to eligible patients. Cases were eligible for inclusion if they were submitted for clinical testing with a completed test requisition form, received a clinically valid gene-expression test result and had completed pre- and post-test surveys. In addition, only cases submitted by pathologists with board certification in dermatopathology who had submitted ≥20 cases were included in this retrospective analysis. All cases were assumed to confer some degree of diagnostic challenge as they were submitted for ancillary diagnostic testing. However, the subset of cases for which the pathologist indicated a pre-test histopathologic diagnosis of ‘indeterminate’ (i.e., neither benign nor malignant could be favored) were considered to be the most representative of the test’s intended use population. As such, analysis was focused on this ‘diagnostically challenging’ subset of cases.

Descriptive statistics were calculated for demographic and other baseline characteristics. Proportions and exact binomial 95% CI were calculated for changes from pre-test recommendations to actual treatment. Changes in treatment were evaluated based on the most invasive treatment selected from the prespecified survey options, with ‘Sentinel lymph node biopsy and/or other evaluation for evidence of metastasis’ considered the most invasive and ‘no further treatment necessary’ considered the least invasive. Treatment upgrades and downgrades (i.e., changes to more or less invasive options) and re-excision of the lesion (a second surgical procedure in which the initial biopsy site and surrounding skin are removed) after obtaining the final pathology report were also assessed, with exclusions of ‘other’ treatment selections and patients who were lost to follow-up during the course of the study.

Results

The final cohort consisted of 315 cases submitted from three eligible pathology practices (one academic group and two community practice groups). The cohort characteristics are summarized in Table 1 and included age, gender, procedure type, anatomic location of the lesion, pre-test diagnosis favored by the submitting pathologist and test result. For 77/315 (24.4%) of the cases, the submitting pathologist’s pre-test diagnosis was ‘indeterminate’. These cases were therefore identified as a ‘diagnostically challenging’ subset of the cohort. The remaining cases were submitted with a pre-test diagnosis of ‘favor benign’ (137/315 cases, 43.5%) or ‘favor malignant’ (101/315 cases, 32.1%).

The gene signature classified 214/315 (67.9%) cases as ‘benign’ and 92/315 (29.2%) as ‘malignant’. Overall, 63.8% (201/315) of cases had a change from dermatopathologist pre-test recommendations to actual treatment by the dermatologist (Table 2). A total of 34/201 cases where ‘other’ was selected as the treatment or that were lost to follow-up were excluded from further analysis. Among the remaining 167 cases, 55.7% were upgraded to more invasive treatment and 44.3% were downgraded to less invasive treatment (Table 2). For cases that received a benign gene signature test result and a change in treatment, 65/125 (52.0%) were downgraded to less invasive treatment and 60/125 (48.0%) cases were upgraded to more invasive treatment. However, 44/60 cases that were upgraded to more invasive treatment also had a pretest diagnosis of benign by the dermatopathologist (Supplementary Table 1). This suggests that the upgrade in treatment was based on clinical factors other than the gene signature test results. Among cases that received a malignant test result and a change in treatment, 28/37 (75.7%) were upgraded to more invasive treatment and 9/37 (24.3%) were downgraded to less invasive treatment. Again, the gene signature test results agreed with the pretest diagnosis of malignant for 4/9 cases that were downgraded to a less invasive treatment (Supplementary Table 1).

Table 2 presents changes from pre-test treatment recommendations to the actual treatment for the 77 ‘diagnostically challenging’ cases. There was a change between the pre-test treatment recommendations that would have been made by the pathologist to the actual treatment carried out by the dermatologist in 55/77 (71.4%) cases. After excluding one case that was lost to follow-up and two cases for which ‘other’ was selected as the treatment, it was found that 16/52 (30.8%) cases with treatment changes were upgraded to more invasive treatment while 36/52 (69.2%) were downgraded to less invasive treatment.

Overall, 39/52 (75.0%) ‘diagnostically challenging’ cases that received a change from pre-test recommendation to actual treatment were treated in a manner consistent with the test result. This included 33 cases with a benign test result for which there was a change to less aggressive treatment, and six cases with a malignant test for which there was a change to more aggressive treatment. Cases that received a benign test result and a change in treatment were statistically more likely to be downgraded to a less aggressive treatment (p < 0.001). There were not enough cases with malignant test results that received a change in treatment to determine the statistical significance of these changes.

All cases for which there was a change in management were also evaluated to determine whether the dermatologist performed a re-excision of the biopsy site (Table 3). Re-excision involves an additional surgical procedure in which the initial biopsy site is removed along with a variably sized ‘margin’ of apparently uninvolved skin in an effort to ensure complete removal of any possible residual lesion that might not have been eradicated by the initial biopsy. When the test result was benign, the number of re-excisions was reduced by 63.2% (55/87) relative to pathologists’ initial, pre-test excision recommendations for the overall cohort. This trend was even more pronounced in the diagnostically challenging subset, with a reduction by 80.5% (33/41). When the test result was malignant, re-excision occurred in 24/50 (48.0%) of all cases (Table 3). Among the diagnostically challenging cases that received a malignant test result, 6/11 (54.5%) cases followed the pre-test recommendation for re-excision, while a decision to forgo re-excision was made in 3/11 (27.3%) cases. The two remaining cases that received malignant test results had ‘Other’ indicated as their actual treatment, with associated comments describing additional surgical procedures in the form of ‘surgical excision’ and ‘Mohs surgery’ (referral to a dermatologic surgeon with special expertise in the excision of cutaneous neoplasms).

Discussion

This study evaluated the impact of a test for melanoma on the treatment 315 patients whose biopsies were submitted for ancillary diagnostic testing with a 23-gene-expression signature. The results showed that the actual treatment performed by the patient’s dermatologists differed from the treatment that would have been recommended without the test by the diagnosing pathologist in 63.8% (201/315) of cases. The change in treatment aligned with the test result in 52.0% of cases that received a benign test result and 75.7% of cases that received a malignant test result. This high frequency of upgrades to more invasive treatment with a malignant test result are consistent with a previous study showing that the use of this gene signature reduces healthcare costs by reducing the number of missed melanomas [17].

Importantly, the trends observed for the overall cohort were more pronounced in the intended use population, where 71.4% (55/77) of diagnostically challenging cases had a change in treatment. In this subset, the majority of treatment changes aligned with the test result and cases with a benign test result were statistically more likely to receive a downgrade in actual treatment relative to the pre-test recommendations. In addition, these changes were accompanied by an 80.5% decrease in the number of re-excisions performed for neoplasms that produced a benign test result.

These data are supported by the results of a prior study which showed that pathologists modify their diagnoses and treatment recommendations in approximately one-half of cases when the test is obtained as part of the diagnostic work-up [18]. Since most dermatologists generally consider their pathologist’s final diagnosis and available treatment recommendations when deciding on treatment, it is reasonable to conclude that an observed change between pre-test recommendations and actual treatment are at least in part attributable to the previously documented influence of the test on pathologists’ decisions.

In the diagnostically challenging subset, the change between pre-test treatment recommendations to actual treatment were made in contradiction to the test result (treatment upgrades despite a benign result, or downgrades despite a malignant result) in 23.3% of cases with a benign score and 33.3% of cases with a malignant score. This is not entirely unexpected given that the test is an adjunctive, rather than absolute, diagnostic tool. Furthermore, factors other than a pathologist’s categorization of a lesion as benign or malignant impact treatment in a certain percentage of cases, including whether the patient has a personal and/or family history of melanoma, patient preference, anatomic location of the lesion, concerning histopathologic and/or clinical features, or prior treatment. For cases in which the treatment change appeared discordant with the test result, it is possible that such factors proved to be of greater significance in rendering a final treatment decision than the test result alone. For example, several diagnostically challenging lesions with a malignant test result that did not undergo re-excision had clear margins on the original biopsy. It therefore seems likely that the dermatologist considered the lesion entirely removed and decided that re-excision was unnecessary in those cases.

The incorporation of an indeterminate reporting range during the course of the clinical experience program represents a limitation of this study. Specifically, only a small proportion of all cases (2.9%) generated a result that fell within the indeterminate reporting range for the test, and none of these occurred within the ‘diagnostically challenging’ subset. This is not representative of results observed in clinical testing, where approximately 10% of all cases will generate a score in the indeterminate range [18]. Additionally, seven of the diagnostically challenging cases that generated a score between -2.0 and -0.1 were reported as benign rather than indeterminate, potentially impacting final treatment decisions in a different way for those cases. Based on the uncertain clinical meaning of an indeterminate result, it might be expected that this result would influence treatment decisions to a lesser degree than more definitive benign or malignant results; however, additional data would be needed to confirm this.

There were some limitations of this analysis. Pre-test surveys were completed by the dermatopathologist and were not conveyed to dermatologists or patients. As such, these surveys may not have been completed with the same considerations as the post-test or actual treatment recommendations. In addition, the pathology report provided to the treating dermatologist after gene-expression testing was not collected as part of this study. As there is no consensus that treatment recommendations should be made by the pathologist, it is unclear whether the dermatopathologists’ treatment recommendations were conveyed to the treating dermatologist. These limitations may confound the analysis of changes in treatment. However, the magnitude and direction of changes in treatment recommendations in the intended use population suggest that final treatment was impacted by the use of the test, despite these confounding variables.

Conclusion

Key stakeholders in the healthcare system are increasingly seeking evidence regarding the clinical utility of LDTs in order to determine their effect on actual clinical practice [14,19]. This study demonstrates that the use of a gene-expression signature-based test by pathologists in the work-up of challenging melanocytic neoplasms impacts the treatment of patients by dermatologists. Additional studies are needed to assess the long-term clinical outcomes associated with these treatment decisions.

Table 1.  Demographic and other baseline characteristics.

Characteristic	Statistic/category	All patients (n = 315)^†	Diagnostically challenging subset (n = 77)
Age (years)	n	315	77
	Mean	49.0	43.9
	SD	18.85	17.32
	Median	51	44
	Min, max	2, 94	5, 81
Gender	Female	166 (52.7%)	42 (54.5%)
	Male	149 (47.3%)	35 (45.5%)
Procedure type	Shave biopsy	274 (87.0%)	72 (93.5%)
	Elliptical excision	23 (7.3%)	3 (3.9%)
	Punch biopsy	17 (5.4%)	1 (1.3%)
	Missing	1 (0.3%)	1 (1.3%)
Anatomical site of lesion	Back/neck	111 (35.2%)	24 (31.2%)
	Extremities	81 (25.7%)	18 (23.4%)
	Abdomen	28 (8.9%)	14 (18.2%)
	Chest	23 (7.3%)	7 (9.1%)
	Face	14 (4.4%)	1 (1.3%)
	Acral	8 (2.5%)	1 (1.3%)
	Scalp	5 (1.6%)	2 (2.6%)
	Other	45 (14.3%)	10 (13.0%)
Pre-test diagnosis	Benign	137 (43.5%)	0
	Malignant	101 (32.1%)	0
	Indeterminate	77 (24.4%)	77 (100.0%)
Gene signature score	N	315	77
	Mean	-3.5	-5.3
	SD	5.37	4.68
	Median	-3.6	-5.2
	Min, max	-16.3, 8.4	-15.6, 4.0
Gene signature result	Benign	214 (67.9%)^‡	64 (83.1%)^‡
	Malignant	92 (29.2%)	13 (16.9%)
	Indeterminate	9 (2.9%)^§	0

^†68/315 (21.6%) patients were tested after the introduction of the indeterminate zone.

^‡Scores between -2.0 and -0.1 were produced by 33 cases prior to the introduction of the indeterminate reporting range, seven of which were in the ‘diagnostically challenging’ subset, and were reported as benign.

^§The small proportion of ‘indeterminate’ test results reflects the incorporation of the indeterminate reporting range during the course of the clinical experience program.

Max: Maximum; Min: Minimum; SD: Standard deviation.

Table 2.  Change between most invasive pre-test treatment recommendations and actual treatment.

Overall change in treatment	All patients (n = 315)		Diagnostically challenging subset (n = 77)
	Fraction	Percentage	Fraction	Percentage
All test results	201/315	63.8	55/77	71.4
Benign test result	136/214	63.6	44/64	68.8
Malignant test result	58/92	63.0	11/13	84.6
Indeterminate test result	7/9	77.8	0	–
Upgrades and downgrades for those who made a change in treatment
All test results^†
Upgrade	93/167	55.7	16/52	30.8
Downgrade	74/167	44.3	36/52	69.2
Benign test result^‡
Upgrade	60/125	48.0	10/43	23.3
Downgrade	65/125	52.0	33/43	76.7
Malignant test result^§
Upgrade	28/37	75.7	6/9	66.7
Downgrade	9/37	24.3	3/9	33.3

^†30 cases were excluded due to selection of ‘other’ under ‘treatment recommendations’ (two in the diagnostically challenging subset) and four cases were lost to follow-up during the course of the study (one in the diagnostically challenging subset).

^‡For benign test results, seven cases were excluded due to selection of ‘other’ under ‘treatment recommendations’ and four cases were lost to follow-up (one in the diagnostically challenging subset).

^§For malignant test results, 21 cases were excluded due to selection of ‘other’ under ‘treatment recommendations’ (two in the diagnostically challenging subset).

Table 3.  Treatment decisions among cases with changes from pre-test recommendations to actual treatment.

All patients^† (n = 201)		Test result
		Benign (n = 136)	Malignant (n = 58)	Total (n = 201)
Pretest treatment recommendation by dermatopathologist	Re-excision	87 (64.0%)	50 (86.2%)	139 (69.2%)
	No re-excision	49 (36.0%)	8 (13.8%)	62 (30.8%)
Actual treatment by dermatologist	Re-excision	42 (30.9%)	27 (46.6%)	71 (35.3%)
	No re-excision	83 (61.0%)	10 (17.2%)	96 (47.8%)
	Other	7 (5.1%)	21 (36.2%)	30 (14.9%)
	Lost to follow-up	4 (2.9%)	0	4 (2.0%)
Diagnostically challenging subset (n = 55)		Benign (n = 44)	Malignant (n = 11)	Total (n = 55)
Pretest treatment recommendation by dermatopathologist	Re-excision	41 (93.2%)	11 (100%)	52 (94.5%)
	No re-excision	3 (6.8%)	0	3 (5.5%)
Actual treatment by dermatologist	Re-excision	9 (20.5%)	6 (54.5%)	15 (27.3%)
	No re-excision	34 (77.3%)	3 (27.3%)	37 (67.3%)
	Other	0	2 (18.2%)	2 (3.6%)
	Lost to follow-up	1 (2.3%)	0	1 (1.8%)

^†Seven lesions with indeterminate test results are only included in total column. None of these lesions were in the diagnostically challenging subset.

Box 1.  List of genes included in the qRT-PCR Assay.

Gene component 1

• PRAME

Gene component 2

• S100A9

• S100A7

• S100A8

• S100A12

• PI3

Gene component 3

• CCL5

• CD38

• CXCL10

• CXCL9

• IRF1

• LCP2

• PTPRC

• SELL

Housekeeping genes

• CLTC

• MRFAPI

• PPP2CA

• PSMA1

• RPL13A

• RPL8

• RPS29

• SLC25A3

• TXNL1

Executive summary

• The change between treatment recommendations by dermatopathologists prior to testing with a gene-expression signature and actual treatment provided by the dermatologists was evaluated.

• A total of 315 cases were submitted by three dermatopathology practices, with 67.9% receiving a benign test result and 29.2% receiving a malignant test result.

• A total of 77 cases were diagnosed by dermatopathologists as indeterminate before testing with the gene-expression signature and were considered ‘diagnostically challenging’.

• Among diagnostically challenging lesions, there was a change from pre-test treatment recommendations to actual treatment in 71.4% of cases.

• The majority of treatment changes were downgrades to less invasive treatment (69.2%), compared with upgrades to more invasive treatment (30.8%).

• Among cases that received a change in management, 75.0% were changes that aligned with the gene-expression test result (i.e., downgrades in treatment in cases with a benign test result).

• For cases that received a benign test result, there was an 80.5% reduction in the number of excisions performed relative to pre-test recommendations.

• Overall, the data presented here show that the gene-expression signature impacted actual treatment of melanocytic lesions by dermatologists in a manner largely consistent with the test result.

Ethical conduct of research

The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.

Financial & competing interests disclosure

B Evans, K Brown, C Rock and L Clarke are employees of Myriad Genetic Laboratories and receive salaries and stock options as compensation. Patents relating to the content of this manuscript have been applied for my Myriad Genetic Laboratories, Inc. C Cockerell and J Tschen received consulting fees from Myriad Genetics. S Billings has no disclosures. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Additional information

Funding

B Evans, K Brown, C Rock and L Clarke are employees of Myriad Genetic Laboratories and receive salaries and stock options as compensation. Patents relating to the content of this manuscript have been applied for my Myriad Genetic Laboratories, Inc. C Cockerell and J Tschen received consulting fees from Myriad Genetics. S Billings has no disclosures. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.