Measurement bias between optical coherence tomography instruments can affect access to treatment: a new lottery

Dear editor

It is with great interest that we read the publication entitled “Critical appraisal of ranibizumab in the treatment of diabetic macular edema” by Stewart.¹ The author emphasized the importance of the vascular endothelial growth factor (VEGF) in the pathophysiology of diabetic macular edema (DME). As highlighted in that article, the anti-VEGF ranibizumab is a superior treatment compared to traditional argon photocoagulation, leading to better anatomical and functional results.

In April 2013, the National Institute for Health and Care Excellence (NICE) of the UK approved the use of ranibizumab as a treatment option to treat diabetic macular edema of the eye if it has a central macular thickness (CMT) of 400 μm or more at the beginning of the treatment.² The guidelines did not specify which optical coherence tomography (OCT) device(s) should be used for this assessment. This is important as, although good consistency has been shown in using the same instrument, there is a known divergence in CMT measurements between different instruments.^3–6 For example, the Spectralis® OCT (Heidelberg Engineering; Carsbad, CA, USA) generally shows higher values of mean CMT in a normal eye compared to most other instruments, in part due to the retinal segmentation algorithm that it employs.⁴ We hypothesized that similar (or increased) differences might be observed in DME, and that for those countries (such as the UK) where a fixed CMT is used to define eligibility for treatment, the “lottery” of OCT instruments may influence eligibility.

In light of this hypothesis, we conducted a preliminary analysis of 24 patients (48 eyes) with suspected DME who had OCT scans performed on the same day using both 3D OCT-1000 (Topcon; Itabashi, Tokyo, Japan) and Spectralis OCT. Matched macular-centered scans were obtained in 42 eyes; scans were not possible in 6 eyes due to media opacity or problems with patient fixation. The mean (standard deviation) CMT in this cohort was 282.0 (89.0) μm with a range of 191–689 μm using the Topcon OCT, and 312.4 (88.8) μm with a range of 224–719 μm using the Spectralis OCT (Figure 1A). Comparing the two instruments in our cohort using a Bland–Altman analysis, there was a bias of +10.73 μm to the Spectralis with a standard deviation of 10.32, and 95% limits of agreement of −9.497 to 30.96 μm (Figure 1B).

Figure 1 Comparison of CMT measurements acquired on Topcon 3D OCT-1000 versus Spectralis® OCT for patients with DME. Direct comparison (A) and Bland– Altman plot (B).

Notes: Spectralis® OCT manufactured by Heidelberg Engineering (Carsbad, CA, USA); 3D OCT-1000 manufactured by Topcon (Itabashi, Tokyo, Japan).
Abbreviations: CMT, central macular thickness; DME, diabetic macular edema; OCT, optical coherence tomography.

Recognizing this issue is important for all those involved in care of patients in countries or institutions where the entry to treatment is limited by a defined CMT level. In the specific example considered here, this finding has a direct clinical impact on patients who have DME with a central macular thickness of 390–410 μm. In our small cohort of matched scans from 42 eyes, there were three whose CMT was >400 μm on the Topcon and four whose CMT was >400 μm on the Spectralis: ie, even in this small study, a patient’s eligibility for treatment depended on which scan was used. “Real-world” studies of OCT will become increasingly important if defined CMT levels are to be used as the “gate-keeper” for treatment, and should include repeatability and inter-instrument variability in defined patient cohorts.

Disclosure

Dr Keane is funded by the Department of Health’s NIHR Biomedical Research Centre for Ophthalmology at Moorfields Eye Hospital and UCL Institute of Ophthalmology. The views expressed in the publication are those of the author and not necessarily those of the Department of Health. None of the authors have any other conflicts of interest to declare.

Author’s replyCorrespondence: Michael W Stewart, Department of Ophthalmology, Mayo Clinic School of Medicine, 4500 San Pablo Rd, Jacksonville, FL 32224, USA, Email [email protected]

Dear editor

We thank Kidess et al for their insightful comments regarding the use of ranibizumab for the treatment of diabetic macular edema (DME). Ranibizumab received US Food and Drug Administration approval for the treatment of DME based primarily on the results of the parallel Phase III registration trials RISE (A Study of Ranibizumab Injection in Subjects With Clinically Significant Macular Edema With Center Involvement Secondary to Diabetes Mellitus) and RIDE (A Study of Ranibizumab Injection in Subjects With Clinically Significant Macular Edema With Center Involvement Secondary to Diabetes Mellitus)¹ and, not surprisingly, reimbursement for this expensive vascular endothelial growth factor (VEGF)-blocking drug is partially based on findings from these trials. For patients with DME, the Centers for Medicare and Medicaid Services (CMS) reimburse for intravitreal ranibizumab regardless of macular thickness, but the National Institute for Health and Care Excellence (NICE) of the UK requires a central macular thickness of at least 400 μm, without stipulating which optical coherence tomography (OCT) machine is used.² By demonstrating that macular thickness measurements depend upon the spectral domain-OCT (SD-OCT) model used, Kidess et al have uncovered a flaw in the treatment guidelines that may arbitrarily restrict access to treatment for patients in the UK and any other country that ties treatment eligibility to macular thickness.

Patients in the RISE and RIDE trials had central macular thickness (CMT) measurements of at least 275 μm by time-domain OCT (TD-OCT),¹ and average CMT of all enrolled patients was 466 μm (with an approximate standard deviation of 158). Previously published studies show that SD-OCT machines consistently give CMT measurements that are 36–74 μm greater (depending on the model of SD-OCT) than those from TD-OCT,^3–5 thereby suggesting that the mean CMT in RISE and RIDE would have ranged from 502 μm (Spectral®; Opko/OTI, Inc, Miami, FL, USA) to 540 μm (Spectralis®; Heidelberg Engineering, Carsbad, CA, USA) if measured with SD-OCT machines. For eyes with CMT near 400 μm, the choice of SD-OCT model is critical to determining eligibility to receive ranibizumab. For example, a CMT measurement of 418 μm with the Spectralis (eligible) would be only 380 μm with the Spectral (ineligible). If we assume that the CMTs in the RISE and RIDE trials followed a normal distribution and are representative of UK patients with DME, then 1.75%–7.5% of eyes could not receive ranibizumab simply because the “wrong” SD-OCT model had been selected.

The ranibizumab eligibility limit has created a second concern that might adversely affect even more patients with DME. Patients were eligible for RISE and RIDE if their CMT was at least 275 μm by TD-OCT, which would equate to 311–349 μm by SD-OCT. Had the RISE and RIDE population been imaged with SD-OCT, according to NICE criteria, 15.8%–25.9% would not have been eligible to receive ranibizumab. If this population of DME patients was imaged with TD-OCT machines, which are still fully functional in many physicians’ offices, the size of the ineligible cohort increases to 33.8%.

What are the consequences of not administering ranibizumab to patients whose macular thicknesses measure less than 400 μm? In this setting, treatment options include observation, laser photocoagulation, intraocular corticosteroids, or bevacizumab. Level II evidence suggests that the off-label use of intravitreal bevacizumab effectively resolves edema and improves visual acuity,⁶ but level I evidence will not be available until completion of the Diabetic Retinopathy Clinical Research Network (DRCR.net) protocol T.⁷ Laser photocoagulation and triamcinolone do not dramatically improve vision,⁸ and delaying the institution of effective anti-VEGF therapy results in months to years of “lost” vision, but the long term consequences have not been determined.⁹

Although we believe that a group of patients with DME will be undertreated because of the 400 μm exclusion limit, the theoretical calculations in this letter have not been tested in clinical trials. We commend Kidess et al for drawing attention to treatment eligibility based upon imaging parameters and encourage them to continue their “real life” studies to help us better understand the consequences of exclusionary policies.

Disclosure

Michael W Stewart is on the advisory boards for Allergan and Regeneron, and is a consultant to Boehringer-Ingelheim.