Ophthalmology Optical Coherence Tomography Databases for Artificial Intelligence Algorithm: A Review

ABSTRACT

Background

Imaging plays a pivotal role in eye assessment. With the introduction of advanced machine learning and artificial intelligence (AI), the focus has shifted to imaging datasets in ophthalmology. While disparities and health inequalities hidden within data are well-documented, the ophthalmology field faces specific challenges to the creation and maintenance of datasets. Optical Coherence Tomography (OCT) is useful for the diagnosis and monitoring of retinal pathologies, making it valuable for AI applications. This review aims to identify and compare the landscape of publicly available optical coherence tomography databases for AI applications.

Methods

We conducted a literature review on OCT and AI articles with publicly accessible datasets, using PubMed, Scopus, and Web of Science databases. The review retrieved 183 articles, and after full-text analysis, 50 articles were included. From the included articles were identified 8 publicly available OCT datasets, focusing on patient demographics and clinical details for thorough assessment and comparison.

Results

The resulting datasets encompass 154,313 images collected from Spectralis, Cirrus HD, Topcon 3D, and Bioptigen devices. These datasets included normal exams, age-related macular degeneration, and diabetic maculopathy, among others. Comprehensive demographic information is available in one dataset and the USA is the most represented population.

Discussion

Current publicly available OCT databases for AI applications exhibit limitations, stemming from their non-representative nature and the lack of comprehensive demographic information. Limited datasets hamper research and equitable AI development. To promote equitable AI algorithmic development in ophthalmology, there is a need for the creation and dissemination of more representative datasets.

KEYWORDS:

• Artificial intelligence
• data
• datasets
• deep learning
• optical coherence tomography

BACKGROUND

Imaging techniques play a crucial role in the field of ophthalmology, as they enable healthcare professionals to visualize and assess the structures of the eye.¹ Traditional imaging methods, such as fundus photography and fundus autofluorescence, have been widely used to diagnose and monitor ocular conditions. However, with the advent of new technologies, such as Optical Coherence Tomography (OCT), novel opportunities have emerged in the field of ophthalmology. OCT enables healthcare professionals to visualize and assess ocular structures with high-resolution, non-invasive, and in vivo imaging, facilitating fast diagnosis and real-time monitoring of ocular conditions.²

Optical Coherence Tomography is used to visualize ocular structures in high resolution, through low-coherence interferometry.^2,3 It provides cross-sectional slabs of the retina, using light waves to create three-dimensional images of the eye’s internal tissues.² OCT scans are particularly useful in diagnosing and monitoring various eye conditions, including diabetic macular edema, glaucoma, and age-related macular degeneration, providing information about the thickness and integrity of the ocular structures.⁴ Additionally, OCT can be used to track changes in ocular diseases over time, assess the effectiveness of treatments, and guide surgical interventions.

The integration of artificial intelligence (AI) into ophthalmic imaging has opened up a landscape of exciting applications.¹ AI algorithms can analyze the vast amount of data generated by imaging modalities and assist ophthalmologists in patient care. One such area where AI shows great potential is in classifying and screening of ocular conditions.^5,6 Furthermore, AI algorithms can assist in screening programs where large populations need to be assessed for eye diseases. This allows for efficient triaging and ensures that at-risk patients receive timely intervention.⁷ In addition to classification and screening, AI can better our understanding of ocular conditions. By analyzing large datasets of imaging data, AI algorithms can identify subtle patterns and correlations that may not be apparent to the human eye.^8,9 This can lead to new insights into disease progression, treatment response, and personalized medicine approaches.

OCT has emerged as a valuable source of data for AI algorithms as numerous publications have explored its use as a data input. Researchers have subsequently developed AI models that can automatically segment different retinal layers, detect and quantify pathological changes, and predict disease progression.^10–12 These models employ deep learning techniques, such as convolutional neural networks, to extract features and make predictions based on OCT images with the goal of improving diagnostics, treatment planning, and patient outcomes in ophthalmology.

Datasets are pivotal for training and developing AI algorithms, aiding in pattern learning, prediction, and insight generation. However, often overlooked is the diversity and quality of said datasets as these are crucial for generalizable and fair algorithms. It is well-documented that much of research is performed in high-income countries, and it is minorities who bear the brunt of this burden.^13,14 Careful curation and balance in datasets are essential to avoid biased samples, and reduce AI biases that may perpetuate inequalities.

The field of ophthalmology faces specific challenges, including technical limitations, accessibility disparities, regulatory complexities, ethical considerations, and patient privacy issues. The need for standardized protocols and practices in ophthalmic imaging limits the interoperability and availability of data. In order to overcome these limitations, the FAIR (Findable, Accessible, Interoperable, and Reusable) principles should be applied in Optical Coherence Tomography (OCT) research.¹⁵ FAIR principles involve processes like standardizing metadata, improving data accessibility, fostering interoperability, and enhancing data reusability. The use of FAIR principles enables data sharing and the development of equitable and accurate algorithms for ophthalmic imaging.

This review aims to identify and compare the landscape of publicly available optical coherence tomography databases.

MATERIALS AND METHODS

This review focused on publicly available Optical Coherence Tomography datasets. The literature review search included PubMed, Scopus, and Web of Science databases, before 10 July 2023. The search query utilized variations of the keywords “optical coherence tomography”, “dataset,” and “publicly available”. Document types were limited to journal papers, conference papers, and data papers with no language, year, or citation exclusion. The exact search queries can be found in Supplemental File 1.

Papers were considered eligible if they met the following criteria: i) Papers were published in academic journals, conference proceedings, and reputable sources; ii) Studies that focus on open OCT datasets; iii) Studies that provide information on the availability, accessibility, and use of open OCT datasets.

Papers were excluded if they did not meet all of the following: i) Non-academic sources such as blog posts, opinion pieces, and news articles; ii) No report of an Open OCT dataset. Analysis of documents and the assessment process for eligibility was performed by three authors (DR, JQ, LFN).

After screening and review of the full text, the applied datasets used in each of the articles were retrieved from the final list of documents. Of these datasets, only the datasets classified by the authors as open were included.

For the purpose of this paper, an open dataset refers to a comprehensive and publicly accessible repository of data that is made available without any restrictions on access, financial barriers, or usage limitations. Such datasets are typically provided in a format that allows researchers to freely download, analyze, modify, and share the data for academic, scientific, and educational purposes. Thus, in order to be considered open, a dataset must meet the following requirements:

• The dataset must have a working link to the data. If the link is no longer working at the time of analysis, the dataset will also not be included in the review.

• There should be no monetary cost for the use of the dataset.

• The use of the dataset does not require extra permissions or requests via email to the authors, that is, it must be in an open repository.

For this review, the variables analyzed of the OCT datasets include the dataset name, device used for imaging, OCT technology employed, the number of images or eyes included in the dataset, demographic information of the subjects, proportions of different diseases present in the dataset, and the ratio of healthy and sick individuals, the count of abnormal slabs detected in the OCT images, the country of origin of the patients, and the labeling methodology employed in the dataset.

RESULTS

The search strategy yielded 183 articles, from which 76 duplicates were removed. From the remaining 107 articles, a primary screening based on the title and abstract assessment excluded 28 papers, and the subsequent full-text analysis excluded 29 papers. A total of 50 articles were used in the final analysis for dataset assessment. (Figure 1)

Figure 1. Flow diagram of open OCT data collection.

From the 50 included articles, a total of 15 datasets were retrieved, 8 publicly available included in this review. The final list of datasets resulting in this review can be seen in Table 1.

Table 1. Open OCT datasets details.

Name	Link	Device	Images	Diseases
Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification	https://data.mendeley.com/datasets/rscbjbr9sj/3	Spectralis	109309	Choroidal neovascularization, diabetic macular edema, drusen, normal
Duke Farsiu	https://people.duke.edu/~sf59/RPEDC_Ophth_2013_dataset.htm	Bioptigen OCT	38400	Intermediate AMD
Duke Fang	https://people.duke.edu/~sf59/Fang_TMI_2013.htm	Spectralis	763	AMD
Duke Chiu	http://www.duke.edu/~sf59/Chiu_BOE_2014_dataset.htm	Spectralis	610	DME
Quantitative analysis of optical coherence tomography for neovascular age-related macular degeneration using deep learning	https://datadryad.org/stash/dataset/doi:10.5061/dryad.2rbnzs7m4	Topcon 3D OCT-2000	2966	AMD
Data on OCT and Fundus Images.	https://data.mendeley.com/datasets/2rnnz5nz74/2	Topcon 3D OCT-1000	50	Glaucomatous
IACL - HCMS John Hopkins	https://iacl.ece.jhu.edu/index.php?title=Main_Page	Spectralis	1745	Multiple sclerosis
OCTID	https://borealisdata.ca/dataverse/OCTID	Cirrus HD OCT	470	Macular hole, AMD, diabetic retinopathy

OCT DEVICES

OCT exams offer detailed cross-sectional images of ocular tissue by reconstructing measurements of light reflection within ocular structures. These scans come in various modes, with the spectral domain and swept source technologies offering enhanced speed and resolution compared to the older time domain OCT systems.

While the capturing and underlying technology are similar, variation across different devices results in distinct images, which can compromise the generalizability of developed algorithms.

The datasets explored in this study encompass a range of OCT equipment and exams. Four datasets utilized exams conducted with the Spectralis, a high-resolution spectral domain OCT system manufactured by Heidelberg Engineering. Another dataset consisted of exams from the Cirrus HD-OCT (non-specified version), a spectral domain OCT. Additionally, two datasets featured exams from the Topcon 3D OCT-2000 and the Topcon 3D OCT-1000, both of which are spectral domain OCT devices. (Figure 2) Lastly, one dataset included exams conducted using the Bioptigen OCT (non-specified version), which is another spectral domain OCT system employed for ophthalmic imaging.

Figure 2. Comparative Samples of OCT Imaging Technologies: A. Cirrus HD OCT from OCTID, B. Topcon 3D OCT-1000 from data on OCT and fundus images, C. Spectralis OCT by Heidelberg Engineering from labeled OCT and chest X-Ray images.

Notably, all datasets included spectral domain OCTs, without any time-domain or swept-source OCT examinations. Additionally, the Topcon 3D–1000 and 2000 were the pioneering models of Topcon’s SD OCTs, released in 2006 and 2009, respectively. However, these models have since been replaced by newer technologies.

DEMOGRAPHICS

There is an increased concern about the potential biases in AI systems when developed utilizing non-representative data.¹⁶ To effectively address and mitigate such biases throughout the AI development process, it becomes essential to incorporate a comprehensive sociodemographic data.¹⁶

Sociodemographic information was not widely available throughout all eight datasets. Patient age information is available in three datasets,^17–19 gender in two datasets,^17,18 and race in one.¹⁸ Only one dataset provided all three.¹⁸

Artificial intelligence systems need to be trained and validated on-site and population-specific targets.²⁰ Of the reviewed datasets, five include patients from the USA, one from China, India, the UK, and Pakistan, showing a clear imbalance in geographic representation, as seen in Figure 3.

Figure 3. Distribution of patients in the OCT datasets per country. Map created adapting naturalearth_lowres’s layer using (c) 2013–2022, GeoPandas developers, an open-source python package created under the liberal terms of the BSD-3-Clause license.²¹.

EXAMS AND DISEASES

A total of 154,313 images were included, although specific counts for the number of eyes and patients remain undisclosed. The examination records encompass a variety of image compressed formats, including JPG, TIF, and MAT, with one dataset presenting OCT information in the form of OCT segmentation voxels, with no dataset in raw DICOM format available.¹⁸

Seven of the datasets are dedicated to retinal diseases and feature macular slabs, while one dataset focuses on glaucoma and includes optic disc-centered slabs. These images include both healthy individuals and those afflicted with a diverse range of conditions such as choroidal neovascularization, diabetic macular edema, drusen, age-related macular degeneration, retinal vascular occlusion, macular holes, and glaucoma.

LABELING

Labeling and annotation are crucial for the exam interpretation and encompass manual annotation of structures within the OCT scan, image segmentation, and classification.

In three of the datasets labeling was done via classifications,^19,22,23 two datasets used layer segmentation,^17,24 one used finding segmentation,¹⁸ one utilized both classifications and layer segmentation,²⁵ and one was without labels and considered undefined.²⁶

When detailing the classification datasets, there are variations in the level of granularity. In two datasets, the eye macular slabs are available, yet the classification itself is performed at the eye-level.^19,22 In contrast, the OCTID dataset offers image-level classification, but it comprises only one fovea-centered image per eye.²⁵ Lastly, the Data on OCT and Fundus Images provides image-level classifications along with the corresponding color fundus photographs.²³

All datasets with labels employed combinations of clinicians, ophthalmology residents, general ophthalmologists, and retina and glaucoma specialists for their labeling criteria. In one dataset, tiered labeling with adjudication is described in.²²

DISCUSSION

The performed review shines light on the current landscape of publicly-available OCT datasets. Of 183 papers yielded from the search strategy, only eight OCT datasets were found that met the requirements to be considered “open.” The OCT is widely implemented as an ancillary exam in ophthalmology to support retinal and glaucoma diagnosis and is considered valuable input for AI-automated systems. Nevertheless, the number of publicly available OCT datasets is limited. The lack of open data in such a widely applied test worldwide hinders and delays the advancement of research in this field.

These eight datasets have restricted geographical representation and limited demographic description. Latin America, Africa, and much of Europe and Southeast Asia are not represented in these datasets, highlighting massive gaps in research and knowledge generation that are multifactorial in nature. Many of these countries do not have the infrastructure to create datasets, educate personnel, and fund research, especially given the highly specialized nature of the equipment, and the countries with the necessary tools are not including the ones without. Additionally, variability in OCT scans is observed across different racial, gender, and age groups.^27,28 For example, in pediatric ophthalmology, in vivo OCT studies provide evidence of ongoing retinal development in early childhood, suggesting that early and timely treatment of certain ophthalmologic conditions may optimize visual function.²⁹ This highlights how demographic information, such as age, in OCT scan data can contribute to an improved utility of the dataset. Ensuring that our data is both representative and diverse is imperative to detect and mitigate potential biases in algorithms relying on OCT exams. Unfortunately, the reviewed datasets often lack comprehensive demographic information. Moving forward, initiatives should be taken to promote the creation and sharing of more diverse and representative OCT datasets.

OCTs exams are not reproducible across devices and scan technology,^30,31 and in clinical practice, it is not recommended for comparison between different devices. In our review, only five OCT devices are represented in OCT datasets, all using spectral domain technology. More diverse data from other devices and comparative studies are needed to assess the generalizability of AI algorithms.

Reliable labeling is a prerequisite for the development of AI algorithms. It is essential that datasets include clear documentation of the labeling protocol, criteria, and adjudication. Regrettably, the reviewed datasets do not provide adequate details regarding the labeling process.In addition, inter- and intra-observer variability should be taken into account to establish a robust dataset. Variations in interpretation and labeling between different clinicians or even the same clinician at different times can significantly impact the reliability of the dataset.

The ideal OCT dataset would encompass several critical criteria for its effectiveness. Firstly, it should be characterized by diversity, both in terms of demographics and devices. Demographic diversity ensures that the dataset represents a wide range of ages, genders, races, and ethnicities, making it applicable to a broader patient population. Similarly, including data from various OCT devices, with different specifications and technologies, enhances the dataset’s versatility and applicability across different clinical settings. Trustable labels are essential, ensuring that the dataset’s annotations and labels are accurate and reliable, often achieved through meticulous manual segmentation by qualified clinicians. A large number of images is crucial to support robust algorithm training and validation, allowing for more accurate and generalizable results. Moreover, the dataset should encompass multiple diseases, covering a spectrum of ocular conditions, to facilitate comprehensive research and diagnostic capabilities. Well-documented report characteristics, such as image formats and labeling methodologies, are vital for researchers to effectively utilize and interpret the dataset’s contents. Lastly, the datasets should also comply with the FAIR principles of findability, accessibility, interoperability, and reusability to allow reliability and usability. The structure of the ideal OCT dataset is suggested in Figure 4.

Figure 4. Key components of the ideal OCT dataset.

Addressing the limitations in ophthalmic datasets requires collaborative efforts among researchers, healthcare institutions, and regulatory bodies. Initiatives that encourage data sharing, establish standardized protocols for data collection and annotation and ensure patient privacy can help alleviate some of these challenges. Furthermore, efforts should be made to actively collect data from diverse patient populations to ensure fairness and equity in the development of AI algorithms for ophthalmology.

Firstly, ophthalmic imaging data tends to be complex and highly specialized. The acquisition of high-quality ophthalmic images requires specialized equipment and expertise, making it challenging to collect large-scale datasets. The availability of diverse datasets can be further hindered by factors such as privacy regulations, limited access to patient records, and data-sharing restrictions. Secondly, ophthalmic conditions may exhibit significant variability within and between individuals. Factors like anatomical differences, disease severity, and treatment responses can contribute to variations in the data. This variability can make it challenging to develop algorithms that generalize well across different patients and ophthalmic conditions.

Furthermore, certain ocular conditions are relatively rare, which can limit the availability of large datasets specifically dedicated to those conditions. This scarcity of data can impede the development and training of AI models for these specific conditions, hindering progress in diagnosis, treatment, and research.

CONCLUSION

In conclusion, this review highlights the critical role of Optical Coherence Tomography in ophthalmology and its integration with artificial intelligence (AI) for enhanced diagnosis and patient care. OCT provides detailed insights into ocular structures, making it a valuable source of data for AI algorithms. AI shows great promise in classifying and screening ocular conditions, contributing to a better understanding of these conditions, and improving diagnostics and treatment planning.

However, the review also underscores several challenges in the field, particularly regarding the characteristics and availability of datasets. The limited diversity, geographical representation, and demographic description reveal a significant gap in research and knowledge generation, especially in regions like Latin America, Europe, and Africa. These challenges, along with the complex nature of ophthalmic imaging data, emphasize the need for collaborative efforts to promote data sharing, standardize data collection and annotation, and ensure patient privacy.

Addressing these limitations and actively collecting diverse datasets will be essential to advance the development of AI algorithms for ophthalmology. Despite these challenges, the integration of AI with OCT imaging holds great promise for enhancing the accuracy, efficiency, and accessibility of diagnosing and managing ocular conditions. As technology and research in this field continue to evolve, the synergy between OCT and AI will likely play a pivotal role in improving patient outcomes and expanding our understanding of eye diseases on a global scale.

While the use of OCT technology in AI is promising, the field is still lacking in terms of diversity and research in validation and identification of bias in OCT datasets. Further studies are needed to address the assets of OCT datasets in terms of enhancing diversity and mitigating bias, thereby ensuring more comprehensive and equitable advancements in the field of AI applied to ophthalmology imaging.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/08820538.2024.2308248.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.