https://orcid.org/0000-0002-1391-5328
ABSTRACT
To evaluate the effectiveness of smartphone ophthalmoscope (SO) in teaching ophthalmoscopy, compared with direct ophthalmoscope (DO). In this cross-over study, 45 final-year medical students attending sessions at a single institution were randomly allocated to two groups (A and B). Both groups attended two training sessions. In the first session, Group A students were taught ophthalmoscopy using DO and Group B students using SO. In the second session, the training sessions were crossed over. A series of eye models with 10 letters placed on the inner surface were designed to assess the students’ skill on ophthalmoscopy. Students performed ophthalmoscopy on the eye models, recorded their findings, and completed a questionnaire of feedback on DO and SO. The main outcome measure was the score of ophthalmoscopy, assessed by the student correctly recording each letter (score 1 for each letter). For Group A, the mean score of ophthalmoscopy on the eye model using DO and SO was 3.9±2.4 and 8.2±2.2, respectively. For Group B, the mean score of ophthalmoscopy on the eye model using SO and DO 8.7±1.8 and 5.7±3.5 . Students scored significantly higher in ophthalmoscopy when using SO than DO (P<0.001). They expressed better visualization of the fundus using SO than DO (4.49±0.65 vs 4.13±0.81, P=0.004). Students’ performance of ophthalmoscopy was better when SO was used compared with DO. The use of SO as an adjunctive tool is recommended to improve the effectiveness of teaching ophthalmoscopy.
Introduction
Eye health and vision have profound implications for health and quality of life. Visually related complaints are common in primary care and emergency room settings [Citation1,Citation2]. Therefore, it is essential to teach essential ophthalmological skills to medical students to prepare them for frontline clinical service.
The retina is vital for vision and a common pathologic site in many blinding eye diseases. It is the only ocular structure where nerves and vessels can be observed non-invasively in vivo. Therefore, visualizing the fundus plays an important role to evaluate patients for ocular diseases and even systemic diseases. It is essential and fundamental in clinical practice and medical education. Direct ophthalmoscopy (DO) is a basic investigative skill not only for ophthalmologists but also for general practitioners and other clinicians at frontline [Citation3]. Traditionally, advantages of DO included a magnified view of the posterior pole, relatively low cost, wide availability, and portability. Its practicality for fundus examination in critical care patients, as well as its potential usefulness in the examination of low-amplitude nystagmus and subtle abnormalities of visual fixation, is also of value [Citation4]. However, mastering DO can be technically challenging. Firstly, due to the short working distance and the limited field of view of the conventional DO, many medical students do not feel confident or competent [Citation5,Citation6]. Secondly, the single eyepiece allows only one examiner to view the image at a time and is inconvenient for teachers to demonstrate the technique, supervise the students, and assess their competency of ophthalmoscopy [Citation7]. Given these factors, teaching direct ophthalmoscopy to medical students has been recognized as difficult. Abolition of direct ophthalmoscopy in the curriculum for medical students has been proposed [Citation8].
The fundus camera can be used to teach visualization of the retina [Citation6]. However, it is a large and expensive equipment not available in many clinics. In recent years, several modes of smartphone ophthalmoscope (SO) have been developed and used for diabetic retinopathy and glaucoma screening [Citation9,Citation10]. Results based on answers in questionnaires have shown that SO could improve the quality and accuracy of ophthalmoscopy and was preferred by medical students, compared with DO [Citation1,Citation7,Citation10]. However, there are still few studies on the usefulness of SO, based on objective assessment tools.
Recently, we have developed a series of eye models for objective assessment of ophthalmoscopic competency and demonstrated the effectiveness of conducting the fundus examination. The objective of this crossover prospective randomized controlled trial was to investigate the utility and effectiveness of SO in teaching ophthalmoscopy to a cohort of final-year medical students, as compared with DO.
Materials and methods
Setting and participants
This prospective randomized study was conducted at the Joint Shantou International Eye Center (JSIEC) of Shantou University and the Chinese University of Hong Kong. This study has obtained ethics approval from the Institutional Review Board of JSIEC, and all participants provided written informed consent. The research adhered to the tenets of the Declaration of Helsinki, and was compliant with clinical trials registration from Chinese Clinical Trial Registry (ChiCTR2100054018).
Final-year medical students, without prior experience of ophthalmoscopy, neither DO nor SO, were consecutively recruited to participate in this crossover study during the academic year 2020–2021. The students had attended a series of lectures on fundamentals of ophthalmology, including anatomy of the eyeball, etiology of major eye diseases, and pathophysiology of retinal diseases. One investigator (X.L.) generated the random sequence, and then equally allocated the participants into two groups (A and B) according to the random number.
Devices
We designed and assembled a prototype of SO. A light-emitting diode (LED), with the maximum power of 0.1-watt, was embedded on an aluminum plate. A fixed resistor (500 Ohm) was used to limit the circuit, and a rotary variable resistor (0 to 10 thousand Ohm) to adjust the LED’s brightness. They were connected to the USB port of the smartphone, which supplied a voltage of 5-volt through the corresponding connector (Type-C, Micro and Lightning, shown in ). All these materials were purchased online (www.taobao.com). The aluminum plate was attached to the student’s smartphone using tape. The LED was placed close to the camera to offer an almost co-axial illumination (). A camera app (Ullman Indirect) with a manual focus was installed in the smartphone (). Various modes of smartphone were used, including iPhone X, Huawei, Samsung, Sony. Conventional DO (Welch Allyn 3.5 V Coaxial ophthalmoscope) was used as a control.
Figure 1. Smartphone ophthalmoscope.
The eye models were manufactured for objective assessment of the students’ skill of ophthalmoscopy as previously described [Citation11]. Briefly, a 26-mm-diameter double-hemispherical brown plastic ball was used to simulate the eyeball; a 6-mm circular opening was drilled on one hemisphere as a pupil, behind which a convex lens was glued to provide the refractive component. Ten pieces of randomized capital letters were placed on the inner surface of the other hemisphere. A total of 4 homogenized eye models with different letters were made to ensure randomization and security.
Procedure and data collection
The study design is shown in . Both groups attended two training sessions on ophthalmoscopy, 45 min for each session. In the first session, Group A participants were taught with DO, while Group B was taught using SO. In the second session, the training was crossed over. Each training involved a 10-min didactic lecture on the working principle of the instrument, a 5-min demonstration and 30-min practice on standardized patients with and without mydriasis under the supervision of a trained ophthalmologist (HXW). , along with additional videos (Appendix 1 and Appendix 2 in Appendices), demonstrated the practice with SO.
Figure 2. Study design.
Figure 3. Trainings of ophthalmoscopy using smartphone ophthalmoscopes.
After training, the students were asked to use the corresponding instrument to view the fundus of one randomly selected eye model (), read the letters and record their findings in a test paper within 10 min. The main outcome measure was the score of ophthalmoscopy, assessed by the student correctly recording each letter (score 1 for each letter). In addition, computer-assisting randomization was conducted to prevent students examining the same eye model in two sessions.
Figure 4. Students using ophthalmoscopes to visualize the fundus of the eye model.
Finally, students were asked to complete a post-training questionnaire (Appendix 3 in Appendices). The questionnaire was designed in a pattern of Likert scale and included feedback on understanding the working principles and confidence of handling the technique with subjective ratings of 1 (‘disagree’) to 5 (‘agree’), which was the secondary outcome measures.
Statistical analyses
All of the test results and questionnaires were processed anonymously. The investigators dealing with data collection and statistics were blind to the participants’ information and the grouping. Statistical analysis was undertaken using SPSS software (Version 23.0).
The required sample size was determined as not less than 20 participants according to the preliminary trial (α = 0.05, β = 0.1, q1=q2 = 0.5, allowable difference was set as 2.5 according to the average individual absolute error of DO score, mean difference score of DO and SO: 3.3, combined standard deviation 1.70).
Descriptive statistics were conducted to analyze the objective scores and the subjective ratings. Center values and error bars were presented as mean and standard deviation (SD). The competency scores of DO and SO were compared using analysis of variance (ANOVA) for crossover design, conducted in SPSS software. Even there is difference in variance across groups (P < 0.001 for Levene’s test), ANOVA for cross-over design is basic on the univariate general linear model which is able to tolerate to low homogeneity of variance. Besides, when analyzing the standardized residuals of the scores from all the participants, there was only two residuals with an absolute value bigger than 2, suggesting that these variables are suitable to undergo ANOVA. This analysis decomposes the overall variance into three parts: the intervention effect (the natural difference between DO and SO), the period effect (the variance caused by different time periods, the average scores of all participants after the 1st session vs that after the 2nd session), and the sequence effect (the variance caused by different sequences, also called group effect, score of Group A vs score of Group B). The intervention and the period were set as fixed effect, while the sequence is set as random effect. Inter-model analysis was conducted by one-way ANOVA to compare the scores of different eye models and verify their homogeneity. The subjective ratings, as discontinuous variable in abnormal distribution, were compared using two-sided Wilcoxon signed-rank test. A P-value of<0.05 was considered to be significant.
Results
A total of 45 final-year medical students were enrolled in the study and were randomly divided into Group A (n = 24) and Group B (n = 21). All participants received intended intervention and were analyzed, without losses and exclusions after randomization. There was no significant difference in age and gender between the groups (). All students were ethnic Han.
Table 1. Baseline characteristics of the study population. Data presented as means ± SD or numbers.
The ophthalmoscopic skill was objectively assessed by the student correctly recording each letter. For Group A, the mean score of ophthalmoscopy using DO and SO was 3.9 ± 2.4 and 8.2 ± 2.2, respectively. For Group B, the mean score of ophthalmoscopy using SO and DO was 8.7 ± 1.8 and 5.7 ± 3.5, respectively. The ANOVA for crossover design () indicated that students attained significantly higher scores when using SO than DO (F = 46.918, P < 0.001). Besides, analysis on sequence effect revealed Group B, firstly trained with SO and then DO, scored significantly higher on the whole than Group A (F = 4.766, P = 0.032). However, analysis on period effect revealed no significant difference when comparing the score in viewing the eye model after the first session and the second session of training (F = 1.348, P = 0.249). Inter-model analysis showed no significant difference in the scores of different eye models, either using DO (P = 0.731) or SO (P = 0.598).
Table 2. Objective assessment of ophthalmoscopy.
The subjective ratings of the students on these two ophthalmoscopes are shown in . Students found they were abler to follow the teacher’s description about the working principles of SO than DO during didactic lectures (4.84 ± 0.36 vs 4.69 ± 0.55, P = 0.020). The students felt it easier to view the fundus using SO than DO (4.49 ± 0.65 vs 4.13 ± 0.81, P = 0.004). As for the confidence in viewing the fundus, the students reported ratings of 4.58 ± 0.71 for SO, slightly higher than the ratings of 4.42 ± 0.71 for DO, although the difference was not statistically significant (P = 0.138). Moreover, students felt more confident to teach fellow students how to use SO than DO (4.71 ± 0.54 vs 4.38 ± 0.82, P = 0.001). When asked to project future popularization of the two ophthalmoscopes, the students thought that SO could provide more help for non-ophthalmologists than DO (4.89 ± 0.31 vs 4.69 ± 0.59, P = 0.021) and expected more popularization of SO than DO among non-ophthalmologists (4.84 ± 0.42 vs 4.51 ± 0.72, P = 0.001). Besides, DO was regarded to have an uncomfortable working distance when compared with SO (4.64 ± 0.52 vs 2.80 ± 0.93, P < 0.001).
Figure 5. Subjective assessment of ophthalmoscopy using DO and SO .
Discussion
We have conducted a prospective crossover randomized controlled trial to compare SO with DO in teaching final-year medical students to view the fundus. All students could equip the SO using their own models of smartphone, and were capable to view the fundus. Objective comparison between SO and DO revealed that students’ performance of ophthalmoscopy was better overall when SO was used over DO.
There are reasons for the superiority of SO over DO. Smartphones are attractive to users in the convenience and effectiveness of image capture, screen display, and real-time sharing. Accordingly, SO is highly applicable in clinical practice and medical education, especially in ophthalmology, in which fine morphological features play an important role in diagnosis [Citation9]. DO has one single inspection window and more difficult working distance in comparison with the SO. With excellent screen display and comfortable doctor-patient distance, medical students could easily keep the device and image stable. Also, the long doctor-patient distance avoids closed contact with the patients, especially in the era of COVID-19. These advantages explained why medical students preferred the SO over the DO. The students also thought that using SO could provide more help for non-ophthalmologists and expected popularization of SO in clinical service.
There are two different types of SO. One is based on the principle of direct ophthalmoscope [Citation12], and the other on the principle of indirect ophthalmoscope [Citation13]. Although the smartphone indirect ophthalmoscope has the advantage of a wide field of view, it requires a hand-held condensing lens and fine bimanual coordination. Also, the images are inverted. Therefore, it needs more time of practice and is not as easy as the smartphone direct ophthalmoscope to teach medical students.
Smartphone direct ophthalmoscope follows the optical principles of conventional direct ophthalmoscope. The almost co-axial illumination by the LED close to the camera provides more coincident retinal area of the illumination system and the observation system. However, completely co-axial design not only results in optical path occlusion by the LED, but also produces an apparent corneal light reflex. In this study, thanks to the incompletely co-axial design and the small surface area of the LED, the small corneal reflex was not in the central field of view, which was acceptable in practice. On the other hand, the illumination system of conventional DO uses a planoconvex lens to change the vergence of lamplight to project a beam of parallel light, and the compensating lens only deals with the uncorrected refractive error from the doctor and patient. However, the LED illumination of SO is divergent. This could be resolved with manual focus of the camera app, which providing continuous and variable focuses to fit the vergence of projecting light and the uncorrected refractive error of the patient, ensuring that fundus images can be captured easily with any refractive status. According to the image-forming principle, the SO provides an erect magnifying image, with a field of view of 20° for a dilated pupil [Citation7].
Several studies have investigated the utility of SO in medical education [Citation1,Citation7,Citation10,Citation14–16]. Uses of subjective questionnaire gave indications on the competency of the medical students in visualizing the fundus. In the current study, we used a verified eye model to objectively investigate competency as the major outcome. Besides, as reported, D-EYE, one of the most frequently used SO [Citation1,Citation7,Citation10,Citation14,Citation15], works specifically with iPhone 5s/SE/6/6s/6Plus/6s/7/8, but not other smartphone models. The new SO designed in the current study could work with any model of smartphone, including iPhone X, Huawei, Samsung, Sony, which also suggested that the SO with high compatibility in our study has great prospects of popularization and application. And the smartphone app was available for both Android and iPhone. The students used their own smartphones to equip the SO, install the app, and perform the ophthalmoscopy, followed by viewing, capturing, and storing the fundus images on their own phones. This do-it-yourself approach aroused great interest and enthusiasm in the students.
On the period effect in crossover design, the overall score after the first training session (6.1 in average) was lower than that after the second session of training (7.0 in average). The difference, though not statistically significant, could be due to cumulative learning effects. Actually, in the setting of a half-day clinical skills training course, there was limited time for students to keep practicing until they mastered the skill, so the difference between these two periods was not obvious in this study.
Notably, when analyzing the sequence effect, we found that the students beginning with the SO (Group B) had a significantly higher score compared with those beginning with the DO (Group A). Given the randomization grouping design and the matched demographic information, this finding indicated that using the SO first may improve the student’s ability of ophthalmoscopy overall, suggesting that the SO may be more helpful to understand the fundus examination and may serve as an adjunctive tool to help teach direct ophthalmoscopy. Even so, in further study, the baseline information about the skill of ophthalmoscopy should be more perfectly matched.
We recognize some limitations in the current study. Firstly, the consistency of being able to visualize the numbers inside the eye model and being able to visualize the fundal signs has not been verified in this study. However, identification of pathologic signs is technically challenging and not suitable for medical students with limited clinical experience. We believe that successful focusing and scanning the retinal area is the fundamental skill of ophthalmoscopy and provides basis of sign identification and disease diagnosis. Also, this was a short-term study based on a single skills session in accordance with the established teaching curriculum. Further longitudinal studies are needed to determine whether the SO can improve medical students’ competency of ophthalmoscopy, especially for human eye, in future careers. In addition, the questionnaire used in the current study has not been previously validated. There may be biases, such as recall bias. We had tried to minimize biases by the crossover study design, randomization of participants and anonymity in data analysis. Last but not least, the position of LED was not standardized because it was fixed with a tape. A better design with a fixed location of the LED should help to improve the performance of the SO. In further study, we should also conduct comparison between our homemade SO with other commercially available SO.
In conclusion, we utilized crossover design, prospective randomization, and the use of both objective and subjective measures to compare students’ use of SO and DO. Students were abler to view the fundus of eye models using the SO, and expected more popularization and application of SO. Further research is needed to assess SO in various clinical and educational settings.
Authors’ contributions
HXW and HYC designed the study. HXW and XLL assembled the devices, conducted the study, and collected the data. HXW and HYC analyzed and interpreted the data. HXW drafted the manuscript. MZZ and CP Pang supervised the study. HYC and CP Pang revised the manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
This study has been approved by the Ethics Committee of Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong. A written informed consent to participate was obtained from the participants.
Consent for publication
Written informed consents were obtained from the participants for publication. A copy of the consent is available for review by the Editor of this journal.
Acknowledgments
We would like to express our deepest gratitude to the study participant. This study was supported by Danny Ng, Mudan Lin, Jingsheng Yi, Fang Deng, Baoyi Tan, Yun Chen, Xiangling Yuan.
Disclosure statement
No potential conflict of interest was reported by the authors.
Data availability statement
All data analyzed or generated during the study are available from the corresponding author on reasonable request.
Additional information
Funding
References
- Kim Y, Chao DL. Comparison of smartphone ophthalmoscopy vs conventional direct ophthalmoscopy as a teaching tool for medical students: the COSMOS study. Clin Ophthalmol. 2019;13:391–9.
- Burton MJ, Ramke J, Marques AP, et al. The lancet global health commission on global eye health: vision beyond 2020. Lancet Glob Health. 2021;9(4):e489–551. DOI:10.1016/S2214-109X(20)30488-5
- Bradley P. A simple eye model to objectively assess ophthalmoscopic skills of medical students. Med Educ. 1999;33(8):592–595.
- Mackay D, Garza P, Bruce B, et al. The demise of direct ophthalmoscopy: a modern clinical challenge. Neurol Clin Pract. 2015;5(2):150–157.
- Levy A, Churchill A. Training and testing competence in direct ophthalmoscopy. Med Educ. 2003;37(5):483–484.
- Chen M, Swinney C, Chen M, et al. Comparing the utility of the non-mydriatic fundus camera to the direct ophthalmoscope for medical education. Hawaii J Med Public Health. 2015;74(3):93–95.
- Nagra M, Huntjens B. Smartphone ophthalmoscopy: patient and student practitioner perceptions. J Med Syst. 2019;44(1):10.
- Hogarty DT, Hogarty JP, Hewitt AW. Smartphone use in ophthalmology: what is their place in clinical practice? Surv Ophthalmol. 2020;65(2):250–262.
- Wintergerst MWM, Jansen LG, Holz FG, et al. Smartphone-based fundus imaging-where are we now? Asia Pac J Ophthalmol (Phila). 2020;9(4):308–314.
- Mamtora S, Sandinha MT, Ajith A, et al. Smart phone ophthalmoscopy: a potential replacement for the direct ophthalmoscope. Eye (Lond). 2018;32(11):1766–1771.
- Wang H, Liao X, Zhang M, et al. A simple eye model for objectively assessing the competency of direct ophthalmoscopy. Eye (Lond). 2022;36(9):1789–1794.
- Shanmugam MP, Mishra DK, Madhukumar R, et al. Fundus imaging with a mobile phone: a review of techniques. Indian J Ophthalmol. 2014;62(9):960–962.
- Haddock LJ, Kim DY, Mukai S. Simple, inexpensive technique for high-quality smartphone fundus photography in human and animal eyes. J Ophthalmol. 2013;2013:518479.
- Wu AR, Fouzdar-Jain S, Suh DW. Comparison study of funduscopic examination using a smartphone-based digital ophthalmoscope and the direct ophthalmoscope. J Pediatr Ophthalmol Strabismus. 2018;55(3):201–206.
- Hakimi AA, Lalehzarian SP, Lalehzarian AS, et al. The utility of a smartphone-enabled ophthalmoscope in pre-clinical fundoscopy training. Acta Ophthalmol. 2019;97(2):e327–328.
- Dunn HP, Kang CJ, Marks S, et al. Perceived usefulness and ease of use of fundoscopy by medical students: a randomised crossover trial of six technologies (eFOCUS 1). BMC Med Educ. 2021;21(1):41.
Appendices
avi. Ophthalmoscopy using the smartphone ophthalmoscope on standardized patients with an undilated pupil. Written informed consent has been obtained for the video to be published
Appendix 2.
avi. Ophthalmoscopy using the smartphone ophthalmoscope on standardized patients with a dilated pupil. Written informed consent has been obtained for the video to be published
Appendix 3.
Questionnaire of feedback on the two ophthalmoscopes
Participant No. : ______________________
Subjectively rates your agreement on the following statements about the direct ophthalmoscope and the smartphone ophthalmoscope, on a scale of 1 (“disagree”) to 5 (“agree”).