Visual Acuity Screening in North Indian Schools: Testing Accuracy and Cost of Alternate Screening Models

ABSTRACT

Purpose

Our study compares the sensitivity, specificity and cost of visual acuity screening as performed by all class teachers (ACTs), selected teachers (STs) and vision technicians (VTs) in north Indian schools.

Methods

Prospective cluster randomized control studies are conducted in schools in a rural block and an urban-slum of north India. Consenting schools, with a minimum of 800 students aged 6 to 17 years, within a defined study region in both locations, were randomised into three arms: ACTs, STs or VTs. Teachers were trained to test visual acuity. Reduced vision was defined as unable to read equivalent of 20/30. Optometrists, who were masked to results of initial screening, examined all children. Costs were measured for all three arms.

Results

The number of students screened were 3410 in 9 ACT schools, 2999 in 9 ST schools and 3071 in 11 VT schools. Vision deficit was found in 214 (6.3%), 349 (11.6%) and 207 (6.7%), (p < .001) children in the ACT, ST and VT arms, respectively. The positive predictive value of VT screening for vision deficit (81.2%) was significantly higher than that of ACTs (42.5%) and STs (30.1%), (p < .001). VTs had significantly higher sensitivity of 93.3% and specificity of 98.7%, compared to ACTs (36.0% and 96.1%) and STs (44.3% and 91.2%). The cost of screening children with actual visual deficit by ACTs, STs and VTs, was found to be $9.35, $5.79 and $2.82 per child, respectively.

Conclusion

Greater accuracy and lower cost favours school visual acuity screening by visual technicians in this setting, when they are available.

KEYWORDS:

• North India
• randomized control trial
• school eye screening
• school teachers
• vision technicians

Introduction

Globally, the prevalence of myopia, hyperopia and astigmatism in paediatric populations has been reported to be 11.7%, 4.6% and 14.9%, respectively.¹ Thus, actively screening school age children remains a high priority. Literature reports effectivity and efficiency of school eye screening (SES) in the form of increased coverage, and decreased dependence on eye care workers.² Schools provide an environment where large number of children can be screened, with a minimum outlay.³

The SES model itself has evolved over time. While the basic model effectively utilizes selected school teachers (STs) to screen for RE and other ocular defects,^4,5 a new model highlights the efficiency of utilizing class teachers as screeners.⁶ This “All Class Teacher” (ACT) model screens children at a third of the cost of the earlier model,⁶ where the cost calculated included the training cost of the screeners and the actual screening cost (manpower cost, equipment cost, travel cost and food cost).⁶ The STs are selected by the school principals while the ACTs belonged to the classes from which children were being screened.

Our organization applies a task-shifting model⁷ and trains students who have just completed secondary school through a two-year accredited program to be professional vision technicians (VTs)- trained to take ocular measurements, carrying out refraction, recognizing ocular pathologies and referring cases needing further management.

In SES, all children with a reduced visual acuity (VA) were referred to an ophthalmic team for refraction. Our study compares sensitivity, specificity, and costs of school VA screening programs across three models of SES, in a north Indian setting. Each model is based on screening conducted by different primary screeners, i.e., ACTs, STs, and VTs. There is no previous study comparing these three models for primary eye screening in schools.

Methods

Study design and period

This is a prospective cluster randomized controlled study, conducted in North India from 2017 to 2018, comparing three different models of SES.

Study setting

Two regions of North India, one in an urban-slum and other in a rural block, where SES was actively being conducted by our organisation, were selected for the study. In the urban-slums, all schools within a five kilometre distance of our vision centre (VC), and in rural area, all schools in the administrative block, were enlisted.

Sample size calculation

To estimate sensitivity and specificity of the diagnosis made by screeners, with a 95% confidence interval of ± 0.10, the minimum required sample sizes were 97 true positive and 97 true negative cases. Given the prevalence of refractive error (RE) of around 7% from our previous experience in similar regions, a minimum of 1386 ( = 97/0.07) students were to be screened. We decided to screen a minimum of around 1500 students in each category at each location to accommodate for a lower prevalence rate.

Inclusion-exclusion criteria and enrolment

A total of 202 schools were shortlisted in the two regions using this initial criterion and schools with enrolment less than 800 were excluded. This was done to maximise screening at one location and limit the number of required training sessions; thus, 47 schools were excluded. A random number was generated for each school using excel and three arms of the study (ACT, ST, and VT) were also randomly assigned numbers 1, 2 or 3. Schools were assigned in a 1:1:1 ratio (Figure 1) to the three groups, starting from the top of randomised list. This process was repeated till the sample size was reached in each arm. The process had to be repeated where the children found in school were less than the numbers mentioned in enrolment.⁸ Details of the program and the study were discussed with school principals and permissions were sought. STs were selected by the school principals, while ACTs belonged to the classes from which children were being screened. Each teacher and VTs screened a maximum of 250 students per day. Teachers and VTs were masked to fact that vision would be re-recorded for all the children by the optometrist, as well as other study details.

Figure 1. Selection process of schools for the study [original].

Training

A single training session was conducted at the school premises by a senior clinical optometrist for STs and ACTs. The session included lectures on the importance of SES, as well as common eye problems like REs, squint, cataract, and ptosis, in the school going age-group, and their recognition using picture charts. Teachers were trained to record VA and were provided kits, which included LogMAR charts, torch, measuring tape, written instructions on how to capture vision and a copy of the sheet to be filled for screened students. Dedicated time (half an hour) was provided to practice the technique on each other, under supervision, and at the end they were supplied eyecare education posters and pictures. VTs did not need any dedicated training as their curriculum includes recording vision and conducting a basic eye examination. They were oriented to the flow and protocol to be followed for screening.

Screening process

VA was measured by primary screeners (ACT, STs and VTs) using full LogMAR chart. We used ETDRS type of LogMAR chart, which has linear optotypes (letters in English) with adjustment for crowding phenomenon. There is a logarithmic decrease in size and spacing from top to bottom in this chart. It was used at the standard recommended distance of 4 meters. Students were asked to occlude one eye at a time with their palms to avoid peeking and read the LogMAR 0.2 (equivalent to 20/32) line of the chart, first from right to left with the right eye and then from left to right with the left eye, to circumvent the chance of memorizing. Only one student could enter the examination classroom at a time. The test was considered as a “pass” if all the letters were read by the student accurately. For children wearing spectacles, vision was recorded using spectacles. An eye not able to read at LogMAR 0.2 was noted as ‘reduced vision’.

Students were then sent to a different room where vision was recorded again by an optometrist using a similar LogMAR chart but by making children read all lines of the chart. The optometrist was masked to the vision recorded by the teacher or VT. All students with vision worse than LogMAR 0.2 underwent refraction. Subsequently, glasses were provided free of cost by delivery at the schools. Those requiring refraction under cycloplegia or further evaluation by an ophthalmologist were referred to the nearest VC. The complete teacher training, screening and diagnosis process is shown in Figure 2.

Figure 2. Teacher training, screening, and diagnosis process [original].

Variables and data collection

Data collected included - the total number of children screened, proportion labelled as ‘could not read 0.2 LogMAR’ by screeners, their VA as recorded by the optometrist and whether glasses were prescribed. The proportion of students who reported at VC within three months of referral were also calculated. Data were entered in the database by a data entry person, masked to the study objective, and then analysed by an independent statistician, not part of the study. A detailed cost estimate was made for the ACTs, STs and VTs. The cost calculated included the cost of training sessions and actual screenings. The cost of training session was calculated for ACTs and STs as no dedicated training for screening was conducted for VTs. This cost included - travel cost of the training team, manpower cost (senior optometrist and coordinator), equipment and food. This was multiplied by the number of training sessions held. Cost of actual screening included - travel cost to schools, manpower cost (primary screener, coordinator, counsellor, and the optometrist), cost of equipment (LogMAR chart, measuring rope, torch, trial set, trial frame, retinoscope used for refraction and stationery) used in screening and the food cost. This was multiplied by the number of SES camps held in each arm. The cost per child detected with actual visual deficit was calculated by dividing the total cost by the number of ‘true positives’ in each arm.

Statistical analysis

Mean values were compared using t-test and Wilcoxon rank sum tests. Proportions were compared using the chi-square test. Stata 11.0 software (StataCorp LP StataCorp, Texas, USA) was used for analysis and p-values less than 0.05 were considered significant.

Ethical considerations

The study was approved by the Ethics Committee of Dr Shroff’s Charity Eye Hospital (SCEHEC/2018/03). It adheres to the tenets in the Declaration of Helsinki. Informed consent was taken from all parents for ophthalmic examination of their child.

Results

Out of a total of 203 schools, 29 schools were included in the study. Sample size was achieved by screening nine ACT schools, nine ST schools, and eleven VT schools. In all three groups, number of boys screened were more than that of the girls; 57.2% (95% CI:54.7%-59.7%) in the ACT group, 54.7% (95% CI:52.1%-57.2%) in the ST group, and 58.1% (95% CI:55.6%-60.5%) in the VT group (p = .021). There was a slight preponderance of children above the age of 10 years in all three groups (Table 1).

Table 1. Demographics details of students screened, across the three types of primary screeners [original].

		All Class Teachers (ACTs)		Selected Teachers (STs)		Vision Technicians (VTs)
		N (%)	95% CI	N (%)	95% CI	N (%)	95% CI
School Student Strength		3410		2999		3071
Age Group	5–10 years	1551 (45.5)	(43–48)	1269 (42.3)	(39.8–44.8)	1240 (40.4)	(37.9–42.8)
	11–17 years	1859 (54.5)	(52–57)	1730 (57.7)	(55.2–60.2)	1831 (59.6)	(57.2–62.1)
Sex	Male	1950 (57.2)	(54.7–59.7)	1639 (54.7)	(52.1–57.2)	1783 (58.1)	(55.6–60.5)
	Female	1460 (42.8)	(40.3–45.3)	1360 (45.3)	(42.8–47.9)	1288 (41.9)	(39.5–44.4)

Children were screened within three months of training in ST schools and within one month in ACT ones. ACTs (87 teachers) screened 3410 students (36.0% of total), STs (14 teachers) screened 2999 students (31.7% of total), and two VTs screened 3071 students (32.3% of total). Average time taken to screen a child was 2.5 minutes for ACTs and STs and 1.5 minutes for VTs. Despite screening by the optometrist being carried out on the same day, eleven students in the ACT group, eight in the ST group, and one in the VT group did not report for repeat check-up. Over the study period, average number of children screened were 39 by each ACT, 215 by each ST, and 1536 by each VT.

ACTs and VTs found significantly fewer students with reduced vision, as compared to STs [214 (6.3%) and 207 (6.7%) versus 349 (11.6%), respectively (p < .001)]. However, VTs found a significantly greater percentage of those with actual visual loss (PPV for vision deficit), as compared to ACTs and STs [168 (81.2%), 91 (42.5%) and 105 (30.1%), respectively, p < .001) (Table 2). Higher proportions of children referred to the hospital in the ST and ACT group complied and reported to the hospital within three months as compared to those in the VT group, although this difference was not found to be statistically significant (p = .304) (Table 2).

Table 2. Screening and referral parameters across the three primary examiners [original].

	ACTs	STs	VTs	p-value (Chi-square)
Total screened	3410	2999	3071
Screened Positive by Screeners (for vision deficit)	214	349	207
Screened other pathology by Screener	39	20	12
Total Referred	253	369	219
Screened Positive Percentage (For Vision deficit)	6.3%	11.6%	6.7%	<0.001
True Positive screened by teachers (for vision deficit)	91	105	168
True Negative screened by teachers (for vision deficit)	3034	2518	2852
False Positive screened by teachers (vision deficit)	123	244	39
False Negative screened by teachers (vision deficit)	162	132	12
Sensitivity (%) (for vision deficit)	36.0%	44.3%	93.3%	<0.001
Specificity (%) (for vision deficit)	96.1%	91.2%	98.7%	<0.001
Positive Predictive Value (%) (for vision deficit)	42.5%	30.1%	81.2%	<0.001
Negative Prediction Value (%) (for vision deficit)	94.9%	95.0%	99.6%	<0.001
Referred to Hospital for Further Examination	70 (2.1)	106 (3.5)	62 (2.0)	<0.001*
Student Compliance Rate for Hospital Referrals	28 (40)	47 (44.3)	20 (32.3)	0.3

*p-value less than 0.05 thus significant.

Many teachers did not conduct torchlight examination of children, preferring instead to only conduct VA examinations. Thus, for the purpose of this study, we have calculated yield, sensitivity, specificity predictive values and cost, based on vision deficit only and no other conditions were considered. Other ocular pathologies for which most referrals were made by the screeners included headache, squint, and itching. Conditions for which children were referred to hospital by optometrist included squint, headache, complex refractions and examination of retina.

The proportion of children found with poor vision by optometrists, was significantly higher in the 11–18 year age group (479 out of 5420, 8.8%) than in the 5–10 year group, (191 out of 4060, 4.7%), p-value <.001). Screening for vision deficit by VTs had a higher sensitivity of 93.3% than STs (44.3%), and ACTs (36.0%). Results from the optometrist were considered gold standard. The difference between VTs and ACTs as well as VTs and STs was found to be statistically significant. All three arms had relatively high specificity with the highest being in the VT arm (98.7%). The differences between the VT arm and the other two arms were found to be statistically significant (Table 2).

Sensitivity was found to vary significantly across schools for STs and ACTs, (8.3–81.2 for STs and 18.0–61.5 for ACTs, p < .05 for both), but not for VTs (84.6–100, p = .08). A mixed-effect multivariate logistic regression analysis was carried out to understand if the prediction accuracy of visual impairment is affected by gender or age of children, category of screener, or individual schools. Schools were assumed to have random effects, and it was significant. The error in the assessment of visual impairment was more likely when the child had poor vision, was 11–18 years old and was female (odds ratios were 19.3, 1.23 and 1.32, respectively). Odds of inaccuracy when screened by ACTs or STs were, respectively, 6 and 10 times the odds of inaccuracy when the children were screened by VTs.

SES by VTs was found to cost less as compared to SES by ACTs and STs. The cost of screening a child was estimated to be $0.25, $0.20, and $0.15 for ACTs, STs, and VTs, respectively. This resulted in a total cost of $850.47, $607.68 and $473.76 for the three groups, respectively. Per child with actual visual deficit, these translated to $9.35, $5.79 and $2.82 in the groups screened by ACTs, STs and VTs, respectively (Table 3).

Table 3. Cost details of screening conducted across three primary screeners [original].

	ACTs	STs	VTs
Total Number of Children Screened	3410	2999	3071
Cost of Screening a Child ($)	0.25	0.20	0.15
Total Cost of Screening ($)	850.47	607.68	473.76
True Positive (for Vision deficit)	91	105	168
Cost per case (identification of Poor Vision) ($)	9.35	5.79	2.82

Discussion

In our study, while STs referred most children (11.6%), VTs made the most positive referrals (over 75%), where the student was suffering from actual visual deficit. Although the data of educational background of STs and ACTs were not collected, we do not expect any significant difference in their backgrounds as the schools were randomly included in the two groups and the teachers in the two groups belonged to varied streams. Screening conducted by VTs also showed a higher sensitivity of around 93%, higher specificity of more than 98%, and they were also found to be the cheaper human resource for student vision screening both in terms of per child, as well as per child with actual visual deficit, when compared to STs and ACTs.

Sensitivity and specificity for teachers in our study was found to be much lower than that found in a study conducted by Saxena R et al.⁹ This could have been due to different profiles of schools in both the studies. In another study on screening by primary school teachers, sensitivity was found to be low, similar to ours.¹⁰ The authors conclude that measures to improve sensitivity need to be taken before SES by teachers is recommended universally.¹⁰ In our study, all schools belonged to either the urban-slum or rural regions. As there were no apparent difference in the socioeconomic status of children enrolled in these schools, we did not categorize them further.

A key strength of our study is that it employs triple masking, unlike a similar study conducted in south India.⁶ The screeners (teachers and VTs), optometrists (gold standard, and the data entry operator) were all masked. Another point in favour of our study is that both rounds of screening are conducted the same day, which ensured a low dropout rate as compared to the study from south India.⁶

Programs for SES, require investments in the form of capital and human resource and the accuracy of screenings affects the performance of the program. Low sensitivity adversely affects the program’s outcomes and impact. Our study shows that the high sensitivity of screening conducted by VTs, does not allow students with low VA to slip out of the system. Low sensitivity in ST and ACT arms would not make them effective screeners. Moreover, all three arms had relatively high specificity with the highest being in the VT arm (98%), highlighting that there was a high probability of accurately detecting children without a vision problem, in all arms. High variation in sensitivity across schools makes the diagnosis of poor vision by STs and ACTs inconsistent and unreliable. Significantly higher inaccuracies were found for children who had confirmed poor vision (diagnosed by the optometrist). This implies that there was a tendency to report more children as normal when they had poor vision. Contrarily, high sensitivity, specificity and low variation in sensitivity make the VT model desirable.

Low proportion of referrals being true-positive, would not only increase the workload at the secondary level of screening,^6,9 it can also increase anxiety within patient groups and their families. Accuracy of ST and ACT model was less despite the combined average of students screened by them during the study period being less than those screened by VTs. Proportion of true positive referrals made by VTs was higher compared to that of the teachers, similar to other studies from south of the country,^6,10 other parts of North India,^4,5 as well as Delhi (one of our study areas).⁹ Yet, when the results of teacher conducted screenings in our study are compared to those from existing literature, they do not fare that well. While the above mentioned studies, including ours, use similar screening cut-offs, a possible reason for the large variation in true positive referral proportions, between the teachers and the VTs can be attributed to differences in technical skill.

VTs are paramedics rigorously trained for two years in refraction, recognition, and referral of ocular pathologies while teachers are a non-clinical resource, conducting screenings with limited training. Our study found that while VTs could screen more patients, their referrals also had a higher PPV- making them a more effective cadre to conduct SES than teachers. Their initial hospital responsibility includes capturing a short history regarding ocular complaints and recording VA. Thus, within a few months of starting their training, VTs can carry out the role of the primary screener efficiently. Even though they undergo training for a period less than that of the optometrists, they possess the technical knowledge required to conduct SES. This makes them a more economical, yet clinically trained, human resource with the flexibility of traveling to field locations, as they face less restrictions on their time away from the hospital, unlike optometrists. They need not be recruited solely for SES by eyecare organizations but could contribute during an ongoing SES. Moreover, VTs may be more motivated due to eye care being their primary occupation, as compared to teachers who have other responsibilities, priorities, and time considerations.

Inter-state disparities between north and south of the country in terms of coverage of public health programs have persisted due to social and geographic factors, as well as seasonal variation.^11,12 Overall, the north seems to fare worse than the south. These factors could also be a reason behind our study’s results differing from the south Indian one.⁶

One of the limitations of our study is that it does not assess other factors affecting teacher’s attitude, training practices and contact with individual students, which could have influenced the results. Another form of bias could have been that of selection, as not all schools provided permission for screening. However, having an established positive reputation in the region enables advocacy, with a high proportion of schools providing permission. Moreover, randomization of schools in the three arms was carried out for both regions separately. While calculating the cost of VT arm, the cost of their training program has not been included as we feel that screening is not their sole or primary function, thus, only the time they spent conducting screenings has been accounted for. Although we captured screening time per child but the overall time of disruption of classes in different models was not included in data collection and hence that cost could not be included in the analysis. As screening was carried out at similar time of the day by all the screeners, we do not anticipate too much difference between the ST and VT model. However, overall time of disruption of classes may have been less for ACTs.

Teachers did not conduct torchlight examination uniformly despite receiving training. Most of the referrals made in children with good vision were due to symptoms of headache and itching by the teachers. Due to this accurate comparison of detection of pathologies other than poor vision could not be carried out in our study. This is one of the limitations of our study. Although uncorrected REs are the most common conditions detected in such programs, SES programs need to be comprehensive.

Conclusion

School vision screening, as conducted by VTs, found significantly greater screen positive children with actual poor vision than those by ACTs and STs at a much lower cost. Our results imply that SES by teachers may require different durations and frequencies of training, in different settings. Also, such programs need to be evaluated for quality using a gold standard with a pilot prior to implementation. More such evaluations are required in different settings before promoting teacher-led SES as a universally accepted model. Future research on the cost-effectiveness of the three models is also recommended. As per the results of our study, the VT model of SES could be by adopted by organizations who have this cadre.

Prior publication

This submission has not been published anywhere previously, and it is not simultaneously being considered for any other publication.

Acknowledgments

The authors also acknowledge the guidance provided by the optometrists Mr Mohammad Adil and Mr Bablu Chaudhary for data collection in the field, as well as the vision technicians Ms Rekha Rani and Ms Ekta Chakravarty, who conducted the screenings. The authors fully acknowledge the cooperation of the school authorities, parents, patients, and the public in enabling this study. The authors would like to thank the SEVA Foundation for their support of children’s vision screening programmes, through which this study could be attempted.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

Raw data were generated at Dr. Shroff’s Charity Eye Hospital. Derived data supporting the findings of this study are available from the corresponding author [Priya] on request.

Additional information

Funding

No funding was received for the study, however the ongoing school eye screening program itself, as conducted by the organisation, is funded by the Seva Foundation.