Evaluating Precision of a Trachomatous Trichiasis (TT) Super Survey with Modulating Sample Sizes in Tanzania

ABSTRACT

As trachoma programs move towards eliminating trachoma as a public health problem, the number of surveys necessary to evaluate the status of trachomatous trichiasis (TT) increases. Currently, the World Health Organization endorses a district-level population-based prevalence survey for trachoma that involves a two-stage cluster design. We explored the validity of implementing this survey design in larger geographic areas to gain cost efficiencies. We evaluated the change in precision due to combining geographically contiguous and homogenous districts into single evaluation units (EUs) and modulating the sample size by running simulations on existing datasets. Preliminary findings from two opportunities in Tanzania show variability in the appropriateness in conducting this survey across larger geographies. These preliminary findings stress the importance of determining what is meant by homogeneity in terms of TT before combining multiple districts into a single EU.

KEYWORDS:

• Trachoma
• Trachomatous Trichiasis
• Survey
• Sample Size
• Cluster Design

Background

Trachoma, a neglected tropical disease, is caused by repeated ocular Chlamydia trachomatis infection leading to chronic inflammation of the tarsal conjunctiva; this inflammation is defined as trachomatous inflammation–follicular (TF).¹ TF may cause conjunctival scarring, which in turn may lead to inward turning of eyelashes that can touch the eye. The eyelashes may damage the cornea, and lead to corneal opacity and blindness. This morbidity stage of trachoma generally occurs in adulthood. The presence of eyelashes touching the eye is called trachomatous trichiasis (TT).

Trachoma is targeted for elimination as a public health problem,² defined as district level prevalence of TF below 5% in children aged 1–9 years and of TT unknown to the health system below 0.2% in adults aged 15 years and older.^3,4 National programs of endemic countries employ the intervention strategy, called SAFE, which involves (S) surgery to correct trichiasis; (A) mass drug administration of antibiotics; and sanitation and hygiene improvements, focusing on (F) facial cleanliness and (E) environmental improvement.²

To inform decision-making and measure against elimination targets, the World Health Organization (WHO) endorses a population-based prevalence survey (PBPS) using two-stage cluster sampling.^5,6 The recommended survey design involves (i) selection of villages as the first sampling stage; and (ii) selection of households as the second sampling stage. This method is powered to estimate prevalence of TF and TT at the evaluation unit (EU) level (typically a district of 100,000–250,00 people). PBPSs require considerable resources, typically costing between 6,500 USD and 11,000 USD per EU.⁷

The 4th Global Scientific Meeting on Trachoma (GSM4) report provides alternative options for country programs to demonstrate that TT elimination targets have been met. These options include conducting full geographic coverage case-finding and trichiasis service delivery, and combining data from adjacent EUs to create a larger EU.⁸The larger geographic area resulting from combining adjacent EUs may contain greater heterogeneity in terms of TT distribution. However, an increased number of clusters across this larger geography may also decrease the sampling error.

The GSM4 report section 5.4.iii, states that “ … . national programmes may use … a combination of data from multiple adjacent evaluation units. Professional statistical

advice should be sought as to the best way to combine data from multiple evaluation units, with guidance subsequently given to national programmes and their partners”.⁸ As trachoma programs move towards eliminating trachoma as a public health problem, the number of surveys necessary to evaluate the status of TT increases – absorbing large portions of country program’s budgets. Here, we explore the validity of conducting the currently endorsed survey design within super EUs (created by combining geographically contiguous and homogenous districts) and evaluated the change in precision seen when reducing the number of villages sampled.

Methods

Data source

With guidance from the Tanzania NTD Control Program, we selected two opportunities to explore the consolidation of data from geographically contiguous districts. The first opportunity combines Babati District Council (DC) and Mbulu DC, both in Manyara Region, into a super EU; the second combines Iringa DC, Kilolo DC and Mufindi DC, all in Iringa Region, into a super EU. Baseline surveys conducted in 2004 (Iringa DC/Kilolo DC), 2012 (Babati DC and Mbulu DC) and 2014 (Mufindi DC) indicated TF below the elimination threshold and TT above the elimination threshold (Table 1).⁹ Additionally, none of these districts had any active TT outreach supported by partners in the years between the baseline surveys and the TT-only survey. These baseline surveys were conducted prior to the refinement of TT indicator to “TT unknown to the health system” and therefore baseline TT prevalence may be overestimated.

Table 1. Baseline TT survey results.

Scenario	District	Year	TF prevalence (%) (95% CI)	TT prevalence (%) (95% CI)
Opportunity 1	Babati DC	2012	0.3 (0.1–0.6)	0.55 (0.13–1.18)
	Mbulu DC	2012	2.8 (1.1–4.5)	0.80 (0.12–1.70)
Opportunity 2	Iringa DC/ Kilolo DC*	2004	<5	0.91 (0.44–1.56)
	Mufindi DC	2014	0.3 (0.1–0.6)	0.35 (0.14–0.63)

*In 2004 these two districts were a single district and surveyed as such. They were divided into two districts in 2014. The 2004 TF estimates have not been published and so the value presented here is from the available categorical data from the global atlas of trachoma (https://www.trachomaatlas.org).

Data used for the analysis presented here were derived from PBPSs conducted in 2018 through the Tropical Data system.¹⁰ They followed a two-stage cluster sampling strategy, and each included 30 villages (Figure 1 and 2).

Figure 1. Distribution of first-stage village data points in Babati DC and Mbulu DC.

Figure 2. Distribution of first-stage village data points in Iringa DC, Kilolo DC, and Mufindi DC.

Analysis

In this analysis, we defined a case of TT as an individual aged at least 15 years old with presence of upper lid trichiasis in either eye.⁸ We defined a case of TT unknown to the health system as a TT case that had not previously been offered surgery or epilation. All analyses were performed in R v3.4.4¹¹. For each village, we calculated the proportion of all TT cases and unknown to the health system found within each five-year age/sex group and weighted these proportions according to the expected proportion of residents with that age and sex. The expected population pyramid was derived from the United Nations World Population Prospective.¹¹ The village level prevalence was calculated as the sum of the weighted proportions for each age/sex group.

To derive 95% confidence intervals for each district, we bootstrapped (randomly sampled with replacement) the village level prevalence dataset with 10,000 replications. The 2.5th and 97.5th centiles of the order means from the bootstrap were used as the lower and upper 95% confidence interval bounds. We then re-classified the villages to create super EUs and again bootstrapped the village level prevalence dataset with 10,000 replications and took the 2.5th and 97.5th centiles of the order means as the lower and upper 95% confidence interval bounds.

We then modulated the sample size of the super EUs through reducing the number of villages in the bootstrap. We bootstrapped with replacement over 10,000 replications, three additional times: first randomly sampling with replacement 100%, then 90%, then 80% of the total number of villages in each resample.

We compared the results by first assessing the programmatic decision that resulted from comparing the mean in both the districts and super EUs with the elimination threshold of 0.2% unknown to the health system. We then assessed the change in precision between the districts and super EUs through evaluating the width of the confidence intervals. Finally, we examined the change in precision across the super EUs with reduced sample sizes.

Results

Opportunity 1

The observed adjusted mean of the district level prevalences are at or above the elimination threshold of 0.2% unknown to the health system in both Babati DC and Mbulu DC (Table 2). The super EU mean estimate is also above the elimination threshold. When the districts are combined into a super EU, the confidence intervals and interquartile range tighten, diluting the influence of outliers.

Table 2. Opportunity 1, 2018 unknown to the health system prevalence estimates.

District	TT prevalence unknown to the health system	Standard Error	95% CI (lower)	95% CI (upper)
Babati DC	0.27%	0.17%	0.01%	0.65%
Mbulu DC	0.20%	0.17%	0.00%	0.57%
Super EU	0.24%	0.12%	0.03%	0.50%

In Opportunity 1, as the sample size was reduced in the super EU, the mean TT prevalence remained stable and the confidence intervals widened (Table 3 and Figure 3). Because the mean point prevalence is used for decision-making, the simulations suggest that reducing the sample sizes does not change the decision to continue intervention. However, as the sample size decreases so does the precision. There is no difference between the mean prevalence point estimate of all TT cases and unknown to the health system cases in this opportunity.

Figure 3. Opportunity 1 distribution of super EU unknown to the health system bootstrap results with modulating sample sizes. Dashed lines represent 95% confidence intervals and the red dashed lines represent the mean; solid lines represent the elimination threshold of 0.2%.

Table 3. Opportunity 1, 2018 super EU unknown to the health system prevalence estimates with modulating sample sizes.

Sample size		TT prevalence	Standard Error	95% CI (lower)	95% CI (upper)
Number of villages	Percent of total villages
60	100%	0.24%	0.12%	0.03%	0.50%
54	90%	0.24%	0.13%	0.02%	0.52%
48	80%	0.24%	0.13%	0.02%	0.52%

The confidence intervals for the 80% simulation are wider than the 90% simulation, due to the third decimal place in the lower bound.

Opportunity 2

The observed adjusted mean of the district level prevalences are below the elimination threshold of 0.2% unknown to the health system in both Kilolo DC and Mufindi DC (Table 4). However, the mean estimate in Iringa DC and the super EU are above the elimination threshold. When the districts are combined into a super EU the confidence intervals and interquartile range tighten, and the higher-prevalence villages in Iringa DC pull the mean to the threshold.

Table 4. Opportunity 2, 2018 prevalence estimates.

	Unknown to the health system				All cases
District	TT prevalence	Standard Error	95% CI (lower)	95% CI (upper)	TT prevalence	Standard Error	95% CI (lower)	95% CI (upper)
Iringa DC	0.30%	0.10%	0.13%	0.51%	0.33%	0.10%	0.16%	0.54%
Kilolo DC	0.15%	0.13%	0.00%	0.43%	0.15%	0.13%	0.00%	0.44%
Mufindi DC	0.15%	0.08%	0.03%	0.32%	0.19%	0.08%	0.05%	0.37%
Super EU	0.20%	0.06%	0.10%	0.33%	0.23%	0.06%	0.12%	0.35%

In Opportunity 2, as the sample size was reduced in the super EU, the mean TT prevalence increases, and the confidence intervals widen. However, the mean remains above the elimination threshold (Table 5 and Figure 4). This suggests that reducing the sample sizes does not change the decision to continue intervention for the super EU.

Figure 4. Opportunity 2 distribution of super EU unknown to the health system bootstrap results with modulating sample sizes. Dashed lines represent 95% confidence intervals and the red dashed lines represent the mean; solid lines represent the elimination threshold of 0.2%.

Table 5. Opportunity 2 super EU prevalence estimates with modulating sample sizes.

Sample size		TT prevalence	Standard Error	95% CI (lower)	95% CI (upper)
Number of villages	Percent of total villages
90	100%	0.20%	0.06%	0.10%	0.33%
81	90%	0.24%	0.11%	0.06%	0.47%
74	80%	0.24%	0.11%	0.05%	0.48%

Discussion

The latest outcomes from the GSM4 provide a variety of options when assessing whether elimination thresholds have been met for TT.⁸ While many country programs chose geographic coverage case finding, there are some situations where a program may wish to use a combination of data from multiple adjacent districts to evaluate their progress. Here we aim to provide guidance on when to combine EUs for country programs who choose this option. To employ this option, programs must first understand the implications of combining districts.

In our opportunities, we observed that combining geographically contiguous districts into a super EU and retaining the prescribed number of villages resulted in increased precision. However, we also found that our super EUs do not trigger consistent programmatic decisions. In Opportunity 1, combining Babati DC and Mbulu DC into a super EU triggers the continuation of implementing TT interventions. However, in Opportunity 2, combining Iringa DC, Kilolo DC and Mufindi DC into as super EU results in unnecessarily continuing TT interventions in Kilolo DC and Mufindi DC. In all cases, however, the bootstrapped 95% confidence intervals cross over the elimination threshold. It is standard practice for trachoma programs to make their decisions based on the observed point estimates rather than the confidence intervals, but they should also consider the loss of precision represented by the broadening of confidence intervals.

In both opportunities, reducing the number of villages included in the super EU did not change the programmatic decision. Therefore, if programs could accurately classify EUs as homogeneous, they might gain some cost efficiencies due to the reduction in the number of clusters in a super EU.

We relied on baseline TF and overall TT prevalence to suggest contemporary TT homogeneity across our districts. At baseline, TF was below the elimination threshold in all included districts. We therefore assumed TT prevalence would reduce at a relatively similar rate across districts. However, our opportunity 2 results suggest that additional factors may influence the rate of reduction in TT prevalence and should be taken into consideration when classifying districts as homogenous. Additional factors to include in future work could be topography, population density, demographics, and infrastructure. Furthermore, exploration of the variation in village-level TT prevalence at baseline could provide valuable insight into what to expect in contemporary surveys.

Understanding both the coverage of and quality of past TT interventions in each area would provide key information to help determine TT homogeneity across districts. Regardless of the challenges associated with reaching patients, a critical component affecting impact is quality of services provided. Numerous studies have highlighted concerns around post-operative TT^12–20 (PTT) and the effect that high rates of PTT in an area can have on community members’ willingness to be screened by a local case finder or by the district health system. A qualitative study in Tanzania found that people were more reluctant to attend a surgical facility due to witnessing poor TT surgical outcomes of their neighbours.²¹ Thus, prevalence of TT unknown to the health system in areas with historically high rates of PTT may reduce more slowly than other districts.

We aimed to provide guidance to country programs who wish to combine districts into super EUs through evaluating change in precision when combining districts and reducing the number of villages. However, this analysis highlighted a more general question of what is meant by homogeneity in terms of TT. Following WHO intervention recommendations, in our super EU opportunities two districts would receive more than required public health interventions. This analysis additionally suggests that while increased precision may be gained by averaging in neighbouring survey data this may not impact the EU’s status in meeting elimination thresholds. GSM4 includes an option of full geographic coverage case finding and service delivery for determining elimination. This approach is in line with providing access to service and so may be the method chosen by many programs. However, if programs wish to combine districts into super EUs, further work is needed to determine if there are additional indicators that could consistently identify TT homogeneity across districts.

Financial support

The TT-only surveys discussed in this paper were funded by the Queen Elizabeth Diamond Jubilee Trust Trachoma Initiative through Sightsavers and Helen Keller International. This analysis was made possible thanks to funding from ENVISION, a global project led by RTI International in partnership with CBM International, The Carter Center, Fred Hollows Foundation, Helen Keller International, IMA World Health, Light for the World, Sightsavers, and World Vision. ENVISION was funded by the United States Agency for International Development under cooperative agreement No. AID-OAA-A-11-00048. The period of performance for ENVISION was September 30, 2011 through September 30, 2019. For more information, go to www.NTDenvision.org.

The authors alone are responsible for the views expressed in this article, which do not necessarily represent the views, decisions, or policies of the institutions with which the authors are affiliated, the United States Agency for International Development, or the United States Government.

Disclosure Statement

Authors have no conflict of interest.

Additional information

Funding

This work was supported by the US Agency for International Development (USAID) and the ENVISION project led by RTI International under cooperative agreement No. AID-OAA-A-11-00048.