ABSTRACT
Purpose: Neglected tropical diseases (NTDs) affect people living in the poorest regions of the world and their debilitating effects perpetuate the poverty cycle. Understanding the distribution of NTDs is crucial for effective intervention delivery. In 2012, the Global Trachoma Mapping Project (GTMP) was initiated to map >1800 suspected trachoma endemic districts by March 2015. This research was carried out to better understand the implementation experience and identify lessons which might inform the GTMP and similar initiatives.
Methods: Using grounded theory methodology, semi-structured interviews were conducted with key informants from six countries with 63% of the global mapping backlog (Ethiopia, Malawi, Mozambique, Nigeria, Solomon Islands, and Yemen). Interviews were transcribed, coded, and findings separated into categories.
Results: Three themes were identified during the research; planning and operations, technical implementation, and governance. The project was felt to be most successful in countries where the Ministry of Health was actively engaged in setting standards, ensuring capacity building for government staff, and guiding the training, data collection, analysis, and interpretation of data. Standardized tools, training platforms, and the use of electronic data capture increased confidence in the reliability of the survey data, informed quality improvement efforts within survey implementation, and the immediate release of results empowered end-user decision-makers. Regional collaboration between endemic countries bolstered program manager competence and confidence, while reinforcing partnerships essential to the success of the GTMP.
Conclusions: We depict how innovative characteristics of the GTMP, and lessons learned from its implementation, can strengthen similar initiatives to map disease prevalence and risk factors.
Introduction
Neglected tropical diseases (NTDs) typically affect people living in the poorest regions of the world and their debilitating effects perpetuate the poverty cycle.Citation1–Citation3 Over the last 2 decades, there have been increased efforts to reduce the burden of NTDs.Citation4,Citation5 In 2012, the World Health Organization (WHO) published a roadmap for the implementation of NTD programs globally; dracunculiasis and yaws were targeted for eradication, while trachoma, lymphatic filariasis, leprosy, and human African trypanosomiasis are targeted for elimination by 2020.Citation6 In response to this global commitment, donors, pharmaceutical companies, research institutions and other partners endorsed the London Declaration on Neglected Tropical Diseases in which they pledged to support countries in their goal of reducing the prevalence of 10 of the 17 NTDs.Citation7
Understanding how NTDs are distributed is crucial for developing effective and efficient management plans, particularly for ensuring that resources are targeted to areas with high disease burdens.Citation8,Citation9 Accordingly, baseline mapping activities are currently underway for several NTDs, but these initiatives are typically undertaken on a country-by-country or region-by-region basis. In July 2012, the United Kingdom’s Department for International Development (DFID) awarded funds to Sightsavers to implement the Global Trachoma Mapping Project (GTMP), on behalf of a consortium of academic institutions and non-governmental development organizations (NGDOs) that proposed to map the remaining suspected trachoma endemic districts, worldwide, by the end of 2015. The project is being implemented using country-specific approaches overlain on standardized methodologies, tools and training systems, with the ownership of activities and data resting with respective ministries of health.Citation10 With plans to map over 1800 suspected endemic districts in 3 years, the project has a distinct emphasis on speed and quality.
The GTMP supports trachoma surveys in multiple countries using a common methodology. As described in detail previously,Citation11 population-based prevalence surveys are undertaken to estimate the district level prevalence of trachomatous inflammation–follicular (TF) in 1–9-year-old children, and trachomatous trichiasis (TT) in adults aged 15 years and older. Household-level data on water and sanitation are also collected. All data are entered into an app on Android smartphones and stored in a Cloud-based server, then reviewed and analyzed online. All data are owned by respective ministries of health.Citation11
The GTMP developed a 3-tier training cascade for trachoma graders to accurately identify signs of trachoma.Citation11 Trainee graders and data recorders are recruited by the local Ministry of Health, then trained using standard GTMP materialsCitation12 over 5 days. A coordinating NGDO known to the Ministry of Health, often a member of the original GTMP consortium, manages relationships with the Ministry of Health (including gaining appropriate approvals and agreements), undertakes overall program planning, management and delivery, and (where necessary) builds an implementing NGDO network. An implementing NGDO (which may be, but is not necessarily, the same as the coordinating NGDO) is responsible for delivering mapping to agreed quality standards in collaboration with the Ministry of Health; in countries for which many surveys needed to be completed, more than one implementing NGDO may be involved. A global advisory committee meets two or three times per year. The GTMP has been a unique venture from a number of different perspectives. It is one of very few epidemiologic initiatives to undertake systematic field-based disease mapping across multiple countries using a standardized approach. Mapping activities were undertaken with the express purpose of planning for the elimination of a disease. Additionally, throughout all phases of work it has included a strong partnership between governments, NGDOs, research institutions and donors and has placed a premium on gathering end-user requirements and perspectives to ensure successful execution and share lessons learned. The use of electronic data capture, while also adopted in other research and program settings,Citation12 has been uniquely deployed by the GTMP to facilitate rapid data collection, cleaning, and analysis, government approval, and display and use of findings for planning of trachoma elimination.
The purpose of this study was to determine how the many innovative characteristics of the GTMP might inform similar initiatives as well as our own planned scale-up to new countries. As goals for eliminating NTDs by 2020 loom, national NTD programs will need the capability to rapidly conduct population-based surveys of disease prevalence, both for baseline mapping and for subsequent monitoring of program performance towards elimination targets.
Materials and methods
The first six countries in which GTMP activities took place were Ethiopia, Malawi, Mozambique, Nigeria, Solomon Islands, and Yemen. To better understand the GTMP implementation experience, semi-structured interviews were conducted with key informants from these six countries, which collectively account for 63% (1152/1818) of the global mapping backlog.Citation13 Three of these countries successfully completed mapping; in Nigeria, Ethiopia, and Yemen, a few districts have to date not been sufficiently secure to map. Overall, in the first six countries, 90% (1032/1152) of targeted districts have been surveyed. A list of Ministry of Health and NGDO officials who were directly involved in the planning and implementation of trachoma mapping activities was generated for each country and 14 individuals were selected for interviews, using non-probability, purposive sampling; participating Ministry of Health officials were those who were in charge of the trachoma elimination programs in their respective countries and participating NGDO officials had been directly responsible for coordinating with the Ministry of Health to implement the mapping activities. Emails were sent to all potential participants and all individuals who responded to the email were interviewed by phone. If an individual did not respond, two follow-up messages were sent and a request was made to identify an alternative individual from the same organization who was equally as involved in the project.
In the social sciences, grounded theory is a systematic qualitative research methodology which emphasizes the generation of theory from data collected in a “bottom-up” or “grounded” approach, via data gathering from informants.Citation14,Citation15 The grounded theory method was used throughout data collection and analysis, with constant cross-checking of data collected in order to identify emerging or unanticipated themes.Citation16 A protocol for interviews was developed to guide their administration and implementation and ensure consistency. Each phone interview was conducted by AH (who was not part of the GTMP team) and started with an explanation of its purpose, confirmation that the conversation would be recorded and collection of verbal consent to proceed. Each interview had a semi-structured format (Box 1) and lasted 45 minutes to 1 hour. All interviews were transcribed, and each paragraph of the transcript coded using concepts derived from the interviews. These concepts were combined into related categories, and links between categories identified and verified against the original data. Within each category, overarching themes were identified. Ethical clearance was given by the University of Cape Town Human Ethics Committee.
Box 1. Semi-structured interview guide for assessing the Global Trachoma Mapping Project (GTMP)
Results
Interviews were carried out with five Ministry of Health officials and seven coordinating NGDO officials. In Nigeria, Malawi, Mozambique, and Yemen, interviews were conducted with one official from the Ministry of Health and one official from the coordinating NGDO. Three NGDO officials were interviewed in Ethiopia, and one Ministry of Health official was interviewed in Solomon Islands. Interviews were conducted with a total of 12 of 14 officials targeted for this study, from all countries implementing the GTMP at the time of the research. The authors concluded that saturation of data was reached with those interviewed, consistent with findings from previous work, which indicate that 12–15 interviews are adequate for smaller, more focused qualitative studies.Citation17,Citation18 Therefore, additional follow-up was not made for the remaining selected officials after two or more failed contact attempts. Responses from participants were placed into three key themes; planning and operations, technical implementation, and governance and coordination (inclusive of communication, collaboration, integration, and partnership).
Planning and operations
When asked how the GTMP could improve its processes for future mapping activities, a number of participants commented on the need for an in-depth situational analysis of the status of the country program and external factors that may affect the implementation of the mapping activities before inception of the project. Political insecurity, inability to reach communities in remote or geographically challenging areas, and the inability to conduct activities during the rainy season can all have a negative impact on the project. It was recommended that contingency plans and budgets should be developed at the beginning of the activity in order to facilitate the process. One participant commented that: “plans should include support for someone to move out [to the districts] before the actual mapping should take place. I have never been to some of the provinces…so when we went there for mapping we sometimes faced many challenges and had to change our plans which affected the budget and the timing of activities.” Additionally, trachoma is typically found in remote, hard to reach areas with little access to health services. To compensate for this: “Additional time should be provided for training graders and recorders in these areas since the health staff typically have less capacity.”
Several respondents recommended that the GTMP should provide additional guidance on how to map remote or hard to reach areas. They recommended that mapping protocols address how to deal with conditions where the standard approach could not be carried out due to inability to reach communities or the lack of accurate community information. While acknowledging that the GTMP had assisted with these issues on a case-by-case basis, respondents felt it would be more efficient to have written guidelines that outlined how to deal with such problems, so that programs could plan and budget for them during the preparatory phases of mapping.
Respondents advised that new(er) national programs, without prior experience in trachoma control and surveys, should start planning at least 2 months in advance of the inception of the surveys. They recommended that mapping teams need adequate time to plan activities with consideration of local context and challenges that need to be addressed, compile census information for sampling, budgeting, and to develop protocols for submission to the Ministry of Health for approval.
The use of Android smartphones also needs considerable pre-planning in some contexts. Respondents recommended that the devices be received in the country several weeks in advance to ensure that, even in the event of delays with importation, they are ready to use on time, that plans for charging the devices in areas with limited or no access to electricity are developed, and that consideration be given to engaging an individual able to provide on-site technical support and troubleshoot issues with either the Android smartphones or the mapping software.
Technical implementation
Respondents stated that the GTMP provided standardized tools and training platforms, such as training for grading trachoma, which increased the reliability of the data. The provision of comprehensive standardized tools for training and programmatic activities was commended by all participants. One participant responded that: “standardized tools for training gave clear guidelines for everyone to follow. This made it easier for us to understand exactly what to do and how to do it and when to do it.”
Respondents stated that the GTMP provided clear guidelines to country programs and a rigorous training and certification process for trachoma graders, increasing the likelihood, according to our respondents, of delivery of a high quality product in all countries participating in the project. They believed that the GTMP methodology provided detailed step-by-step guidance for carrying out mapping activities. Respondents indicated that it is important that country programs follow the tools as presented in order to ensure standardization between all countries participating in the GTMP.
Participants noted that the use of Android smartphones to record survey data provided nearly instant access to the raw survey data on a secure website. The data were immediately cleaned and the results of the survey were available for approval within days of the conclusion of the activity. One participant stated: “The standardization [of training] and use of [Android smartphones] led to the collection of high quality data that can be confidently used for planning interventions, and now that the data [are] available we can advocate for the government and all concerned stakeholders to help solve the big problem of blinding trachoma in our country.”
Participants agreed that the use of Android smartphones for data capture was one of the most successful aspects of the program; “Electronic data collection has revolutionized trachoma mapping and I see us using this strategy for more program activities. We will start using it to collect treatment data, for the impact assessment, and I see it having more opportunities to be used in terms of health. It is innovative and is going to change the way surveys are done in our country.” The use of electronic data capture increases the speed with which the program can work and provides real time data that can be very rapidly employed for decision making; “Its essential programs move from paper to electronic data collection. With [paper data collection] time is wasted and energy is wasted. Migration to electronic data collection is essential.” Participants also noted that the benefits of electronic data capture may not be felt if the program does not have plans in place for troubleshooting technical issues and providing facilities for keeping the devices charged.
Governance and coordination
Inter-program collaboration, the sharing of ideas and practical lessons learned between NTD country programs, reinforced solidarity between countries while giving program managers opportunities to learn from colleagues implementing trachoma control programs under similar circumstances: “There were opportunities for programs to learn from the successes and failures of other teams, as those who have gone through the process are the best teachers.” Participants from various countries converged at one training site in a highly trachoma endemic country to participate in a training of trainers for grading trachoma, providing an opportunity to learn from mapping experiences in that region; “I got to see how they were getting started so I was able to adapt quickly. I think that was much more effective than the training in [my country]. “Respondents from the highly endemic country that was mapped at the beginning of the process “shared their experience with all partners on budgeting, training, use of the Android devices [and implementing the mapping activities]. [Other countries] had the opportunity to learn from us because we shared our experiences during training and our experiences were used to update the GTMP tools. This was important as the new countries did not have the challenge of starting from zero.”
One participant noted that the official from their country was well briefed by the GTMP during an international training session, which facilitated scale up: “The [international] training gave [the representative] a lot of confidence, and she came back with that confidence [to implement the project] so in that respect it’s been fantastic. It has been a learning process for all of us all along but I feel like the key support has always been there throughout the program.” Additionally, participants from countries where a GTMP staff member visited the country at the beginning of the project stated that this visit helped to clearly define roles and responsibilities of all parties involved. It was also noted by one participant that a lack of communication in-country caused challenges during the planning stages of the program. The participant felt that there could have been more effort to streamline communication between various partners and the Ministry of Health but they were able to work through the challenges. In some instances, “being left out of an email or not copying the right individual caused some unnecessary delays in starting the program. It would be good if everyone knows who is responsible for what at the beginning of the program.”
Some participants also felt that opportunities for mapping other NTDs had been missed. Several programs had mapped yaws or conducted a Guinea worm case search alongside trachoma mapping. Since the methodologies for conducting baseline surveys for other NTDs are quite different from that of trachoma, many respondents felt they were not able to coordinate activities without recommended guidelines from the WHO. The lack of clear guidelines on coordinated NTD mapping and the speed at which the GTMP activities took place prevented these programs from coordinating mapping activities with other diseases. However, in countries where coordination took place it was typically through the use of shared logistics between programs. One respondent noted that: “Mapping NTDs separately places a large amount of strain on the Ministry’s human resources. The same individuals are called away from their normal duties to conduct consecutive mapping activities. This has a strain on the country program and needs to be considered when planning.” Respondents encouraged the GTMP and other country programs to consider how trachoma mapping activities can be coordinated with other NTD mapping in the future.
Participants were asked to share advice for other countries starting a GTMP collaboration and the most common response was that the Ministry of Health should lead the entire process. Participants strongly felt there was a need to have buy-in from government and partners from the beginning of the program and that it is important to cast a wide net for partners at the beginning of the project: “This includes individuals from the water and sanitation sector and any relevant research institutes. If they are engaged early they will be on board and ready to move forward once the survey data [are] available.” Some respondents also expressed the need to increase the capacity of government staff to plan and carry out survey activities. One respondent stated “as much as possible the GTMP should consider strengthening the existing health systems so there is capacity within the government to carry out subsequent mapping activities.” Additionally, most participants agreed that clearly defined roles and responsibilities and coordination between the government, partners, and researchers at the beginning of the program are essential for ensuring buy-in from all stakeholders. The majority of participants felt the process of implementing the GTMP in their country helped hone their program management skills. Participants noted an improved ability to network and manage projects with multiple stakeholders and they felt the project forced them to focus on improving their financial management, planning and coordination, and communication skills.
Discussion
Overall, participants indicated that the GTMP was successful because the resource availability placed trachoma, at least temporarily, at the forefront of the Ministry of Health agenda. Baseline prevalence data are needed to start implementing trachoma elimination activities, but most programs and their NGDO partners have not previously had the financial resources available to map all suspected endemic districts. The GTMP has provided the opportunity for countries to rapidly map trachoma prevalence, allowing national programs to plan for implementation, at scale, in a relatively short period of time. However, many global health initiatives start with solid funding, but do not have the collaborations/partnerships, the appropriate technology, nor the evidence-based planning and proven interventions to achieve success, at scale.
The GTMP is an ambitious project that, out of necessity, has made a rapid start to implementation, commencing field work in seven countries in its first year. Moving a project forward at high speed can provide opportunities for poor coordination or fragmentation, low quality, a lack of standardization, disenfranchisement, or damaged relationships between partners, and we therefore wished to undertake a detailed qualitative examination of the project’s performance. While this analysis is of importance to our own future efforts within the GTMP, its lessons can also be applied elsewhere, both within the trachoma field and in data collection exercises for other diseases.
Perspectives on planning and operations are a reminder that, regardless of the format for data collection, sufficient time for planning is critical for success. Planning also has a bearing on governance and if the planning period is too short, engagement of the key personnel at government and partner levels can be inadequate. In some countries undertaking trachoma mapping, the project provided the first opportunity for personnel responsible for trachoma control to visit trachoma-suspect areas. Planning any mapping activity requires an appreciation of the situation on the ground; ensuring that one or more members of the planning team have a good understanding of the areas to be surveyed is essential. On the basis of this study, future efforts may consider requiring micro-planning, informed by onsite assessment, prior to survey implementation, especially within challenging administrative areas.
New technologies, such as data collection on Android smartphones, which may be attractive to funders and academics, may be viewed with suspicion by some in government, and adequate time is needed to discuss the advantages and disadvantages of adopting them. While provision of standardized tools such as training packages, data collection systems, and reporting formats, is attractive from an international perspective and can significantly improve quality control, it can also be received as being somewhat critical of previous local data collection efforts. These scenarios suggest that consulting with all relevant stakeholders over multiple sessions, so that concerns can be raised and addressed, requires both time and sensitivity.
The ability to collect data efficiently and accurately has been key to the success of the GTMP and is considered a highlight of its work by respondents from both government and NGDOs. Near-instantaneous access to data allows the programs to monitor data collection and adjust fieldwork in a timely manner to collect more accurate data. In cases where good connectivity was available in the field, program managers were able to provide feedback to graders and recorders during the mapping exercise, ultimately saving time and improving quality of data collection. In addition, the use of mobile and cloud technologies enabled timely and efficient review of findings and approval of results. Electronic data collection has been applied by other NTD programs, including in mapping of podoconiosis and lymphatic filariasis.Citation19 Other surveys currently being rolled out would also likely benefit. Within the field of trachoma, the adaptation of the GTMP data entry template to facilitate impact surveys is strongly being considered. Applying similar tools for monitoring program activities, such as trichiasis surgery and antibiotic distribution and inventory levels would likely lead to greater efficiency and more accurate and timely reporting. Creation of electronic data capture tools for monitoring program activities will likely require significant development time to align with national and sub-national reporting systems. Despite the use of mHealth systems in many countries, there has been limited evidence to inform effective implementation and scale-up of electronic data capture systems,Citation20 and some anecdotal evidence to suggest that the proliferation of different systems may be detrimental. Survey recorders during the GTMP project had several days of training, a dedicated support system within the project and the surveys were time-limited. For programs endeavoring to use sub-district health workers or community members for routine electronic data collection, consideration of the most appropriate system and format, and the level of training and supervision of recorders, will all be critical.
The strong coordination required for successful implementation of the GTMP, as for all standardized survey efforts, requires the active leadership of government. As prevalence category data, once approved by government, were uploaded onto the Trachoma AtlasCitation21 information sharing became an essential component for the success of GTMP. According to the WHO, health systems strengthening includes initiatives and strategies that improve the functions of the health system, ultimately leading to better health through improvements in access, coverage, quality, or efficiency.Citation22 The facilitation of the GTMP was most successful in countries where the Ministry of Health was actively engaged in setting standards, ensuring capacity building within government staff, and guiding the training, data collection, analysis, and interpretation of data. However, implementation at this scale needs a well-coordinated effort with multiple stakeholders. In many countries, the implementation of mapping activities took place over the span of several months and external collaborators spent relatively little time supervising day-to-day activities. This required training a large cadre of staff from local government and partners to ensure good supervision, proper logistics, and support mechanisms.
Epidemiologic mapping at this global scale understandably creates tensions between speed, quality and ownership of the process. Perspectives from the personnel responsible at the national and sub-national level help to identify the specific tensions and to explore ways to address them. Even before trachoma endemic countries finalize their mapping, focus begins to shift to the application of data to planning for trachoma elimination, as there is considerable pressure from governments and non-governmental organizations to apply the findings without delay in order to reach the target of global elimination of trachoma as a public health problem by the year 2020. Scale-up of planning and implementation will likely be slower than scale-up of mapping.
However, efforts to implement elimination interventions and monitor for impact, as well as future global efforts to conduct standardized surveys, may consider drawing from the key lessons learned from the GTMP experience: (1) Micro-planning to customize training and survey implementation for challenging sampling units is essential; (2) standardized, electronic data capture and centralized processing promotes relevance to end-users, because both results and suggestions to improve performance can be fed back quickly; and (3) upfront efforts to empower and build capacity of government leadership can result in strong coordination, collaboration and execution by the partnership.
Declaration of interest
AWS is the Chief Scientist and PC is a Technical Advisor for the GTMP. There is no conflict of interest for AH.
Funding
The GTMP is funded by a grant from the United Kingdom’s Department for International Development (DFID) (ARIES: 203145) to Sightsavers, who is leading a consortium of non-governmental organizations and academic institutions to complete the global baseline trachoma map; this grant also supported the study described in this paper. Fieldwork costs in the first year of GTMP operation were funded through the DFID GTMP grant to Sightsavers (Ethiopia, Malawi, Mozambique, Nigeria, Solomon Islands, Yemen); and by the United States Agency for International Development (USAID) through the ENVISION project implemented by RTI International under cooperative agreement number AID-OAA-A-11-00048 (Mozambique), and the END in Asia project implemented by FHI360 under cooperative agreement number OAA-A-10-00051 (Laos). AWS was supported by a Wellcome Trust Intermediate Clinical Fellowship (098521) at the London School of Hygiene & Tropical Medicine. None of the funders had any role in study design, in study implementation or analysis or interpretation of data, in the decisions on where, how or when to publish in the peer reviewed press, or in preparation of the manuscript.
References
- Hotez PJ, Ottesen E, Fenwick A, et al. The neglected tropical diseases. The ancient afflictions of stigma and poverty and the prospects for their control and elimination. Adv Exp Biol Med 2006;582:22–33.
- Frick KD, Hanson CL, Jacobson GA. Global burden of trachoma and economics of disease. Am J Trop Med Hyg 2003;69(5):1–10.
- Hoetz PJ, Molyneux DH, Fenwick A, et al. Control of neglected tropical diseases. N Engl J Med 2007;375:1018–1027.
- Anonymous. Neglected tropical diseases: becoming less neglected. Lancet 2014;383(9925):1269.
- Uniting to combat NTDs. Delivering on promises and driving progress: the second report on uniting to combat NTDs. 2013. Retrieved from Uniting to Combat NTDs website: http://unitingtocombatntds.org/resource/download-report
- World Health Organization. Accelerating work to overcome the global impact of neglected tropical diseases: a roadmap for implementation. France: WHO, 2012.
- Uniting to Combat Neglected Tropical Diseases. London Declaration on Neglected Tropical Diseases. 2012. Retrieved from the Uniting to Combat Neglected Tropical Diseases website: http://unitingtocombatntds.org/downloads/press/ntd_event_london_declaration_on_ntds.pdf
- Broker SJ, Smith JL. Mapping neglected tropical diseases: a global overview. Community Eye Health 2013;26(82):32.
- Broker SJ, Hoetz PJ, Bundy DAP. The global atlas of helminth infection: mapping the way forward in neglected tropical disease control. PLoS Negl Trop Dis 2010;4:779.
- International Coalition for Trachoma Control. Global Trachoma Mapping Project: raining for mapping of trachoma. 2013. Retrieved from the International Coalition for Trachoma Control website: http://www.trachomacoalition.org/resources/global-trachoma-mapping-project-training-mapping-trachoma
- Solomon AW, Pavluck AL, Courtright P, et al. The Global Trachoma Mapping Project: methodology of a 34-country population-based study. Ophthalmic Epidemiol 2015;22:214–225.
- Pavluck A, Chu B, Mann Flueckiger R, et al. Electronic data capture tools for global health programs: evolution of LINKS, an Android-, web-based system. PLoS Negl Trop Dis 2014;8:e2654.
- The International Coalition for Trachoma Control. The end in sight: 2020 INSight. Atlanta, GA: Author, 2011.
- Martin PY & Turner BA. Grounded theory and organizational research. J Appl Behav Sci 1986;22:141.
- Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. New Brunswick, NJ: Aldine Transaction, Inc., 1967.
- Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. London: Sage Publications, 1998.
- Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Meth 2006;18:59–82.
- Francis J, Johnston M, Robertson C, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health 2010;25:1229–1245.
- Sime H, Deribe K, et al. Integrated mapping of lymphatic filariasis and podoconiosis: lessons learnt from Ethiopia. Parasites Vectors 2014;7:397.
- Tomlinson M, Rotheram-Borus MJ, Swartz L, et al. Scaling up mHealth: where is the evidence? PLoS Me 2013;10:e1001382.
- Smith JL, Haddad D, Broker SJ, et al. Mapping the global distribution of trachoma: why an updated atlas is needed. PLoS Negl Trop Dis 2011;5:e973.
- World Health Organization. Everybody’s business: strengthening health systems to improve health outcomes. Switzerland: World Health Organization, 2007.