NGO perspectives on the challenges and opportunities for real-world evaluation: a qualitative study

ABSTRACT

Background

The move towards robust monitoring and evaluation (M&E) has been increasing in global health, motivated by both an accountability agenda and to increase learning from M&E activities. Many international non-governmental organizations (NGOs) receive funding from one or more large institutional donors.

Objective

To understand NGOs’ perspective on their own role in terms of accountability to both donors and the populations they serve.

Methods

We conducted a series of in-depth interviews with M&E staff in 11 NGOs with projects related to maternal and child health to better understand how M&E is being implemented in these organizations. We then examined the data based on a priori identified themes.

Results

We found that despite flexibility from some donors, rigid reporting structures remain a barrier for NGOs to fully communicate the impact of their projects. While NGOs do utilize M&E findings, their use is limited by low staff capacity. The primary audience for the results remains the donor agency, and the primary motivation for M&E remains donor reporting. Reporting remains a burdensome affair, with ongoing limitations around streamlining results for donors. To reduce the burden of reporting for individual projects, the participants in our study suggested placing greater emphasis on process evaluations rather than impact evaluations. Participants also suggested increased data sharing between organizations working in the same regions and making better use of secondary data sources; in both cases to reduce the need for primary data collection.

Conclusion

We carried out this work to advance the conversation on how NGOs currently manage their M&E – a conversation which should involve NGOs, donors, local health system actors, and the communities with whom they work. More flexibility from donors, increased use of technology, and more transparency on if and how data is being used would help NGOs with their M&E process.

KEYWORDS:

• Evaluation
• NGOs
• donors
• non-governmental organizations
• maternal and child health

Responsible Editor

Maria Nilsson

Responsible Editor

Maria Nilsson

Acknowledgments

We would like to thank Jonathan Alcazar for his support in formatting this manuscript. We also thank the reviewers whose comments contributed to strengthening this manuscript.

Author contributions

TSL, RB and TR designed the analysis. RB carried out individual interviews. TSL wrote the first draft of this manuscript. RB and TR provided comments. All authors have read and approve of the final version of the manuscript.

Disclosure statement

Authors are funded by one of the donors affiliated with the study. The donor had no impact on study design, data collection or data interpretation.

Paper context

Increasing attention is being paid to monitoring and evaluation in the global health arena – particularly within international non-governmental organization. However, little research has been done on these organizations’ own perception of their role and challenges. This paper shines light on what is working well, what is working poorly and what could be improved from their perspective. These insights serve to advance the discussion on how to improve accountability to both donors and populations.

Additional information

Funding

This work was funded by Global Affairs Canada (Johns Hopkins University project CA-3-D001627001).