Demand driven replication research: an overview of financial services for the poor replication research

ABSTRACT

Spurred on by the ‘reproducibility crisis’, social scientists are starting to adopt research transparency practices. Research funders are largely unaware that replication work could strengthen the reliability, rigour, and relevance of their investments. The Gates Foundation commissioned the International Initiative for Impact Evaluation (3ie) to award and quality assure the strongest financial services for the poor evidence. By working with the Gates Foundation to identify the studies, screen the applicants, and quality assure the seven replication research, 3ie ensured policy relevant papers. By publishing this special issue, 3ie is ensuring that the replication research is appreciated by the development community.

KEYWORDS:

• Replication
• transparency
• funding
• financial services
• development

Acknowledgments

The authors would like to thank valuable contributions from Stefan Lhachimi and Emmanuel Jimenez in conceptualising and encouraging this paper. We would also like to thank all of the replication researchers and original authors who made these replication studies possible. All errors are our own.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Open Science Collaboration (2015).

2. 3ie has consistently served as a leader in the research transparency movement around impact evaluations in development. Over the 10 years of 3ie’s existence, it has maintained a steadfast commitment to transparency, including requiring data sharing for 3ie commissioned research (Goel, Khatua, and Gaarder 2019). From creating a replication program (Brown, Cameron, and Wood 2014) to expanding the program to incorporate push button replication (Wood, Müller, and Brown 2018), 3ie staff defined, implemented, and published replication research. Currently, 3ie’s research transparency policy (3ie 2018), including the push button replication of all final reports and the requiring of open data and code within 6 months of completing the research, is on the cutting edge of transparency requirements across development organisations.

3. Sukumar and Metoyer (2019), p.2.

4. http://www.3ieimpact.org/evidence-hub/publications/systematic-reviews.

Additional information

Funding

We greatly appreciate funding from the Bill and Melinda Gates Foundation that faciliated the replication studies in this speical issue.

Notes on contributors

Benjamin DK. Wood

Benjamin DK. Wood, Heifer International, formerly International Initiative for Impact Evaluation, 3ie.

Abed Mutemi

Abed Mutemi, Amazon, formerly Bill & Melinda Gates Foundation, Financial Services for the Poor.

Marie M. Gaarder

Marie M. Gaarder, International Initiative for Impact Evaluation, 3ie.