Microfinance trials on trial

Figures & data

Table 1. Where might the microfinance trials have gone wrong?

Key factors in preparing for a trial in ‘medicine’	What would medicine say?	Augsburg et al. (2015)	Tarozzi et al. (2015)	Banerjee, Karlan, et al. (2015)	Angelucci et al. (2015)	Attanasio et al. (2015)	Crepon et al. (2015)	D. Karlan and Zinman (2011)
Generalisability Maybe make this representativeness and move it down the list) (MRC Trials) (MRC Complex Interventions)	Usually, a representative patient group can be captured in the clinical setting. If inclusion criteria are too restrictive, … . with little by way of cross-contamination. Quasi-experimental studies are also recommended, especially for assessing longer-term impacts in the community.	X	X	X (again recruited ‘at the margin’, people of lower interest to provider)	X	Y (for those pre-disposed)	Y (for those pre-disposed, which is perhaps what we want to know about?)	X (only generalisable to marginal borrowers)
Community engagement (MRC Trials) (MRC Complex Interventions)	Investigators should seek to establish relationships with communities from which study subjects will be drawn to discuss various aspects of study design, implementation and communication	? (Worked with provider but not community)	? (Discussed with provider but not community)	? (Discussed with provider but not community)	X	? (Partly – as targeted loans at those who had expressed an interest and explained probabilistic access)	? (Discussed with provider but not community)	? (Worked with provider on credit-scoring algorithm, but not with community)
Uptake and drop-out (MRC Complex Interventions)	Uptake, drop-out, and thus capacity to achieve required samples sizes, would be assessed in a prior feasibility study.	Y (pilot mentioned but not reported on. Good reporting of [low] attrition)	? (Reasonable uptake in experimental group but different panels of ‘before’ and ‘after’ groups, but no feasibility study)	? (Ditto, used estimates from provider which proved optimistic)	X	X	Y (reported on a pilot)	? (Tried to mitigate with credit scoring, but still no pilot as such)
Consent (MRC Trials) (MRC Complex Interventions)	Consent would be a requirement, ethically and practically, and, again, would be assessed in a prior feasibility study.	Y (not explicitly mentioned but individuals knew they only had a probability of receipt of a loan)	X	X	X	Y (interested people could sign up)	X	X
Equipoise (MRC Trials, but don’t cover it explicitly. More now about ensuring benefits outweigh the risks)	It is unethical to conduct an RCT without being in equipoise. (MRC – consider alternative designs)	Y (seems to have been maintained as indicated by analyses of control group access to other sources of loans)	X (clear uptake in control group)	X (clear uptake in control group)	X	X (microfinance already available)	Y (in sense that microcredit did not exist in areas covered by the study)	Y in sense that uptake in control group was low)
Exclusion	If, for some reason, people are excluded from RCTs in medicine, it is challenging to leak into the intervention group as access is controlled by clinicians.	Y (ditto)	Y	Y (in sense of being recognised by study team)	Y		Y (covered this issue well, showing borrowers were those pre-disposed)	Y (seemed to manage this as indicated by low uptake in control group)
Comparators (MRC Trials)	In medicine, the comparator against which a new intervention is assessed, is normally best practice. This helps with ethics, too, as we do not know if the new treatment is better and also, for those not getting the new treatment, they are receiving best (known) practice.	? (unclear as with all studies, but good on ‘what would otherwise happen?’)	?	X (acknowledged that this is messy and was not anticipated)	?	Y (strong on comparing group with individual lending, but comparisons are unclear wrt no lending group)	Y (but only in sense that ‘control’ was largely areas where there was no microcredit)	? (other borrowing is documented, but authors unsure of data and control ‘picture’ is unclear)

Augsburg, B., De Haas, R., Glennerster, R., & Kinnan, C. (2015). The impacts of microcredit: Evidence from Bosnia and Herzegovina. American Economic Journal Applied Economics, 7(1), 183–203. https://doi.org/10.1257/app.20130272

 Web of Science ®Google Scholar

Tarozzi, A., Desai, J., & Johnson, K. (2015). The impacts of microcredit: Evidence from Ethiopia. American Economic Journal Applied Economics, 7(1), 54–89. https://doi.org/10.1257/app.20130475

 Web of Science ®Google Scholar

Banerjee, A., Karlan, D., & Zinman, J. (2015). Six randomized evaluations of microcredit: Introduction and further steps. American Economic Journal Applied Economics, 7(1), 1–21. https://doi.org/10.1257/app.20140287

 Web of Science ®Google Scholar

Angelucci, M., Karlan, D., & Zinman, J. (2015). Microcredit impacts: Evidence from a randomized microcredit program placement experiment by compartamos banco. American Economic Journal Applied Economics, 7(1), 151–182. https://doi.org/10.1257/app.20130537

 Web of Science ®Google Scholar

Attanasio, O., Augsburg, B., De Haas, R., Fitzsimons, E., & Harmgart, H. (2015). The impacts of microfinance: Evidence from joint-liability lending in Mongolia. American Economic Journal Applied Economics, 7(1), 90–122. https://doi.org/10.1257/app.20130489

 Web of Science ®Google Scholar

Crepon, B., Fiorencia, D., Dufflo, E., & Pariente, W. (2015). Estimating the impact of microcredit on those who take it up: Evidence from a randomised experiment. American Economic Journal Applied Economics, 7, 173–150. https://doi.org/10.2139/ssrn.2438038

 Web of Science ®Google Scholar

Karlan, D., & Zinman, J. (2011). Microcredit in theory and practice: Using randomized credit scoring for impact evaluation. Science, 332(6035), 1278–1284. https://doi.org/10.1126/science.1200138

 PubMed Web of Science ®Google Scholar