Generalizing the Results from Social Experiments: Theory and Evidence from India

Abstract

How informative are treatment effects estimated in one region or time period for another region or time? In this article, I derive bounds on the average treatment effect in a context of interest using experimental evidence from another context. The bounds are based on (a) the information identified about treatment effect heterogeneity due to unobservables in the experiment and (b) using differences in outcome distributions across contexts to learn about differences in distributions of unobservables. Empirically, using data from a pair of remedial education experiments carried out in India, I show the bounds are able to recover average treatment effects in one location using results from the other while the benchmark method cannot.

Keywords:

• External validity
• Partial identification
• Sensitivity analysis

Supplementary Materials

This supplement provides (1) the Online Appendix referenced in the main text and (2) the code and data needed to reproduce all figures and tables in the paper.

Acknowledgments

This article is a revised version of the first chapter of my Ph.D. dissertation. I am grateful to my advisors Dilip Mookherjee, Hiroaki Kaido and Kevin Lang for guidance, support and encouragement. I wish to thank the Editor, Associate Editor, and anyonomyous referees for comments which have greatly improved the paper. I also thank Kehinde Ajayi, Manuel Arellano, Sam Bazzi, Xavier d’Haultfoeuille, Rajeev Dehejia, Isabella Dobrescu, Iván Fernández-Val, Claudio Ferraz, Andy Foster, Xin Geng, Patrik Guggenberger, Rema Hanna, Stefan Hoderlein, Asim Khwaja, Horacio Larreguy, David Lam, Leigh Linden, Michael Manove, Shanthi Nataraj, Rohini Pande, Justin Sandefur, Johannes Schmieder, Jeff Smith, Duncan Thomas, Nate Young, and numerous seminar and conference participants for useful conversations and feedback. Part of this work was completed while I was at the Markets, Trade, and Institutions Division of IFPRI. I thank IFPRI and Maximo Torero in particular for supporting this project. Gloria Sarmiento-Becerra provided excellent research assistance.

Disclosure Statement

No potential conflict of interest was reported by the author.

Additional information

Funding

Financial support from the Boston University Department of Economics is gratefully acknowledged.