Replicating Experimental Impact Estimates with Nonexperimental Methods in the Context of Control-Group Noncompliance

References

• Abdulkadiroğlu, A., Angrist, J. D., Dynarski, S. M., Kane, T. J., and Pathak, P. A. (2011), “Accountability and Flexibility in Public Schools: Evidence from Boston's Charters and Pilots,” The Quarterly Journal of Economics, 126, 699–748.
   Web of Science ®Google Scholar
• Angrist, J. D., Cohodes, S. A., Dynarski, S. M., Parak, P. A., and Walters, C. D. (2013), Charter Schools and the Road to College Readiness: The Effects on College Preparation, Attendance, and Choice, Boston, MA: The Boston Foundation and the NewSchools Venture Fund.
   Google Scholar
• Angrist, J. D., Imbens, G. W., and Rubin, D. B. (1996), “Identification of Causal Effects Using Instrumental Variables,” Journal of the American Statistical Association, 91, 444–455.
   Web of Science ®Google Scholar
• Angrist, J. D., Pathak, P. A., and Walters, C. D. (2011), “Explaining Charter School Effectiveness,” National Bureau of Economic Research, NBER Working Paper 17332.
   Google Scholar
• Bifulco, R. (2012), “Can Nonexperimental Estimates Replicate Estimates Based on Random Assignment in Evaluations of School Choice? A Within-Study Comparison,” Journal of Policy Analysis and Management, 31, 729–751.
   Web of Science ®Google Scholar
• Cook, T. D., Shadish, W. R., and Wong, V. C. (2008), “Three Conditions Under Which Experiments and Observational Studies Produce Comparable Causal Estimates: New Findings from Within-Study Comparisons,” Journal of Public Analysis and Management, 27, 724–750.
   Web of Science ®Google Scholar
• Dobbie, W., and Fryer, R. G. (2011a), “Are High-Quality Schools Enough to Increase Achievement Among the Poor? Evidence from the Harlem Children's Zone,” American Economic Journal: Applied Economics, 3, 158–187.
   Web of Science ®Google Scholar
• ——— (2011b), “Getting Beneath the Veil of Effective Schools: Evidence from New York City,” National Bureau of Economic Research, NBER Working Paper 17632.
   Google Scholar
• Fortson, K., Verbitzy-Savitz, N., Kopa, E., and Gleason, P. (2012), Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates (NCEE Technical Methods Report 2012–4019), Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• Furgeson, J., Gill, B., Haimson, J., Killewald, A., McCullough, M., Nichols-Barrer, I., Teh, B., Verbitsky-Savitz, N., Bowen, M., Demeritt, A., Hill, P., and Lake, R. (2012), Charter-School Management Organizations: Diverse Strategies and Diverse Student Impacts, Cambridge, MA: Mathematica Policy Research.
   Google Scholar
• Glazerman, S., Levy, D. M., and Myers, D. (2003), “Nonexperimental Versus Experimental Estimates of Earnings Impacts,” The Annals of the American Academy of Political and Social Science, 589, 63–93.
   Web of Science ®Google Scholar
• Gleason, P., Clark, M., Tuttle, C. C., and Dwoyer, E. (2010), The Evaluation of Charter School Impacts: Final Report (NCEE 2010–4029), Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• Hollis, S., and Campbell, F. (1999), “What is Meant by Intention to Treat Analysis? Survey of Published Randomised Controlled Trials,” BMJ, 391, 670–674.
   Google Scholar
• Krueger, A. (1999), “Experimental Estimates of Education Production Functions,” Quarterly Journal of Economics, 104, 497–532.
   Web of Science ®Google Scholar
• McKenzie, D., Gibson, J., and Stillman, S. (2007), How Important Is Selection? Experimental Versus Nonexperimental Measures of Income Gains from Migration, Washington, DC: World Bank.
   Google Scholar
• Murnane, R. J., and Willet, J. B. (2010), Methods Matter: Improving Causal Inference in Educational and Social Science Research, New York: Oxford University Press.
   Google Scholar
• Permutt, T., and Hebel, J. R. (1989), “Simultaneous Equation Estimation in a Clinical Trial of the Effect of Smoking on Birth Weight,” Biometrics, 45, 619–622.
   PubMed Web of Science ®Google Scholar
• Rosenbaum, P. R., and Rubin, D. B. (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects,” Biometrika, 70, 41–45.
   Web of Science ®Google Scholar
• Schochet, P., Burghardt, J., and Glazerman, S. (1999), The National Job Corps Study: Short Term Impacts of Job Corps on Participants' Employment and Related Outcomes, Princeton, NJ: Mathematica Policy Research.
   Google Scholar
• Shadish, W. R., Cook, T. D., and Campbell, D. T. (2002), Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Boston, MA: Houghton Mifflin.
   Google Scholar
• Stecher, B., and Bohrnstedt, G. (2001), “Class-Size Reduction in California: A Story of Hope, Promise, and Unintended Consequences,” Phi Delta Kappan, 82, 670–674.
   Web of Science ®Google Scholar
• Tuttle, C. C., Gleason, P., and Clark, M. (2012), “Using Lotteries to Evaluate Schools of Choice: Evidence from a National Study of Charter Schools,” Economics of Education Review, 31, 237–253.
   Web of Science ®Google Scholar
• Tuttle, C. C., Teh, B., Nichols-Barrer, I., Gill, B. P., and Gleason, P. (2011), Student Characteristics and Achievement in 22 KIPP Middle Schools, Washington, DC: Mathematica Policy Research.
   Google Scholar
• Woodworth, J. L., and Raymond, M. E. (2013), Charter School Growth and Replication, Stanford, CA: CREDO.
   Google Scholar