Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System

References

• Anand, P., Mizala, A., and Repetto, A. (2009), “Using School Scholarships to Estimate the Effect of Government Subsidized Private Education on Academic Achievement in Chile,” Economics of Education Review, 28, 370–381.
   Web of Science ®Google Scholar
• Arpino, B., and Mealli, F. (2011), “The Specification of the Propensity Score in Multilevel Observational Studies,” Computational Statistics & Data Analysis, 55, 1770–1780.
   Web of Science ®Google Scholar
• Bachelet, M. (2013), “Programa de Gobierno MICHELLE BACHELET 2014–2018,” available at http://michellebachelet.cl/wp-content/uploads/2013/10/Reforma-Educacional-14-21.pdf.
   Google Scholar
• Bellman, R. (1957), Dynamic Programming, Princeton, NJ: Princeton University Press.
   Google Scholar
• Bertsekas, D. P. (2005), Dynamic Programming and Optimal Control, (Vol. I), Belmont, MA: Athena Scientific.
   Google Scholar
• Borman, G. D., Slavin, R. E., Cheung, A. C., Chamberlain, A. M., Madden, N. A., and Chambers, B. (2007), “Final Reading Outcomes of the National Randomized Field Trial of Success for All,” American Educational Research Journal, 44, 701–731.
   Web of Science ®Google Scholar
• Crump, R. K., Hotz, V. J., Imbens, G. W., and Mitnik, O. A. (2009), “Dealing With Limited Overlap in Estimation of Average Treatment Effects,” Biometrika, 96, 187–199.
   Web of Science ®Google Scholar
• Dehejia, R., and Wahba, S. (1999), “Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs,” Journal of the American Statistical Association, 94, 1053–1062.
   Web of Science ®Google Scholar
• Editorial Projects in Education Research Center (2004), “Issues A–Z: Comprehensive School Reform,” available at http://www.edweek.org/ew/issues/comprehensive-school-reform.
   Google Scholar
• Feller, A., and Gelman, A. (2015), “Hierarchical Models for Causal Effects,” in Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource. Available at https://protect-us.mimecast.com/s/oXA1BRUw8MJ8Hd?domain=onlinelibrary.wiley.com.
   Google Scholar
• Hansen, B. B., and Bowers, J. (2009), “Attributing Effects to A Clustered Randomized Get-Out-The-Vote Campaign,” Journal of the American Statistical Association, 104, 873–885.
   Web of Science ®Google Scholar
• Hansen, B. B., Rosenbaum, P. R., and Small, D. S. (2014), “Clustered Treatment Assignments and Sensitivity to Unmeasured Biases in Observational Studies,” Journal of the American Statistical Association, 109, 133–144.
   Web of Science ®Google Scholar
• Hartman, E., Grieve, R., Ramsahai, R., and Sekhon, J. S. (2015), “From SATE to PATT: Combining Experimental With Observational Studies to Estimate Population Treatment Effects,” Journal of the Royal Statistical Society, Series A, forthcoming.
   Google Scholar
• Hodges, J. L., and Lehmann, E. (1963), “Estimates of Location Based on Ranks,” The Annals of Mathematical Statistics, 34, 598–611.
   Google Scholar
• Hong, G., and Raudenbush, S. W. (2006), “Evaluating Kindergarten Retention Policy: A Case of Study of Causal Inference for Multilevel Data,” Journal of the American Statistical Association, 101, 901– 910.
   Web of Science ®Google Scholar
• Hsieh, C.-T., and Urquiola, M. (2006), “The Effects of Generalized School Choice on Achievement and Stratification: Evidence from Chile’s Voucher Program,” Journal of Public Economics, 90, 1477–1503.
   Web of Science ®Google Scholar
• Hsu, J. Y., Zubizarreta, J. R., Small, D. S., and Rosenbaum, P. R. (2015), “Strong Control of the Family-Wise Error Rate in Observational Studiesthat Discover Effect Modification by Exploratory Methods,” Biometrika, 102, 767–782.
   Google Scholar
• Imbens, G. W., and Rubin, D. B. (2015), Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction, Cambridge, UK: Cambridge University Press.
   Google Scholar
• Kilcioglu, C., and Zubizarreta, J. R. (2016), “Maximizing the Information Content of a Balanced Matched Sample With a Flexible Matching Ratio: A Study ofthe Economics of Green Buildings,” Annals of Applied Statistics, 10, 1997–2020.
   Google Scholar
• Lara, B., Mizala, A., and Repetto, A. (2011), “The Effectiveness of Private Voucher Education: Evidence From Structural School Switches,” Educational Evaluation and Policy Analysis, 33, 119–137.
   Web of Science ®Google Scholar
• Lee, V. E., and Bryk, A. S. (1989), “A Multilevel Model of The Social Distribution of High School Achievement,” Sociology of Education, 62, 172–192.
   Web of Science ®Google Scholar
• Li, F., Zaslavsky, A. M., and Landrum, M. B. (2013), “Propensity Score Weighting With Multilevel Data,” Statistics in Medicine, 32, 3373–3387.
   PubMed Web of Science ®Google Scholar
• Lipsey, M. W. (1990), Design Sensitivity: StatisticalPower for Experimental Research (Vol. 19), Newbury Park, CA: Sage.
   Google Scholar
• Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., Roberts, M., Anthony, K. S., and Busick, M. D. (2012), “Translating the Statistical Representation of the Effects of Education Interventions into MoreReadily Interpretable Forms,” Technical Report 2013-3000, National Center for Special Education Research, Washington, D.C.
   Google Scholar
• Lu, B., Greevy, R., Xu, X., and Beck, C. (2011), “Optimal Nonbipartite Matching and its Statistical Applications,” The American Statistician, 65, 21–30.
   PubMed Web of Science ®Google Scholar
• McEwan, P. J. (2001), “The Effectiveness of Public, Catholic, and Non-Religious Private Schools in Chile’s Voucher System,” Education Economics, 9, 183–219.
   Google Scholar
• MINEDUC (2015a), “Estadísticas de la Educación 2014,” available at http://www.comunidadescolar.cl/documentacion/LeyInclusionEscolar/presentacion_sostenedores.pdf.
   Google Scholar
• ——— (2015b), “Ley de Inclusión Escolar,” available at http://centroestudios.mineduc.cl/tp_enlaces/portales/tp5996f8b7cm96/uploadImg/File/Estadisticas/Anuario_2014.pdf.
   Google Scholar
• Mizala, A., and Romaguera, P. (2001), “Factors Explaining Secondary Education Outcomes in Chile,” El Trimestre Economico, 272, 515–549.
   Google Scholar
• Neyman, J. (1923), “On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9,” Statistical Science, 5, 465–472. (Trans. Dorota M. Dabrowska and Terence P. Speed (1990)).
   Google Scholar
• Pimentel, S. D., Kelz, R. R., Silber, J. H., and Rosenbaum, P. R. (2015), “Large, Sparse Optimal Matching With Refined Covariate Balance in an Observational Study of the Health Outcomes Produced by New Surgeons,” Journal of the American Statistical Association, 110, 515–527.
   PubMed Web of Science ®Google Scholar
• Rosenbaum, P. R. (1987), “Sensitivity Analysis for Certain Permutation Inferences in Matched Observational Studies,” Biometrika, 74, 13–26.
   Web of Science ®Google Scholar
• ——— (1989), “Optimal Matching for Observational Studies,” Journal of the American Statistical Association, 84, 1024–1032.
   Web of Science ®Google Scholar
• ——— (2002), Observational Studies (2nd ed.), New York: Springer.
   Google Scholar
• ——— (2003), “Exact Confidence Intervals for Nonconstant Effects by Inverting the Signed Rank Test,” The American Statistician, 57, 132–138.
   Web of Science ®Google Scholar
• ——— (2008), “Testing Hypotheses in Order,” Biometrika, 95, 248–252.
   Web of Science ®Google Scholar
• ——— (2010), Design of Observational Studies, New York: Springer-Verlag.
   Google Scholar
• ——— (2012a), “Optimal Matching of an Optimally Chosen Subset in Observational Studies,” Journal of Computational and Graphical Statistics, 21, 57–71.
   Web of Science ®Google Scholar
• ——— (2012b), “Testing One Hypothesis Twice in Observational Studies,” Biometrika, 99, 763–774.
   Web of Science ®Google Scholar
• Rosenbaum, P. R., Ross, R. N., and Silber, J. H. (2007), “Minimum Distance Matched Sampling with Fine Balance in an Observational Study of Treatment for Ovarian Cancer,” Journal of the American Statistical Association, 102, 75–83.
   Web of Science ®Google Scholar
• Rosenbaum, P. R., and Rubin, D. B. (1983), “The Central Role of Propensity Scores in Observational Studies for Causal Effects,” Biometrika, 76, 41–55.
   Web of Science ®Google Scholar
• ——— (1985), “Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score,” The American Statistician, 39, 33–38.
   Web of Science ®Google Scholar
• Rosenbaum, P. R., and Silber, J. H. (2009), “Sensitivity Analysis for Equivalence and Difference in an Observational Study of Neonatal Intensive Care Units,” Journal of the American Statistical Association, 104, 501–511.
   Web of Science ®Google Scholar
• Rubin, D. B. (1974), “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies,” Journal of Educational Psychology, 6, 688–701.
   Web of Science ®Google Scholar
• ——— (1986), “Which Ifs Have Causal Answers,” Journal of the American Statistical Association, 81, 961–962.
   Web of Science ®Google Scholar
• Sapelli, C., and Vial, B. (2002), “The Performance of Private and Public Schools in the Chilean Voucher System,” Cuadernos De Economia, 39, 423–454.
   Google Scholar
• ——— (2005), “Private vs Public Voucher Schools in Chile: New Evidence on Efficiency and Peer Effects,” Working Paper 289, Catholic University of Chile, Instituto de Economia.
   Google Scholar
• Shadish, W. R., Cook, T. D., and Campbell, D. T. (2002), Experimental and Quasi-Experimental Designs for GeneralizedCausal Inference, Boston, MA: Houghton Mifflin.
   Google Scholar
• Silber, J. H., Rosenbaum, P. R., Kelz, R. R., Gaskin, D. J., Ludwig, J. M., Ross, R. N., Niknam, B. A., Hill, A., Wang, M., Even-Shoshan, O.et. al, (2015), “Examining Causes of Racial Disparities in General Surgical Mortality,” MedicalCare, 53, 619–629.
   PubMed Web of Science ®Google Scholar
• Small, D., and Rosenbaum, P. R. (2008), “War and Wages: The Strength of Instrumental Variables and Their Sensitivity to Unobserved Biases,” Journal of the American Statistical Association, 103, 924–933.
   Web of Science ®Google Scholar
• Small, D. S., Have, T. R. T., and Rosenbaum, P. R. (2008), “Randomization Inference in a Group–Randomized Trial of Treatments for Depression: Covariate Adjustment, Noncompliance, and Quantile Effects,” Journal of the American Statistical Association, 103, 271–279.
   Web of Science ®Google Scholar
• Stuart, E. A. (2010), “Matching Methods for Causal Inference: A Review and a Look Forward,” Statistical Science, 25, 1–21.
   PubMed Web of Science ®Google Scholar
• Stuart, E. A., Cole, S. R., Bradshaw, C. P., and Leaf, P. J. (2011), “The Use of Propensity Scores to Assess the Generalizability of Results From Randomized Trials,” Journal of the Royal Statistical Society, Series A, 174, 369–386.
   Google Scholar
• The Economist (2011), “The Fraught Politics of the Classroom,” available at http://www.economist.com/node/21534785.
   Google Scholar
• The New York Times (2011), “With Kiss-Ins and Dances, Young Chileans Push for Reform,” available at http://www.nytimes.com /2011/08/05/world/americas/05chile.html.
   Google Scholar
• Zubizarreta, J. R. (2012), “Using Mixed Integer Programming for Matching in an Observational Study of Kidney Failure After Surgery,” Journal of the American Statistical Association, 107, 1360– 1371.
   Web of Science ®Google Scholar
• Zubizarreta, J. R., and Kilcioglu, C. (2016), Designmatch: Construction of Optimally Matched Samples for Randomized Experiments and ObservationalStudies That are Balanced by Design, R Package Version 0.1.1.
   Google Scholar
• Zubizarreta, J. R., Paredes, R. D., and Rosenbaum, P. R. (2014), “Matching for Balance, Pairing for Heterogeneity in an Observational Study of the Effectiveness of For-Profit and Not-for-Profit High Schools in Chile,” Annals of Applied Statistics, 8, 204–231.
   Web of Science ®Google Scholar
• Zubizarreta, J. R., Reinke, C. E., Kelz, R. R., Silber, J. H., and Rosenbaum, P. R. (2011), “Matching for Several Sparse Nominal Variables in a Case-Control Study of Readmission Following Surgery,” The American Statistician, 65, 229–238.
   PubMed Web of Science ®Google Scholar