Adding Design Elements to Improve Time Series Designs: No Child Left Behind as an Example of Causal Pattern-Matching

REFERENCES

• Angrist, J.D., & Pischke, J.S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton, NJ: Princeton University Press.
   Google Scholar
• Belasen, A.R., & Dai, C. (2011). When oceans attack: Using a generalized difference-in-difference technique to assess the impact of hurricanes on localized taxable sales. Unpublished manuscript, Southern Illinois University, Carbondale, IL.
   Google Scholar
• Belasen, A.R., & Polachek, S.W. (2008). How hurricanes affect wages and employment in local Labor Markets. The American Economic Review, 98(2), 49–53.
   Web of Science ®Google Scholar
• Campbell, D.T. (1966). Pattern matching as an essential in distal knowing. In K.R. Hammond (Ed.), The psychology of Egon Brunswik. New York, NY: Holt, Rinehart, & Winston.
   Google Scholar
• Campbell, D.T., & Erlebacher, A. (1970). How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful. Compensatory Education: A National Debate, 3, 185–210.
   Google Scholar
• Center on Education Policy. (2007). Answering the question that matters most: Has student achievement increased since No Child Left Behind? Washington, DC: Author.
   Google Scholar
• Chetty, R., Looney, A., & Kroft, K. (2009). Salience and taxation: Theory and evidence. American Economic Review, 99, 1145–1177.
   Web of Science ®Google Scholar
• Cochran, W.G. (1965). The planning of observational studies of human populations (with discussion). Journal of the Royal Statistical Society. Series A 128, 134–155.
   Web of Science ®Google Scholar
• Cook, T.D. (1985). Post-positivist critical multiplism. In R.L. Shotland & M.M. Mark (Eds.), Social science and social policy (pp. 21–62). Beverly Hills, CA: Sage.
   Google Scholar
• Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design and analysis for field settings. Chicago, IL: Rand McNally.
   Google Scholar
• Cook, T.D., & Shadish, W.R. (1994). Social experiments: Some developments over the past fifteen years. Annual Review of Psychology, 45, 545–579.
   Web of Science ®Google Scholar
• Corrin, W.J., & Cook, T.D. (1998). Design elements of quasi-experiments. In A.J. Reynolds & H.J. Walberg (Eds.), Advances in educational productivity (Vol. 7; pp. 91–112). Greenwich, CT: JAI Press.
   Google Scholar
• Dee, T.S., & Jacob, B. (2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management, 30, 418–446.
   Web of Science ®Google Scholar
• Ewing, B.T., & Kruse, J.B. (2005). Hurricanes and unemployment (Center for Natural Hazards Research). Greenville, NC: East Carolina University.
   Google Scholar
• Finnigan, K.S., & Gross, B. (2007). Do accountability policy sanctions influence teacher motivation? Lessons from Chicago's low performing schools. American Educational Research Journal, 44, 594–630.
   Web of Science ®Google Scholar
• Fisher, R.A. (1925). Statistical methods for research workers. Edinburgh, Scotland: Oliver & Boyd.
   Google Scholar
• Fisher, R.A. (1935). The design of experiments. Edinburgh, Scotland: Oliver & Boyd.
   Google Scholar
• Fordham Foundation. (2005). The state of State math standards. Washington, DC: Author.
   Google Scholar
• Fuller, B., Wright, J., Gesicki, K., & Kang, E. (2007). Gauging growth: How to judge No Child Left Behind. Educational Researcher, 36, 268–278.
   Google Scholar
• Goertz, M.E. (2005). Implementing the No Child Left Behind Act: Challenges for the states. Peabody Journal of Education, 80, 73–89.
   Google Scholar
• Hill, C., Bloom, H., Black, A., & Lipsey, M. (2007). Empirical benchmarks for interpreting effect sizes in research. New York, NY: Manpower Demonstration Research Corporation.
   Google Scholar
• Imbens, G., & Wooldridge, J. (2007). What's new in econometrics (Lecture notes). NBER Summer Institute, Cambridge, MA.
   Google Scholar
• Keigher, A. (2009). Characteristics of public, private, and Bureau of Indian Education elementary and secondary Schools in the United States: Results from the 2007–08 Schools and Staffing Survey (NCES 2009–321). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• Lazer, S. (2004). Innovations in instrumentation and dissemination. In L. Jones& I. Olkin (Eds.), The Nation's report card: Evolution and perspectives (pp. 469–487). Bloomington, IN: Phi Delta Kappa Educational Foundation and American Educational Research Association.
   Google Scholar
• Lechner, M. (2010). The estimation of causal effects by difference-in-difference methods. Foundation and Trends in Econometrics, 4, 165–224.
   Google Scholar
• Lee, J. (2006). Tracking achievement gaps and assessing the impact of NCLB on the gaps: An in-depth look into National and State reading and math outcome trends. Cambridge, MA: Harvard Civil Rights Project.
   Google Scholar
• Mayo, D.G. (1996). Error and the growth of experimental knowledge. Chicago, IL: The University of Chicago Press.
   Google Scholar
• Meyer, B.D. (1995). Natural and quasi-experiments in economics. Journal of Business & Economic Statistics, 13, 151–161.
   Web of Science ®Google Scholar
• Michalopoulos, C., Bloom, H.S., & Hill, C.J. (2004). Can propensity-score methods match the findings from a random assignment evaluation of mandatory welfare-to-work programs? The Review of Economics and Statistics, 86, 156–179.
   Web of Science ®Google Scholar
• Milligan, K., & Stabile, M. (2009). Child benefits, maternal employment, and children's health: Evidence from Canadian child benefit expansions. American Economic Review, 99, 128–132.
   PubMed Web of Science ®Google Scholar
• National Center for Education Statistics. (2009). How the samples of schools and students are selected for the main assessments (State and National). Retrieved from http://nces.ed.gov/nationsreportcard/about/nathow.asp
   Google Scholar
• National Council of Teachers of Mathematics. (2008, January/February). Rise in NAEP math scores coincides with NCTM standards. NCTM News Bulletin, pp. 1–2.
   Google Scholar
• Neal, D., & Schanzenbach, D.W. (2010). Left behind by design: Proficiency counts and test-based accountability. The Review of Economics and Statistics, 92, 263–283.
   Web of Science ®Google Scholar
• Popper, K.R. (1983). Postscript: Vol. 1. Realism and the aim of science. Totowa, NJ: Rowman & Littlefield.
   Google Scholar
• Rogers, W.H. (1993). Regression standard errors in clustered samples. Stata Technical Bulletin, 13, 19–23.
   Google Scholar
• Rosenbaum, P.R. (2005). Observational studies. In B.S. Everitt & D.C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol., pp. 1451–1462). Chichester, UK: Wiley & Sons.
   Google Scholar
• Rosenbaum, P.R. (2009). Observational studies (Edition 2). New York, NY: Springer-Verlag.
   Google Scholar
• Rosenbaum, P.R. (2011). Some approximate evidence factors in observational studies. Journal of the American Statistical Association, 106, 285–295.
   Web of Science ®Google Scholar
• Rosenberg, M.S., Sindelar, P.T., & Hardman, M.L. (2004). Preparing highly qualified teachers for students with emotional or behavioral disorders: The impact of NCLB and IDEA. Behavioral Disorders, 29, 266–278.
   Web of Science ®Google Scholar
• Shadish, W., Cook, T.D., & Campbell, D. (2002). Experimental and quasi-experimental design for generalized causal inference. Boston, MA: Houghton Mifflin.
   Google Scholar
• Smith, T.M., Desimone, L.M., & Ueno, K. (2005). Highly qualified to do what? The relationship between NCLB teacher quality mandates and the use of reform-oriented instruction in middle school mathematics. Educational Evaluation and Policy Analysis, 20, 75–109.
   Web of Science ®Google Scholar
• Solé-Ollé, A., & Sorribas-Navarro, P. (2008). The effects of partisan alignment on the allocation of intergovernmental transfers. Differences-in-differences estimates for Spain. Journal of Public Economics, 92, 2302–2319.
   Web of Science ®Google Scholar
• Stullich, S., Eisner, E., & McCrary, J. (2007). National assessment of Title I: Final report. Washington, DC: Department of Education.
   Google Scholar
• U. S. Department of Education. (2007). Private school participants in Federal program under the No Child Left Behind Act and the Individuals with Disabilities Education Act. Washington, DC: Author.
   Google Scholar
• U. S. Department of Education. (2008). No Child Left Behind Act of 2001, Public Law print of PL 107–110. Retrieved from http://www.ed.gov/policy/elsec/leg/esea02/107-110.pdf.
   Google Scholar
• Wong, V.C. (2011). Games that schools play: Manipulation of the assignment mechanism by schools under No Child Left Behind. Washington, DC: Society for Research on Educational Effectiveness.
   Google Scholar
• Wong, V.C., Cook, T.D., Barnett, W.S., & Jung, K. (2008). An effectiveness based evaluation of five state pre kindergarten programs. Journal of Policy Analysis and Management, 27, 122–154.
   Web of Science ®Google Scholar