Defining Program Effects: A Distribution-Based Perspective

References

• American Educational Research Association (2015), “AERA Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs,” Educational Researcher, 44, 448–452.
   Web of Science ®Google Scholar
• American Statistical Association (2014), ASA Statement on Using Value-Added Models for Educational Assessment, Alexandria, VA: American Statistical Association. Available at http://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf
   Google Scholar
• Antoniou, P., and Kyriakides, L. (2011), “The Impact of a Dynamic Approach to Professional Development on Teacher Instruction and Student Learning: Results From an Experimental Study,” School Effectiveness and School Improvement, 22, 291–311.
   Web of Science ®Google Scholar
• Ballou, D., Sanders, W., and Wright, P. (2004), “Controlling for Student Background in Value-Added Assessment of Teachers,” Journal of Educational and Behavioral Statistics, 29, 37–65.
   Web of Science ®Google Scholar
• Barrett, N., Butler, J., and Toma, E. F. (2012), “Do Less Effective Teachers Choose Professional Development Does it Matter?” Evaluation Review, 36, 346–374.
   PubMed Web of Science ®Google Scholar
• Biancarosa, G., Bryk, A. S., and Dexter, E. R. (2010), “Assessing the Value-Added Effects of Literacy Collaborative Professional Development on Student Learning,” The Elementary School Journal, 111, 7–34.
   Web of Science ®Google Scholar
• Broatch, J., and Lohr, S. (2012), “Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes,” Journal of Educational and Behavioral Statistics, 37, 256–277.
   Web of Science ®Google Scholar
• Carlson, D., Borman, G. D., and Robinson, M. (2011), “A Multistate District-Level Cluster Randomized Trial of the Impact of Data-Driven Reform on Reading and Mathematics Achievement,” Educational Evaluation and Policy Analysis, 33, 378–398.
   Web of Science ®Google Scholar
• Dimitrov, D. M. (2009), “Intermediate Trends in Math and Science Partnership-Related Changes in Student Achievement with Management Information System Data,” Journal of Educational Research & Policy Studies, 9, 97–138.
   Google Scholar
• Everson, K. C. (2017), “Value-Added Modeling and Educational Accountability: Are We Answering the Real Questions?” Review of Educational Research, 87, 35–70.
   Web of Science ®Google Scholar
• Foster, J. M., Toma, E. F., and Troske, S. P. (2013), “Does Teacher Professional Development Improve Math and Science Outcomes and is it Cost Effective?” Journal of Education Finance, 38, 255–275.
   Google Scholar
• Glazerman, S., Isenberg, E., Dolfin, S., Bleeker, M., Johnson, A., Grider, M., Jacobus, M., et al. (2010), “Impacts of Comprehensive Teacher Induction: Final Results From a Randomized Controlled Study,” (NCEE 2010-4028). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• Goldhaber, D. (2013), What Do Value-Added Measures of Teacher Preparation Programs Tell Us? Stanford, CA: Carnegie Knowledge Network. Available at http://www.carnegieknowledgenetwork.org/briefs/teacher_prep/
   Google Scholar
• Goldhaber, D., Liddle, S., and Theobald, R. (2013), “The Gateway to the Profession: Assessing Teacher Preparation Programs Based on Student Achievement,” Economics of Education Review, 34, 29–44.
   Web of Science ®Google Scholar
• Harris, D. N., and Sass, T. R. (2011), “Teacher Training, Teacher Quality and Student Achievement,” Journal of Public Economics, 95, 798–812.
   Web of Science ®Google Scholar
• Hedges, L. V., and Borenstein, M. (2014), “Conditional Optimal Design in Three-and Four-Level Experiments,” Journal of Educational and Behavioral Statistics, 39, 257–281.
   Web of Science ®Google Scholar
• Heller, J. I., Daehler, K. R., Wong, N., Shinohara, M., and Miratrix, L. W. (2012), “Differential Effects of Three Professional Development Models on Teacher Knowledge and Student Achievement in Elementary Science,” Journal of Research in Science Teaching, 49, 333– 362.
   Web of Science ®Google Scholar
• Johnson, C. C., and Fargo, J. D. (2014), “A Study of the Impact of Transformative Professional Development on Hispanic Student Performance on State Mandated Assessments of Science in Elementary School,” Journal of Science Teacher Education, 25, 845–859.
   Google Scholar
• La Paz, S. D., Malkus, N., Monte-Sano, C., and Montanaro, E. (2011), “Evaluating American History Teachers’ Professional Development: Effects on Student Learning,” Theory & Research in Social Education, 39, 494–540.
   Google Scholar
• Lockwood, J., and McCaffrey, D. F. (2007), “Controlling for Individual Heterogeneity in Longitudinal Models, With Applications to Student Achievement,” Electronic Journal of Statistics, 1, 223–252.
   Web of Science ®Google Scholar
• Lockwood, J., McCaffrey, D. F., Mariano, L. T., and Setodji, C. (2007), “Bayesian Methods for Scalable Multivariate Value-Added Assessment,” Journal of Educational and Behavioral Statistics, 32, 125–150.
   Web of Science ®Google Scholar
• Lohr, S. L. (2015), “Red Beads and Profound Knowledge: Deming and Quality of Education,” Education Policy Analysis Archives, 23, 1–21.
   Google Scholar
• Louis, T. A. (1984), “Estimating a Population of Parameter Values Using Bayes and Empirical Bayes Methods,” Journal of the American Statistical Association, 79, 393–398.
   Web of Science ®Google Scholar
• Mariano, L. T., McCaffrey, D. F., and Lockwood, J. (2010), “A Model for Teacher Effects From Longitudinal Data Without Assuming Vertical Scaling,” Journal of Educational and Behavioral Statistics, 35, 253–279.
   Web of Science ®Google Scholar
• McCaffrey, D. F., Lockwood, J., Koretz, D., Louis, T. A., and Hamilton, L. (2004), “Models for Value-Added Modeling of Teacher Effects,” Journal of Educational and Behavioral Statistics, 29, 67–101.
   PubMed Web of Science ®Google Scholar
• McCaffrey, D. F., Lockwood, J., Koretz, D. M., and Hamilton, L. S. (2003), Evaluating Value-Added Models for Teacher Accountability, Santa Monica, CA: RAND Corporation. Available at http://www.rand.org /content/dam/rand/pubs/monographs/2004/RAND_MG158.pdf
   Google Scholar
• Morris, J. S. (2002), “The BLUPs are Not Best When it Comes to Bootstrapping,” Statistics & Probability Letters, 56, 425–430.
   Web of Science ®Google Scholar
• Penuel, W. R., Gallagher, L. P., and Moorthy, S. (2011), “Preparing Teachers to Design Sequences of Instruction in Earth Systems Science: A Comparison of Three Professional Development Programs,” American Educational Research Journal, 48, 996–1025.
   Web of Science ®Google Scholar
• Raudenbush, S. W., and Bryk, A. S. (2002), Hierarchical Linear Models: Applications and Data Analysis Methods (2nd ed.), Thousand Oaks, CA: Sage.
   Google Scholar
• Robinson, G. K. (1991), “That BLUP is a Good Thing: The Estimation of Random Effects,” Statistical Science, 6, 15–32.
   Google Scholar
• Sanders, W. L., Saxton, A. M., and Horn, S. P. (1997), “The Tennessee Value-Added Assessment System: A Quantitative Outcomes-Based Approach to Educational Assessment,” in Grading Teachers, Grading Schools: Is Student Achievement a Valid Evaluation Measure, ed. J. Millman, Thousand Oaks, CA: Corwin Press, Inc., pp. 137–162.
   Google Scholar
• Shen, W., and Louis, T. A. (1998), “Triple-Goal Estimates in Two-Stage Hierarchical Models,” Journal of the Royal Statistical Society, Series B, 60, 455–471.
   Google Scholar
• Sleeter, C. (2014), “Toward Teacher Education Research that Informs Policy,” Educational Researcher, 43, 146–153.
   Web of Science ®Google Scholar
• Stroup, W. W. (2013), Generalized Linear Mixed Models: Modern Concepts, Methods and Applications, Boca Raton, FL: CRC Press.
   Google Scholar
• Supovitz, J. A., and Turner, H. M. (2000), “The Effects of Professional Development on Science Teaching Practices and Classroom Culture,” Journal of Research in Science Teaching, 37, 963–980.
   Web of Science ®Google Scholar
• Tate, R. L. (2004), “A Cautionary Note on Shrinkage Estimates of School and Teacher Effects,” Florida Journal of Educational Research, 42, 1–21.
   Google Scholar
• Tatto, M. T., Savage, C., Liao, W., Marshall, S. L., Goldblatt, P., and Contreras, L. M. (2016), “The Emergence of High-Stakes Accountability Policies in Teacher Preparation: An Examination of the U.S. Department of Education’s Proposed Regulations,” Education Policy Analysis Archives, 24, 1–57.
   Google Scholar
• Wayne, A. J., Yoon, K. S., Zhu, P., Cronen, S., and Garet, M. S. (2008), “Experimenting With Teacher Professional Development: Motives and Methods,” Educational Researcher, 37, 469–479.
   Web of Science ®Google Scholar
• Wright, S. P., White, J. T., Sanders, W. L., and Rivers, J. C. (2010), “SAS® EVAAS® for K-12 Statistical Models,” SAS White Paper, SAS Institute, Inc.
   Google Scholar