Sample Selection in Randomized Experiments: A New Method Using Propensity Score Stratified Sampling

REFERENCES

• Borman , G. D. , Dowling , N. M. and Schneck , C. 2008 . A multi-site cluster randomized field trial of Open Court Reading . Educational Evaluation and Policy Analysis , 30 : 389 – 407 .
   Web of Science ®Google Scholar
• Borman , G. D. , Slavin , R. E. , Cheung , A. , Chamberlain , A. M. , Madden , N. A. and Chambers , B. 2005 . Success for All: First-year results from the National Randomized Field Trial . Educational Evaluation and Policy Analysis , 27 : 1 – 22 .
   Web of Science ®Google Scholar
• Boruch , R. F. 1996 . Randomized experiments for planning and evaluation: A practical guide (Applied Social Research Methods Series, Vol. 44) , Thousand Oaks , CA : Sage .
   Google Scholar
• Cochran , W. G. 1968 . The effectiveness of adjustment by subclassification in removing bias in observational studies . Biometrics , 24 : 295 – 313 .
   PubMed Web of Science ®Google Scholar
• Cook , T. D. 1993 . “ A quasi-sampling theory of the generalization of causal relationships ” . In Understanding causes and generalizing about them: New directions for program evaluation , Edited by: Sechrest , L. and Scott , A. G. 39 – 82 . San Francisco , CA : Jossey-Bass .
   Google Scholar
• Cornfield , J. and Tukey , J. W. 1956 . Average values of mean squares in factorials . The Annals of Mathematical Statistics , 27 : 907 – 949 .
   Google Scholar
• Deming , W. E. and Stephan , F. F. 1940 . On a least squares adjustment of a sampled frequency table when the expected marginal totals are known . The Annals of Mathematical Statistics , 11 : 427 – 444 .
   Google Scholar
• Duncan , G. J. 2008 . When to promote, and when to avoid, a population perspective . Demography , 45 : 763 – 784 .
   PubMed Web of Science ®Google Scholar
• Fienberg , S. E. and Tanur , J. M. 1987 . Experimental and sampling structures: Parallels diverging and meeting . International Statistical Review , 55 : 75 – 96 .
   Web of Science ®Google Scholar
• Groves , R. , Dillman , D. , Eltinge , J. and Little , R. J. A. 2002 . Survey nonresponse , New York , NY : Wiley .
   Google Scholar
• Groves , R. M. , Fowler , F. J. , Couper , M. P. , Lepkowski , J. M. , Singer , E. and Tourangeau , R. 2009 . Survey methodology (2nd ed.) , Hoboken , NJ : Wiley .
   Google Scholar
• Hedges , L. V. and O’Muircheartaigh , C. A. 2011 . Improving generalizations from designed experiments (Working paper) Evanston , IL Northwestern University
   Google Scholar
• Imai , K. , King , G. and Stuart , E. A. 2008 . Misunderstandings among experimentalists and observationalists about causal inference . Journal of the Royal Statistical Society, Series A (Statistics in Society) , 171 : 481 – 502 .
   Web of Science ®Google Scholar
• Institute for Education Sciences . 2011 . Funding opportunities: Search funded research and contracts . Retrieved from http://ies.ed.gov/funding/grantsearch
   Google Scholar
• Kalton , G. 1968 . Standardization: A technique to control for extraneous variables . Applied Statistics , 17 : 118 – 136 .
   Web of Science ®Google Scholar
• Kish , L. 1987 . Statistical design for research , New York , NY : Wiley . (Wiley Series in Probability and Mathematical Statistics)
   Google Scholar
• Kitagawa , E. M. 1964 . Standardized comparisons in population research . Demography , 1 : 296 – 315 .
   Web of Science ®Google Scholar
• Kruskal , W. H. and Mosteller , F. 1979 . Representative sampling. III. The current statistical literature . International Statistical Review , 47 : 245 – 265 .
   Web of Science ®Google Scholar
• Little , R. J. A. and Rubin , D. B. 2002 . Statistical analysis with missing data , New York , NY : Wiley . (Wiley Series in Probability and Mathematical Statistics)
   Google Scholar
• Lohr , S. 1999 . Sampling: Design and analysis. , New York , NY : Duxbury .
   Google Scholar
• National Center for Education Statistics . 2011 . Common core of data. Retrieved from http://nces.ed.gov/ccd
   Google Scholar
• National Reading Panel . 2000 . Report of the National Reading Panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. , Washington , DC : National Institute of Child Health and Human Development .
   Google Scholar
• Neyman , J. 1934 . On the two different aspects of the representative method: The method of stratified sampling and of purposive selection . Journal of the Royal Statistical Society , 97 : 558 – 625 .
   Google Scholar
• Oaxaca , R. 1973 . Male–female wage differentials in urban labor markets . International Economic Review , 14 : 693 – 709 .
   Google Scholar
• Olsen , R. B. , Orr , L. L. , Bell , S. H. and Stuart , E. A. 2012 . External validity in policy evaluations that choose sites purposively . Journal of Policy Analysis and Management. , Advance online publication. doi:10.1002/pam.21660
   Web of Science ®Google Scholar
• Raudenbush , S. W. and Bryk , A. S. 2002 . Hierarchical linear models (2nd ed.) , Thousand Oaks , CA : Sage .
   Google Scholar
• Rosenbaum , P. R. and Rubin , D. B. 1983 . The central role of the propensity score in observational studies for causal effects . Biometrika , 70 : 41 – 55 . doi: 10.1093/biomet/70.1.41
   Web of Science ®Google Scholar
• Rosenbaum , P. R. and Rubin , D. B. 1984 . Reducing bias in observational studies using subclassification on the propensity score . Journal of the American Statistical Association , 79 : 516 – 524 . Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/20199225
   Web of Science ®Google Scholar
• Rosenberg , P. 1962 . Test factor standardization as a method of interpretation . Social Forces , 41 : 53 – 61 .
   Web of Science ®Google Scholar
• Rubin , D. B. 1976 . Inference and missing data . Biometrika , 63 : 581 – 592 .
   Web of Science ®Google Scholar
• Schneider , B. and McDonald , S. K. 2007 . Scale-up in education: Ideas in principle (Vol. 1) , Lanham , MD : Rowman & Littlefield .
   Google Scholar
• Shadish , W. R. , Cook , T. D. and Campbell , D. T. 2002 . Experimental and quasi-experimental designs for generalized causal inference. , Boston , MA : Houghton-Mifflin .
   Google Scholar
• Slavin , R. E. and Lake , C. 2007 . Effective programs in elementary mathematics: A best- evidence synthesis. , Baltimore , MD : Johns Hopkins University .
   Google Scholar
• Spybrook , J. and Raudenbush , S. W. 2009 . An examination of the precision and technical adequacy of the first wave of group randomized trials funded by the Institute of Education Sciences . Educational Evaluation and Policy Analysis , 31 : 298 – 318 .
   Web of Science ®Google Scholar
• Stuart , E. A. , Cole , S. R. , Bradshaw , C. P. and Leaf , P. J. 2011 . The use of propensity scores to assess the generalizability of results from randomized trials . Journal of the Royal Statistical Society, Series A , Part 2 : 369 – 386 .
   Web of Science ®Google Scholar
• Tipton , E. 2013 . Improving generalizations from experiments using propensity score subclassification: Assumptions, properties, and contexts . Journal of Educational and Behavioral Statistics , 38 : 239 – 266 .
   Web of Science ®Google Scholar
• Tipton , E. in press . Stratified sampling using cluster analysis: A balanced-sampling strategy for improved generalizations from experiments. Evaluation review
   Google Scholar
• Valliant , R. , Dorfman , A. H. and Royall , R. M. 2000 . Finite population sampling and inference: A prediction approach , New York , NY : Wiley . (Wiley Series in Probability and Statistics)
   Google Scholar
• What Works Clearinghouse . 2007 . Elementary school math. Retrieved from http://ies.ed.gov/ncee/wwc/reports/elementary_math/topic/references.asp
   Google Scholar
• What Works Clearinghouse . 2010 . Intervention: Everyday mathematics. Retrieved from http://ies.ed.gov/ncee/wwc/reports/elementary_math/eday_math/
   Google Scholar