Evaluating Dosage Effects in a Social-Emotional Skills Training Program for Children: An Application of Generalized Propensity Scores

REFERENCES

• Abadie, A., & Imbens, G. (2002). Simple and bias-corrected matching estimators for average treatment effects (Technical Working Paper 283). Retrieved from http://www.nber.org/papers/t0283.pdf?new_window=1
   Google Scholar
• Alkon, A., Ramler, M., & MacLennan, K. (2003). Evaluation of mental health consultation in child care centers. Early Childhood Education Journal, 31, 91–99. doi:10.1023/B:ECEJ.0000005307.00142.3c
   Google Scholar
• Andrade, A., Lambert, E. W., & Bickman, L. (2000). Dose effect in child psychotherapy: Outcomes associated with negligible treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 161–168. doi:10.1097/00004583-200002000-00014
   PubMed Web of Science ®Google Scholar
• Angrist, J. D. (2006). Instrumental variables methods in experimental criminological research: What, why, and how. Journal of Experimental Criminology, 2, 23–44. doi:10.1007/s11292-005-5126-x
   Google Scholar
• Barnow, B. S., Cain, G. G., & Goldberger, A. S. (1980). Issues in the analysis of selectivity bias. Evaluation Studies, 5, 42–59. Retrieved from http://www.eric.ed.gov/PDFS/
   Google Scholar
• Behrman, J. R., Cheng, Y. M., & Todd, P. E. (2004). Evaluating preschool programs when length of exposure to the program varies: A nonparametric approach. Review of Economics and Statistics, 86, 108–132. doi:10.1162/003465304323023714
   Web of Science ®Google Scholar
• Bickman, L., Andrade, A. R., & Lambert, E. W. (2002). Dose response in child and adolescent mental health services. Mental Health Services Research, 4, 57–70. doi:10.1023/A:1015210332175 ED202979.pdf
   PubMedGoogle Scholar
• Bross, I. D. J. (1966). Spurious effects from an extraneous variable. Journal of Chronic Disease, 19, 637–647. doi:10.1016/0021-9681(66)90062-2
   PubMedGoogle Scholar
• Bross, I. D. J. (1967). Pertinency of an extraneous variable. Journal of Chronic Disease, 20, 487–495. doi:10.1016/0021-9681(67)90080-X
   PubMedGoogle Scholar
• Brumback, B. A., Hernan, M. A., Haneuse, S. J. P. A., & Robins, J. (2004). Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Statistics in Medicine, 23, 749–764. doi:10.1002/sim.1657
   PubMed Web of Science ®Google Scholar
• Cochran, W. G. (1983). Planning and analysis of observational studies. New York, NY: Wiley. doi:10.1002/9780470316542
   Google Scholar
• Cook, T., & Campbell, D. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.
   Google Scholar
• Cornfield, J., Haenszel, W., Hammond, E. C., Lilienfeld, A. M., Shimkin, M. B., & Wynder, E. L. (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. Journal of the National Cancer Institute, 22, 137–203. doi:10.1093/ije/dyp289
   Web of Science ®Google Scholar
• Crump, R. K., Hotz, V. J., Imbens, G. W., & Mitnik, O. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96, 187–199. doi:10.1093/biomet/asn055
   Web of Science ®Google Scholar
• Dawid, A. P. (1979). Conditional independence in statistical theory. Journal of the Royal Statistical Society: Series B (Methodological), 41, 1–31. Retrieved from http://www.jstor.org/stable/2984718
   Web of Science ®Google Scholar
• Dehejia, R. H., & Wahba, S. (1999). Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American Statistical Association, 94, 1053–1062. Retrieved from http://www.jstor.org/stable/2669919
   Web of Science ®Google Scholar
• Drake, C. (1993). Effects of misspecification of the propensity score on estimators of treatment effect. Biometrics, 49, 1231–1236. doi:10.2307/2532266
   Web of Science ®Google Scholar
• Fitzgerald, J., Gottschalk, P., & Moffitt, R. (1998). An analysis of sample attrition in panel data: The Michigan Panel Study of Income Dynamics. Journal of Human Resources, 33, 251–299. doi:10.2307/146433
   Web of Science ®Google Scholar
• Flores, C. A., Flores-Lagunes, A., Gonzalez, A., & Neumann, T. C. (2010). Estimating the effects of length of exposure to instruction in a training program: The case of Job Corps. Retrieved from http://ftp.iza.org/dp2846.pdf
   Google Scholar
• Foster, E. M. (2000). Is more better than less? An analysis of children's mental health services. Health Services Research, 35, 1135–1158. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1089167/pdf/hsresearch00014-0080.pdf
   PubMed Web of Science ®Google Scholar
• Foster, E. M. (2003). Propensity score matching an illustrative analysis of dose response.Medical Care, 41, 1183–1192. doi:10.1097/01.MLR.0000089629.62884.22
   PubMed Web of Science ®Google Scholar
• Fraser, M. W., Guo, S. Y., Ellis, A. R., Day, S. H., Li., J. L., & Wike, T. L. (2009). Social and character development in elementary school: Effects from a controlled trial. Unpublished manuscript, School of Social Work, University of North Carolina, Chapel Hill. Retrieved from http://sowkweb.usc.edu/download/research/social-and-character-development-elementary-school.pdf
   Google Scholar
• Fraser, M. W., Guo, S. Y., Ellis, A. R., Thompson, A. M., Wike, T. L., & Li., J. L. (2011). Outcome studies of social, behavioral, and educational interventions: Emerging issues and challenges. Research on Social Work Practice, 21, 619–635. doi:10.1177/1049731511406136
   Web of Science ®Google Scholar
• Guo, S., & Fraser, M. W. (2015). Propensity score analysis: Statistical methods and applications (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Harada, M. (2012). Generalized sensitivity analysis.Retrieved from https://files.nyu.edu/mh166/public/docs/quick_guide_gsa.pdf
   Google Scholar
• Heckman, J. J., & Hotz, V. J. (1989). Choosing among alternative nonexperimental methods for estimating the impact of social programs: The case of manpower training. Journal of the American Statistical Association, 84, 862–874. Retrieved from http://www.jstor.org/stable/2290059
   Web of Science ®Google Scholar
• Heckman, J. J., Ichimura, H., Petra, E., & Todd, P. E. (1997). Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. Review of Economic Studies, 64, 605–654. doi:10.2307/2971733
   Web of Science ®Google Scholar
• Hirano, K., & Imbens, G. W. (2004). The propensity score with continuous treatments. In A. Gelman & X. L. Meng (Eds.), Applied Bayesian modeling and causal inference from incomplete-data perspectives (pp. 73–84). New York, NY: John Wiley.
   Google Scholar
• Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15, 199–236. doi:10.1093/pan/mpl013
   Web of Science ®Google Scholar
• Howard, K. I., Kopta, S. M., Krause, M. S., & Orlinsky, D. E. (1986). The dose–effect relationship in psychotherapy. American Psychologist, 41, 159–164. doi:10.1037/0003-066X.41.2.159
   PubMed Web of Science ®Google Scholar
• Ichino, A., Mealli, F., & Nannicini, T. (2008). From temporary help jobs to permanent employment: What can we learn from matching estimators and their sensitivity?Journal of Applied Econometrics, 23, 305–327. doi:10.1002/jae.998
   Web of Science ®Google Scholar
• Imai, K., & Van Dyk, D. A. (2004). Causal inference with general treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association, 99, 854–866. doi:10.1198/016214504000001187
   Web of Science ®Google Scholar
• Imbens, G. W. (2000). The role of propensity score in estimating dose–response functions.Biometrika, 87, 706–710. doi:10.1093/biomet/87.3.706
   Web of Science ®Google Scholar
• Imbens, G. W. (2003). Sensitivity to exogeneity assumptions in program evaluation. American Economic Review, 93, 126–132. doi:10.1257/000282803321946921
   Web of Science ®Google Scholar
• Jato, M. N., Simbakalia, C., Tarasevich, J. M., Awasum., D. N., Kihinga, C. N. B., & Ngirwamungu, E. (1999). The impact of multimedia family planning promotion on the contraceptive behavior of women in Tanzania. International Family Planning Perspectives, 25, 60–67. doi:10.2307/2991943
   Google Scholar
• Joffe, M. M., & Rosenbaum, P. R. (1999). Invited commentary: Propensity scores. American Journal of Epidemiology, 150, 327–333. doi:10.1093/oxfordjournals.aje.a010011
   PubMed Web of Science ®Google Scholar
• Kiess, H. O. (2002). Statistical concepts for the behavioral sciences (3rd ed.). Boston, MA: Allyn & Bacon.
   Google Scholar
• King, G., & Zeng, L. (2006). The dangers of extreme counterfactuals. Political Analysis, 14, 131–159. doi:10.1093/pan/mpj004
   Web of Science ®Google Scholar
• King, G., & Zeng, L. (2007). When can history be our guide? The pitfalls of counterfactual inference. International Studies Quarterly, 51, 183–210. doi:10.1111/j.1468-2478.2007.00445.x
   Web of Science ®Google Scholar
• Kluve, J., Schneider, H., Uhlendorff, A., & Zhao, Z. (2012). Evaluating continuous training programmes by using the generalized propensity score. Journal of the Royal Statistical Society, 175, 587–617. doi:10.1111/j.1467-985X.2011.01000.x
   Google Scholar
• Lechner, M. (1999). Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. Retrieved from http://ideas.repec.org/p/iza/izadps/dp91.html
   Google Scholar
• Lechner, M., & Melly, B. (2010). Partial identification of wage effects of training programs. Retrieved from http://www.brown.edu/Departments/Economics/Papers/2010/2010-8_paper.pdf
   Google Scholar
• Lin, D. Y., Psaty, B. M., & Kronmal, R. A. (1998). Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics, 54, 948–963. doi:10.2307/2533848
   PubMed Web of Science ®Google Scholar
• Lochman, J. E., Boxmeyer, C., Powell, N., Roth, D. L., & Windle, M. (2006). Masked intervention effects: Analytic methods for addressing low dosage of intervention. New Directions for Evaluation, 110, 19–32. doi:10.1002/ev.184
   Google Scholar
• Lu, B., Zanutto, E., Hornik, R., & Rosenbaum, P. R. (2001). Matching with doses in an observational study of a media campaign against drug abuse. Journal of the American Statistical Association, 96, 1245–1253. doi:10.1198/016214501753381896
   PubMed Web of Science ®Google Scholar
• Macgowan, M. J., Nash, J. K., & Fraser, M. W. (2002). The Carolina Child Checklist of risk and protective factors for aggression. Research on Social Work Practice, 12, 253–276.
   Web of Science ®Google Scholar
• Malti, T., Ribeaud, D., & Eisner, M. P. (2011). The effectiveness of two universal preventive interventions in reducing children's externalizing behavior: A cluster randomized controlled trial. Journal of Clinical Child & Adolescent Psychology, 40, 677–692. doi:10.1080/15374416.2011.597084
   PubMed Web of Science ®Google Scholar
• McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society: Series B (Methodological), 42, 109–142. Retrieved from http://people.csail.mit.edu/jrennie/papers/other/mccullagh-ordinal-80.pdf
   Web of Science ®Google Scholar
• Miller, B. C., & Dyk, P. H. (1991). Community of caring effects on adolescent mothers: A program evaluation case study. Family Relations, 40, 386–395. doi:10.2307/584895
   Web of Science ®Google Scholar
• Orwin, R., Hornik, R., Judkins, D., Zador, P., Sridharan, S., & Baskin, R. (2003). Innovative design and analysis strategies in the evaluation of the National Youth Anti-Drug Media Campaign: Propensity scores and counterfactual projection weights in a national probability survey. Retrieved from https://fcsm.sites.usa.gov/files/2014/05/2003FCSM_Orwin.pdf
   Google Scholar
• Pearson, R. K. (2003). Generalized sensitivity analysis: A framework for evaluating data analysis results.Retrieved from http://epubs.siam.org/doi/pdf/10.1137/1.9781611972733.20
   Google Scholar
• Peck, L. R. (2003). Subgroup analysis in social experiments: Measuring program impacts based on post-treatment choice. American Journal of Evaluation, 24, 157–187.
   Web of Science ®Google Scholar
• Rosenbaum, P. R. (1987a). The role of a second control group in an observational study. Statistical Science, 2, 292–306. doi:10.1214/ss/1177013232
   Google Scholar
• Rosenbaum, P. R. (1987b). Sensitivity analysis for certain permutation inferences in matched observational studies. Biometrika, 74, 13–26. doi:10.1093/biomet/74.1.13
   Web of Science ®Google Scholar
• Rosenbaum, P. R. (1989). Optimal matching for observational studies. Journal of the American Statistical Association, 84, 1024–1032. Retrieved from http://www.jstor.org/stable/2290079
   Web of Science ®Google Scholar
• Rosenbaum, P. R. (1991). Discussing hidden bias in observational studies. Annals of Internal Medicine, 115, 901–905. Retrieved from http://annals.org/article.aspx? volume=115&page=901
   PubMed Web of Science ®Google Scholar
• Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York, NY: Springer.
   Google Scholar
• Rosenbaum, P. R. (2003). Does a dose–response relationship reduce sensitivity to hidden bias? Biostatistics, 4, 1–10. doi:10.1093/biostatistics/4.1.1
   PubMed Web of Science ®Google Scholar
• Rosenbaum, P. R. (2005). Sensitivity analysis in observational studies. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 4, pp. 1809–1814). Hoboken, NJ: John Wiley & Sons. doi:10.1002/0470013192.bsa606
   Google Scholar
• Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. doi:10.1093/biomet/70.1.41
   Web of Science ®Google Scholar
• Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516–524. Retrieved from http://personal.psc.isr.umich.edu/yuxie-web/files/soc710/Rosenbaum-Rubin1984.pdf
   Web of Science ®Google Scholar
• Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization.Annals of Statistics, 6, 34–58. doi:10.1214/aos/1176344064
   Web of Science ®Google Scholar
• Rubin, D. B. (1980). Discussion of ‘Randomization Analysis of Experimental Data in the Fisher Randomization Test’ by Basu. Journal of the American Statistical Association, 75, 591–593. doi:10.2307/2287653
   Web of Science ®Google Scholar
• Rubin, D. B. (1990). Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5, 472–480. doi:10.1214/ss//1177012032
   Google Scholar
• Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757–763. Retrieved from http://annals.org/article.aspx? volume=127&page=757
   PubMed Web of Science ®Google Scholar
• Salzer, M. S., Bickman, L., & Lambert, E. W. (1999). Dose–effect relationship in children's psychotherapy services. Journal of Clinical and Consulting Psychology, 66, 270–279. doi:10.1037/0022-006X.67.2.228
   Web of Science ®Google Scholar
• Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13, 279–313. doi:10.1037/a0014268
   PubMed Web of Science ®Google Scholar
• Spreeuwenberg, M. D., Bartak, A., Croon, M. A., Hagenaars, J. A., Busschbach, J. J. V., Andrea, H., … Stijnen, H. (2010). The multiple propensity score as control for bias in the comparison of more than two treatment arms: An introduction from a case study in mental health. Medical Care, 48, 166–174. doi:10.1097/MLR.0b013e3181c1328f
   PubMed Web of Science ®Google Scholar
• Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1–21. doi:10.1214/09-STS313
   PubMed Web of Science ®Google Scholar
• Waters, E., & Sroufe, L. A. (1983). Social competence as a developmental construct. Developmental Review, 3, 79–97.
   Google Scholar
• Weissberg, R. P., & Greenberg, M. T. (1998). School and community competence-enhancement and prevention programs. In W. Damon (Series Ed.) & I. E. Sigel & K. A. Renninger (Vol. Eds.), Handbook of child psychology: Vol 4. Child psychology in practice (5th ed., pp. 877–954). New York, NY: John Wiley & Sons.
   Google Scholar
• Zanutto, E., Lu, B., & Hornik, R. (2005). Using propensity score subclassification for multiple treatment doses to evaluate a national antidrug media campaign.Journal of Educational and Behavioral Statistics, 20, 59–73. doi:10.3102/10769986030001059
   Web of Science ®Google Scholar
• Zhai, F. H., Raver, C. C., Jones, S. M., Li-Grining, C. P., Pressler, E., & Gao, Q. (2010). Dosage effects on school readiness: Evidence from a randomized classroom-based intervention. Social Service Review, 84, 615–655. doi:10.1086/657988
   PubMed Web of Science ®Google Scholar