216
Views
0
CrossRef citations to date
0
Altmetric
Research Articles

Propensity Score Weighting with Missing Data on Covariates and Clustered Data Structure

ORCID Icon

References

  • Arpino, B., & Mealli, F. (2011). The specification of the propensity score in multilevel observational studies. Computational Statistics & Data Analysis, 55(4), 1770–1780. https://doi.org/10.1016/j.csda.2010.11.008
  • Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399–424. https://doi.org/10.1080/00273171.2011.568786
  • Asparouhov, T. (2006). General multi-level modeling with sampling weights. Communications in Statistics - Theory and Methods, 35(3), 439–460. https://doi.org/10.1080/03610920500476598
  • Aydin, B., Leite, W. L., & Algina, J. (2016). The effects of including observed means or latent means as covariates in multilevel models for cluster randomized trials. Educational and Psychological Measurement, 76(5), 803–823. https://doi.org/10.1177/0013164415618705
  • Bandalos, D. L., & Leite, W. L. (2015). The use of Monte Carlo studies in structural equation modeling research. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (2nd ed., pp. 625–666). https://books.google.com/books?hl=en&lr=&id=VfonDwAAQBAJ&oi=fnd&pg=PA385&dq=Bandalos,+D.+L.+%26+Leite,+W.+L.+(2013).+Use+of+Monte+Carlo+Studies+in+Structural+Equation+Modeling+Research.+In+G.+R.+Hancock+%26+R.+O.+Mueller+(Eds.),+Structural+equation+modeling:+A+second+course+(2nd+Ed.).+Greenwich,+CT:+Information+Age+Publishing.&ots=-Nx7qmHeWD&sig=qv8K1ls8nWFSipNWrs1EWqzTijU
  • Bates, D., Maechler, M., Bolker aut, B., cre, Walker, S., Christensen, R. H. B., Singmann, H., Dai, B., Scheipl, F., Grothendieck, G., Green, P., Fox, J., Bauer, A., & Tanaka, E. (2011). lme4: Linear Mixed-Effects Models using “Eigen“ and S4 (1.1-35.1) [Computer software]. https://cran.r-project.org/web/packages/lme4/index.html
  • Bloom, H. S., Richburg-Hayes, L., & Black, A. R. (2007). Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29(1), 30–59. https://www.jstor.org/stable/30128044 https://doi.org/10.3102/0162373707299550
  • Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Stürmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163(12), 1149–1156. https://doi.org/10.1093/aje/kwj149
  • Caliendo, M., & Kopeinig, S. (2008). Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys, 22(1), 31–72. https://doi.org/10.1111/j.1467-6419.2007.00527.x
  • Carpenter, J. R., Roger, J. H., & Kenward, M. G. (2013). Analysis of longitudinal trials with protocol deviation: A framework for relevant, accessible assumptions, and inference via multiple imputation. Journal of Biopharmaceutical Statistics, 23(6), 1352–1371. https://doi.org/10.1080/10543406.2013.834911
  • Cham, H., & West, S. G. (2016). Propensity score analysis with missing data. Psychological Methods, 21(3), 427–445. https://doi.org/10.1037/met0000076
  • Chang, T.-H., & Stuart, E. A. (2022). Propensity score methods for observational studies with clustered data: A review. Statistics in Medicine, 41(18), 3612–3626. https://doi.org/10.1002/sim.9437
  • Choi, J., Dekkers, O. M., & Le Cessie, S. (2019). A comparison of different methods to handle missing data in the context of propensity score analysis. European Journal of Epidemiology, 34(1), 23–36. https://doi.org/10.1007/s10654-018-0447-z
  • Choi, J., Dekkers, O. M., & Le Cessie, S. (2020). Authors’ Reply: A comparison of different methods to handle missing data in the context of propensity score analysis. European Journal of Epidemiology, 35(1), 89–91. https://doi.org/10.1007/s10654-019-00553-y
  • Chung, H., & Beretvas, S. N. (2012). The impact of ignoring multiple membership data structures in multilevel models. The British Journal of Mathematical and Statistical Psychology, 65(2), 185–200. https://doi.org/10.1111/j.2044-8317.2011.02023.x
  • Coffman, D. L., Zhou, J., & Cai, X. (2020). Comparison of methods for handling covariate missingness in propensity score estimation with a binary exposure. BMC Medical Research Methodology, 20(1), 168. https://doi.org/10.1186/s12874-020-01053-4
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. (2nd ed.) L. Erlbaum Associates.
  • D’Agostino Jr, R., Lang, W. E. I., Walkup, M., Morgan, A. T., & Karter, A. (2001). Examining the impact of missing data on propensity score estimation in determining the effectiveness of self-monitoring of blood glucose (SMBG). Health Services and Outcomes Research Methodology, 2(3/4), 291–315. https://doi.org/10.1023/A:1020375413191
  • D’Agostino, R. B., Jr., & Rubin, D. B. (2000). Estimating and using propensity scores with partially missing data. Journal of the American Statistical Association, 95(451), 749–759. https://doi.org/10.1080/01621459.2000.10474263
  • Drechsler, J. (2015). Multiple Imputation of Multilevel Missing Data—Rigor Versus Simplicity. Journal of Educational and Behavioral Statistics, 40(1), 69–95. https://doi.org/10.3102/1076998614563393
  • Du, H., Alacam, E., Mena, S., & Keller, B. T. (2022). Compatibility in imputation specification. Behavior Research Methods, 54(6), 2962–2980. https://doi.org/10.3758/s13428-021-01749-5
  • Eiset, A. H., & Frydenberg, M. (2022). Considerations for using multiple imputation in propensity score-weighted analysis – A tutorial with applied example. Clinical Epidemiology, 14, 835–847. https://doi.org/10.2147/CLEP.S354733
  • Enders, C. K. (2022). Applied missing data analysis. Guilford Publications.
  • Enders, C. K. (2023). Missing data: An update on the state of the art. Psychological Methods, https://doi.org/10.1037/met0000563
  • Enders, C. K., Du, H., & Keller, B. T. (2020). A model-based imputation procedure for multilevel regression models with random coefficients, interaction effects, and nonlinear terms. Psychological Methods, 25(1), 88–112. https://doi.org/10.1037/met0000228
  • Enders, C. K., Mistler, S. A., & Keller, B. T. (2016). Multilevel multiple imputation: A review and evaluation of joint modeling and chained equations imputation. Psychological Methods, 21(2), 222–240. https://doi.org/10.1037/met0000063
  • Fuentes, A., Lüdtke, O., & Robitzsch, A. (2022). Causal inference with multilevel data: A comparison of different propensity score weighting approaches. Multivariate Behavioral Research, 57(6), 916–939. https://doi.org/10.1080/00273171.2021.1925521
  • Goldstein, H., Carpenter, J. R., & Browne, W. J. (2014). Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non-linear terms. Journal of the Royal Statistical Society Series A: Statistics in Society,)177(2), 553–564. https://doi.org/10.1111/rssa.12022
  • Graham, J. W. (2012). Multiple imputation and analysis with multilevel (cluster) data. In J. W. Graham (Ed.), Missing data: Analysis and design (pp. 133–150). Springer. https://doi.org/10.1007/978-1-4614-4018-5_6
  • Granger, E., Sergeant, J. C., & Lunt, M. (2019). Avoiding pitfalls when combining multiple imputation and propensity scores. Statistics in Medicine, 38(26), 5120–5132. https://doi.org/10.1002/sim.8355
  • Greifer, N., & Greifer, M. N. (2020). Package ‘cobalt.’ R Package Version, 4.0. https://cran.r-project.org/package=cobalt
  • Grund, S., Lüdtke, O., & Robitzsch, A. (2018). Multiple imputation of missing data for multilevel models: Simulations and recommendations. Organizational Research Methods, 21(1), 111–149. https://doi.org/10.1177/1094428117703686
  • Grund, S., Lüdtke, O., & Robitzsch, A. (2021). Multiple imputation of missing data in multilevel models with the R package mdmb: A flexible sequential modeling approach. Behavior Research Methods, 53(6), 2631–2649. https://doi.org/10.3758/s13428-020-01530-0
  • Guo, S., & Fraser, M. W. (2014). Propensity score analysis: Statistical methods and applications. (Vol. 11) SAGE publications.
  • Hammon, A., & Zinn, S. (2020). Multiple Imputation of Binary Multilevel Missing not at Random Data. Journal of the Royal Statistical Society Series C: Applied Statistics, 69(3), 547–564. https://doi.org/10.1111/rssc.12401
  • Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87. https://doi.org/10.3102/0162373707299706
  • Hernan, M., & Robins, J. (2020). Causal Inference: What if. Chapman & Hill/CRC.
  • Hill, J. (2004). Reducing bias in treatment effect estimation in observational studies suffering from missing data (ISERP Working Papers). https://doi.org/10.7916/D8B85G11
  • Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15(3), 199–236. https://doi.org/10.1093/pan/mpl013
  • Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. https://doi.org/10.1080/01621459.1986.10478354
  • Hong, G., & Raudenbush, S. W. (2005). Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics. Educational Evaluation and Policy Analysis, 27(3), 205–224. https://doi.org/10.3102/01623737027003205
  • Hong, G., & Raudenbush, S. W. (2006). Evaluating Kindergarten Retention Policy. Journal of the American Statistical Association, 101(475), 901–910. https://doi.org/10.1198/016214506000000447
  • Hoshino, T., Kurata, H., & Shigemasu, K. (2006). A propensity score adjustment for multiple group structural equation modeling. Psychometrika, 71(4), 691–712. https://doi.org/10.1007/s11336-005-1370-2
  • Ingels, S. J., Pratt, D. J., Rogers, J. E., Siegel, P. H., Stutts, E. S. (2005). Education Longitudinal Study of 2002: Base-Year to First Follow-Up Data File Documentation. NCES 2006-344. ED Pubs, P. https://eric.ed.gov/?id=ED489083
  • Jalali, A., Tamimi, R. M., McPherson, S. M., & Murphy, S. M. (2022). Econometric issues in prospective economic evaluations alongside clinical trials: Combining the nonparametric bootstrap with methods that address missing data. Epidemiologic Reviews, 44(1), 67–77. https://doi.org/10.1093/epirev/mxac006
  • Keller, B. T., & Enders, C. K. (2019). Blimp user’s guide. (version 2.0)
  • Keller, B. T., & Enders, C. K. (2023). An Investigation of Factored Regression Missing Data Methods for Multilevel Models with Cross-Level Interactions. Multivariate Behavioral Research, 58(5), 938–963. https://doi.org/10.1080/00273171.2022.2147049
  • Kim, G.-S., Paik, M. C., & Kim, H. (2017). Causal inference with observational data under cluster-specific non-ignorable assignment mechanism. Computational Statistics & Data Analysis, 113, 88–99. https://doi.org/10.1016/j.csda.2016.10.002
  • Kreft, I. G. G., & de Leeuw, J. (1998). Introducing multilevel modeling. SAGE.
  • Langworthy, B., Wu, Y., & Wang, M. (2023). An overview of propensity score matching methods for clustered data. Statistical Methods in Medical Research, 32(4), 641–655. https://doi.org/10.1177/09622802221133556
  • Lee, B. K., Lessler, J., & Stuart, E. A. (2011). Weight Trimming and Propensity Score Weighting. PloS One, 6(3), e18174. https://doi.org/10.1371/journal.pone.0018174
  • Lee, Y., Leite, W. L., & Leroux, A. J. (2023). Propensity score matching with cross-classified data structures: A comparison of methods. The Journal of Experimental Education, 0(0), 1–18. https://doi.org/10.1080/00220973.2023.2164843
  • Lee, Y., Nguyen, T. Q., & Stuart, E. A. (2021). Partially pooled propensity score models for average treatment effect estimation with multilevel data. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(4), 1578–1598. https://doi.org/10.1111/rssa.12741
  • Leite, W. L. (2007). A comparison of latent growth models for constructs measured by multiple items. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 581–610. https://doi.org/10.1080/10705510701575438
  • Leite, W. L. (2016). Practical Propensity Score Methods Using R. SAGE Publications.
  • Leite, W. L., Aydin, B., & Cetin-Berber, D. D. (2021). Imputation of missing covariate data prior to propensity score analysis: A tutorial and evaluation of the robustness of practical approaches. Evaluation Review, 0193841X2110202. https://doi.org/10.1177/0193841X211020245
  • Leite, W. L., Aydin, B., & Gurel, S. (2019). A comparison of propensity score weighting methods for evaluating the effects of programs with multiple versions. The Journal of Experimental Education, 87(1), 75–88. https://doi.org/10.1080/00220973.2017.1409179
  • Leite, W. L., Jimenez, F., Kaya, Y., Stapleton, L. M., MacInnes, J. W., & Sandbach, R. (2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50(3), 265–284. https://doi.org/10.1080/00273171.2014.991018
  • Leyrat, C., Seaman, S. R., White, I. R., Douglas, I., Smeeth, L., Kim, J., Resche-Rigon, M., Carpenter, J. R., & Williamson, E. J. (2019). Propensity score analysis with partially observed covariates: How should multiple imputation be used? Statistical Methods in Medical Research, 28(1), 3–19. https://doi.org/10.1177/0962280217713032
  • Li, F., Zaslavsky, A. M., & Beth, M. (2013). Propensity score weighting with multilevel data. Statistics in Medicine, 32(19), 3373–3387. https://doi.org/10.1002/sim.5786
  • Little, R. J., & Rubin, D. B. (2019). Statistical analysis with missing data. (Vol. 793) John Wiley & Sons.
  • Liu, X., Liu, F., Miller-Graff, L., Howell, K. H., & Wang, L. (2023). Causal inference for treatment effects in partially nested designs. Psychological Methods, Advanced Online Publication. https://doi.org/10.1037/met0000565.supp
  • Lunceford, J. K., & Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: A comparative study. Statistics in Medicine, 23(19), 2937–2960. https://doi.org/10.1002/sim.1903
  • Lüdtke, O., Robitzsch, A., & Grund, S. (2017). Multiple imputation of missing data in multilevel designs: A comparison of different strategies. Psychological Methods, 22(1), 141–165. https://doi.org/10.1037/met0000096
  • Lüdtke, O., Robitzsch, A., & West, S. G. (2020). Regression models involving nonlinear effects with missing data: A sequential modeling approach using Bayesian estimation. Psychological Methods, 25(2), 157–181. https://doi.org/10.1037/met0000233
  • Maas, C. J., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86–92. https://doi.org/10.1027/1614-2241.1.3.86
  • Malla, L., Perera-Salazar, R., McFadden, E., Ogero, M., Stepniewska, K., & English, M. (2018). Handling missing data in propensity score estimation in comparative effectiveness evaluations: A systematic review. Journal of Comparative Effectiveness Research, 7(3), 271–279. https://doi.org/10.2217/cer-2017-0071
  • Murray, D. M., & Blistein, J. L. (2003). Methods to reduce the impact of intraclass correlation in group-randomized trials. Evaluation Review, 27(1), 79–103. https://doi.org/10.1177/0193841X02239019
  • Muthén, L. K., & Muthén, B. O. (2002). How to use a monte carlo study to decide on sample size and determine power. Structural Equation Modeling: A Multidisciplinary Journal, 9(4), 599–620. https://doi.org/10.1207/S15328007SEM0904_8
  • Muthén, B., & Muthén, L. (2017). Mplus. Chapman and Hall/CRC.
  • Nguyen, T. Q., & Stuart, E. A. (2023). Multiple imputation for propensity score analysis with covariates missing at random: Some clarity on within and across methods (arXiv:2301.07066). arXiv. https://doi.org/10.48550/arXiv.2301.07066
  • Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods, 8(4), 434–447. https://doi.org/10.1037/1082-989X.8.4.434
  • Qu, Y., & Lipkovich, I. (2009). Propensity score estimation with missing values using a multiple imputation missingness pattern (MIMP) approach. Statistics in Medicine, 28(9), 1402–1414. https://doi.org/10.1002/sim.3549
  • Quartagno, M., Grund, S., & Carpenter, J. (2019). Jomo: A flexible package for two-level joint modelling multiple imputation. The R Journal, 11(2), 205–228. https://doi.org/10.32614/RJ-2019-028
  • Reed, D. K., Aloe, A. M., Reeger, A. J., & Folsom, J. S. (2019). Defining summer gain among elementary students with or at risk for reading disabilities. Exceptional Children, 85(4), 413–431. https://doi.org/10.1177/0014402918819426
  • Robins, J. M., Hernán, M. Á., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology (Cambridge, Mass.), 11(5), 550–560. https://www.jstor.org/stable/3703997 https://doi.org/10.1097/00001648-200009000-00011
  • Rosenbaum, P. R. (2020). Design of observational studies. (2nd ed.) Springer International. https://doi.org/10.1007/978-3-030-46405-9
  • Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
  • Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79(387), 516–524. https://doi.org/10.1080/01621459.1984.10478078
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350
  • Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581–592. https://doi.org/10.1093/biomet/63.3.581
  • Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 6(1), 34–58. http://www.jstor.org/stable/2958688 https://doi.org/10.1214/aos/1176344064
  • Rubin, D. B. (1980). Randomization analysis of experimental data: The Fisher randomization test comment. Journal of the American Statistical Association, 75(371), 591–593. https://doi.org/10.2307/2287653
  • Rubin, D. B. (1986). Statistics and causal inference: Comment, Which ifs have causal answers. Journal of the American Statistical Association, 81(396), 961–962. https://doi.org/10.1080/01621459.1986.10478355
  • Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. John Wiley & Sons. https://doi.org/10.1002/9780470316696
  • Schafer, J. L., & Yucel, R. M. (2002). Computational strategies for multivariate linear mixed-effects models with missing values. Journal of Computational and Graphical Statistics, 11(2), 437–457. https://doi.org/10.1198/106186002760180608
  • Schafer, J. L. (1997). Analysis of incomplete multivariate data. CRC Press.
  • Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313. https://doi.org/10.1037/a0014268
  • Schomaker, M., & Heumann, C. (2018). Bootstrap inference when using multiple imputation. Statistics in Medicine, 37(14), 2252–2266. https://doi.org/10.1002/sim.7654
  • Schuler, M. S., Chu, W., & Coffman, D. (2016). Propensity score weighting for a continuous exposure with multilevel data. Health Services & Outcomes Research Methodology, 16(4), 271–292. https://doi.org/10.1007/s10742-016-0157-5
  • Seaman, S., & White, I. (2014). Inverse probability weighting with missing predictors of treatment assignment or missingness. Communications in Statistics - Theory and Methods, 43(16), 3499–3515. https://doi.org/10.1080/03610926.2012.700371
  • Shadish, W. R., & Steiner, P. M. (2010). A primer on propensity score analysis. Newborn and Infant Nursing Reviews, 10(1), 19–26. https://doi.org/10.1053/j.nainr.2009.12.010
  • Shin, Y., & Raudenbush, S. W. (2013). Efficient analysis of q-level nested hierarchical general linear models given ignorable missing data. The International Journal of Biostatistics, 9(1), 109–133. https://doi.org/10.1515/ijb-2012-0048
  • Steiner, P. M., Kim, J.-S., & Thoemmes, F. (2013). Matching strategies for observational multilevel data. JSM Proceedings, 5020–5032.
  • Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science: A Review Journal of the Institute of Mathematical Statistics, 25(1), 1–21. https://doi.org/10.1214/09-STS313
  • Suk, Y., & Kang, H. (2023). Tuning random forests for causal inference under cluster-level unmeasured confounding. Multivariate Behavioral Research, 58(2), 408–440. https://doi.org/10.1080/00273171.2021.1994364
  • Thoemmes, F. J., & Kim, E. S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46(1), 90–118. https://doi.org/10.1080/00273171.2011.540475
  • Thoemmes, F. J., & West, S. G. (2011). The use of propensity scores for nonrandomized designs with clustered data. Multivariate Behavioral Research, 46(3), 514–543. https://doi.org/10.1080/00273171.2011.569395
  • van Buuren, S. (2011). Multiple imputation of multilevel data. Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203848852-14/multiple-imputation-multilevel-data-stef-van-buuren
  • van Buuren, S. (2018). Flexible imputation of missing data (2nd ed.). CRC Press. https://stefvanbuuren.name/fimd/
  • van Buuren, S., & Groothuis-Oudshoorn, K. (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1–67. https://doi.org/10.18637/jss.v045.i03
  • Whittaker, T. A. (2020). The comparison of latent variable propensity score models to traditional propensity score models under conditions of covariate unreliability. Multivariate Behavioral Research, 55(4), 625–646. https://doi.org/10.1080/00273171.2019.1663136
  • Yang, S. (2018). Propensity score weighting for causal inference with clustered data. Journal of Causal Inference, 6(2)Article, 20170027. https://doi.org/10.1515/jci-2017-0027
  • Yucel, R. M. (2008). Multiple imputation inference for multivariate multilevel continuous data with ignorable non-response. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 366(1874), 2389–2403. https://doi.org/10.1098/rsta.2008.0038
  • Zubizarreta, J. R., Stuart, E. A., Small, D. S., & Rosenbaum, P. R. (2023). Handbook of matching and weighting adjustments for causal inference. CRC Press.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.