Estimating propensity scores using neural networks and traditional methods: a comparative simulation study

References

• Alam, S., E. Moodie, and D. Stephens. 2019. Should a propensity score model be super? The utility of ensemble procedures for causal adjustment. Statistics in Medicine 38 (9):1690–702. doi:10.1002/sim.8075.
   PubMed Web of Science ®Google Scholar
• Altman, D. G. 1990. Practical statistics for medical research. Boca Raton, FL: CRC Press.
   Google Scholar
• Arpino, B., and F. Mealli. 2011. The specification of the propensity score in multilevel observational studies. Computational Statistics & Data Analysis 55 (4):1770–80. doi:10.1016/j.csda.2010.11.008.
   Web of Science ®Google Scholar
• Austin, P. C. 2012. Using ensemble-based methods for directly estimating causal effects: An investigation of tree-based G-computation. Multivariate Behavioral Research 47 (1):115–35. doi:10.1080/00273171.2012.640600.
   PubMed Web of Science ®Google Scholar
• Bembom, O., and M. J. Van der Laan. 2008. Data-adaptive selection of the truncation level for inverse-probability-of-treatment-weighted estimators. U.C. Berkeley Division of Biostatistics Working Paper Series. http://biostats.bepress.com/ucbbiostat/paper230
   Google Scholar
• Cannas, M., and B. Arpino. 2019. A comparison of machine learning algorithms and covariate balance measures for propensity score matching and weighting. Biometrical Journal 61 (4):1049–72. doi:10.1002/bimj.201800132.
   PubMed Web of Science ®Google Scholar
• Choi, B., C. Wang, J. Michalek, and J. Gelfond. 2019. Power comparison for propensity score methods. Computational Statistics 34 (2):743–61. doi:10.1007/s00180-018-0852-5.
   Web of Science ®Google Scholar
• Collier, Z. K., and W. L. Leite. 2020. A tutorial on artificial neural networks in propensity score analysis. The Journal of Experimental Education. Advance online publication. doi:10.1080/00220973.2020.1854158.
   PubMed Web of Science ®Google Scholar
• Collier, Z. K., W. L. Leite, and A. Karpyn. 2021. Neural networks to estimate generalized propensity scores for continuous treatment doses. Evaluation Review. Advance online publication. doi:10.1177/0193841X21992199.
   PubMed Web of Science ®Google Scholar
• Colombet, I., A. Ruelland, G. Chatellier, F. Gueyffier, P. Degoulet, and M. C. Jaulent. 2000. Models to predict cardiovascular risk: Comparison of CART, multilayer perceptron and logistic regression. In Proceedings of the AMIA Symposium (p. 156). American Medical Informatics Association.
   Google Scholar
• Core Team, R. 2017. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-progject.org/.
   Google Scholar
• Drake, C. 1993. Effects of misspecification of the propensity score on estimators of treatment effect. Biometrics 49 (4):1231–6. doi:10.2307/2532266.
   Web of Science ®Google Scholar
• Fallah, N., H. Gu, K. Mohammad, S. A. Seyyedsalehi, K. Nourijelyani, and M. R. Eshraghian. 2009. Nonlinear Poisson regression using neural networks: A simulation study. Neural Computing and Applications 18 (8):939–43. doi:10.1007/s00521-009-0277-8.
   Web of Science ®Google Scholar
• Finne, P., R. Finne, A. Auvinen, H. Juusela, J. Aro, L. Määttänen, M. Hakama, S. Rannikko, T. L. J. Tammela, and U.-H. Stenman. 2000. Predicting the outcome of prostate biopsy in screen-positive men by a multilayer perceptron network. Urology 56 (3):418–22. doi:10.1016/S0090-4295(00)00672-5.
   PubMed Web of Science ®Google Scholar
• Freund, Y., and R. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55 (1):119–39. doi:10.1006/jcss.1997.1504.
   Web of Science ®Google Scholar
• Friedman, J. H. 2001. Greedy function approximation: A gradient boosting machine. Annals of Statistics 29:1189–232.
   Web of Science ®Google Scholar
• Friedman, J. H., T. Hastie, and R. Tibshirani. 2000. Additive logistic regression: A statistical view of boosting. Annals of Statistics 28:337–74.
   Web of Science ®Google Scholar
• Géron, A. 2019. Hands-on machine learning with scikit-learn, keras, and tensorflow: concepts, tools, and techniques to build intelligent systems. Newton, MA: O’Reilly Media.
   Google Scholar
• Goodfellow, I., Y. Bengio, and A. Courville. 2016. Deep learning. Cambridge, MA: MIT Press.
   Google Scholar
• Gurel, S. 2015. Dealing with selection bias in multilevel observational studies: An evaluation of propensity score and direct estimation procedures (Unpublished doctoral dissertation). University of Florida, Gainesville, FL.
   Google Scholar
• Hasan, A., W. Zhiyu, and A. S. Mahani. 2014. Fast estimation of multinomial logit models: R package mnlogit. arXiv:1404.3177.
   Google Scholar
• Hill, J. L., C. Weiss, and F. Zhai. 2011. Challenges with propensity score matching in a high-dimensional setting and a potential alternative. Multivariate Behavioral Research 46 (3):477–513. doi:10.1080/00273171.2011.570161.
   PubMed Web of Science ®Google Scholar
• Hinton, G. E., and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks). Science (New York, N.Y.) 313 (5786):504–7. doi:10.1126/science.1127647.
   PubMed Web of Science ®Google Scholar
• Hirano, K., and G. W. Imbens. 2001. Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Services and Outcomes Research Methodology 2 (3/4):259–78. doi:10.1023/A:1020371312283.
   Google Scholar
• Imai, K., and D. A. Van Dyk. 2004. Causal inference with general treatment regimes. Journal of the American Statistical Association 99 (467):854–66. doi:10.1198/016214504000001187.
   Web of Science ®Google Scholar
• Imbens, G. W. 2000. The role of the propensity score in estimating dose-response functions. Biometrika 87 (3):706–10. doi:10.1093/biomet/87.3.706.
   Web of Science ®Google Scholar
• Indira, V., R. Vasanthakumari, R. Jegadeeshwaran, and V. Sugumaran. 2015. Determination of minimum sample size for fault diagnosis of automobile hydraulic brake system using power analysis. Engineering Science and Technology: An International Journal 18 (1):59–69.
   Google Scholar
• Jain, A., K. Nandakumar, and A. Ross. 2005. Score normalization in multimodal biometric systems. Pattern Recognition 38 (12):2270–85. doi:10.1016/j.patcog.2005.01.012.
   Web of Science ®Google Scholar
• Ju, C., J. Schwab, and M. J. van der Laan. 2019. On adaptive propensity score truncation in causal inference. Statistical Methods in Medical Research 28 (6):1741–60. doi:10.1177/0962280218774817.
   PubMed Web of Science ®Google Scholar
• Kim, J., and M. Seltzer. 2007. Causal inference in multilevel settings in which selection processes vary across schools. Los Angeles, CA: Center for Study of Evaluation (CSE).
   Google Scholar
• Krogh, A., and J. Vedelsby. 1995. Neural network ensembles, cross validation, and active learning. In Advances in neural information processing systems, 231–238.
   Google Scholar
• Kurt, I., M. Ture, and A. T. Kurum. 2008. Comparing performances of logistic regression, classification and regression tree, and neural networks for predicting coronary artery disease. Expert Systems with Applications 34 (1):366–374. doi:10.1016/j.eswa.2006.09.004.
   Web of Science ®Google Scholar
• LeCun, Y., L. Bottou, G. B. Orr, and K. R. Müller. 2012. Efficient backprop. In Neural networks: Tricks of the trade, 9–48. Berlin, Heidelberg: Springer.
   Google Scholar
• Lee, B. K., J. Lessler, and E. A. Stuart. 2010. Improving propensity score weighting using machine learning. Statistics in Medicine 29 (3):337–346. doi:10.1002/sim.3782.
   PubMed Web of Science ®Google Scholar
• Leite, W. 2017. Practical propensity score methods using R. Thousand Oaks, CA: SAGE Publications.
   Google Scholar
• Leite, W. L., F. Jimenez, Y. Kaya, L. M. Stapleton, J. W. MacInnes, and R. Sandbach. 2015. An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research 50 (3):265–284. doi:10.1080/00273171.2014.991018.
   PubMed Web of Science ®Google Scholar
• Manel, S., H. C. Williams, and S. J. Ormerod. 2001. Evaluating presence–absence models in ecology: The need to account for prevalence. Journal of Applied Ecology 38 (5):921–931. doi:10.1046/j.1365-2664.2001.00647.x.
   Web of Science ®Google Scholar
• McCaffrey, D. F., B. A. Griffin, D. Almirall, M. E. Slaughter, R. Ramchand, and L. F. Burgette. 2013. A tutorial on propensity score estimation for multiple treatments using generalized boosted models. Statistics in Medicine 32 (19):3388–3414. doi:10.1002/sim.5753.
   PubMed Web of Science ®Google Scholar
• McCaffrey, D. F., G. Ridgeway, and A. R. Morral. 2004. Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods 9 (4):403–425. doi:10.1037/1082-989X.9.4.403.
   PubMed Web of Science ®Google Scholar
• Olejnik, S., and J. Algina. 2003. Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods 8 (4):434–447. doi:10.1037/1082-989X.8.4.434.
   PubMed Web of Science ®Google Scholar
• Pedregosa, F., G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, … J. Vanderplas. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12 (Oct):2825–2830.
   Google Scholar
• Peduzzi, P.,. J. Concato, E. Kemper, T. R. Holford, and A. R. Feinstein. 1996. A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology 49 (12):1373–1379. doi:10.1016/s0895-4356(96)00236-3.
   PubMed Web of Science ®Google Scholar
• Petersen, M. L., K. E. Porter, S. Gruber, Y. Wang, and M. J. Van Der Laan. 2012. Diagnosing and responding to violations in the positivity assumption. Statistical Methods in Medical Research 21 (1):31–54. doi:10.1177/0962280210386207.
   PubMed Web of Science ®Google Scholar
• Pirracchio, R., M. L. Petersen, and M. van der Laan. 2015. Improving propensity score estimators' robustness to model misspecification using super learner. American Journal of Epidemiology 181 (2):108–119. doi:10.1093/aje/kwu253.
   PubMed Web of Science ®Google Scholar
• Plunkett, K., and J. L. Elman. 1997. Exercises in rethinking innateness: A handbook for connectionist simulations. Cambridge, MA: MIT Press.
   Google Scholar
• Ridgeway, G. 1999. The state of boosting. Computing Science and Statistics 31:172–181.
   Google Scholar
• Rosenbaum, P. R., and D. B. Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70 (1):41–55. doi:10.1093/biomet/70.1.41.
   Web of Science ®Google Scholar
• Rosenbaum, P. R., and D. B. Rubin. 1984. Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association 79 (387):516–524. doi:10.1080/01621459.1984.10478078.
   Web of Science ®Google Scholar
• Setoguchi, S.,. S. Schneeweiss, M. A. Brookhart, R. J. Glynn, and E. F. Cook. 2008. Evaluating uses of data mining techniques in propensity score estimation: A simulation study. Pharmacoepidemiology and Drug Safety 17 (6):546–555. doi:10.1002/pds.1555.
   PubMed Web of Science ®Google Scholar
• Stuart, E. A. 2010. Matching methods for causal inference: A review and a look forward. Statistical Science: A Review Journal of the Institute of Mathematical Statistics 25 (1):1–21. doi:10.1214/09-STS313.
   PubMed Web of Science ®Google Scholar
• Subasi, A., and E. Ercelebi. 2005. Classification of EEG signals using neural network and logistic regression. Computer Methods and Programs in Biomedicine 78 (2):87–99. doi:10.1016/j.cmpb.2004.10.009.
   PubMed Web of Science ®Google Scholar
• Thoemmes, F. J., and E. S. Kim. 2011. A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research 46 (1):90–118. doi:10.1080/00273171.2011.540475.
   PubMed Web of Science ®Google Scholar
• Tu, C. 2019. Comparison of various machine learning algorithms for estimating generalized propensity score. Journal of Statistical Computation and Simulation 89 (4):708–19. doi:10.1080/00949655.2019.1571059.
   Web of Science ®Google Scholar
• Tu, J. 1996. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. Journal of Clinical Epidemiology 49 (11):1225–1231. doi:10.1016/S0895-4356(96)00002-9.
   PubMed Web of Science ®Google Scholar
• Van Rossum, G., and F. L. Drake Jr. 1995. Python reference manual. Amsterdam: Centrum voor Wiskunde en Informatica.
   Google Scholar
• Westreich, D., J. Lessler, and M. J. Funk. 2010. Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression. Journal of Clinical Epidemiology 63 (8):826–833. doi:10.1016/j.jclinepi.2009.11.020.
   PubMed Web of Science ®Google Scholar
• Witten, I. H., E. Frank, M. A. Hall, C. J. Pal, and DATA MINING. 2005. Practical machine learning tools and techniques. In DATA MINING (Vol. 2, p. 4).
   Google Scholar
• Yee, T. W. 2010. The VGAm package for categorical data analysis. Journal of Statistical Software 32 (10):1–34. doi:10.18637/jss.v032.i10.
   Web of Science ®Google Scholar