Fit for a Bayesian: An Evaluation of PPP and DIC for Structural Equation Modeling

Abstract

Despite its importance to structural equation modeling, model evaluation remains underdeveloped in the Bayesian SEM framework. Posterior predictive p-values (PPP) and deviance information criteria (DIC) are now available in popular software for Bayesian model evaluation, but they remain underutilized. This is largely due to the lack of recommendations for their use. To address this problem, PPP and DIC were evaluated in a series of Monte Carlo simulation studies. The results show that both PPP and DIC are influenced by severity of model misspecification, sample size, model size, and choice of prior. The cutoffs PPP < 0.10 and ∆DIC > 7 work best in the conditions and models tested here to maintain low false detection rates and misspecified model selection rates, respectively. The recommendations provided in this study will help researchers evaluate their models in a Bayesian SEM analysis and set the stage for future development and evaluation of Bayesian SEM fit indices.

Keywords:

• Bayesian
• deviance information criteria
• model fit
• posterior predictive p-values
• structural equation modeling

Supplemental data

Supplemental data for this article can be accessed here.

Notes

¹ $\mathrm{S}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{r}\mathrm{d}\mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}(\mathrm{S}\mathrm{E})=\sqrt{\left(p\left(1-p\right)\right)/r}$, where $r$ is the number of replications and $p$ is the proportion. Using a proportion of 0.50 yields a standard error of 0.016, the largest possible standard error given $r$ = 1000.