Bayesian Meta-Analytic SEM: A One-Stage Approach to Modeling Between-Studies Heterogeneity in Structural Parameters

Abstract

Meta-analytic structural equation modeling (MASEM) refers to a set of meta-analysis techniques for combining and comparing structural equation modeling (SEM) results from multiple studies. Existing approaches to MASEM cannot appropriately model between-studies heterogeneity in structural parameters because of missing correlations, lack model fit assessment, and suffer from several theoretical limitations. In this study, we address the major shortcomings of existing approaches by proposing a novel Bayesian multilevel SEM approach. Simulation results showed that the proposed approach performed satisfactorily in terms of parameter estimation and model fit evaluation when the number of studies and the within-study sample size were sufficiently large and when correlations were missing completely at random. An empirical example about the structure of personality based on a subset of data was provided. Results favored the third factor structure over the hierarchical structure. We end the article with discussions and future directions.

Keywords:

• Bayesian approach
• MASEM
• meta-analysis
• multilevel SEM
• structural equation modeling

Acknowledgments

We gratefully acknowledge Timo Gnambs for permitting us to use his data in the real data example. We thank Betsy Becker for her constructive and valuable comments on earlier versions of the paper. Part of the project was presented in the 2016 International Meeting of the Psychometric Society and the 2016 American Psychological Association’s Annual Convention.

Notes

¹ If structural parameters are assumed to be different across groups of studies and invariant within groups, then by using the multi-group SEM technique, correlation-based MASEM is possible to handle this type of between-studies heterogeneity. However, if each study is assumed to have its own set of population SEM parameter values and missing correlations exist, which often occur in reality, correlation-based MASEM is not applicable with the multi-group SEM technique.

² The credibility intervals in FIMASEM are different from the credible intervals of Bayesian methods. Consider a random SEM parameter $\theta$ that varies across studies and follows a normal distribution, ${\theta }_i\sim N\left({\theta }_0,{\sigma }_{\theta }^2\right)$ . The end points of the credibility interval for $\theta$ in FIMASEM are estimates of the corresponding quantiles of $N\left({\theta }_0,{\sigma }_{\theta }^2\right)$ . However, the credible interval under the Bayesian framework for $\theta$ consists of corresponding quantiles of the posterior distribution of ${\theta }_0$ . The latter would normally be much narrower than the former. Bayesian methods can also provide FIMASEM’s credibility intervals.

³ Equations (4) and (6.2) are two examples of the nonlinearity of $\mathbf{g}\left(\cdot \right)$ . For most structural equation models, $\mathbf{g}\left(\cdot \right)$ is nonlinear.

⁴ Our notations are slightly different from the book. Specifically, ${\mathbf{\Lambda }}_i=\left[\begin{array}{cc}{\mathbf{\Lambda }}_{xi}& \mathbf{0}\\ \mathbf{0}& {\mathbf{\Lambda }}_{yi}\end{array}\right]$, ${\mathit{B}}_i=\left[\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ {\mathbf{\Gamma }}_i^{\star }& {\mathit{B}}_i^{\star }\end{array}\right]$, and ${\mathbf{\Psi }}_i=$ diag(${\mathbf{\Phi }}_i^{\star }$,${\mathbf{\Psi }}_i^{\star }$) where ${\mathbf{\Gamma }}_i^{\star }$, ${\mathit{B}}_i^{\star }$, ${\mathbf{\Phi }}_i^{\star }$, and ${\mathbf{\Psi }}_i^{\star }$ are the matrices ${\mathbf{\Gamma }}_i$, ${\mathit{B}}_i$, ${\mathbf{\Phi }}_i$, and ${\mathbf{\Psi }}_i$ in the book.

⁵ Despite similarity in their estimates, the MLE and Bayesian methods still differ conceptually. The maximum likelihood approach is to find the set of parameter values that maximizes the probability of observing the sample data if the parameter values under consideration are the population values. The Bayesian approach, however, is to find the set of typical parameter values given the observed data.

⁶ A detailed description of the conditional posterior distributions is available in Section 1 of the supplemental materials.

⁷ The widely used SEM-based fit indices and statistics such as the Chi-square test statistic are not used because they are not suitable for many multilevel models, including the multilevel model for MASEM with random SEM coefficients where there is no saturated model (see Wu et al., 2009 for a review on fit indices and statistics for multilevel models).

⁸ Using different ${\text{σ}}_{\omega ,i}^2$ for different studies makes the model not identified. To cope with this issue and for the sake of simplicity, we assume the covariance matrix of ${\mathit{\omega }}_i$ to be diagonal with the same diagonal elements within a certain study, and different across studies by a factor of the same size. Specifically, the larger the sample size for a study becomes, the smaller the ${\text{σ}}_{\omega ,i}^2$ is.

⁹ A factor of 100 is used because typical sample sizes for studies using SEM roughly range from 100 to 500. Therefore, $d$ is expected to be between 1.2$p^{\star }$ to $2$ $p^{\star }$ in typical situations. By setting $d$ to this value, the resulting prior for ${\sigma }_{\omega }^{-2}$ would be neither so informative so that parameter estimation is dominated by the prior, nor so uninformative so that the model fit of misspecified models improves substantially.

¹⁰ This lower limit is obtained by running a small simulation study where 100,000 datasets are simulated from the binomial distribution $Bin\left(100,.95\right)$ and the sample proportion of 1 is calculated for each dataset. The 2.5% percentile of those sample proportions across datasets is used as the lower limit for empirical CRs. The obtained lower limit is slightly different from that obtained using the normal approximation method: $.95-1.96\times \sqrt{.95\times .05/R}\approx .907$ . The difference is likely due to the small number of replications (i.e., $R=$ 100). In such case, normal approximation is not sufficiently accurate.

¹¹ This prior does not preclude estimates greater than one. Because all variables are standardized, factor loadings larger than one in magnitude are sometimes treated as “inadmissible” estimates. Despite that the probability is slim, to strictly avoid inadmissible estimates, researchers may use uniform priors for elements of ${\text{λ}}_0$, for example, U[−1,1].

¹² Population values are slightly different from Cheung and Chan (2005) to ensure that under the heterogeneous case, the probability of generating positive definite population correlation matrix for each study is sufficiently high (see the discussion in the Simulation Design subsection in Study 2).

¹³ For conditions with $\bar{N}$ = 100 and 200, the standard deviations of $N_i$ are roughly 23 and 45 respectively. The ratio of the standard deviation of $N_i$ to $\bar{N}$ is therefore approximately 0.23, which is close to the ratio used in Cheung and Chan (2009).

¹⁴ Example R code and OpenBUGS model scripts are available on https://github.com/zijunke/Bayesian-MASEM.

¹⁵ Results under other conditions generally share a similar pattern. Differences are noted here. Specific results are available in Section 2 of the supplemental materials for brevity.

¹⁶ Results under other conditions generally share a similar pattern. Differences are noted here. Specific results are available in Section 2 of the supplemental materials for brevity.

¹⁷ R code is available in Section 5 in the supplemental materials.

¹⁸ Models 2SF and 2SFG fitted by FIMASEM were not identical to those fitted by the proposed method because FIMASEM does not allow fixing parameters (loadings of factors $\alpha$ and $\beta$) to be invariant across studies.

¹⁹ Followed by Yu, Downes, Carter, and O'Boyle (2018), models were fitted using normal-theory-based MLE via R package OpenMx (Neale et al., 2016).

²⁰ Results without ${\omega }_i$ are available in Section 3 in the supplemental materials.

²¹ Here random parameters are assumed to be uncorrelated. For the situation with correlated random parameters, please refer to https://github.com/zijunke/Bayesian-MASEM.

Additional information

Funding

This work was supported by the National Natural Science Foundation of China [31700986].