A Comparison of Fixed-Effects and Random-Effects Models for Multivariate Meta-Analysis Using an SEM Approach

References

• Ackerman, D. L., & Greenland, S. (2002). Multivariate meta-analysis of controlled drug studies for obsessive-compulsive disorder. Journal of Clinical Psychopharmacology, 22(3), 309–317. doi:10.1097/00004714-200206000-00012
   PubMed Web of Science ®Google Scholar
• Ades, A. E., Lu, G., & Higgins, J. P. (2005). The interpretation of random-effects meta-analysis indecision models. Medical Decision Making, 25(6), 646–654. doi:10.1177/0272989X05282643
   PubMed Web of Science ®Google Scholar
• Aloe, A. M., Amo, L. C., & Shanahan, M. E. (2014). Classroom management self-efficacy and burnout: A multivariate meta-analysis. Educational Psychology Review, 26(1), 101–126. doi:10.1007/s10648-013-9244-0
   Web of Science ®Google Scholar
• Aloe, A. M., Shisler, S. M., Norris, B. D., Nickerson, A. B., & Rinker, T. W. (2014). A multivariate meta-analysis of student misbehavior and teacher burnout. Educational Research Review, 12, 30–44. doi:10.1016/j.edurev.2014.05.003
   Web of Science ®Google Scholar
• Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), The handbook of applied multivariate statistics and mathematical modeling (pp. 499–525). San Diego: Academic Press.
   Google Scholar
• Becker, B. J., & Schram, C. M. (1994). Examining explanatory models through research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 357–381). New York, NY: Russell Sage Foundation.
   Google Scholar
• Berkey, C. S., Anderson, J., & Hoaglin, D. C. (1996). Multiple-outcome meta-analysis of clinical trials. Statistics in Medicine, 15(5), 537–557. doi:10.1002/(SICI)1097-0258(19960315)15:5 < 537::AID-SIM176 > 3.0.CO;2-S
   PubMed Web of Science ®Google Scholar
• Berkey, C. S., Antczak-Bouckoms, A., Hoaglin, D. C., Mosteller, F., & Pihlstrom, B. L. (1995). Multiple outcomes meta-analysis of treatments for periodontal disease. Journal of Dental Research, 74(4), 1030–1039. doi:10.1177/00220345950740040201
   PubMed Web of Science ®Google Scholar
• Berkey, C. S., Hoaglin, D. C., Antczak-Bouckoms, A., Mosteller, F., & Colditz, G. A. (1998). Meta-analysis of multiple outcomes by regression with random effects. Statistics in Medicine, 17(22), 2537–2550. doi:10.1002/(SICI)1097-0258(19981130)17:22 < 2537::AID-SIM953 > 3.0.CO;2-C
   PubMed Web of Science ®Google Scholar
• Boca, S. M., Pfeiffer, R. M., & Sampson, J. N. (2017). Multivariate meta-analysis with an increasing number of parameters. Biometrical Journal, 59(3), 496–510. doi:10.1002/bimj.201600013
   PubMed Web of Science ®Google Scholar
• Borenstein, M., Hedges, H. L., Higgins, J., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: Wiley.
   Google Scholar
• Burton, A., Altman, D. G., Royston, P., & Holder, R. L. (2006). The design of simulation studies in medical statistics. Statistics in Medicine, 25(24), 4279–4292. doi:10.1002/sim.2673
   PubMed Web of Science ®Google Scholar
• Card, N. A. (2012). Applied meta-analysis for social science research. New York, NY: The Guilford Press.
   Google Scholar
• Cheung, M. W. L. (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychological Methods, 13(3), 182–202. doi:10.1037/a0013163
   PubMed Web of Science ®Google Scholar
• Cheung, M. W. L. (2013). Multivariate meta-analysis as structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 20(3), 429–454. doi:10.1080/10705511.2013.797827
   Web of Science ®Google Scholar
• Cheung, M. W. L. (2015a). Meta-analysis: A structural equation modeling approach. Chichester: Wiley.
   Google Scholar
• Cheung, M. W. L. (2015b). metaSEM: An R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5, 1521. doi:10.3389/fpsyg.2014.01521
   PubMed Web of Science ®Google Scholar
• Cheung, M. W. L. (2018). Computing multivariate effect sizes and their sampling covariance matrices with structural equation modeling: Theory, examples, and computer simulations. Frontiers in Psychology, 9, 1387. doi:10.3389/fpsyg.2018.01387
   PubMed Web of Science ®Google Scholar
• Cheung, M. W. L., & Chan, W. (2004). Testing dependent correlation coefficients via structural equation modeling. Organizational Research Methods, 7(2), 206–223. doi:10.1177/1094428104264024
   Web of Science ®Google Scholar
• Cheung, M. W. L., & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10(1), 40–64. doi:10.1037/1082-989X.10.1.40
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
   Google Scholar
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. doi:10.1037/0033-2909.112.1.155
   PubMed Web of Science ®Google Scholar
• Conn, V. S., Hafdahl, A. R., Brown, S. A., & Brown, L. M. (2008). Meta-analysis of patient education interventions to increase physical activity among chronically ill adults. Patient Education and Counseling, 70(2), 157–172. doi:10.1016/j.pec.2007.10.004
   PubMed Web of Science ®Google Scholar
• Dear, K. B. (1994). Iterative generalized least squares for meta-analysis of survival data at multiple times. Biometrics, 50(4), 989–1002. doi:10.2307/2533438
   PubMed Web of Science ®Google Scholar
• Field, A. P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed-and random-effects methods. Psychological Methods, 6(2), 161. doi:10.1037/1082-989X.6.2.161
   PubMed Web of Science ®Google Scholar
• Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8. doi:10.3102/0013189X005010003
   Google Scholar
• Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage Publications.
   Google Scholar
• Gleser, L. J., & Olkin, I. (1994). Stochastically dependent effect sizes. In H. M. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 339–355). New York: Russell Sage Foundation.
   Google Scholar
• Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. M. Cooper, L. V. Hedges & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 357–376). New York, NY: Russell Sage Foundation.
   Google Scholar
• Hafdahl, A. R. (2001). Multivariate meta-analysis for exploratory factor analytic research (Doctoral dissertation), The University of North Carolina, Chapel Hill.
   Google Scholar
• Hafdahl, A. R. (2008). Combining heterogeneous correlation matrices: Simulation analysis of fixed-effects methods. Journal of Educational and Behavioral Statistics, 33(4), 507–533. doi:10.3102/1076998607309472
   Web of Science ®Google Scholar
• Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
   Google Scholar
• Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random effects models in meta-analysis. Psychological Methods, 3(4), 486–504. doi:10.1037//1082-989X.3.4.486
   Web of Science ®Google Scholar
• Higgins, J. P., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1), 137–159. doi:10.1111/j.1467-985X.2008.00552.x
   PubMed Web of Science ®Google Scholar
• Hong, C., Salanti, G., Morton, S., Riley, R., Chu, H., Kimmel, S. E., & Chen, Y. (2018). Testing small study effects in multivariate meta-analysis. arXiv Preprint arXiv: 1805.09876.
   Google Scholar
• Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling an overview and a meta-analysis. Sociological Methods & Research, 26, 329–367. doi:10.1177/0049124198026003003
   Web of Science ®Google Scholar
• Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Beverly Hills, CA: Sage.
   Google Scholar
• Hunter, J. E., & Schmidt, F. L. (2000). Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge. International Journal of Selection and Assessment, 8(4), 275–292. doi:10.1111/1468-2389.00156
   Web of Science ®Google Scholar
• Ishak, K. J., Platt, R. W., Joseph, L., & Hanley, J. A. (2008). Impact of approximating or ignoring within-study covariances in multivariate meta-analyses. Statistics in Medicine, 27(5), 670–686. doi:10.1002/sim.2913
   PubMed Web of Science ®Google Scholar
• Jackson, D., Riley, R., & White, I. R. (2011). Multivariate meta-analysis: Potential and promise. Statistics in Medicine, 30(20), 2509–2498. doi:10.1002/sim.4172
   Web of Science ®Google Scholar
• Jackson, D., White, I. R., & Thompson, S. G. (2010). Extending DerSimonian and Laird’s methodology to perform multivariate random effects meta-analyses. Statistics in Medicine, 29(12), 1282–1297. doi:10.1002/sim.3602
   PubMed Web of Science ®Google Scholar
• Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1(3), 227–235. doi:10.1037/1082-989X.1.3.227
   Web of Science ®Google Scholar
• Ke, Z., Zhang, Q., & Tong, X. (2019). Bayesian meta-analytic SEM: A one-stage approach to modeling between-studies heterogeneity in structural parameters. Structural Equation Modeling: A Multidisciplinary Journal, 26(3), 348–370. doi:10.1080/10705511.2018.1530059
   Web of Science ®Google Scholar
• Kim, R.-S., & Becker, B. J. (2010). The degree of dependence between multiple treatment effect sizes. Multivariate Behavioral Research, 45(2), 213–238. doi:10.1080/00273171003680104
   PubMed Web of Science ®Google Scholar
• Kirkham, J. J., Riley, R. D., & Williamson, P. R. (2012). A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews. Statistics in Medicine, 31(20), 2179–2195. doi:10.1002/sim.5356
   PubMed Web of Science ®Google Scholar
• Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis (Vol. 49). Thousand Oaks: Sage.
   Google Scholar
• Little, R. J. A., & Rubin, D. B. (2014). Statistical analysis with missing data (2nd ed.). Hoboken, NJ: Wiley.
   Google Scholar
• Nam, I.-S., Mengersen, K., & Garthwaite, P. (2003). Multivariate meta-analysis. Statistics in Medicine, 22(14), 2309–2333. doi:10.1002/sim.1410
   PubMed Web of Science ®Google Scholar
• National Research Council. (1992). Combining information: Statistical issues and opportunities for research. Washington, DC: National Academy of Sciences Press.
   Google Scholar
• Overton, R. C. (1998). A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator variable effects. Psychological Methods, 3(3), 354–379. doi:10.1037/1082-989X.3.3.354
   Web of Science ®Google Scholar
• Preacher, K. J. (2006). Testing complex correlational hypotheses with structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 13(4), 520–543. doi:10.1207/s15328007sem1304_2
   Web of Science ®Google Scholar
• R Development Core Team. (2018). R: A language and environment for statistical computing, Vienna. Retrieved from http://www.R-project.org/.
   Google Scholar
• Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(4), 789–811. doi:10.1111/j.1467-985X.2008.00593.x
   Web of Science ®Google Scholar
• Riley, R. D., Abrams, K. R., Lambert, P. C., Sutton, A. J., & Thompson, J. R. (2007). An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Statistics in Medicine, 26(1), 78–97. doi:10.1002/sim.2524
   PubMed Web of Science ®Google Scholar
• Riley, R. D., Gates, S., Neilson, J., & Alfirevic, Z. (2011). Statistical methods can be improved within Cochrane pregnancy and childbirth reviews. Journal of Clinical Epidemiology, 64(6), 608–618. doi:10.1016/j.jclinepi.2010.08.002
   PubMed Web of Science ®Google Scholar
• Rosenthal, R. (1991). Meta-analytic procedures for social research (2nd ed.). Newbury Park, CA: Sage.
   Google Scholar
• Schmidt, F. L., Oh, I. S., & Hayes, T. L. (2009). Fixed- versus random- effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62(1), 97–128. doi:10.1348/000711007X255327
   PubMed Web of Science ®Google Scholar
• Schroll, J. B., Moustgaard, R., & Gotzsche, P. C. (2011). Dealing with substantial heterogeneity in Cochrane reviews. Cross-sectional study. Medical Research Methodology, 11(1), 22. doi:10.1186/1471-2288-11-22
   Web of Science ®Google Scholar
• Sheng, Z., Kong, W., Cortina, J. M., & Hou, S. (2016). Analyzing matrices of meta-analytic correlations: Current practices and recommendations. Research Synthesis Methods, 7(2), 187–208. doi:10.1002/jrsm.1206
   PubMed Web of Science ®Google Scholar
• Stram, D. O. (1996). Meta-analysis of published data using a linear mixed-effects model. Biometrics, 52(2), 536–544. doi:10.2307/2532893
   PubMed Web of Science ®Google Scholar
• Trikalinos, T. A., Hoaglin, D. C., & Schmid, C. H. (2014). An empirical comparison of univariate and multivariate meta-analyses for categorical outcomes. Statistics in Medicine, 33(9), 1441–1459. doi:10.1002/sim.6044
   PubMed Web of Science ®Google Scholar
• Wei, Y., & Higgins, J. P. (2013). Estimating within-study covariances in multivariate meta-analysis with multiple outcomes. Statistics in Medicine, 32(7), 1191–1205. doi:10.1002/sim.5679
   PubMed Web of Science ®Google Scholar