821
Views
10
CrossRef citations to date
0
Altmetric
Articles

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling

ORCID Icon, , , , &

References

  • Bollen, K. A., & Stine, R. A. (1992). Bootstrapping goodness-of-fit measures in structural equation models. Sociological Methods and Research, 21, 205–229. doi:10.1177/0049124192021002004.
  • Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. doi:10.1037/0033-2909.105.3.456.
  • Cohen, J. (1994). The earth is round (p<. 05). American Psychologist, 49(12), 997–1003. doi:10.1037/0003-066X.49.12.997
  • Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., & Paxton, P. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research, 36(4), 462–494. doi:10.1177/0049124108314720.
  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. doi:10.1080/10705510701301834.
  • Cheung, G. W., & Rensvold, R. B. (1998). Cross-cultural comparisons using non-invariant measurement items. Applied Behavioral Science Review, 6(1), 93–110. doi:10.1016/S1068-8595(99)80006-3.
  • Cheung, G. W., & Rensvold, R. B. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of management, 25(1), 1–27. doi:10.1177/014920639902500101.
  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. doi:10.1207/S15328007SEM0902_5.
  • Dienes, Z. (2011). Bayesian Versus Orthodox Statistics: Which Side Are You On? Perspectives on Psychological Science, 6(3), 274–290. doi:10.1177/1745691611406920.
  • DiStefano, C., & Morgan, G. B. (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 425–438. doi:10.1080/10705511.2014.915373.
  • Depaoli, S. (2013). Mixture class recovery in GMM under varying degrees of class separation: frequentist versus Bayesian estimation. Psychological Methods, 18(2), 186–219. doi:10.1037/a0031609.
  • Depaoli, S. (2014). The impact of inaccurate “informative” priors for growth parameters in Bayesian growth mixture modeling. Structural Equation Modeling: A Multidisciplinary Journal, 21(2), 239–252. doi:10.1080/10705511.2014.882686.
  • Depaoli, S., Yang, Y., & Felt, J. (2017). Using bayesian statistics to model uncertainty in mixture models: a sensitivity analysis of priors. Structural Equation Modeling: A Multidisciplinary Journal, 24(2), 198–215. doi:10.1080/10705511.2016.1250640.
  • Dunnett, C. W., & Gent, M. (1977). Significance testing to establish equivalence between treatments, with special reference to data in the form of 2 x 2 tables. Biometrics, 33, 593–602. doi:10.2307/2529457.
  • Edwards, M. C. (2013). Purple unicorns, true models, and other things I’ve never seen. Measurement: Interdisciplinary Research & Perspective, 11(3), 107–111. doi:10.1080/15366367.2013.835178.
  • Edwards, M. C., Cheavens, J. S., Heiy, J. E., & Cukrowicz, K. C. (2010). A reexamination of the factor structure of the Center for Epidemiologic Studies Depression Scale: Is a one-factor model plausible? Psychological Assessment, 22(3), 711–715. doi:10.1037/a0019917.
  • Efron, B., & Tibshirani, R. (1993). An introduction to the bootstrap. New York, NY: Chapman & Hall.
  • Falk, C. F. (2018). Are Robust standard errors the best approach for interval estimation with nonnormal data in structural equation modeling? Structural Equation Modeling: A Multidisciplinary Journal, 25(2), 244–266. doi:10.1080/10705511.2017.1367254.
  • Finch, W. H., & French, B. F. (2018). A simulation investigation of the performance of invariance assessment using equivalence testing procedures. Structural Equation Modeling: A Multidisciplinary Journal, 25, 1–14. doi:10.1080/10705511.2018.1431781.
  • French, B., & Finch, W. H. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 13(3), 378–402. doi:10.1207/s15328007sem1303_3.
  • French, B. F., & Finch,W. H. (2016). Factorial invariance testing under different levels of partial loading invariance within a multiple group confirmatory factor analysis model. Journal of Modern Applied Statistical Methods, 15(1), 511–538. doi:10.22237/jmasm/1462076700.
  • Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. doi:10.1214/ss/1177011136
  • Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003). Bayesian data analysis (2nd ed.). London: Chapman and Hall.
  • Gill, J. (2014). Bayesian methods: A social and behavioral sciences approach. Boca Raton, FL: CRC.
  • Hox, J. J., van de Schoot, R., & Matthijsse, S. (2012). How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Methods, 6, 87–93. doi:10.18148/srm/2012.v6i2.5033.
  • Horn, J. L., & Mcardle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144. doi:10.1080/03610739208253916.
  • Jiang, Z., & Carter, R. (2018). Using Hamiltonian Monte Carlo to estimate the log-linear cognitive diagnosis model via Stan. Behavior Research Methods, 1–12. doi:10.3758/s13428-018-1069-9.
  • Jiang, Z., & Skorupski, W. (2017). A Bayesian approach to estimating variance components within a multivariate generalizability theory framework. Behavior Research Methods, 1–22. doi:10.3758/s13428-017-0986-3.
  • Jiang, Z., & Templing, J. (2018). Constructing Gibbs samplers for Bayesian logistic item response models. Multivariate Behavioral Research, 53(1), 132–133. doi:10.1080/00273171.2017.1404897.
  • Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409–426. doi:10.1007/BF02291366.
  • Johnson, E. C., Meade, A. W., & DuVernet, A. M. (2009). The role of referent indicators in tests of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 16(4), 642–657. doi:10.1080/10705510903206014.
  • Kaplan, D. (2014). Bayesian statistics for the social sciences. New York: Guilford.
  • Kaplan, D., & Depaoli, S. (2012). Bayesian structural equation modeling. In R. Hoyle (Ed.), Handbook of structural equation modeling (pp. 650–673). New York: Guilford Press.
  • Kim, E. S., & Yoon, M. (2011). Testing measurement invariance: A comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling: A Multidisciplinary Journal, 18(2), 212–228. doi:10.1080/10705511.2011.557337.
  • Kim, E. S., Yoon, M., & Lee, T. (2012). Testing measurement invariance using MIMIC: Likelihood ratio test with a critical value adjustment. Educational and Psychological Measurement, 72(3), 469–492. doi:10.1177/0013164411427395.
  • Kim, S.-Y., Suh, Y., Kim, J.-S., Albanese, M., & Langer, M. M. (2013). Single and multiple ability estimation in the SEM framework: A noninformative Bayesian estimation approach. Multivariate and Behavioral Research, 48(4), 563–591. doi:10.1080/00273171.2013.802647.
  • Kruschke, J. K. (2011). Bayesian assessment of null values via parameter estimation and model comparison. Perspectives on Psychological Science, 6(3), 299–312. doi:10.1177/1745691611406925.
  • Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142(2), 573–603. doi:10.1037/a0029146.
  • Kruschke, J. K. (2014). Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan. Burlington, MA: Academic Press.
  • Kruschke, J. K., Aguinis, H., & Joo, H. (2012). The time has come: Bayesian methods for data analysis in the organizational sciences. Organizational Research Methods, 15(4), 722–752. doi:10.1177/1094428112457829.
  • Kruschke, J. K., & Liddell, T. M. (2018). The Bayesian new statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25(1), 178–206. doi:10.3758/s13423-016-1221-4.
  • Kruschke, J. K., & Meredith, M. (2015). Package “BEST.” Retrieved from https://cran.r-project.org/web/packages/BEST/BEST.pdf
  • Lai, M. H., Kwok, O. M., Yoon, M., & Hsiao, Y. Y. (2017). Understanding the impact of partial factorial invariance on selection accuracy: An R script. Structural Equation Modeling: A Multidisciplinary Journal, 24(5), 783–799. doi:10.1080/10705511.2017.1318703.
  • Lee, S. Y. (2007). Structural Equation Modeling: A Bayesian Approach. West Sussex, England: Wiley.
  • MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149. doi:10.1037/1082-989X.1.2.130.
  • MacCallum, R. C., Edwards, M. C., & Cai, L. (2012). Hopes and cautions in implementing Bayesian structural equation modeling. Psychological Methods, 17(3), 340–345. doi:10.1037/a0027131.
  • Marcoulides, K. M., & Yuan, K. H. (2017). New ways to evaluate goodness of fit: A note on using equivalence testing to assess structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 24(1), 148–153. doi:10.1080/10705511.2016.1225260.
  • Martin, A. (2008). Bayesian analysis. Retrieved from https://deepblue.lib.umich.edu/bitstream/handle/2027.42/116259/Bayesian_Analysis.pdf?sequence =1&isAllowed=y
  • Maydeu-Olivares, A. (2017). Assessing the size of model misfit in structural equation models. Psychometrika, 82(3), 533–558. doi:10.1007/s11336-016-9552-7.
  • Maydeu-Olivares, A., & Cai, L. (2006). A cautionary note on using G2 (dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research,41(1), 55–64. doi:10.1207/s15327906mbr4101_4.
  • Maydeu-Olivares, A., Shi, D. & Rosseel, Y. (2018). Assessing fit in structural equation models: A Monte Carlo evaluation of RMSEA vs. SRMR confidence intervals and tests of close fit. Structural Equation Modeling: A Multidisciplinary Journal. 25(3), 389–402. doi:10.1080/10705511.2017.1389611.
  • McGraw, B., & Jöreskog, K. G. (1971). Factorial invariance of ability measures in groups differing in intelligence and social economic status. British Journal of Mathematical and Statistical Psychology, 24(2), 154–168. doi:10.1111/j.2044-8317.1971.tb00463.x.
  • Meade, A. W. (2010). A taxonomy of effect size measures for the differential functioning of items and scales. Journal of Applied Psychology, 95(4), 728–743. doi:10.1037/a0018966.
  • Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. doi:10.1037/0021-9010.93.3.568.
  • Meade, A. W., & Lautenschlager, G. J. (2004). A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance. Organizational Research Methods, 7(4), 361–388. doi:10.1177/1094428104268027.
  • Meade, A. W., & Wright, N. A. (2012). Solving the measurement invariance anchor item problem in item response theory. Journal of Applied Psychology, 97(5), 1016–1031. doi:10.1037/a0027934.
  • Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. doi:10.1007/BF02294825.
  • Millsap, R. E. (2011). Statistical approaches to measurement invariance. New York: Routledge.
  • Millsap, R. E., & Kwok, O. M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9(1), 93–115. doi:10.1037/1082-989X.9.1.93.
  • Muthén, B., & Asparouhov, T. (2012a). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313–335. doi:10.1037/a0026802.
  • Muthén, B., & Asparouhov, T. (2012b). New developments in Mplus Version 7. Retrieved from http://www.statmodel.com/download/handouts/MuthenV7Part1.pdf
  • Muthén, L.K. and Muthén, B.O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén
  • Nye, C. D., Bradburn, J., Olenick, J., Bialko, C., & Drasgow, F. (2018). How big are my effects? Examining the magnitude of effect sizes in studies of measurement equivalence. Organizational Research Methods, 1–32. doi:10.1037/a0022955.
  • Nye, C. D., & Drasgow, F. (2011). Effect size indices for analyses of measurement equivalence: Understanding the practical importance of differences between groups. Journal of Applied Psychology, 96(5), 966–980. doi:10.1037/a0022955.
  • R Development Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
  • Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. doi:10.1177/014662167700100306.
  • Raykov, T., Marcoulides, G. A., & Millsap, R. E. (2013). Factorial invariance in multiple populations: A multiple testing procedure. Educational and Psychological Measurement, 73(4), 713–727.
  • Rivas, G. E., Stark, S., & Chernyshenko, O. S. (2009). The effects of referent item parameters on differential item functioning detection using the free baseline likelihood ratio test. Applied Psychological Measurement, 33(4), 251–265. doi:10.1177/0146621608321760.
  • Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. doi:10.1037/a0029315.
  • Saris, W. E., Satorra, A., & van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling: A Multidisciplinary Journal, 16(4), 561–582. doi:10.1080/10705510903203433.
  • Satorra, A., & Bentler, P. M. (1988). A scaled difference chi-square test statistic for moment structure analysis. In 1998 Proceedings of the Business & Statistics Section (pp. 308–313). Alexandria, VA: American Statistical Association.
  • Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variable analysis: Applications for developmental research (pp. 399–419). Thousand Oaks, CA: Sage.
  • Savalei, V. (2012). The relationship between root mean square error of approximation and model misspecification in confirmatory factor analysis models. Educational and Psychological Measurement, 72(6), 910–932. doi:10.1177/0013164412452564.
  • Shi, D. (2016). Resolving three important issues on measurement invariance using bayesian structural equation modeling (BSEM) (Unpublished Doctoral Dissertation). University of Oklahoma, Norman, OK.
  • Shi, D., Maydeu-Olivares, A. & Distefano, C. (2018). The relationship between the standardized root mean square residual and model misspecification in factor analysis models. Multivariate Behavioral Research. doi:10.1080/00273171.2018.1476221.
  • Shi, D., Song, H., & Lewis, M. D. (2017). The impact of partial factorial invariance on cross-group comparisons. Assessment, 1–17. doi:10.1177/1073191117711020.
  • Shi, D., Song, H., Liao, X., Terry, R., & Snyder, L. A. (2017). Bayesian SEM for specification search problems in testing factorial invariance. Multivariate Behavioral Research, 52(4), 430–444. doi:10.1080/00273171.2017.1306432.
  • Shi, D., Lee, T., & Maydeu-Olivares, A. (2018). Understading the model size effect on SEM fit indices. Educational and Psychological Measurement, 1–25. doi:10.1177/0013164418783530.
  • Shi, D., & Tong, X. (2017). The impact of prior information on Bayesian latent basis growth model estimation. SAGE Open, 7(3), 1–14. doi:10.1177/2158244017727039.
  • Song, X. Y., & Lee, S. Y. (2012). Basic and advanced Bayesian structural equation modeling: with applications in the medical and behavioral sciences. Hoboken, NJ: Wiley.
  • Stark, S., Chernyshenko, O. S., & Drasgow, F. (2004). Examining the effects of differential item (functioning and differential) test functioning on selection decisions: when are statistically significant effects practically important? Journal of Applied Psychology, 89(3), 497–508. doi:10.1037/0021-9010.89.3.497.
  • Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. Journal of Applied Psychology, 91(6), 1292–306. doi:10.1037/0021-9010.91.6.1292.
  • Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing Measurement Invariance in Cross-National Consumer Research. Journal of Consumer Research, 25(1), 78–107. doi:10.1086/209528.
  • Steiger, J. H. (2002). When constraints interact: A caution about reference variables, identification constraints, and scale dependencies in structural equation modeling. Psychological Methods, 7(2), 210–227. doi:10.1037/1082-989X.7.2.210.
  • Steiger, J. H., Shapiro, A., & Browne, M. W. (1985). On the multivariate asymptotic distribution of sequential Chi-square statistics. Psychometrika, 50(3), 253–263. doi:10.1007/BF02294104.
  • Steinberg, L., & Thissen, D. (2006). Using effect sizes for research reporting: examples using item response theory to analyze differential item functioning. Psychological Methods, 11(4), 402–15. doi:10.1037/1082-989X.11.4.402.
  • Tanner, M. A., & Wong, W. H. (1987). The calculation of posterior distributions by data augmentation: Rejoinder. Journal of the American Statistical Association, 82(398), 548–550. doi:10.2307/2289463.
  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. doi:10.1177/109442810031002.
  • Verhagen, A. J., & Fox, J. P. (2013). Bayesian tests of measurement invariance. British Journal of Mathematical and Statistical Psychology, 66, 383–401. doi:10.1111/j.2044-8317.2012.02059.x.
  • Verhagen, J., Levy, R., Millsap, R. E., & Fox, J. P. (2016). Evaluating evidence for invariant items: A Bayes factor applied to testing measurement invariance in IRT models. Journal of Mathematical Psychology, 72, 171–182. doi:10.1016/j.jmp.2015.06.005.
  • Wellek, S. (2010). Testing statistical hypotheses of equivalence and noninferiority (2nd ed.). Boca Raton, FL: Chapman & Hall/CRC.
  • Woods, C. M. (2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33(1), 42–57. doi:10.1177/0146621607314044.
  • Xie, Y., & Hu, J. (2014). An introduction to the China Family Panel Studies (CFPS). Chinese Sociological Review, 47(1), 3–29. doi:10.2753/CSA2162-0555470101.
  • Yoon, M., & Millsap, R. E. (2007). Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling, 14(3), 435–463. doi:10.1080/10705510701301677.
  • Yuan, K. H., & Bentler, P. M. (2004). On chi-square difference and z tests in mean and covariance structure analysis when the base model is misspecified. Educational and Psychological Measurement, 64(5), 737–757. doi:10.1177/0013164404264853.
  • Yuan, K. H., & Chan, W. (2016). Measurement invariance via multigroup SEM: Issues and solutions with chi-square-difference tests. Psychological Methods, 21(3), 405–426. doi:10.1037/met0000080.
  • Yuan, K. H., Chan, W., Marcoulides, G. A., & Bentler, P. M. (2016). Assessing structural equation models by equivalence testing with adjusted fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 23(3), 319–330. doi:10.1080/10705511.2015.1065414
  • Zondervan-Zwijnenburg, M., Peeters, M., Depaoli, S., & Van de Schoot, R. (2017). Where do priors come from? Applying guidelines to construct informative priors in small sample research. Research in Human Development, 14(4), 305–320. doi:10.1080/15427609.2017.1370966.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.