Specification Searches in Multilevel Structural Equation Modeling: A Monte Carlo Investigation

REFERENCES

• Akaike , H. 1973 . “ Information theory and the extension of the maximum likelihood principle ” . In 2nd International Symposium on Information Theory , Edited by: Petrov , B. N. and Csaki , F. 267 – 281 . Budapest, Hungary : Akademia Kiado .
   Google Scholar
• Berger , M. P. 1978 . A note on the use of simultaneous test procedures . Psychological Bulletin , 85 : 895 – 897 .
   Google Scholar
• Bollen , K. A. 1989 . Structural equations with latent variables , New York : Wiley .
   Google Scholar
• Boomsma , A. 1983 . On the robustness of LISREL (maximum likelihood estimation) against small sample size and nonnormality , Amsterdam : Sociometric Research Foundation .
   Google Scholar
• Burt , S. A. , McGue , M. and Krueger , R. F. 2007 . Environmental contributions to adolescent delinquency: A fresh look at the shared environment . Journal of Abnormal Child Psychology , 35 : 787 – 800 .
   PubMedGoogle Scholar
• Chou , C.-P. and Bentler , P. M. 1990 . Model modification in covariance structure modeling: A comparison among likelihood ratio, Wald, and Lagrange multiplier tests . Multivariate Behavioral Research , 25 : 115 – 136 .
   PubMed Web of Science ®Google Scholar
• Collins , L. M. , Schafer , J. L. and Kam , C.-M. 2001 . A comparison of inclusive and restrictive strategies in modern missing data procedures . Psychological Methods , 6 : 330 – 351 .
   PubMed Web of Science ®Google Scholar
• Cronbach , L. J. and Webb , N. 1975 . Between-class and within-class effects of repeated aptitude x treatment interaction: Re-analysis of a study by G. L. Anderson . Journal of Educational Psychology , 67 : 717 – 724 .
   Web of Science ®Google Scholar
• Fan , X. and Wang , L. 1998 . Effects of potential confounding factors on fit indices and parameter estimates for true and miss-specified models . Structural Equation Modeling , 5 : 701 – 735 .
   Google Scholar
• Fan , X. , Wang , L. and Thompson , B. 1997 . Effects of data non-normality and other factors on fit indices and parameter estimates for the true and miss-specified SEM models . Paper presented at the annual meeting of the American Educational Research Association . April 1997 , Chicago.
   Google Scholar
• Finch , J. F. , West , S. G. and MacKinnon , D. P. 1997 . Effects of sample size and non-normality on the estimation of mediated effects in latent variable models . Structural Equation Modeling , 4 : 87 – 105 .
   Web of Science ®Google Scholar
• Green , S. B. and Babyak , M. A. 1997 . Control of Type I errors with multiple tests of constraints in structural equation modeling . Multivariate Behavioral Research , 32 : 39 – 51 .
   PubMed Web of Science ®Google Scholar
• Green , S. B. , Thompson , M. S. and Babyak , M. A. 1998 . A Monte Carlo investigation of methods for controlling Type-I errors with specification searches in structural equation modeling . Multivariate Behavioral Research , 33 : 365 – 383 .
   PubMed Web of Science ®Google Scholar
• Green , S. B. , Thompson , M. S. and Poirier , J. 1999 . Exploratory analyses to improve model fit: Errors due to misspecification and a strategy to reduce their occurrence . Structural Equation Modeling , 6 : 113 – 126 .
   Web of Science ®Google Scholar
• Green , S. B. , Thompson , M. S. and Poirier , J. 2001 . An adjusted Bonferroni method for elimination of parameters in specification addition searches . Structural Equation Modeling , 8 : 18 – 39 .
   Web of Science ®Google Scholar
• Harnqvist , K. 1978 . Primary mental abilities of collective and individual levels . Journal of Educational Psychology , 70 : 706 – 716 .
   Web of Science ®Google Scholar
• Heck , R. H. 2001 . “ Multilevel modeling with SEM ” . In New developments and techniques in structural equation modeling , Edited by: Marcoulides , G. A. and Schumacker , R. E. 89 – 127 . Mahwah, NJ : Lawrence Erlbaum Associates, Inc. .
   Google Scholar
• Henry , G. T. 1990 . Practical sampling , Newbury Park, CA : Sage .
   Google Scholar
• Holm , S. 1979 . A simple sequentially rejective multiple test procedure . Scandinavian Journal of Statistics , 6 : 65 – 70 .
   Web of Science ®Google Scholar
• Hox , J. J. 1993 . “ Factor analysis of multilevel data: Gauging the Muthén model ” . In Advances in longitudinal and multivariate analysis in the behavioral sciences , Edited by: Oud , J. H. L. and van Blokland-Vogelesang , R. A. W. 141 – 156 . Nijmegen, The Netherlands : ITS. .
   Google Scholar
• Hox , J. J. 1994 . Applied multilevel analysis , Amsterdam : TT-Publikaties .
   Google Scholar
• Hox , J. J. 2002 . Multilevel analysis: Techniques and applications , Mahwah, NJ : Lawrence Erlbaum Associates, Inc. .
   Google Scholar
• Hox, J. J. (2004). Multilevel multivariate and structural equation models: Some missing links. Unpublished manuscript. Retrieved June 1, 2007, from http://www.fss.uu.nl/ms/jh/papers/papers.htm
   Google Scholar
• Hox , J. J. and Maas , C. J. M. 2001 . The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples . Structural Equation Modeling , 8 : 157 – 174 .
   Web of Science ®Google Scholar
• Jöreskog , K. G. and Sörbom , D. 1984 . LISREL-VI: Analysis of linear structural relationships by the method of maximum likelihood , Mooresville, IL : Scientific Software .
   Google Scholar
• Jöreskog , K. G. and Sörbom , D. 1989 . LISREL 7: A guide to the program and applications , Chicago, IL : SPSS .
   Google Scholar
• Julian , M. W. 2001 . The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling . Structural Equation Modeling , 8 : 325 – 352 .
   Web of Science ®Google Scholar
• Kaplan , D. 1988 . The impact of specification error on the estimation, testing, and improvement of structural equation models . Multivariate Behavioral Research , 23 : 69 – 86 .
   PubMed Web of Science ®Google Scholar
• Kaplan , D. 2000 . Structural equation modeling: Foundations and extensions , Thousand Oaks, CA : Sage .
   Google Scholar
• Kaplan , D. and Elliott , P. R. 1997a . A didactic example of multilevel structural equation modeling applicable to the study of organizations . Structural Equation Modeling , 4 : 1 – 24 .
   Web of Science ®Google Scholar
• Kaplan , D. and Elliott , P. R. 1997b . A model-based approach to validating education indicators using multilevel structural equation modeling . Journal of Educational and Behavioral Statistics , 22 : 323 – 347 .
   Web of Science ®Google Scholar
• Kaplan , D. and Kreisman , M. B. 2000 . On the validation of indicators of mathematics education using TIMMS: An application of multilevel covariance structure modeling . International Journal of Educational Policy, Research, & Practice , 1 : 217 – 242 .
   Google Scholar
• Kaplan , D. and Wenger , R. N. 1993 . Asymptotic independence and separability in covariance structure models. Implications for specification error, power, and model modification . Multivariate Behavioral Research , 28 : 467 – 482 .
   PubMed Web of Science ®Google Scholar
• Kline , R. B. 2004 . Principles and practices of structural equation modeling , 2 New York : Guilford .
   Google Scholar
• Leite , W. L. , Huang , I.-C. and Marcoulides , G. A. 2008 . Item selection for the development of short-forms of scales using an ant colony optimization algorithm . Multivariate Behavioral Research , 43 : 411 – 431 .
   PubMed Web of Science ®Google Scholar
• Li , F. , Duncan , T. E. , Harmer , P. , Acock , A. and Stoolmiller , M. 1998 . Analyzing measurement models of latent variables through multilevel confirmatory factor analysis and hierarchical linear modeling approaches . Structural Equation Modeling , 5 : 294 – 306 .
   Web of Science ®Google Scholar
• Maas, C. J. M., & Hox, J. J. (2004). Robustness of multilevel parameter estimates against small sample sizes. Unpublished manuscript. Retrieved July 11, 2007, from http://www.fss.uu.nl/ms/jh/papers/papers.htm
   Google Scholar
• MacCallum , R. 1986 . Specification searches in covariance structure modeling . Psychological Bulletin , 100 : 107 – 120 .
   Web of Science ®Google Scholar
• Marcoulides , G. A. and Drezner , Z. 2003 . Model specification searches using Ant Colony optimization algorithms . Structural Equation Modeling , 10 : 154 – 164 .
   Web of Science ®Google Scholar
• Marcoulides , G. A. , Drezner , Z. and Schumacker , R. E. 1998 . Model specification searches in structural equation modeling using Tabu search . Structural Equation Modeling , 5 : 365 – 376 .
   Web of Science ®Google Scholar
• Micceri , T. 1989 . The unicorn, the normal curve, and other improbable creatures . Psychological Bulletin , 105 : 156 – 166 .
   Web of Science ®Google Scholar
• Muthén , B. O. 1989 . Latent variable modeling in heterogeneous populations . Psychometrika , 54 : 557 – 585 .
   Web of Science ®Google Scholar
• Muthén , B. O. 1990 . Means and covariance structure analysis of hierarchical data , UCLA Statistics Series No. 62 Los Angeles, CA : UCLA. .
   Google Scholar
• Muthén , B. O. 1991 . Multilevel factor analysis of class and student achievement components . Journal of Educational Measurement , 28 : 338 – 354 .
   Web of Science ®Google Scholar
• Muthén , B. O. 1994 . Multilevel covariance structure analysis . Sociological Methods and Research , 22 : 376 – 398 .
   Web of Science ®Google Scholar
• Muthén , B. O. and Satorra , A. 1989 . “ Multilevel aspects of varying parameters in structural models ” . In Multilevel analysis of educational data , Edited by: Bock , R. D. 87 – 99 . San Diego, CA : Academic .
   Google Scholar
• Muthén , B. O. and Satorra , A. 1995 . “ Complex sample data in structural equation modeling ” . In Sociological methodology 1955 , Edited by: Marsden , P. V. 267 – 316 . Oxford, UK : Blackwell .
   Google Scholar
• Muthén , L. K. and Muthén , B. O. 1998–2006 . Mplus users guide , 4 Los Angeles : Muthén & Muthén .
   Google Scholar
• Olsson , U. H. , Foss , T. , Troye , S. V. and Howell , R. D. 2000 . The performance of ML, GLS, and WLS estimation in structural equation modeling under conditions of misspecification and nonnormality . Structural Equation Modeling , 7 : 557 – 595 .
   Web of Science ®Google Scholar
• Olsson , U. H. , Troye , S. V. and Howell , R. D. 1999 . Theoretic fit and empirical fit: The performance of maximum likelihood versus generalized least squares estimation in structural equation models . Multivariate Behavioral Research , 34 : 31 – 58 .
   PubMed Web of Science ®Google Scholar
• Paxton , P. , Curran , P. J. , Bollen , K. A. , Kirby , J. and Chen , F. 2001 . Monte Carlo experiments: Design and implementation . Structural Equation Modeling , 8 : 287 – 312 .
   Web of Science ®Google Scholar
• Raftery , A. E. 1988 . Approximate Bayes factors for generalized linear models , Seattle, WA : University of Washington, Department of Statistics . Tech. Rep. No. 121
   Google Scholar
• Rao , P. S. R. S. 2000 . Sampling methodologies with applications: Texts in statistical science , New York : Chapman & Hall/CRC .
   Google Scholar
• Raudenbush , S. W. and Bryk , A. S. 2002 . Hierarchical linear models: Applications and data analysis methods , 2 Thousand Oaks, CA : Sage .
   Google Scholar
• Roth , D. L. , Wiebe , D. J. , Fillingim , R. B. and Shay , K. A. 1989 . Life events, fitness, hardiness, and health: A simultaneous analysis of proposed stress-resistance effects . Journal of Personality and Social Psychology , 57 : 136 – 142 .
   PubMed Web of Science ®Google Scholar
• Rowe , K. J. 2003 . “ Estimating interdependent effects among multilevel composite variables in psychosocial research: An example of the application of multilevel structural equation modeling ” . In Multilevel modeling: Methodological advances, issues and applications , Edited by: Reise , S. P. and Duan , N. 255 – 284 . Mahwah, NJ : Lawrence Erlbaum Associates, Inc. .
   Google Scholar
• Ryu , E. 2008 . Evaluation of model fit in multilevel structural equation modeling: Level-specific model fit evaluation and the robustness to nonnormality , Tempe, AZ : Arizona State University . Unpublished doctoral dissertation
   Google Scholar
• Silvia , E. S. M. and MacCallum , R. C. 1988 . Some factors affecting the success of specification searches in covariance structure modeling . Multivariate Behavioral Research , 23 : 297 – 326 .
   PubMed Web of Science ®Google Scholar
• Sörbom , D. 1989 . Model modification . Psychometrika , 54 : 371 – 384 .
   Web of Science ®Google Scholar
• Stapleton , L. M. 2006 . “ Using multilevel structural equation modeling techniques with complex sample data ” . In A second course in structural equation modeling , Edited by: Hancock , G. R. and Mueller , R. 345 – 384 . Greenwich, CT : Information Age .
   Google Scholar