Review of Sample Size for Structural Equation Models in Second Language Testing and Learning Research: A Monte Carlo Approach

REFERENCES

• Akbari , R. , Behzadpoor , F. and Dadvand , B. 2010 . Development of English language teaching reflection inventory . System , 38 : 211 – 227 . **
   Web of Science ®Google Scholar
• Allison , P. D. 2003 . Missing data techniques for structural equation modeling . Journal of Abnormal Psychology , 112 : 545 – 557 .
   PubMed Web of Science ®Google Scholar
• American Psychological Association . 2010 . Publication manual of the American Psychological Association (6th ed.) , Washington , DC : Author .
   Google Scholar
• Arbuckle , J. L. 2009 . Amos (Version 18.0.0) [Computer software] , Crawfordville , FL : Amos Development Corporation .
   Google Scholar
• Bachman , L. F. 1982 . The trait structure of cloze test scores . TESOL Quarterly , 16 : 61 – 70 . **
   Web of Science ®Google Scholar
• Bachman , L. F. and Palmer , A. S. 1981 . The construct validation of the FSI oral interview . Language Learning , 31 : 67 – 86 . **
   Web of Science ®Google Scholar
• Bachman , L. F. and Palmer , A. 1982 . The construct validation of some components of communicative proficiency . TESOL Quarterly , 16 : 449 – 465 . **
   Web of Science ®Google Scholar
• Bachman , L. F. and Palmer , A. 1989 . The construct validation of self-ratings of communicative language ability . Language Testing , 6 : 14 – 29 . **
   Google Scholar
• Bae , J. and Bachman , L. F. 1998 . A latent variable approach to listening and reading: Testing factorial invariance across two groups of children in the Korean/English Two-Way Immersion Program . Language Testing , 15 : 380 – 414 . **
   Google Scholar
• Bae , J. and Bachman , L. F. 2010 . An investigation of four writing traits and two tasks across two languages . Language Testing , 27 : 213 – 234 . **
   Web of Science ®Google Scholar
• Bae , J. and Lee , Y.-S. 2010 . The validation of parallel test forms: “Mountain” and “beach” picture series for assessment of language skills . Language Testing , 28 : 155 – 177 . **
   Web of Science ®Google Scholar
• Barrett , P. 2007 . Structural equation modelling: Adjudging model fit . Personality and Individual Differences , 42 : 815 – 824 .
   Web of Science ®Google Scholar
• Bentler , P. and Yuan , K.-H. 1999 . Structural equation modeling with small samples: Test statistics . Multivariate Behavioral Research , 34 : 181 – 197 .
   PubMed Web of Science ®Google Scholar
• Blais , J. G. and Laurier , M. D. 1995 . The dimensionality of a placement test from several analytical perspectives . Language Testing , 12 : 72 – 98 . *
   Google Scholar
• Brown , T. A. 2006 . Confirmatory factor analysis for applied research , New York, NY : Guilford .
   Google Scholar
• Byrne , B. M. 2012 . Structural equation modeling with Mplus: Basic concepts, applications, and programming , New York, NY : Routledge .
   Google Scholar
• Byrne , B. M. , Stewart , S. M. , Kennard , B. D. and Lee , P. W. H. 2007 . The Beck Depression Inventory-II: Testing for measurement equivalence and factor mean differences across Hong Kong and American adolescents . International Journal of Testing , 7 : 293 – 309 .
   Google Scholar
• Carr , N. T. 2006 . The factor structure of test task characteristics and examinee performance . Language Testing , 23 : 269 – 289 . **
   Google Scholar
• Chen , J. F. , Warden , C. A. and Chang , H.-T. 2005 . Motivators that do not motivate: The case of Chinese EFL learners and the influence of culture on motivation . TESOL Quarterly , 39 : 609 – 633 . **
   Web of Science ®Google Scholar
• Choi , I.-C. , Kim , K. S. and Boo , J. 2003 . Comparability of a paper-based language test and a computer-based language test . Language Testing , 20 : 295 – 320 . **
   Google Scholar
• Cohen , J. 1988 . Statistical power analysis for the behavioral sciences (2nd ed.) , Hillsdale , NJ : Erlbaum .
   Google Scholar
• Csizér , K. and Dörnyei , Z. 2005 . The internal structure of language learning motivation and its relationship with language choice and learning effort . Modern Language Journal , 89 : 19 – 36 . **
   Web of Science ®Google Scholar
• Csizér , K. and Kormos , J. 2009 . Modelling the role of inter-cultural contact in the motivation of learning English as a foreign language . Applied Linguistics , 30 : 166 – 185 . *
   Web of Science ®Google Scholar
• Curran , P. J. , Bollen , K. A. , Paxton , P. , Kirby , J. and Chen , F. 2002 . The noncentral chi-square distribution in misspecified structural equation models: Finite sample results from a Monte Carlo simulation . Multivariate Behavioral Research , 37 : 1 – 36 .
   PubMed Web of Science ®Google Scholar
• Ding , L. , Velicer , W. F. and Harlow , L. L. 1995 . Effects of estimation methods, number of indicators per factor and improper solutions on structural equation modeling fit indices . Structural Equation Modeling , 2 : 119 – 143 .
   Web of Science ®Google Scholar
• Eckes , T. and Grotjahn , R. 2006 . A closer look at the construct validity of C-tests . Language Testing , 23 : 290 – 325 . **
   Google Scholar
• Ellis , R. and Loewen , S. 2007 . Confirming the operational definitions of explicit and implicit knowledge in Ellis (2005): Responding to Isemonger . Studies in Second Language Acquisition , 29 : 119 – 126 . **
   Web of Science ®Google Scholar
• Enders , C. K. and Bandalos , D. L. 2001 . The relative performance of full information maximum likelihood estimation for missing data in structural equation models . Structural Equation Modeling , 8 : 430 – 457 .
   Web of Science ®Google Scholar
• Fan , X. , Thompson , B. and Wang , L. 1999 . Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes . Structural Equation Modeling , 6 : 56 – 83 .
   Web of Science ®Google Scholar
• Fulcher , G. 1996 . Invalidating validity claims for the ACTFL oral rating scale . System , 24 : 163 – 172 . **
   Google Scholar
• Fushino , K. 2010 . Causal relationships between communication confidence, beliefs about group work, and willingness to communicate in foreign language group work . TESOL Quarterly , 44 : 700 – 724 . **
   Web of Science ®Google Scholar
• Gardner , R. C. , Tremblay , P. F. and Masgoret , A. M. 1997 . Towards a full model of second language learning: An empirical investigation . Modern Language Journal , 81 : 344 – 362 . **
   Web of Science ®Google Scholar
• Gorsuch , G. J. 2000 . EFL educational policies and educational cultures: Influences on teachers’ approval of communicative activities . TESOL Quarterly , 34 : 675 – 710 . **
   Web of Science ®Google Scholar
• Hiromori , T. 2009 . A process model of L2 learners’ motivation: From the perspectives of general tendency and individual differences . System , 37 : 313 – 321 . *
   Web of Science ®Google Scholar
• Hsiao , T.-Y. and Oxford , R. L. 2002 . Comparing theories of language learning strategies: A confirmatory factor analysis . Modern Language Journal , 86 : 368 – 383 . *
   Web of Science ®Google Scholar
• In’nami , Y. 2006 . The effects of test anxiety on listening test performance . System , 34 : 317 – 340 . **
   Google Scholar
• Isemonger , I. and Watanabe , K. 2007 . The construct validity of scores on a Japanese version of the perceptual component of the Style Analysis Survey . System , 35 : 134 – 147 . *
   Google Scholar
• Kim , K. 2005 . The relation among fit indexes, power, and sample size in structural equation modeling . Structural Equation Modeling , 12 : 368 – 390 .
   Web of Science ®Google Scholar
• Kim , S.-Y. 2012 . Sample size requirements in single- and multiphase growth mixture models: A Monte Carlo simulation study . Structural Equation Modeling , 19 : 457 – 476 .
   Web of Science ®Google Scholar
• Kline , R. B. 2005 . Principles and practice of structural equation modeling (2nd ed.) , New York, NY : Guilford .
   Google Scholar
• Kormos , J. , Kiddle , T. and Csizer , K. 2011 . Systems of goals, attitudes, and self-related beliefs in second-language-learning motivation . Applied Linguistics , 32 : 495 – 516 . *
   Web of Science ®Google Scholar
• Kunnan , A. J. 1994 . Modelling relationships among some test-taker characteristics and performance on EFL tests: An approach to construct validation . Language Testing , 11 : 225 – 250 . **
   Google Scholar
• Lee , S.-Y. 2005 . Facilitating and inhibiting factors in English as a foreign language writing performance: A model testing with structural equation modeling . Language Learning , 55 : 335 – 374 . *
   Web of Science ®Google Scholar
• Lei , P.-W. 2009 . Evaluating estimation methods for ordinal data in structural equation modeling . Quality & Quantity , 43 : 495 – 507 .
   Web of Science ®Google Scholar
• Llosa , L. 2007 . Validating a standards-based classroom assessment of English proficiency: A multitrait-multimethod approach . Language Testing , 24 : 489 – 515 . **
   Google Scholar
• MacCallum , R. C. and Austin , J. T. 2000 . Application of structural equation modeling in psychological research . Annual Review of Psychology , 51 : 201 – 226 .
   PubMed Web of Science ®Google Scholar
• MacCallum , R. C. , Browne , M. W. and Sugawara , H. M. 1996 . Power analysis and determination of sample size for covariance structure modeling . Psychological Methods , 1 : 130 – 149 .
   Web of Science ®Google Scholar
• Matsumura , S. 2003 . Modelling the relationships among interlanguage pragmatic development, L2 proficiency, and exposure to L2 . Applied Linguistics , 24 : 465 – 491 . **
   Web of Science ®Google Scholar
• McIntosh , C. N. 2007 . Rethinking fit assessment in structural equation modelling: A commentary and elaboration on Barrett (2007) . Personality and Individual Differences , 42 : 859 – 867 .
   Web of Science ®Google Scholar
• Mundfrom , D. J. , Shaw , D. G. and Ke , T. L. 2005 . Minimum sample size recommendations for conducting factor analyses . International Journal of Testing , 5 : 159 – 168 .
   Google Scholar
• Muthén , B. O. 2000, February 9 . Mplus discussion > Categorical data modeling > Monte Carlo simulation [Online forum comment] . Retrieved from http://www.statmodel.com/discussion/messages/23/57.html?1322159573
   Google Scholar
• Muthén , B. O. 2007, May 20 . Mplus discussion > Structural equation modeling > Sample size [Online forum comment] . Retrieved from http://www.statmodel.com/discussion/messages/11/456.html?1272399263
   Google Scholar
• Muthén , L. K. and Muthén , B. O. 2002 . How to use a Monte Carlo study to decide on sample size and determine power . Structural Equation Modeling , 9 : 599 – 620 .
   Web of Science ®Google Scholar
• Muthén , L. K. and Muthén , B. O. 2007 . Mplus user's guide (5th ed.) , Los Angeles, CA : Muthén & Muthén .
   Google Scholar
• Muthén , L. K. and Muthén , B. O. 2011 . Mplus (Version 6.12) [Computer software] , Los Angeles, CA : Muthén & Muthén .
   Google Scholar
• Nevitt , J. and Hancock , G. R. 2004 . Evaluating small sample approaches for model test statistics in structural equation modeling . Multivariate Behavioral Research , 39 : 439 – 478 .
   Web of Science ®Google Scholar
• Ockey , G. 2011 . Self-consciousness and assertiveness as explanatory variables of L2 oral ability: A latent variable approach . Language Learning , 61 : 968 – 989 . **
   Web of Science ®Google Scholar
• Onwuegbuzie , A. J. , Bailey , P. and Daley , C. E. 2000 . The validation of three scales measuring anxiety at different stages of the foreign language learning process: The input anxiety scale, the processing anxiety scale, and the output anxiety scale . Language Learning , 50 : 87 – 117 . *
   Web of Science ®Google Scholar
• Pae , T.-I. and Park , G.-P. 2006 . Examining the relationship between differential item functioning and differential test functioning . Language Testing , 23 : 475 – 496 . *
   Google Scholar
• Papi , M. 2010 . The L2 motivational self system, L2 anxiety, and motivated behavior: A structural equation modeling approach . System , 38 : 467 – 479 . **
   Web of Science ®Google Scholar
• Peng , J.-E. and Woodrow , L. 2010 . Willingness to communicate in English: A model in the Chinese EFL classroom context . Language Learning , 60 : 834 – 876 . **
   Web of Science ®Google Scholar
• Phakiti , A. 2008a . Construct validation of Bachman and Palmer's (1996) strategic competence model over time in EFL reading tests . Language Testing , 25 : 237 – 272 . **
   Web of Science ®Google Scholar
• Phakiti , A. 2008b . Strategic competence as a fourth-order factor model: A structural equation modeling approach . Language Assessment Quarterly , 5 : 20 – 42 . **
   Web of Science ®Google Scholar
• Preacher , K. J. and Coffman , D. L. 2006 . Computing power and minimum sample size for RMSEA [Computer software] . Retrieved from http://quantpsy.org/
   Google Scholar
• Preacher , K. J. and Merkle , E. C. 2012 . The problem of model selection uncertainty in structural equation modeling . Psychological Methods , 17 : 1 – 14 .
   PubMed Web of Science ®Google Scholar
• Purpura , J. E. 1997 . An analysis of the relationships between test takers’ cognitive and metacognitive strategy use and second language test performance . Language Learning , 47 : 289 – 325 . **
   Web of Science ®Google Scholar
• Purpura , J. E. 1998 . Investigating the effects of strategy use and second language test performance with high- and low-ability test takers: A structural equation modelling approach . Language Testing , 15 : 333 – 379 . **
   Google Scholar
• Raykov , T. and Marcoulides , G. A. 2006 . A first course in structural equation modeling (2nd ed.) , Mahwah , NJ : Erlbaum .
   Google Scholar
• Ross , S. J. 2005 . The impact of assessment method on foreign language proficiency growth . Applied Linguistics , 26 : 317 – 342 . *
   Web of Science ®Google Scholar
• Sang , F. , Schmitz , B. , Vollmer , H. J. , Baumert , J. and Roeder , P. M. 1986 . Models of second language competence: A structural equation approach . Language Testing , 3 : 54 – 79 . **
   Google Scholar
• Sasaki , M. 1993 . Relationships among second language proficiency, foreign language aptitude, and intelligence: A structural equation modeling approach . Language Learning , 43 : 313 – 344 . *
   Web of Science ®Google Scholar
• Satorra , A. and Saris , W. E. 1985 . Power of the likelihood ratio test in covariance structure analysis . Psychometrika , 50 : 83 – 90 .
   Web of Science ®Google Scholar
• Savalei , V. 2010 . Small sample statistics for incomplete nonnormal data: Extensions of complete data formulae and a Monte Carlo comparison . Structural Equation Modeling , 17 : 241 – 264 .
   Web of Science ®Google Scholar
• Sawaki , Y. 2007 . Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile and a composite . Language Testing , 24 : 355 – 390 . **
   Google Scholar
• Sawaki , Y. , Stricker , L. J. and Oranje , A. H. 2009 . Factor structure of the TOEFL Internet-based test . Language Testing , 26 : 5 – 30 . *
   Web of Science ®Google Scholar
• Schoonen , R. 2005 . Generalizability of writing scores: An application of structural equation modeling . Language Testing , 22 : 1 – 30 . *
   Google Scholar
• Schoonen , R. , Hulstijn , J. and Bossers , B. 1998 . Metacognitive and language-specific knowledge in native and foreign language reading comprehension: An empirical study among Dutch students in grades 6, 8 and 10 . Language Learning , 48 : 71 – 106 . *
   Web of Science ®Google Scholar
• Schoonen , R. , van Gelderen , A. , de Glopper , K. , Hulstijn , J. , Simis , A. , Snellings , P. and Stevenson , M. 2003 . First language and second language writing: The role of linguistic knowledge, speed of processing, and metacognitive knowledge . Language Learning , 53 : 165 – 202 . *
   Web of Science ®Google Scholar
• Schoonen , R. , van Gelderen , A. , Stoel , R. D. , Hulstijn , J. and de Glopper , K. 2011 . Modeling the development of L1 and EFL writing proficiency of secondary school students . Language Learning , 61 : 31 – 79 . *
   Web of Science ®Google Scholar
• Schoonen , R. , Vergeer , M. and Eiting , M. 1997 . The assessment of writing ability: Expert readers versus lay readers . Language Testing , 14 : 157 – 184 . *
   Google Scholar
• Shin , S.-K. 2005 . Did they take the same test? Examinee language proficiency and the structure of language tests . Language Testing , 22 : 31 – 57 . **
   Google Scholar
• Shiotsu , T. and Weir , C. J. 2007 . The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance . Language Testing , 24 : 99 – 128 . **
   Google Scholar
• Song , M.-Y. 2008 . Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach . Language Testing , 25 : 435 – 464 . **
   Web of Science ®Google Scholar
• Tremblay , P. F. and Gardner , R. C. 1995 . Expanding the motivation construct in language learning . Modern Language Journal , 79 : 505 – 518 . *
   Web of Science ®Google Scholar
• Tseng , W.-T. , Dörnyei , Z. and Schmitt , N. 2006 . A new approach to assessing strategic learning: The case of self-regulation in vocabulary acquisition . Applied Linguistics , 27 : 78 – 102 . *
   Web of Science ®Google Scholar
• Tseng , W.-T. and Schmitt , N. 2008 . Toward a model of motivated vocabulary learning: A structural equation modeling approach . Language Learning , 58 : 357 – 400 . **
   Web of Science ®Google Scholar
• Turner , C. E. 1989 . The underlying factor structure of L2 close test performance in Francophone, university-level students: Causal modelling as an approach to construct validation . Language Testing , 6 : 172 – 197 . **
   Google Scholar
• Ullman , J. E. 2007 . “ Structural equation modeling ” . In Using multivariate statistics (5th ed. , Edited by: Tabachnick , B. G. and Fidell , L. S. 676 – 780 . Boston, MA : Allyn and Bacon .
   Google Scholar
• Vandergrift , L. , Goh , C. C. M. , Mareschal , C. J. and Tafaghodtari , M. H. 2006 . The metacognitive awareness listening questionnaire: Development and validation . Language Learning , 56 : 431 – 462 . **
   Web of Science ®Google Scholar
• Wilkinson , L. and Task Force on Statistical Inference . 1999 . Statistical methods in psychology journals: Guidelines and explanations . American Psychologist , 54 : 594 – 604 .
   Web of Science ®Google Scholar
• Woodrow , L. J. 2006a . A model of adaptive language learning . Modern Language Journal , 90 : 297 – 319 . **
   Web of Science ®Google Scholar
• Woodrow , L. J. 2006b . Anxiety and speaking English as a second language . RELC Journal , 37 : 308 – 328 . **
   Google Scholar
• Xi , X. 2005 . Do visual chunks and planning impact performance on the graph description task in the SPEAK exam? . Language Testing , 22 : 463 – 508 . **
   Google Scholar
• Xi , X. 2010 . Aspects of performance on line graph description tasks: Influenced by graph familiarity and different task features . Language Testing , 27 : 73 – 100 . **
   Web of Science ®Google Scholar
• Yashima , T. 2002 . Willingness to communicate in a second language: The Japanese EFL context . Modern Language Journal , 86 : 54 – 66 . **
   Web of Science ®Google Scholar
• Yashima , T. , Zenuk-Nishide , L. and Shimizu , K. 2004 . The influence of attitudes and affect on willingness to communicate and second language communication . Language Learning , 54 : 119 – 152 . **
   Web of Science ®Google Scholar
• Zumbo , B. D. 2003 . Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests . Language Testing , 20 : 136 – 147 . *
   Google Scholar