Linear Mixed-Effects Models for Dependent Data: Power and Accuracy in Parameter Estimation

References

• Aarts, E., Verhage, M., Veenvliet, J. V., Dolan, C. V., & van Der Sluis, S. (2014). A solution to dependency: Using multilevel analysis to accommodate nested data. Nature Neuroscience, 17(4), 491–496. https://doi.org/10.1038/nn.3648
   PubMed Web of Science ®Google Scholar
• Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Caski (Eds.), Proceedings of the Second International Symposium on Information Theory (pp. 267–281) Akademiai Kiado.
   Google Scholar
• Albers, C. J., & Bringmann, L. F. (2020). Inspecting gradual and abrupt changes in emotion dynamics with the time-varying change point autoregressive model. European Journal of Psychological Assessment, 36(3), 492–499. https://doi.org/10.1027/1015-5759/a000589
   Web of Science ®Google Scholar
• Arend, M. G., & Schäfer, T. (2019). Statistical power in two-level models: A tutorial based on Monte Carlo simulation. Psychological Methods, 24(1), 1–19. https://doi.org/10.1037/met0000195
   PubMed Web of Science ®Google Scholar
• Asparouhov, T., Muthén, B. O., & Morin, A. J. (2015). Bayesian structural equation modeling with cross-loadings and residual covariances: Comments on Stromeyer et al. Sage.
   Google Scholar
• Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/j.jml.2007.12.005
   Web of Science ®Google Scholar
• Baek, E., Beretvas, S. N., Van den Noortgate, W., & Ferron, J. M. (2020). Brief research report: Bayesian versus REML estimations with non-informative priors in multilevel single-case data. The Journal of Experimental Education, 88(4), 698–710. https://doi.org/10.1080/00220973.2018.1527280
   Web of Science ®Google Scholar
• Baek, E., & Ferron, J. M. (2020). Modeling heterogeneity of the level-1 error covariance matrix in multilevel models for single-case data using Bayesian estimation. Methodology, 16(2), 166–185. https://doi.org/10.5964/meth.2817
   Web of Science ®Google Scholar
• Bahník, Š., & Vranka, M. A. (2017). If it’s difficult to pronounce, it might not be risky: The effect of fluency on judgment of risk does not generalize to new stimuli. Psychological Science, 28(4), 427–436. https://doi.org/10.1177/0956797616685770
   PubMed Web of Science ®Google Scholar
• Baird, R., & Maxwell, S. E. (2016). Performance of time-varying predictors in multilevel models under an assumption of fixed or random effects. Psychological Methods, 21(2), 175–188. https://doi.org/10.1037/met0000070
   PubMed Web of Science ®Google Scholar
• Baldwin, S. A., & Fellingham, G. W. (2013). Bayesian methods for the analysis of small sample multilevel data with a complex variance structure. Psychological Methods, 18(2), 151–164. https://doi.org/10.1037/a0030642
   PubMed Web of Science ®Google Scholar
• Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001
   Web of Science ®Google Scholar
• Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixedeffects models using lme4. Journal of Statistical Software, 67, 1–48.
   Google Scholar
• Boisgontier, M. P., & Cheval, B. (2016). The anova to mixed model transition. Neuroscience and Biobehavioral Reviews, 68, 1004–1005. https://doi.org/10.1016/j.neubiorev.2016.05.034
   PubMed Web of Science ®Google Scholar
• Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31(2), 144–152. https://doi.org/10.1111/j.2044-8317.1978.tb00581.x
   Web of Science ®Google Scholar
• Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychological Methods, 23(3), 389–411. https://doi.org/10.1037/met0000159
   PubMed Web of Science ®Google Scholar
• Cain, M. K., & Zhang, Z. (2019). Fit for a Bayesian: An evaluation of PPP and DIC for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 26(1), 39–50. https://doi.org/10.1080/10705511.2018.1490648
   Web of Science ®Google Scholar
• Chen, X., Wang, S., & Hartsuiker, R. J. (2022). Error-based structure prediction in language comprehension: Evidence from verb bias effects in a visual-world structural priming paradigm for Mandarin Chinese. Journal of Experimental Psychology. Learning, Memory, and Cognition, 48(1), 60–71. https://doi.org/10.1037/xlm0001048
   PubMed Web of Science ®Google Scholar
• Cho, S. J., De Boeck, P., & Lee, W. Y. (2017). Evaluating testing, profile likelihood confidence interval estimation, and model comparisons for item covariate effects in linear logistic test models. Applied Psychological Measurement, 41(5), 353–371. https://doi.org/10.1177/0146621617692078
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Erlbaum.
   Google Scholar
• De Boeck, P. (2008). Random item IRT models. Psychometrika, 73(4), 533–559. https://doi.org/10.1007/s11336-008-9092-x
   Web of Science ®Google Scholar
• Depaoli, S., & Clifton, J. P. (2015). A Bayesian approach to multilevel structural equation modeling with continuous and dichotomous outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 22(3), 327–351. https://doi.org/10.1080/10705511.2014.937849
   Web of Science ®Google Scholar
• Depaoli, S., Lai, K., & Yang, Y. (2021). Bayesian model averaging as an alternative to model selection for multilevel models. Multivariate Behavioral Research, 56(6), 920–940. https://doi.org/10.1080/00273171.2020.1778439
   PubMed Web of Science ®Google Scholar
• Ferron, J. M., Bell, B. A., Hess, M. R., Rendina-Gobioff, G., & Hibbard, S. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41(2), 372–384. https://doi.org/10.3758/BRM.41.2.372
   PubMed Web of Science ®Google Scholar
• Frömer, R., Maier, M., & Abdel Rahman, R. (2018). Group-level EEG-processing pipeline for flexible single trial-based analyses including linear mixed models. Frontiers in Neuroscience, 12, 48–62. https://doi.org/10.3389/fnins.2018.00048
   PubMed Web of Science ®Google Scholar
• Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2013). Bayesian data analysis (3rd ed.). Chapman and Hall/CRC.
   Google Scholar
• Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7(4), 457–472. https://doi.org/10.1214/ss/1177011136
   Google Scholar
• Greven, S., & Kneib, T. (2010). On the behaviour of marginal and conditional AIC in linear mixed models. Biometrika, 97(4), 773–789. https://doi.org/10.1093/biomet/asq042
   Web of Science ®Google Scholar
• Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018). A brief introduction to mixed effects modelling and multi‐model inference in ecology. PeerJ. 6, e4794. https://doi.org/10.7717/peerj.4794
   PubMed Web of Science ®Google Scholar
• Jiang, Z. (2018). Using the linear mixed-effect model framework to estimate generalizability variance components in R: A lme4 package application. Methodology, 14(3), 133–142. https://doi.org/10.1027/1614-2241/a000149
   Web of Science ®Google Scholar
• Joo, S. H., Ferron, J. M., Moeyaert, M., Beretvas, S. N., & Van den Noortgate, W. (2019). Approaches for specifying the level-1 error structure when synthesizing single-case data. The Journal of Experimental Education, 87(1), 55–74. https://doi.org/10.1080/00220973.2017.1409181
   Web of Science ®Google Scholar
• Judd, C. M., Westfall, J., & Kenny, D. A. (2017). Experiments with more than one random factor: Designs, analytic models, and statistical power. Annual Review of Psychology, 68(1), 601–625. https://doi.org/10.1146/annurev-psych-122414-033702
   PubMed Web of Science ®Google Scholar
• Kawakubo, Y., & Kubokawa, T. (2014). Modified conditional AIC in linear mixed models. Journal of Multivariate Analysis, 129, 44–56. https://doi.org/10.1016/j.jmva.2014.03.017
   Web of Science ®Google Scholar
• Kenward, M. G., & Roger, J. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53(3), 983–997.
   PubMed Web of Science ®Google Scholar
• Koch, T., Schultze, M., Eid, M., & Geiser, C. (2014). A longitudinal multilevel CFA-MTMM model for interchangeable and structurally different methods. Frontiers in Psychology, 5, 311. https://doi.org/10.3389/fpsyg.2014.00311
   PubMed Web of Science ®Google Scholar
• Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2015). Package ‘lmertest’. R Package Version, 2, 734.
   Google Scholar
• Kwok, O. M., West, S. G., & Green, S. B. (2007). The impact of misspecifying the within-subject covariance structure in multiwave longitudinal multilevel models: A Monte Carlo study. Multivariate Behavioral Research, 42(3), 557–592. https://doi.org/10.1080/00273170701540537
   Web of Science ®Google Scholar
• Lai, M. H. (2021). Bootstrap confidence intervals for multilevel standardized effect size. Multivariate Behavioral Research, 56(4), 558–578. https://doi.org/10.1080/00273171.2020.1746902
   PubMed Web of Science ®Google Scholar
• Lee, W. Y. (2018). [generalized linear mixed effect models with crossed random effects for experimental designs having non-repeated items: Model specification and selection] [Doctoral dissertation]. Vanderbilt University.
   Google Scholar
• Levy, R., & Mislevy, A. R. J. (2016). Bayesian psychometric modeling. Chapman and Hall/CRC.
   Google Scholar
• Liu, Y., Luo, F., Zhang, D., & Liu, H. (2017). Comparison and robustness of the REML, ML, MIVQUE estimators for multilevel random mediation model. Journal of Applied Statistics, 44(9), 1644–1661. https://doi.org/10.1080/02664763.2016.1221904
   Web of Science ®Google Scholar
• Luke, S. G. (2017). Evaluating significance in linear mixed effects models in R. Behavior Research Methods, 49(4), 1494–1502. https://doi.org/10.3758/s13428-016-0809-y
   PubMed Web of Science ®Google Scholar
• Martínez-Huertas, J. Á., Olmos, R., & Ferrer, E. (2022). Model selection and model averaging for mixed-effects models with crossed random effects for subjects and items. Multivariate Behavioral Research, 57(4), 603–619. https://doi.org/10.1080/00273171.2021.1889946
   PubMed Web of Science ®Google Scholar
• Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59(1), 537–563. https://doi.org/10.1146/annurev.psych.59.103006.093735
   PubMed Web of Science ®Google Scholar
• Moeyaert, M., Rindskopf, D., Onghena, P., & Van den Noortgate, W. (2017). Multilevel modeling of single-case data: A comparison of maximum likelihood and Bayesian estimation. Psychological Methods, 22(4), 760–778. https://doi.org/10.1037/met0000136
   PubMed Web of Science ®Google Scholar
• Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2016). The misspecification of the covariance structures in multilevel models for single-case data: A Monte Carlo simulation study. The Journal of Experimental Education, 84(3), 473–509. https://doi.org/10.1080/00220973.2015.1065216
   Web of Science ®Google Scholar
• OECD. (2018). PISA 2018 technical report. OECD Publishing.
   Google Scholar
• Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Proceedings of the 3rd International Workshop on Distributed Statistical Computing, 124(125.10), 1–10.
   Google Scholar
• R Development Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.R-project.org/
   Google Scholar
• Raaijmakers, J. G., Schrijnemakers, J. M., & Gremmen, F. (1999). How to deal with “the language-as-fixed-effect fallacy”: Common misconceptions and alternative solutions. Journal of Memory and Language, 41(3), 416–426. https://doi.org/10.1006/jmla.1999.2650
   Web of Science ®Google Scholar
• Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage.
   Google Scholar
• Rights, J. D., & Sterba, S. K. (2019). Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures. Psychological Methods, 24(3), 309–338. https://doi.org/10.1037/met0000184
   PubMed Web of Science ®Google Scholar
• Schultzberg, M., & Muthén, B. (2018). Number of subjects and time points needed for multilevel time-series analysis: A simulation study of dynamic structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 495–515. https://doi.org/10.1080/10705511.2017.1392862
   Web of Science ®Google Scholar
• Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136
   Web of Science ®Google Scholar
• Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Linde, A. (2014). The deviance information criterion: 12 years on. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(3), 485–493. https://doi.org/10.1111/rssb.12062
   Web of Science ®Google Scholar
• Van Erp, S., & Browne, W. J. (2021). Bayesian multilevel structural equation modeling: An investigation into robust prior distributions for the doubly latent categorical model. Structural Equation Modeling: A Multidisciplinary Journal, 28(6), 875–893.
   Google Scholar
• Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology. General, 143(5), 2020–2045. https://doi.org/10.1037/xge0000014
   PubMed Web of Science ®Google Scholar
• Yuan, Y., & MacKinnon, D. P. (2009). Bayesian mediation analysis. Psychological Methods, 14(4), 301–322. https://doi.org/10.1037/a0016972
   PubMed Web of Science ®Google Scholar
• Zhang, Z., Lai, K., Lu, Z., & Tong, X. (2013). Bayesian inference and application of robust growth curve models using student’s t distribution. Structural Equation Modeling: A Multidisciplinary Journal, 20(1), 47–78. https://doi.org/10.1080/10705511.2013.742382
   Web of Science ®Google Scholar