Determining Reliability of Daily Measures: An Illustration with Data on Teacher Stress

References

• Asparouhov, T., & Muthen, B. O. (2012, November 15). Multiple group multilevel analysis. Mplus Web Notes: No. 16.
   Google Scholar
• Asparouhov, T., & Muthén, B. (2005). Multivariate statistical modeling with survey data. Proceedings of the Federal Committee on Statistical Methodology (FCSM) Research Conference. http://statmodel.com/resrchpap.shtml.
   Google Scholar
• Bolger, N., & Laurenceau, J. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research. doi:https://doi.org/10.1177/1049731513495458
   Google Scholar
• Brennan, R. L. (2001). Generalizability theory. New York, NY: Springer.
   Google Scholar
• Brown, T. A. (2015). Methodology in the social sciences. Confirmatory factor analysis for applied research (2nd ed.). New York: The Guilford Press.
   Google Scholar
• Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. doi:https://doi.org/10.1177/0049124192021002005
   Web of Science ®Google Scholar
• Cranford, J. A., Shrout, P. E., Iida, M., Rafaeli, E., Yip, T., & Bolger, N. (2006). A procedure for evaluating sensitivity to within-person change: Can mood measures in diary studies detect change reliably? Personality and Social Psychology Bulletin, 32(7), 917–929. doi:https://doi.org/10.1177/0146167206287721
   PubMed Web of Science ®Google Scholar
• Cronbach, L. J. (1951). Coefficient Alpha and the Internal Structure of Tests. Psychometrika, 16, 297–334. doi:https://doi.org/10.1007/BF02310555
   Web of Science ®Google Scholar
• Folkman, S. (2008). The Case for Positive Emotions in the Stress Process. Anxiety, Stress and Coping, 21(1), 3–14. https://doi.org/https://doi.org/10.1080/10615800701740457
   PubMed Web of Science ®Google Scholar
• Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. doi:https://doi.org/10.1037/a0032138
   PubMed Web of Science ®Google Scholar
• Hox, J. J., Moerbeek, M., & van de Schoot, R. (2017). Multilevel Analysis: Techniques and Applications. New York, NY: Routledge.
   Google Scholar
• Hu, L., & Bentler, P. M. (1999). Cutoff Criteria for Fit Indexes in Covariance Structure Analysis : Conventional Criteria Versus New Alternatives Cutoff Criteria for Fit Indexes in Covariance Structure Analysis : Conventional Criteria Versus New Alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/https://doi.org/10.1080/10705519909540118
   Web of Science ®Google Scholar
• Hubley, A. M. (2014). Discriminant validity. In A. C. Michalos (Ed.), Encyclopedia of quality of life and well-being research (pp. 1664–1667). doi:https://doi.org/10.1007/978-94-007-0753-5_751
   Google Scholar
• Jak, S. (2019). Cross-Level Invariance in Multilevel Factor Models. Structural Equation Modeling, 26(4), 607–622. https://doi.org/https://doi.org/10.1080/10705511.2018.1534205.
   Web of Science ®Google Scholar
• Jak, S & Jorgensen, T.D. (2017). Relating Measurement Invariance, Cross-Level Invariance, and Multilevel Reliability. Frontiers in Psychology, 8(OCT), 1–9. https://doi.org/https://doi.org/10.3389/fpsyg.2017.01640.
   Web of Science ®Google Scholar
• Jak, S , Oort, F.J. , Dolan, C.V. (2013). A Test for Cluster Bias: Detecting Violations of Measurement Invariance Across Clusters in Multilevel Data Structural Equation Modeling, 20, 265-282.
   Google Scholar
• Jak, S , Oort, F.J. & Dolan, C.V. (2014). Measurement Bias in Multilevel Data. Structural Equation Modeling, 21(1), 31–39. https://doi.org/https://doi.org/10.1080/10705511.2014.856694.
   Web of Science ®Google Scholar
• Kline, R. B. (2011). Principles and practice of structural equation modeling. (3rd ed.). doi:https://doi.org/10.5840/thought194520147
   Google Scholar
• Kolenikov, S., & Bollen, K. A. (2012). Testing negative error variances: Is a heywood case a symptom of misspecification? Sociological Methods & Research, 41(1), 124–167. doi:https://doi.org/10.1177/0049124112442138
   Web of Science ®Google Scholar
• Lai, M. H. C. (2021). Composite reliability of multilevel data: It’s about observed scores and construct meanings. Psychological Methods, 26(1), 90–102. doi:https://doi.org/10.1037/met0000287
   PubMed Web of Science ®Google Scholar
• Lazarus, R. S. (1999). Stress and Emotion: A new synthesis. Springer Publishing Co
   Google Scholar
• Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13(3), 203. doi:https://doi.org/10.1037/a0012869
   PubMed Web of Science ®Google Scholar
• MacCallum, R. (1986). Specification searches in covariance structure modeling. Psychological Bulletin, 100(1), 107–120. https://doi.org/https://doi.org/10.1037/0033-2909.100.1.107
   Web of Science ®Google Scholar
• Marsh, H. W., Balla, J. R., & Mcdonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391–410. doi:https://doi.org/10.1037/0033-2909.103.3.391
   Web of Science ®Google Scholar
• McDonald, R. P. (1999). Test Theory: A Unified Treatment. Mahwah, NJ: Erlbaum
   Google Scholar
• McNeish, D., & Hamaker, E. L. (2019, December). A primer on two-level dynamic structural equation models for intensive longitudinal data in Mplus. Psychological Methods. doi:https://doi.org/10.1037/met0000250
   Google Scholar
• Mehl, M. R., Conner, T. S., & Csikszentmihalyi, M. (2012). Handbook of research methods for studying daily life. New York, NY: The Guilford Press.
   Google Scholar
• Mehta, P. D., & Neale, M. C. (2005). People Are Variables Too: Multilevel Structural Equations Modeling. Psychological Methods, 10(3), 259–284.
   PubMed Web of Science ®Google Scholar
• Muthén, B. (1990). Mean and Covariance Structure Analysis Of Hierarchical Data. Los Angeles, CA: UCLA.
   Google Scholar
• Muthén, B. O. (1994). Multilevel Covariance Structure Analysis. Sociological Methods & Research, 22, 376–398. doi:https://doi.org/10.1177/0049124194022003006
   Web of Science ®Google Scholar
• Peters, G. (2014). The alpha and the omega of scale reliability and validity: Why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality. The European Health Psychologist, 16(2), 56–69. doi:https://doi.org/10.31234/osf.io/h47fv
   Google Scholar
• R Core Team (2021). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
   Google Scholar
• Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized Multilevel Structural Equation Modeling. Psychometrika, 69(2), 167–190. https://doi.org/https://doi.org/10.1007/BF02295939
   Web of Science ®Google Scholar
• Raykov, T., & Shrout, P. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9(2), 195–212. doi:https://doi.org/10.1207/S15328007SEM0902_3
   Web of Science ®Google Scholar
• Rhemtulla, M., & Hancock, G. R. (2016). Planned missing data designs in educational psychology research. Educational Psychologist, 51(3–4), 305–316. doi:https://doi.org/10.1080/00461520.2016.1208094
   Web of Science ®Google Scholar
• Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/https://doi.org/10.18637/jss.v048.i02
   Web of Science ®Google Scholar
• Rosseel, Y. (2021). Evaluating the observed log-likelihood function in two-level structural equation modeling with missing data: From formulas to R code. Psych, 3(2), 197–232. doi:https://doi.org/10.3390/psych3020017
   Google Scholar
• Sadikaj, G., Wright, A., Dunkley, D., Zuroff, D., & Moskowitz, D. S. (2019). Multilevel structural equation modeling for intensive longitudinal data: A practical guide for personality researchers. doi:https://doi.org/10.31234/osf.io/hwj9r
   Google Scholar
• Satorra, A., & Bentler, P. M. (1994). Corrections to Test Statistics and Standard Errors in Covariance Structure Analysis. In A. von Eye, and C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (p. 399–419). Sage Publications, Inc.
   Google Scholar
• Schmidt, W. H. (1969). Covariance structure analysis of the multivariate random effects model (Doctoral dissertation). University of Chicago, Department of Education.
   Google Scholar
• Shrout, P. E., & Lane, S. P. (2012). Psychometrics. In Handbook of Research Methods for Studying Daily Life (pp. 302–320). The Guilford Press.
   Google Scholar
• Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London, United Kingdom: Sage Publishers.
   Google Scholar
• Spielberger, C. D. (2010). State-trait anxiety inventory. The Corsini Encyclopedia of Psychology. doi:https://doi.org/10.1002/9780470479216.corpsy0943
   Google Scholar
• Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct Meaning in Multilevel Settings. Journal of Educational and Behavioral Statistics, 41(5), 481–520. https://doi.org/https://doi.org/10.3102/1076998616646200
   Web of Science ®Google Scholar
• Stoel, R. D., Garre, F. G., Dolan, C., & van den Wittenboer, G. (2006). On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints. Psychological Methods, 11(4), 439–455. doi:https://doi.org/10.1037/1082-989X.11.4.439
   PubMed Web of Science ®Google Scholar
• van der Ploeg, H. M. (1982). De Zelf-Beoordelings Vragenlijst (STAI-DY). Tijdschrift Voor Psychiatrie, 24(9), 576–588. Retrieved from http://tijdschriftvoorpsychiatrie.be/issues/368/articles/2438
   Google Scholar