Estimating the Reliability of Aggregated and Within-Person Centered Scores in Ecological Momentary Assessment

REFERENCES

• Bentler , P. M. 2009 . Alpha, dimension-free, and model-based internal consistency reliability . Psychometrika , 74 : 137 – 143 .
   PubMed Web of Science ®Google Scholar
• Biemer , P. P. , Christ , S. L. and Wiesen , C. A. 2009 . A general approach for estimating scale score reliability for panel survey data . Psychological Methods , 14 : 400 – 412 .
   PubMed Web of Science ®Google Scholar
• Bolger , N. , Davis , A. and Rafaeli , E. 2003 . Diary methods: Capturing life as it is lived . Annual Review of Psychology , 54 : 579 – 616 .
   PubMed Web of Science ®Google Scholar
• Casella , G. and Berger , R. L. 2002 . Statistical Inference, , 2nd ed. New York , NY : Duxbury. .
   Google Scholar
• Cheung , M. W. L. 2009 . Constructing approximate confidence intervals for parameters with structural equation models . Structural Equation Modeling , 16 : 267 – 294 .
   Web of Science ®Google Scholar
• Cole , D. A. , Martin , N. C. and Steiger , J. H. 2005 . Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model . Psychological Methods , 10 : 3 – 20 .
   PubMed Web of Science ®Google Scholar
• Cronbach , L. J. 1951 . Coefficient alpha and the internal structure of tests . Psychometrika , 16 : 297 – 334 .
   Web of Science ®Google Scholar
• Csikszentmihalyi , M. and Larson , R. 1987 . Validity and reliability of the experience sampling method . The Journal of Nervous and Mental Disease , 175 : 526 – 536 .
   PubMed Web of Science ®Google Scholar
• Curran , P. J. and Bauer , D. J. 2011 . The disaggregation of within-person and between-person effects in longitudinal models of change . Annual Review of Psychology , 62 : 583 – 619 .
   PubMed Web of Science ®Google Scholar
• Efron , B. and Tibshirani , R. 1993 . An Introduction to the bootstrap. New York , NY : Chapman & Hall. .
   Google Scholar
• Egloff , B. , Schmukle , S. C. , Kohlman , C. W. , Burns , L. R. and Hock , M. 2003 . Facets of dynamic positive affect: Differentiation joy, interest and anticipation in the positive and negative affect schedule (PANAS) . Journal of Personality and Social Psychology , 85 : 528 – 540 .
   PubMed Web of Science ®Google Scholar
• Everitt , B. S. 2002 . Cambridge dictionary of statistics, , 2nd ed. Cambridge , , UK : Cambridge University Press. .
   Google Scholar
• Feldt , L. S. and Brennan , R. L. 1989 . “ Reliability. ” . In Educational measurement, , 3rd ed. Edited by: Linn , R. L. 105 – 146 . New York , NY : American Council on Education and Macmillan. .
   Google Scholar
• Guttman , L. 1945 . A basis for analyzing test-retest reliability . Psychometrika , 10 : 255 – 282 .
   PubMed Web of Science ®Google Scholar
• Haertel , E. H. 2006 . “ Reliability. ” . In Educational measurement, , 4th ed. Edited by: Brennan , R. L. 65 – 110 . Westport , CT : American Council on Education and Praeger. .
   Google Scholar
• Hektner , J. M. , Schmidt , J. A. and Csikszentmihalyi , M. 2007 . Experience sampling method: Measuring the quality of everyday life. Thousand Oaks , CA : Sage. .
   Google Scholar
• Huang , P. H. 2009 . Depression and mood congruent thoughts in everyday life: A diary study. , Master's thesis Taipei , , Taiwan : National Chengchi University. .
   Google Scholar
• Hufford , M. R. 2007 . “ Special methodological challenges and opportunities in ecological momentary assessment. ” . In The science of real-time data capture: Self-reports in health research Edited by: Stone , A. A. , Shiffman , S. , Atienza , A. and Nebeling , L. 54 – 75 . New York , NY : Oxford University Press. .
   Google Scholar
• Jahng , S. , Wood , P. K. and Trull , T. J. 2008 . Analysis of affective instability in ecological momentary assessment: Indices using successive difference and group comparison via multilevel modeling . Psychological Methods , 13 : 354 – 375 .
   PubMed Web of Science ®Google Scholar
• J⊘reskog , K. G. 1971 . Statistical analysis of sets of congeneric tests . Psychometrika , 36 : 109 – 133 .
   Web of Science ®Google Scholar
• Kenny , D. A. and Judd , C. M. 1986 . Consequences of violating the independence assumption in analysis of variance . Psychological Bulletin , 99 : 422 – 431 .
   Web of Science ®Google Scholar
• Kollo , T. and von Rosen , D. 2005 . Advanced multivariate statistics with matrices. Dordrecht , , The Netherlands : Springer. .
   Google Scholar
• Laenen , A. 2008 . Psychometric validation of continuous rating scales from complex data. , Doctoral thesis Diepenbeek , , Belgium : Universiteit Hasselt. .
   Google Scholar
• Laenen , A. , Alonso , A. and Molenberghs , G. 2007 . A measure for the reliability of a rating scale based on longitudinal clinical trial data . Psychometrika , 72 : 443 – 448 .
   Web of Science ®Google Scholar
• Laenen , A. , Alonso , A. , Molenberghs , G. and Vangeneugden , T. 2009 . Reliability of a longitudinal sequence of scale ratings . Psychometrika , 74 : 49 – 64 .
   Web of Science ®Google Scholar
• Laenen , A. , Vangeneugden , T. , Geys , H. and Molenberghs , G. 2006 . Generalized reliability estimation using repeated measurements . British Journal of Mathematical and Statistical Psychology , 59 : 113 – 131 .
   PubMed Web of Science ®Google Scholar
• Lee , S. Y. 1990 . Multilevel analysis of structural equation models . Biometrika , 77 : 763 – 772 .
   Web of Science ®Google Scholar
• Lord , F. M. and Novick , M. 1968 . Statistical theories of mental test scores. Reading , MA : Addison-Wesley. .
   Google Scholar
• Maas , C. and Hox , J. 2005 . Sufficient sample sizes for multilevel modeling . Methodology: European Journal of Research Methods for the Behavioral and Social Sciences , 1 : 86 – 92 .
   Google Scholar
• McDonald , R. P. and Goldstein , H. 1989 . Balanced versus unbalanced designs for linear structural relations in two-level data . British Journal of Mathematical and Statistical Psychology , 42 : 215 – 232 .
   Web of Science ®Google Scholar
• Mehta , P. D. and West , S. G. 2000 . Putting the individual back into individual growth curves . Psychological Methods , 5 : 23 – 43 .
   PubMed Web of Science ®Google Scholar
• Merz , E. L. and Roesch , S. C. 2011 . Modeling trait and state variation using multilevel factor analysis with PANAS daily diary data . Journal of Research in Personality , 45 : 2 – 9 .
   PubMed Web of Science ®Google Scholar
• Molenaar , P. C. M. 2004 . A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever . Measurement , 2 : 201 – 218 .
   Google Scholar
• Muthén , B. O. 1989 . Latent variable modeling in heterogeneous populations . Psychometrika , 54 : 557 – 585 .
   Web of Science ®Google Scholar
• Muthén , B. and Satorra , A. 1995 . “ Complex sample data in structural equation modeling. ” . In Sociological methodology Edited by: Marsden , P. V. 267 – 316 . Washington , DC : American Sociological Association. .
   Google Scholar
• Neale , M. C. , Boker , S. M. , Xie , G. and Maes , H. H. 2004 . Mx: Statistical modeling, , 5th ed. Richmond : Virginia Commonwealth University, Department of Psychiatry. .
   Google Scholar
• Neale , M. C. and Miller , M. B. 1997 . The use of likelihood-based confidence intervals in genetic models . Behavior Genetics , 27 : 113 – 120 .
   PubMed Web of Science ®Google Scholar
• R Development Core Team. 2010 . R: A language and environment for statistical computing. Vienna , , Austria : R Foundation for Statistical Computing. .
   Google Scholar
• Raudenbush , S. W. and Bryk , A. S. 2002 . Hierarchical linear models, , 2nd ed. Newbury Park , CA : Sage. .
   Google Scholar
• Raudenbush , S. W. , Rowan , B. and Kang , S. J. 1991 . A multilevel, multivariate model for studying school climate in secondary schools with estimation via the EM algorithm . Journal of Educational Statistics , 16 : 295 – 330 .
   Google Scholar
• Raykov , T. and du Toit , S. H. C. 2005 . Evaluation of reliability for multiple component measuring instruments in hierarchical designs . Structural Equation Modeling , 12 : 536 – 550 .
   Web of Science ®Google Scholar
• Raykov , T. and Shrout , P. E. 2002 . Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach . Structural Equation Modeling , 9 : 195 – 212 .
   Web of Science ®Google Scholar
• Roesch , S. C. , Aldridge , A. A. , Stocking , S. N. , Villodas , F. , Leung , Q. , Bartley , C. E. and Black , L. J. 2010 . Multilevel factor analysis and structural equation modeling of daily diary coping data: Modeling trait and state variation . Multivariate Behavioral Research , 45 : 767 – 789 .
   PubMed Web of Science ®Google Scholar
• Ryu , E. and West , S. G. 2009 . Level-specific evaluation of model fit in multilevel structural equation modeling . Structural Equation Modeling , 16 : 583 – 601 .
   Web of Science ®Google Scholar
• Schwartz , J. E. and Stone , A. A. 1998 . Data analysis for EMA studies . Health Psychology , 17 : 6 – 16 .
   PubMed Web of Science ®Google Scholar
• Schwartz , J. E. and Stone , A. A. 2007 . “ The analysis of real-time momentary data: A practical guide. ” . In The science of real-time data capture: Self-report in health research Edited by: Stone , A. A. , Shiffman , S. , Atienza , A. and Nebeling , L. 76 – 113 . New York , NY : Oxford University Press. .
   Google Scholar
• Schwarz , N. 2007 . “ Retrospective and concurrent self-reports: The rationale for real-time data capture. ” . In The science of real-time data capture: Self-reports in health research Edited by: Stone , A. A. , Shiffman , S. S. , Atienza , A. and Nebeling , L. 11 – 26 . New York , NY : Oxford University Press. .
   Google Scholar
• Shapiro , A. 1984 . A note on the consistency of estimators in the analysis of moment structures . British Journal of Mathematical and Statistical Psychology , 37 : 84 – 88 .
   Web of Science ®Google Scholar
• Shiffman , S. 2007 . “ Designing protocols for ecological momentary assessment. ” . In The science of real-time data capture: Self-reports in health research Edited by: Stone , A. A. , Shiffman , S. , Atienza , A. and Nebeling , L. 27 – 53 . New York , NY : Oxford University Press. .
   Google Scholar
• Shiffman , S. M. and Stone , A. A. 1998 . Introduction to the special section: Ecological momentary assessment in health psychology . Health Psychology , 17 : 3 – 5 .
   Web of Science ®Google Scholar
• Shiffman , S. , Stone , A. A. and Hufford , M. R. 2008 . Ecological momentary assessment . Annual Review of Clinical Psychology , 4 : 1 – 32 .
   PubMed Web of Science ®Google Scholar
• Shiyko , M. P. and Ram , N. 2011 . Conceptualizing and estimating process speed in studies employing ecological momentary assessment designs: A multilevel variance decomposition approach . Multivariate Behavioral Research , 46 : 875 – 899 .
   PubMed Web of Science ®Google Scholar
• Snijders , T. A. B. and Bosker , R. J. 1999 . Multilevel analysis: An introduction to basic and advanced multilevel modeling. London , , UK : Sage. .
   Google Scholar
• Stone , A. A. and Shiffman , S. 1994 . Ecological momentary assessment (EMA) in behavioral medicine . Annals of Behavioral Medicine , 16 : 199 – 202 .
   Google Scholar
• Stone , A. A. , Shiffman , S. , Atienza , A. and Nebeling , L. 2007 . The science of real-time data capture: Self-reports in health research. New York , NY : Oxford University Press. .
   Google Scholar
• Tennen , H. , Affleck , G. and Armeli , S. 2005 . Personality and daily experience revisited . Journal of Personality , 73 : 1 – 19 .
   PubMed Web of Science ®Google Scholar
• Trull , J. T. and Ebner-Priemer , U. W. 2009 . Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: Introduction to the special section . Psychological Assessment , 21 : 457 – 462 .
   PubMed Web of Science ®Google Scholar
• Verbeke , G. and Molenberghs , G. 2000 . Linear mixed models for longitudinal data. New York , NY : Springer. .
   Google Scholar
• Watson , D. , Clark , L. A. and Tellegen , A. 1988 . Development and validation of brief measures of positive and negative affect: The PANAS scales . Journal of Personality and Social Psychology , 54 : 1063 – 1070 .
   PubMed Web of Science ®Google Scholar
• Wenze , S. J. and Miller , I. V. 2010 . Use of ecological momentary assessment in mood disorders research . Clinical Psychology Review , 30 : 794 – 804 .
   PubMed Web of Science ®Google Scholar
• Yang , Y. and Green , S. B. 2010 . A note on structural equation modeling estimates of reliability . Structural Equation Modeling , 17 : 66 – 81 .
   Web of Science ®Google Scholar