Analyzing Complete Generalizability Theory Designs Using Structural Equation Models

ABSTRACT

We illustrate how to analyze complete generalizability theory (GT) designs using structural equation modeling software (lavaan in R), compare results to those obtained from numerous ANOVA-based packages, and apply those results in practical ways using data obtained from a large sample of respondents, who completed the Self-Perception Profile for College Students (Neemann & Harter, 2012) on two occasions. Results revealed that estimates of variance components, generalizability coefficients, dependability coefficients, and proportions of measurement error derived from lavaan were essentially equivalent to those produced by the GT packages GENOVA and gtheory in R and variance component programs in SPSS, SAS, and R. Within the article and extended online Supplemental Material, we illustrate how indices obtained from these resources can be used for either norm- and criterion-referencing purposes and for estimating effects of changes made to measurement procedures. We further describe ways to use structural equation models for applications of GT beyond what conventional ANOVA-based packages would typically permit.

Acknowledgments

We thank the Iowa Measurement Research Foundation for providing research assistant support for this project and Tingting Chen for her assistance in creating factor model diagrams and proof reading drafts of the article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/08957347.2023.2274573

Additional information

Funding

The work was supported by the Iowa Measurement Research Foundation.