The Nature of Rater Effects and Differences in Multilevel MTMM Latent Variable Models

ABSTRACT

Measuring and understanding the nature of informant/rater effects and differences (Level 1) on a common trait when the target of measurement is at the organizational level (Level 2) involves a number of methodological considerations. Although previous research has discussed single-level latent variable applications of the correlated trait-correlated method minus one (CT-C(M-1)) model for multitrait-multimethod (MTMM) designs, only recently was the CT-C(M-1) model derived for multilevel data structures. The current paper examines research on the development of multilevel latent variable measurement models when both structurally different and interchangeable raters are involved in measuring Level 2 constructs. In so doing, we review and articulate often overlooked design and interpretive considerations unique to these models. We describe these elements within the context of a real-world multilevel CT-C(M-1) latent variable model that measures school climate for 294 public high schools using individual-level ratings from students (N = 58,018), teachers (N = 12,236), and health professionals (N = 742).

KEYWORDS:

• Multiple Informants
• MTMM design
• multilevel SEM
• school climate
• method effects
• rater effects
• correlated trait-correlated method minus one
• CT-C(M-1)
• latent variable models

Additional information

Funding

This work was supported by the Office of Juvenile Justice and Delinquency Prevention [#2012-JF-FX-0062]; U. S. Department of Justice [#NIJ 2014-CK-BX-0004]. The opinions, findings, and conclusions or recommendations expressed in this report are those of the authors and do not necessarily reflect those of the Office of Juvenile Justice and Delinquency Prevention or the U.S. Department of Justice.