A Comparison of Method Effects in Two Confirmatory Factor Models for Structurally Different Methods

Abstract

Multimethod data analysis is a complex procedure that is often used to examine the degree to which different measures of the same construct converge in the assessment of this construct. Several authors have called for a greater understanding of the definition and meaning of method effects in different models for multimethod data. In this article, we compare 2 recently proposed approaches for modeling data with structurally different methods with regard to the definition and meaning of method effects, the restricted CT-C(M – 1) model (Geiser, Eid, & Nussbeck, 2008) and the latent difference model (Lischetzke, Eid, & Nussbeck, 2002). We also introduce the concepts of individual, conditional, and general method bias and show how these types of biases are represented in the models. An application to a multirater data set (N = 199) as well as recommendations for the application and interpretation of each model are provided.

Keywords:

• confirmatory factor analysis
• CT-C(M – 1)
• latent difference
• method effects
• multitrait–multimethod analysis

Notes

¹Throughout this article, we assume that the latent variable units of measurement and latent variable means are identified by fixing the loading of one indicator per T_m factor to 1.0 and the intercept of the same indicator to 0.0.

²The difference between the restricted CT-C(M – 1) model and the original CT-C(M – 1) model (as discussed in Eid et al., 2003; Eid et al., 2008) is that the original CT-C(M – 1) model estimates the loadings of the nonreference indicators on the reference factors (λ* _im , m ≠ r) freely, whereas the restricted CT-C(M – 1) model imposes the following constraint on these loadings: λ* _im = λ _im β _mr . Furthermore, if a mean structure is included in the restricted CT-C(M – 1) model, the intercepts of the nonreference indicators, α* _im , are constrained to equal α* _im = α _im + λ _im γ _mr , where α _im is equal to the intercept in the baseline TMU model.

³These two properties are logical consequences of the definition of this factor as a residual variable. See Steyer (1988) and Zimmerman (1975) for a detailed discussion of the properties of residual variables.

⁴Given that hypotheses regarding convergent validity are unidirectional, one-tailed tests of significance were used for these tests.

^aModel contains no method factors.

^bSelf-report was selected as reference method.

^cThe latent differences from the self-report were considered.

^dStandardized loadings > |1| are admissible in the LD model, because reference and method factors can be correlated in this model so that these loadings represent partial regression coefficients, which are not restricted to a [−1, +1] interval (e.g., Deegan, 1978).

*p < .05.

**p < .01.

***p < .001.

^aSelf-report was selected as reference method.

^bThe latent differences from the self-report were considered. All parameters set to 0 by definition are represented by a dash.

*p < .05.

**p < .01.

***p < .001.