Multitrait–Multimethod Assessment of Giftedness: An Application of the Correlated Traits–Correlated (Methods – 1) Model

Abstract

Although the use of multiple criteria and informants is one of the most universally agreed on practices in the identification of gifted children, few studies to date have examined the convergent validity of multiple informants and objective ability tests in gifted identification. In this study, we illustrate the use of the correlated traits–correlated (methods – 1) or CT–C(M – 1) model (Eid, Lischetzke, Nussbeck, & Trierweiler, 2003) to examine the convergent validity of self, parent, and teacher ratings relative to objective cognitive ability tests in a sample of 145 4th to 6th graders. The CT–C(M – 1) analyses revealed that teacher ratings showed the highest convergence with the objective assessments, whereas self-ratings had the lowest reliabilities and insufficient validity. Parent ratings were more reliable and valid than self-reports, but were outperformed by teacher ratings for most abilities. Overall, the CT–C(M – 1) analyses showed that the convergent validity of the ratings relative to the objective test battery was highest for numerical and lowest for creative abilities. Furthermore, whereas part of the shared variance between parent and teacher ratings reflected true convergent validity, agreement between parent and self-reports was entirely due to a shared rater variance. Our analyses demonstrate the usefulness and proper interpretation of the CT–C(M – 1) approach for examining convergent validity and method effects in multitrait–multimethod data.

Keywords:

• CT–C(M – 1) model
• gifted identification
• intelligence
• multiple informants
• multitrait–multimethod
• confirmatory factor analysis

Notes

¹ An alternative explanation of a high correlation between raters that is not shared with objective ability tests is that the ability tests are invalid and that the ratings in fact reflect valid ability variance. That is, the ratings migt reflect true aspects of giftedness that are not (or not fully) captured by the tests. Researchers who want to study this alternative hypothesis could include additional variables (e.g., criterion measures) into the model to examine whether the rater variance can predict relevant outcomes that cannot be predicted by the tests.

² In addition, a researcher could formally test the hypothesis of no true convergent validity by constraining the loadings of the nonreference indicators on the reference factor to zero and comparing the fit of this constrained model to a model in which the loadings are freely estimated. If there is no true convergent validity, then the model with constrained loadings should not fit the data worse than the model with freely estimated loadings.

³ In our analyses, we used the CT–C(M – 1) model with a single general trait factor. Note that in cases in which a researcher uses heterogeneous indicators that reflect different facets of a construct and that are consistent across methods, a model with indicator-specific traits often provides a better fit to the data. For details, see Eid et al. (2008) or Grigorenko, Geiser, Slobodskaya, and Francis (2010).

⁴ Note that this approach is not without limitations, given that the information about the reliability coefficient and test score variance was derived from the same data to which we fit the model. Nonetheless, using reliability estimates from a different sample or population could have been even more problematic, given the dependence of the reliability coefficient on the amount of variability in the specific sample. In general, the accuracy of the results depends on how accurate a researcher’s estimate of the reliability of the reference indicator is for the data set at hand.