Testing the convergent and discriminant validity of the Systemic Therapy Inventory of Change Initial scales

Abstract

Objective: The Systemic Therapy Inventory of Change (STIC^®) is the first multi-systemic and multi-dimensional measurement and feedback system designed for assessment in family, couple, and individual functioning. Patients fill out the STIC Initial before the first session to identify treatment targets and provide starting values for subsequent assessments of trajectories of change. This study tested the construct validity of five of the six STIC Initial scales. Methods: We administered both the STIC Initial and a set of validity measures to a relatively large sample of patients. Convergent and discriminant validity were tested using both an examination of observed correlations and confirmatory factor analysis (CFA). Results: The correlations among the observed measures showed that the convergent validity coefficients were generally large, whereas the discriminant validity coefficients were moderate to small. Similarly, CFAs suggested that the STIC total scales and subscales are good indicators of the factors they were intended to measure and that the STIC total scales and subscales are weakly related to the factors they were intended to not measure. Conclusion: The results supported the convergent and discriminant validity of the five scales of the STIC Initial.

Clinical or methodological significance of this article: The clinical significance of this article is that it demonstrates that the STIC Initial should be useful for identifying treatment targets including both which systems, in addition to the facets within each system, that require targeting. The methodological significance is twofold. First, the use of CFA for testing convergent and discriminant validity is still relatively rare. Second, we demonstrated how to use CFA for a more stringent test of discriminant validity compared with the original approach described by Cole (1987).

Keywords:

• psychotherapy outcome measure
• multi-systemic
• construct validity
• convergent validity
• discriminant validity

Notes

1 The STIC is wholly owned by the Epstein Center for Psychotherapy Change, which is a limited liability corporation of the Family Institute, which is a not-for-profit 501c3 corporation. The STIC is copyrighted and trademarked by The Family Institute. None of the authors or investigators have any proprietary interest in the STIC. The STIC Initial, STIC Intersession, and Integrative Psychotherapy Alliance scales are free for anyone to use. To help defray some of the costs of maintaining the Psychotherapychange.org website, The Family Institute does charge for use of the website.

2 We do not report the results of a cross-sectional, item-level CFA of the STIC Initial in this article. The primary reason for this is that a manuscript in preparation (Quirk, et al., 2017) tests whether the factor structure holds longitudinally in the current sample demonstrates adequate invariance of the factor structure over time and this provides an even more stringent test of the replicability of the factor structure in this sample than would a cross-sectional replication.

3 Cattell (1952) and Thurstone (1947) discussed the importance of factor loadings that essentially equal zero in factor analysis and the loadings that essentially equal zero may be thought of as discriminant validity coefficients. That is, they argued that an essential part of the definition of a construct is the identification of what the construct does not correlate with.

4 When participants reported more than one child’s adjustment as measured by the SDQ-P, RWC, and CPS, their responses about one child were randomly selected and included in the data analyses.

5 Some correlations in these analyses (and factor loadings in the subsequent factor analyses) were negative because some of the validation measures are scored in the opposite direction. Hence, we focus here on the absolute value of the correlations.