An SEM Approach for the Evaluation of Intervention Effects Using Pre-Post-Post Designs

Abstract

This study analyzes latent change scores using latent curve models (LCMs) for evaluation research with pre–post–post designs. The article extends a recent article by Willoughby, Vandergrift, Blair, and Granger (2007) on the use of LCMs for studies with pre–post–post designs, and demonstrates that intervention effects can be better tested using different parameterizations of LCMs. This study illustrates how to test the overall mean of a latent variable at the time of research interest, not just at baseline, as well as means of latent change variables between assessments, and introduces how individual differences in the referent outcome (i.e., Level 2 random effects) and measurement-specific residuals (i.e., Level 1 residuals) can be modeled and interpreted. Two intervention data examples are presented. This LCM approach to change is more advantageous than other methods for its handling of measurement errors and individual differences in response to treatment, avoiding unrealistic assumptions, and being more powerful and flexible.

ACKNOWLEDGMENTS

This study was funded by the National Institute on Drug Abuse (DA 17552) as part of the Rutgers Transdiscplinary Prevention Research Center. We thank Tenko Raykov for helpful comments, and Tom Morgan, Lisa Laitman, Barbara Kachur, Brian Kaye, Malina Spirito, Sara Fink, Corey Grassl, Lisa Pugh, Kelly Pugh, and Adam Thacker for their help with data collection.

Notes

¹The intercept term noted by Willoughby et al. (2007) and also adopted in this study is an arbitrary latent variable name to indicate the referent time point. It is different from the intercept terms ν _y and ν _x in the y and x measurement equations y = ν _y + Λ _y η + ε and x = ν _x + Λ _x ξ + δ, respectively. The latter terms indicate the expected values of y and x when their respective latent variables η and ξ are zero. In LCMs, ν _y and ν _x are typically specified to be zero.

²Models are equivalent if they have the same number of independent parameters, the same fitted covariance matrix and residuals, and the same goodness-of-fit statistics. Equivalent models occur when “there is a one-to-one transformation between the two sets of parameters” (Jöreskog & Sörbom, 1996, pp. 272–275).

³To have unbiased estimates, ν _y and ν _x need to be specified to be zero.

⁴The identification of covariance elements, ψ₂₂ (latent variable variance for Change 1) and ψ₃₃ (latent variable variance for Change 2) depends on the residual variances of Time 1 and Time 2 measures being fixed to zero, and vice versa. Although it is possible to constrain either latent variable variance elements or residual variances to a constant rather than to zero to achieve identification (Raykov, 1994; cf. Jöreskog & Sörbom, 1988), the unique distinction between latent and observed variable variances is elusive for nonreferent measurements. For the purpose of model identification, the unconditional LCMs in which the residual variances are constrained to be equal (θ _e = σ² I) and all three variances of latent variables are estimated also have 2 degrees of freedom and are, therefore, identified. These LCMs are equivalent to the homogeneous Level 1 variance model in multilevel linear models. However, this parameterization approach requires the assumption that the three time measurements are interchangeable, which is unlikely for the data collected from a pre–post–post design. Therefore, this alternative parameterization (or the homogeneous Level 1 variance, multilevel linear model) will more likely poorly fit the data collected for evaluation research using pre–post–post designs.

⁵Two dummy-code variables that reflect change patterns from the referent need to be created and entered as predictors in multilevel linear models to produce equivalent results. Two dummy-code variables to be entered are the same as the last two columns of the respective factor loading matrix.

⁶In the current LCMs, estimating the residual variance of the observed baseline will have the same effect.

*p < .05.

⁷With the available degrees of freedom, we examined correlations among latent variables one by one in the unconditional models. For Sample 1, covariance estimates between the intercept and Change 1, between the intercept and Change 2, and between Change 1 and Change 2 were −.04 (SE = .02), .01 (SE = .02), and .01 (SE = .02), respectively, and none was statistically significant at p < .05. For Sample 2, covariance estimates between the intercept and Change 1, between the intercept and Change 2, and between Change 1 and Change 2 were −.05 (SE = .03), .02 (SE = .03), and −.03 (SE = .02), respectively, and none was statistically significant at p < .05. Insignificant minimal correlations among latent variables were also observed in the conditional models. For Sample 1, covariance estimates between the intercept and Change 1, between the intercept and Change 2, and between Change 1 and Change 2 were −.04 (SE = .22), .01 (SE = .02), and −.04 (SE = .02), respectively, and none was statistically significant at p < .05. For Sample 2, covariance estimates between the intercept and Change 1, between the intercept and Change 2, and between Change 1 and Change 2 were −.04 (SE = .03), .01 (SE = .03), and −.03 (SE = .02), respectively, and none was statistically significant at p < .05.

*p < .05.

⁸Testing treatment effects utilizing the multigroup SEM approach can be done by not imposing any equality constraints on the kappa (or alpha) matrix using LISREL, and the unconditional model can be tested by constraining elements in the kappa (or alpha) matrix to be equal across groups. The chi-square model fit comparisons can gauge whether the model improves after constraints are successively or simultaneously freed.