Using Modification Indexes to Detect Turning Points in Longitudinal Data: A Monte Carlo Study

Abstract

Some nonlinear developmental phenomena can be represented by using a simple piecewise procedure in which 2 linear growth models are joined at a single knot. The major problem of using this piecewise approach is that researchers have to optimally locate the knot (or turning point) where the change in the growth rate occurs. A relatively simple way to detect the location of the knot or turning point is to freely estimate the time-specific factor loadings using the linear latent growth model framework. The major goal of this simulation study was to examine the effectiveness of using modification indexes (MIs) to detect potential turning points in longitudinal data. The results showed that when using a restricted search strategy with an adequate number of both observations (210) and measurement waves (8), MIs performed well in detecting a medium change in the growth rate between two linear models at the turning point. Implications of the findings and limitations are discussed.

Notes

¹One study (Lagattuta & Wellman, 2002) contained a very large number of repeated measures (i.e., more than 50). This study was treated as an outlier and excluded from the proportion calculation.

²The reason we used the centered time predictor for our simulation study is to reduce the potential nonessential multicollinearity. Biesanz, Deeb-Sossa, Papadakis, Bollen, and Curran (2004) indicated that the coding of the time predictor should depend on the purpose of the research question. More detailed information and suggestions can be found in Biesanz et al. (2004).

³The effect size values were obtained by the following effect size equation (Raudenbush & Liu, 2001):

where δ is the standardized effect size, β₁ is the average linear growth trend, and τ₁₁ is the variance of the random effect associated with the growth parameter, which indicates the differences between individual growth trends and the average growth trend. Cohen (1988) provided some standardized effect size guidelines in which small effect size (i.e., δ) is equal to .20 and medium effect size is equal to .50. Raudenbush and Liu (2001) proposed similar guidelines for the size of τ₁₁, where .05 was for small τ₁₁ and .10 was for medium τ₁₁. Given the values of δ and τ₁₁, the corresponding β₁ could be easily computed. β₀, the model intercept, was fixed to a constant value in all conditions (i.e., β₀ = .10).

⁴By using Raudenbush and Liu's (2001) effect size equation, a small growth rate is .05 and a medium growth rate is .16. We set the growth rate for the first piece model as .16 (a medium growth rate). We can translate this medium growth rate into “degrees”: tan⁻¹(.16) ≈ 9.09°. Similarly, we can translate the small growth rate into degrees: tan⁻¹(.05) ≈ 2.86°. Hence, the growth rates of the second piece are: tan (9.09° – 2.86°) = (6.23°) = .11 given that a small change in the growth rate (i.e., 2.86° reduction), and tan (9.09° – 9.09°) = .00 given that a medium change in the growth rate (i.e., 9.09° reduction).

^aEmpirical power.

⁵We did not examine the turning point at the first two time points. We constrained the loadings of the first two time points to 0 and 1, respectively, to identify the model (Bollen & Curran, 2006; Meredith & Tisak, 1990). Changing the identifying constraints would alter the interpretation of the unstandardized model. If the turning point is suspected to occur early in the series, the hierarchical procedure can use an alternate parameterization of the model such as setting the factor loading of the (J – 1)th wave to 0 and the Jth (final) wave to 1.

^aEmpirical type I error rate.

⁶Generally, the Type I error rate a is typically set to be .05 so that the 95% CI for the nominal Type I error rate is: .05 ± 1.96 * , where N is the number of converged replications within each condition. For this study, the 95% CI for the nominal Type I error rate was [.03, .07] with N = 500 replications per condition.