Testing Measurement Invariance over Time with Intensive Longitudinal Data and Identifying a Source of Non-invariance

Abstract

Longitudinal measurement invariance (LMI) is a critical prerequisite to assessing change over time with intensive longitudinal data (ILD). For LMI testing with ILD, we propose cross-classified factor analysis (CCFA) to detect non-invariant item parameters and alignment optimization (AO) to detect non-invariant time points as a supplement to CCFA. In addition, we use a covariate in CCFA to identify a source of non-invariance. To evaluate the proposed models under unique features of ILD, such as autoregression (AR), we conducted a Monte Carlo simulation study. The results showed CCFA can be an excellent tool for ILD LMI testing regardless of simulation factors even when AR was misspecified and can identify a source of non-invariance using a covariate. AO can supplement CCFA to find non-invariant time points although AO requires a large number of persons. We provide detailed discussions and practical suggestions.

Keywords:

• Alignment
• autoregression
• cross-classified
• longitudinal
• measurement invariance

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 For identification, the within-level factor variance is fixed at one.

2 The interval-contingent design is in contrast to signal-contingent and event-contingent designs (Bolger & Laurenceau, 2013). With signal-contingent designs participants provide data at time points randomly signaled by the researcher and in event-contingent designs, participants provide data when a predefined event has occurred (e.g., smoking event). Each of these designs has advantages and disadvantages and the decision to use a particular design is usually determined by a number of factors that include the nature of the construct under investigation (e.g., amount of variability), research questions, and practical considerations (e.g., participant compliance). Papini et al.’s (2020) systematic review reported that the studies they examined (n = 29) were predominantly based on either random prompts (n = 13) or fixed time intervals (n = 10). The examples of ILD data we observed in the literature are presented in the online supplements: https://osf.io/8pf6k/?view_only=a0e99ef8916b433baabceae49a725eeb.

3 Because participants in ILD studies need to respond to survey prompts numerous times (e.g., five times a day for 10 days), the length and complexity of a survey needs to be very limited (Shiffman et al., 2008) and it would be easier to have fewer items for one latent construct. For example, the average number of items was 2.75 in the range of 1–9 in a systematic review of ILD studies of suicidal ideation (Rabasco & Sheehan, 2022).

4 In McNeish et al. (2021), the between-time factor variance was 0.012 while the between-person factor variance was 0.913. In our empirical investigation with one set of ILD, the between-time factor variance was 1.359 while the between-person factor variance was 1.074.

5 We used R package parallel for simultaneous running of multiple models to maximize the computer capacity. With 10 cores utilized, running four conditions of the largest sample size took about 3 days.

6 We did not consider using the 95% credible interval (CI) for a variance (i.e., whether the CI includes zero or not given that the CI with zero included indicates zero variance or measurement invariance). In Mplus with a Bayes estimator, the credible interval for a variance never includes zero because its posterior distribution always consists of positive values only (Asparouhov & Muthén, 2021).

7 We also examined the proportion of non-invariant time points (i.e., the number of time points that are detected as MNI divided by the total number of time points). We did not use this proportion as a detection rate because it depends on the proportion of MNI and cannot reach 1.0 (.50 maximum with 50% MNI; .25 maximum with 25% MNI).

8 https://osf.io/8pf6k/?view_only=a0e99ef8916b433baabceae49a725eeb.