389
Views
0
CrossRef citations to date
0
Altmetric
Research Articles

The Effect of Noninvariance on the Estimation of the Mediated Effect in the Two-Wave Mediation Model

Pages 908-919 | Received 21 Oct 2020, Accepted 13 Apr 2022, Published online: 10 Jun 2022
 

Abstract

The two-wave mediation model is the most suitable model for examining mediation effects in a randomized intervention and includes measures taken at pretest and posttest. When using self-report measures, the meaning of responses may change for the treatment group over the course of the intervention and result in noninvariance across groups at posttest, a phenomenon referred to as response shift. We investigate how the mediated effect would be impacted by noninvariance when using sum scores (i.e., assuming invariance). In a Monte Carlo simulation study, the magnitude and proportion of items that had noninvariant intercepts, the direction of noninvariance, number of items, effect size of the mediated effect and sample size were varied. Results showed increased Type I and Type II errors due to a biased estimate of the intervention effect on the mediator resulting from noninvariance. Thus, measurement noninvariance could lead to erroneous conclusions about the process underlying the intervention.

Notes

1 Prior to treatment, M1 and Y1 are assumed to be correlated to form a joint confounder of the M2 to Y2 relation (i.e., the b-path component of the mediated effect ab5) via the temporal stabilities of the mediator (b2) and the outcome (b1) (Mayer et al., Citation2014; Valente and MacKinnon, Citation2020), and if present, via the cross-lags (b3 and b4). It is important to adjust for M1 and Y1 to adjust for potential confounder bias of the mediated effect.

2 The literature on measurement invariance methodology largely focuses on reflective measurement models (Bollen & Diamantopoulos, Citation2017, p. 607) in which changes in the latent construct result in changes to the observed indicators rather than the other way around. Therefore, under a reflective model, differences in any of the measurement parameters are understood as measurement noninvariance rather than indicating change in the construct. Throughout this paper, we make the explicit assumption that the measurement model for the construct is reflective. If the measurement model is not reflective, but a reflective model is fit, true change could appear as noninvariance. Therefore, we urge researchers to ensure that a reflective measurement model is justifiable for their mediator construct. Alternative measurement models include causal-formative indicators (e.g., Bollen & Bauldry, Citation2011; Rhemtulla, van Bork & Borsboom, Citation2020) and psychometric network models (e.g., Epskamp, Rhemtulla, & Borsboom, Citation2017).

3 For example, if the intercepts for the control group were (0.9, 0.4, 0.5, 0.5, 0.7, 0.4), and the intercepts for the treatment group were (0.9, 0.4, 0.6, 0.4, 0.7, 0.4), there is noninvariance for items 3 and 4 in different directions. However, because τC*=3.4 and τT* = 3.4, per Equation 6, the a-path will not be biased.

4 In using multiple group model for data generation, the a-path was required to enter the model as an intercept in the regression of M2 on M1 because the X variable representing group is not a variable in the structural model. This intercept was then allowed to differ between groups. Thus, if the true a=0.285, the regression in the data-generating model for the treatment group was: M2=0.285 +0 .985M1 and for the control group M2=0.985M1.

5 This would apply best for situations in which it is not possible to fit a MIMIC model due to a small sample size and a large number of items. If it is not possible to fit a latent variable model to test for invariance, researchers could consider simpler techniques for testing for noninvariance, such as observed variable methods (Millsap, Citation2011, Ch. 8).

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.