A Computationally More Efficient and More Accurate Stepwise Approach for Correcting for Sampling Error and Measurement Error

ABSTRACT

Over the last decade or two, multilevel structural equation modeling (ML-SEM) has become a prominent modeling approach in the social sciences because it allows researchers to correct for sampling and measurement errors and thus to estimate the effects of Level 2 (L2) constructs without bias. Because the latent variable modeling software Mplus uses maximum likelihood (ML) by default, many researchers in the social sciences have applied ML to obtain estimates of L2 regression coefficients. However, one drawback of ML is that covariance matrices of the predictor variables at L2 tend to be degenerate, and thus, estimates of L2 regression coefficients tend to be rather inaccurate when sample sizes are small. In this article, I show how an approach for stabilizing covariance matrices at L2 can be used to obtain more accurate estimates of L2 regression coefficients. A simulation study is conducted to compare the proposed approach with ML, and I illustrate its application with an example from organizational research.

KEYWORDS:

• Multilevel modeling
• structural equation modeling
• maximum likelihood
• stepwise estimation
• stabilization

Notes

1 Analysis of variance (ANOVA) estimates (e.g., Searle et al., 1992) can be derived as (2) $${\widehat{\sigma }}_{B,yx}=\frac{{\sum }_j\left({\bar{y}}_{•j}-{\bar{y}}_{••}\right)\left({\bar{x}}_{•j}-{\bar{x}}_{••}\right)}{J-1}-\frac{1}{n}\frac{{\sum }_j{\sum }_i\left(y_{ij}-{\bar{y}}_{•j}\right)\left(x_{ij}-{\bar{x}}_{•j}\right)}{N-J}$$(2) (3) $${\widehat{\sigma }}_{B,x}^2=\frac{{\sum }_j{\left({\bar{x}}_{•j}-{\bar{x}}_{••}\right)}^2}{J-1}-\frac{1}{n}\frac{{\sum }_j{\sum }_i{\left(x_{ij}-{\bar{x}}_{•j}\right)}^2}{N-J},$$(3) where N = Jn is the total number of observations.

2 Applying the ANOVA method yields (6) $${\widehat{\sigma }}_{y\xi }=\frac{{\sum }_i\left(y_i-{\bar{y}}_{•}\right)\left({\bar{x}}_{•i}-{\bar{x}}_{••}\right)}{N-1}$$(6) (7) $${\widehat{\sigma }}_{\xi }^2=\frac{{\sum }_i{\left({\bar{x}}_{•i}-{\bar{x}}_{••}\right)}^2}{N-1}-\frac{1}{K}\frac{{\sum }_k{\sum }_i{\left(x_{ki}-{\bar{x}}_{•i}\right)}^2}{(K-1)N}.$$(7)

3 K_r is the number of indicators of the rth predictor variable.

4 The covariances of the measurement errors are assumed to be zero and thus need not to be estimated.

5 $\tilde{n}$ is a rather difficult to interpret. However, it may help to recognize that $\tilde{n}$ is close to the average number $\bar{n}=\frac{1}{J}{\sum }_{j=1}^J{n}_j$ of persons per group when the data are nearly balanced (see Hox, 2010).

6 Note that this does not necessarily mean that degenerate matrices are completely useless in practice (see Savalei & Kolenikov, 2008, for the argument that degenerate matrices can be useful in the evaluation of model fit).

7 d must be chosen such that $\frac{J}{d}$ is an integer. Typically, d is much smaller than J (see Simulation Study). Once d has been selected, subsamples can be created by the following algorithm: First, divide (1, …, J) into $\frac{J}{d}$ subsets that do not overlap. For example, (d + 1, …, J), …, (1, …, J − d). Next, use these subsets to create the subsamples. More specifically, create the first subsample by maintaining groups d + 1, …, J, the second subsample by maintaining groups 1, …, d and 2d + 1, …, J, the third subsample by maintaining groups 1, …, 2d and 3d + 1, …, J, and so forth.

8 It should be noted that focusing on the smaller of the two between-group regression coefficients (β_{B, 2}) instead would not have changed these findings.

9 Mplus runs a maximum of 500 iterations by default and uses a criterion of 0.001 to assess convergence (Muthén & Muthén, 2012, pp. 590–629).

10 In an additional analysis, I also investigated the statistical properties of the estimates of the within-group regression coefficient. With an absolute value of the estimated relative bias below 0.05, all approaches to estimating the ML-SEM were unbiased except the manifest model. They did not differ with respect to the relative RMSE, and the coverage rates were all close to the nominal 95% level.

11 Please find the equations that were used to compute these estimates as well as further details about their derivations in the Appendix.