The Impact of Over-Simplifying the Between-Subject Covariance Structure on Inferences of Fixed Effects in Modeling Nested Data

Abstract

This study discusses the effects of oversimplifying the between-subject covariance structure on inferences for fixed effects in modeling nested data. Linear and quadratic growth curve models (GCMs) with both full and simplified between-subject covariance structures were fit to real longitudinal data. The results were contradictory to the statement that using oversimplified between-subject covariance structures (e.g., uni-level analysis) leads to underestimated standard errors of fixed effect estimates and thus inflated Type I error rates. We analytically derived simple mathematical forms to systematically examine the oversimplification effects for the linear GCMs. The derivation results were aligned with the real data analysis results and further revealed the conditions under which the standard errors of the fixed-effect intercept and slope estimates could be underestimated or overestimated for over-simplified linear GCMs. Therefore, our results showed that the underestimation statement is a myth and can be misleading. Implications are discussed and recommendations are provided.

Keywords:

• fixed effect inference
• growth curve modeling
• ignoring nested data structure
• misspecified between-subject covariance structure
• multilevel modeling

Appendix

Changes in standard error estimates of fixed effect estimates with a simplified between-subject covariance matrix in linear growth curve modeling

The true model is Model 2 with the full between-subject covariance structure (i.e., both $b_1$ and $b_2$ are included). Throughout this Appendix, we use the following notations: $S$ is the sum of the squared values of the centered time variable, $T^{\ast }=T$(number of time points) for ML estimation, and $T^{\ast }=T+(T-2)/(N-1)$ for REML estimation.

Notes

¹ For normal data, GEE reduces to ML (Liang & Zeger, 1986).

² Using effect size measures described in Singer and Willett (2003), we found that 89.96% of the within-person variability can be explained by a linear trend function. The number became 90.03% when a quadratic trend function is used. In addition, the correlation between predicted and observed outcome scores was 0.968 from the unconditional linear models with the full between-subject covariance matrix and the squared correlation was 0.937. The numbers stayed for the quadratic growth model with the full between-subject covariance matrix. So, a linear or quadratic model can explain most of the within-person variation in the real data.

Additional information

Funding

Lijuan Wang is grateful for the financial support from the National Institutes of Health [Grant Number 1R01HD087319].