Estimating Variance Components and Random Effects Using the Box-Cox Transformation in the Linear Mixed Model

Abstract

The linear mixed model assumes normality of its two sources of randomness: the random effects and the residual error. Recent research demonstrated that a simple transformation of the response targets normality of both sources simultaneously. However, estimating the transformation can lead to biased estimates of the variance components. Here, we provide guidance regarding this potential bias and propose a correction for it when such bias is substantial. This correction allows for accurate estimation of the random effects when using a transformation to achieve normality. The utility of this approach is demonstrated in a study of sleep-wake behavior in preterm infants.

Keywords:

• Linear mixed model
• Longitudinal data
• Normality
• Random effects
• Transformation

Mathematics Subject Classification:

• Primary 62J05
• Secondary 62P10

Notes

*The variable associated with each of the states is a percentage of time in that state during a four-hour observation of the infant.

SE = Monte Carlo standard error of the model parameter estimates.

, “naive” and “adjusted” fixed effect age estimate.

, = Estimated “naive” variance component parameter estimates.

, = Estimated “adjusted” variance component parameter estimates.

, component (VC) parameter estimates from the untransformed model.

, = “Naive” VC estimates from the transformed model.

, =“Adjusted” VC estimates from transformed model.

), ), estimates: untransformed model, and transformed model (naive and adjusted).

*As determined by the EB estimates of the random intercepts, calculated from each of the three sets of variance component estimators above (those with values less than the 10th percentile). ID's in bold indicate subjects not identified by both the untransformed and transformed model (in this case, using the adjusted variance component estimates did not influence the identification of outlying subjects).