On the Common but Problematic Specification of Conflated Random Slopes in Multilevel Models

Abstract

For multilevel models (MLMs) with fixed slopes, it has been widely recognized that a level-1 variable can have distinct between-cluster and within-cluster fixed effects, and that failing to disaggregate these effects yields a conflated, uninterpretable fixed effect. For MLMs with random slopes, however, we clarify that two different types of slope conflation can occur: that of the fixed component (termed fixed conflation) and that of the random component (termed random conflation). The latter is rarely recognized and not well understood. Here we explain that a model commonly used to disaggregate the fixed component—the contextual effect model with random slopes—troublingly still yields a conflated random component. Negative consequences of such random conflation have not been demonstrated. Here we show that they include erroneous interpretation and inferences about the substantively important extent of between-cluster differences in slopes, including either underestimating or overestimating such slope heterogeneity. Furthermore, we show that this random conflation can yield inappropriate standard errors for fixed effects. To aid researchers in practice, we delineate which types of random slope specifications yield an unconflated random component. We demonstrate the advantages of these unconflated models in terms of estimating and testing random slope variance (i.e., improved power, Type I error, and bias) and in terms of standard error estimation for fixed effects (i.e., more accurate standard errors), and make recommendations for which specifications to use for particular research purposes.

Keywords:

• Multilevel modeling
• random slopes
• contextual effect models
• group mean centering

Acknowledgments

The authors would like to thank Kristopher Preacher, Sun-Joo Cho, and Andrew Tomarken for their helpful comments. Portions of this manuscript are based on the lead author’s dissertation. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institutions or the Natural Science and Engineering Research Council of Canada is not intended and should not be inferred.

Conflict of Interest Disclosures

Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles

The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Role of the funders/sponsors

None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Notes

1 Grand-mean centering (or centering by any constant value) has no impact on whether level-specific effects are disaggregated or conflated. Hence we do not distinguish between uncentered vs. grand-mean-centered predictors.

2 Throughout this paper, to facilitate the distinction of fixed and random conflation, we define the overall random slope of x_ij (shorthand: “the x_ij slope”) as the combination of the fixed component—i.e., the $\gamma$ term—and the random component—i.e., the u term (thus, the random slope, as a whole, can be subject to either fixed or random conflation). This nomenclature is consistent with that found in the MLM literature (see, e.g., Snijders & Bosker, 2012, p. 92), but as a point of clarification, in practice, often the term random slope refers specifically to what we term here the random component of the slope.

3 Note that we reserve the use of the word “conflation” for equality constraints between parameters dealing with the within-cluster and the between-cluster portion of x_ij; the substitution done in Eq. (2) is simply a mathematically convenient way of demonstrating such implicit equality constraints. Practically speaking, any model implicitly places innumerable constraints, but only those specifically involving $x_{ij}-x_{·j}$ and $x_{·j}$ are relevant to the current discussion.

4 Prior authors have, however, recognized that, despite the likelihood equivalency of the fixed-slope contextual effect model and fixed-slope cluster-mean centered model (defined later), the random-slope contextual effect model (Equation 3) is not likelihood equivalent to the random-slope cluster-mean-centered model (equal to Equation 3 when replacing $x_{ij}$ with $x_{ij}-x_{·j}$). Methodologists have noted this nonequivalence and have suggested differing implications thereof (see, e.g., Kreft et al., 1995; Enders & Tofighi, 2007; Raudenbush & Bryk, 2002; Snijders & Bosker, 2012). We discuss such explanations later, and explain how the results of the current paper add to the understanding of the differences between these two models.

5 Indeed, it is commonly taught in introductory MLM courses that one can never include random slopes of level-2 predictors in a two-level model—the intuition being that if each cluster supplies exactly one observation of $x_{·j},$ it does not make sense to estimate a “cluster-specific” effect of $x_{·j}.$ However, an inherent aspect to multilevel modeling is that estimation procedures pool across all observations/ clusters when estimating parameters (Gelman & Hill, 2007; Raudenbush & Bryk, 2002), which, in turn, allows estimation of variances and covariances associated with random components of level-2 predictor slopes, as demonstrated later via simulation and via empirical examples.

6 Though here, and in Appendix A, the cluster-specific intercept ${\beta }_{0j}$is defined as conditional on the cluster mean of the level-1 predictor (following Raudenbush, 1989), and is thus equivalent to model-implied cluster mean of the outcome for cluster j, the intercept alternatively could be defined unconditionally as ${\gamma }_{00}+u_{0j}.$ The former definition is useful for explicating issues associated with random conflation; however, these issues arise regardless of which intercept definition is used.

7 In theory the intercept variance could follow some other function (e.g., linear, cubic, etc.) of $x_{·j}$ and/or could vary as a function of some other level-2 variable. Because our goal is to explicate the issues associated with random conflation, here we restrict focus to the intercept variance structure specifically implied by the conventional random-slope contextual effect model.

8 This statement was made in comparing the random-slope cluster-mean-centered model with fixed between effect and the conventional random-slope contextual effect model. Breaking this statement down and focusing on the random portion of the model (as the fixed portion of both models are analytically equivalent), if $x_{ij}$ is more strongly related to $y_{ij}$ than $(x_{ij}-x_{·j})$ is related to $y_{ij},$ this implies that $x_{·j}$ is related to $y_{ij}$ via a random component, since the difference between $x_{ij}$ and $(x_{ij}-x_{·j})$ is exactly $x_{·j}.$ However, by using the conventional random-slope contextual effect model, the random component of the slopes of both $(x_{ij}-x_{·j})$ and $x_{·j}$ are implicitly assumed to be equivalent. Hence, the analytic result of the current paper shows that this recommendation is logically inconsistent—if, on the random side of the model, $x_{ij}$ is more strongly related to $y_{ij}$ than $(x_{ij}-x_{·j}),$ then the underlying random components of $(x_{ij}-x_{·j})$ and $x_{·j}$ cannot be the same (if they were the same, as the random conflated model assumes, then $x_{ij}$ and $(x_{ij}-x_{·j})$ would have the same strength of relation with $y_{ij}$).

9 The set of population effect sizes (in the form of variance explained/ R-squared) for this and all subsequent conditions are provided in Online Appendix A (Rights & Sterba, 2019; Rights & Sterba, 2021).

10 When any model failed to converge, that sample was excluded. Across all simulations and all conditions, at least 96% of samples were retained.

11 Note that the ICC (i.e., proportion of outcome variance that is between-cluster) is not constant across all conditions in the original simulation. This is because the $u_{bj}$ component contributes to between-cluster outcome variance, and thus the ICC is larger when $\mathrm{var}(u_{bj})$ is larger.

12 In order to disaggregate the fixed component of this cross-level interaction, we additionally included a fixed slope of school-mean SES × school-mean parent education in both the unconflated and random-conflated models. This level-2 interaction itself is not central to the research question (i.e., why the slope of parent’s education differs across school), and was non-significant in both models (see Table 4).

13 If, for the conflated model, one were to ignore the non-significance of the slope variability in the reduced model and nonetheless assess the change in the slope variance after adding the cross-level interaction, one would see the slope variance go to nearly 0 (from 1.66 to <0.01). Importantly, however, this decrease is driven not only by the impact of the product term, but also the extent to which there is random conflation, compromising the utility and interpretability of the result.

Additional information

Funding

This work was supported by Grant RGPIN-2020-06132 from the Natural Science and Engineering Research Council of Canada.