Diagnosing Global Case Influence on MAR Versus MNAR Model Comparisons

Abstract

When missingness is suspected to be not at random (MNAR) in longitudinal studies, researchers sometimes compare the fit of a target model that assumes missingness at random (here termed a MAR model) and a model that accommodates a hypothesized MNAR missingness mechanism (here termed a MNAR model). It is well known that such comparisons are only interpretable conditional on the validity of the chosen MNAR model’s assumptions about the missingness mechanism. For that reason, researchers often perform a sensitivity analysis comparing the MAR model to not one, but several, plausible alternative MNAR models. In the social sciences, it is not widely known that such model comparisons can be particularly sensitive to case influence, such that conclusions drawn could depend on a single case. This article describes two convenient diagnostics suited for detecting case influence on MAR–MNAR model comparisons. Both diagnostics require much less computational burden than global influence diagnostics that have been used in other disciplines for MNAR sensitivity analyses. We illustrate the interpretation and implementation of these diagnostics with simulated and empirical latent growth modeling examples. It is hoped that this article increases awareness of the potential for case influence on MAR–MNAR model comparisons and how it could be detected in longitudinal social science applications.

Keywords:

• case influence
• diagnostics
• growth models
• missing not at random
• nonignorable missingness

Notes

¹ For instance, Verbeke, Lessafre, et al. (2001) interpreted a fit comparison that yielded a better fit for the MNAR model than the MAR model as follows: “conditional on the validity of [the] model, there is a lot of evidence for nonrandom dropout” (p. 426).

² Whereas source code (e.g., in GAUSS) for local influence diagnostics is available directly from authors for certain model specifications (see Verbeke, Molenberghs, et al., 2001), in general this approach might require substantial programming for other specifications.

³ Other types of MNAR models, such as pattern mixture models, employ different factorizations of the joint distribution of the repeated measures and missingness indicators that do not give as ready access to inferences involving parameters of the marginal distribution of repeated measures.

⁴ Depending on the MNAR model, such constraints could include fixing to 0 the effects of certain predictors on the probability of missingness. These predictors could be outcome scores at the time of dropout or latent growth coefficients from the outcome-generating model (as described later, in Examples 1 and 2). Another possible constraint involves fixing to 0 the covariance between random effects from the outcome-generating and the missingness mechanisms.

⁵ exactly equals and exactly equals when parameters are fixed to their estimates from the full N analyses for Models A and B when calculating and . One degenerate special case in which the rank orders of these indices will diverge is discussed in Sterba and Pek (2012), but it is very unlikely to be seen in practice.

⁶ Previous model comparisons involving this milk yield example used an LRT. We replicated their pattern of results using the diagnostics described in this article. For instance, in the full sample ΔBIC = 1.508, favoring the MNAR model. But Cows 4 and 5 were each flagged as potentially influential on model ranking using Equation 5: for Cow 4 and for Cow 5 . To confirm their influence, first Cow 4 was deleted, which indeed reversed support at the sample level to the MAR model, = –1.17; additionally, deleting Cow 5 increased sample-level support for the MAR model from weak to strong (= –4.63).

⁷ Parameters for the outcome-generating mechanism were: Parameters for the missingness mechanism included: Parameters were chosen to give rise to a pattern of observed and missing data with features related to Neri et al. (2013), Barry et al. (2005), Pan et al. (2005), Yancy et al. (2010), or Grober et al. (2008) in the manner described in the text.

⁸ Technically no missingness model is needed under MAR, and as such the estimates for the outcome-generating process parameters in the MAR model should stay the same regardless of whether this logistic dropout model is included or not. Nonetheless, it is conventional to include the logistic dropout model conditional on observables in the MAR specification when it is to be compared to a more elaborated logistic dropout model, in the MNAR specification, so the likelihoods are on the same metric (see Muthén et al., 2011).

⁹ and index plots will only differ by a constant so we only present one of them here.

¹⁰ Here we are not interested in comparing the MAR model from Example 2 to a Diggle-Kenward MNAR model because these models differ in parameters pertaining to not only MNAR (the effect of y_it on dropout) but also MAR (the effect of y_it−1 on dropout). If this comparison were of interest, note that ΔAIC and ΔBIC support the MNAR model (ΔAIC = 22.29 and ΔBIC = 16.02, where Δdf = 2) and no influential cases are flagged.