Examining the Missing Completely at Random Mechanism in Incomplete Data Sets: A Multiple Testing Approach

Abstract

A multiple testing procedure for examining implications of the missing completely at random (MCAR) mechanism in incomplete data sets is discussed. The approach uses the false discovery rate concept and is concerned with testing group differences on a set of variables. The method can be used for ascertaining violations of MCAR and disproving this mechanism in empirical behavioral and social research. The procedure can also be employed when locating violations of MCAR in observed measures is of interest. The outlined approach is illustrated with data from a cognitive intervention study.

Keywords:

• false discovery rate
• incomplete data
• missing completely at random
• multiple testing

Notes

¹For simplicity, the remainder of the discussion assumes that none of the y variables has a missing value. If this is not the case, however, and data are MAR on the y variables with missing values, an application of the group identity testing method in Raykov and Marcoulides (2010) can be used to obtain the associated p values in Equation 6 for the pertinent null hypotheses (group identity of the parameters π₁, π₂, …, π_m for the distributions of the y variables); these p values are eventually the quantities needed for an application of the multiple testing approach of this article.

²Under normality, the means and covariance matrix represent sufficient statistics (e.g., Casella & Berger, 2002), and thus their group equality is equivalent to that of the pertinent variable distributions (e.g., of the eight-dimensional vector with pretest measures (a) through (f) in this empirical example). In the data set used in this section, already when testing mean and variance group equality one concludes that the MCAR mechanism is to be rejected, but in other data sets such a conclusion might or might not be reached until variable covariances are examined for such equality (if it is reached). Testing group identity in the latter parameters can be carried out using the approach in Raykov and Marcoulides (2010) under the MAR assumption. When normality is not plausible, testing group equality in means, variances, and covariances can be conducted using the robust maximum likelihood method with the same approach (under MAR; see also Muthén & Muthén, 2010).