Hypothesis Testing with Complex Survey Data: The Use of Classical Quadratic Test Statistics with Particular Reference to Regression Problems

Abstract

Sample surveys often have complex sample designs with multistage cluster sampling, stratification, and differential selection probabilities. This article is concerned with testing the null hypothesis H ₀: θ = θ, where the p-dimensional parameter θ = g( μ ) and μ is a q-dimensional vector of means. The asymptotic framework that consists of a sequence of increasing finite populations is used to define μ as the limit of finite population means. As part of the inference, we use replicated estimates of variances that take into account the complex sample design. The Wald statistic can be used to test H ₀. But inference for θ based on the Wald statistic can have low power. Thus an alternative to using a Wald test is pursued in this article. First, define a classical quadratic test statistic that would be used if one had a simple random sample of the population. Second, treating this quadratic form as a population parameter, use design-based methods to estimate it from the observed survey data. Last, use a replication method to approximate the distribution of this estimated quadratic form to perform the hypothesis test. Specific applications of this general approach have been used previously in contingency table analysis. For small numbers of sampled first-stage clusters and large p, modified versions of the Fay procedure are proposed. Simulations show that these modified procedures maintain nominal levels better than the original Fay and the Rao-Scott procedures for testing a vector of means and a vector of regression coefficients. An application is given for testing whether design-based regression coefficients differ from ordinary least squares regression coefficients.

Key Words:

• Balanced half-sample repeated replication
• Complex survey sample data
• Fay jackknife chi-squared test
• Multiple linear regression
• Rao-Scott tests
• Wald test