Differences Between RDD Telephone and ABS Mail Survey Design: Coverage, Unit Nonresponse, and Measurement Error

Abstract

Using list-assisted random digit dialing (RDD) with telephone data collection and address-based sampling (ABS) with mail questionnaires are two survey designs that yield probability based inference, yet they are so different that they can yield entirely different results. The 2007 Health Information National Trends Survey (HINTS) provides a unique opportunity to evaluate the effect of these different designs on a variety of survey estimates and, even more importantly, the effect on individual sources of survey error. Understanding the difference in error structure between the two designs is important to survey practitioners in order to select the optimum design, and to data users who can anticipate which results may be affected and how. We first compared estimates between the two designs and then estimated the different sources of error. In addition to identified differences in estimates, we found that for some estimates the two designs can yield similar results merely due to the effect of similar biases. The error components were quite different between the two designs––while the ABS design yields almost complete coverage of the population compared to the RDD design, it was subjected to substantially higher nonresponse bias.

Notes

¹This is not the poststratified weight available in the publicly used data but rather repoststratified after dropping households without a landline telephone.

Note: Bolded lines are where at least one difference is significant at α = .05.