Effects of screening questionnaires on response in a two-phase postal survey

Abstract

For surveys targeting specific population groups, the two-phase postal approach (screener followed by a topical survey sent to eligible households) has been demonstrated to be more effective at identifying population domains of interest than random digit dial telephone methods considering cost, coverage, and response. An important question is how best to motivate screener response from eligible households. In 2011, we conducted a large-scale field test to empirically test a number of methods for motivating response. We fielded screening surveys that varied content-influencing relevance, and also switched screener questionnaires for following up nonrespondents to the initial postal survey – an approach we have labeled responsive tailoring. In another experiment, we tested the effect of asking for first names in the screener questionnaire. In this article, we describe the effects of these experimental treatments on response to both the screener and the topical survey.

Keywords:

• nonresponse follow-up
• sensitive items
• tailoring

Acknowledgments

The authors would like to thank NCES and members of the NHES Technical Review Panel, who offered valuable input during the design process influencing the final outcome implemented in this field test. The members of the Technical Review Panel were: Nancy Bates, Paul Beatty, Johnny Blair, Stephen Blumberg, Mick Couper, Don Dillman, Bob Groves, Scott Keeter, Kristen Olson, Roger Tourangeau, Clyde Tucker, and Gordon Willis. The data were collected under a contract with the National Center for Education Statistics.

Notes

1. All sample sizes presented in this paper have been rounded to the nearest 10 in accordance with NCES publication standards.

2. The questionnaires and initial cover letters are included as a supplement available on the journal’s website.

3. Throughout the article, when the rates are noted as different it means that they are statistically significant when tested with a two-sided t-test with a significance level of .05 using the appropriately computed sampling errors.

4. All of the NHES experimental treatments have eligibility rates at least slightly less than the 35% from the American Community Survey.