Now, later, or never? Using response-time patterns to predict panel attrition

ABSTRACT

Preventing panel members from attriting is a fundamental challenge for panel surveys. Research has shown that response behavior in earlier waves (response or nonresponse) is a good predictor of panelists’ response behavior in upcoming waves. However, response behavior can be described in greater detail by considering the time until the response is returned. In the present study, we investigated whether respondents who habitually return their survey late and respondents who switch between early and late response in multiple waves are more likely to attrit from a panel. Using data from the GESIS Panel, we found that later response is related to a higher likelihood of attrition (AME = 0.087) and that response-time stability is related to a lower likelihood of attrition (AME = −0.013). Our models predicted most cases of attrition; thus, survey practitioners could potentially predict future attriters by applying these models to their own data.

KEYWORDS:

• Reluctant response
• data quality
• panel survey
• paradata
• late response

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Our choice of 10 waves was not arbitrary. We estimated our model – which is described later in the Method section – multiple times by varying the number of waves (pattern lengths). To determine the optimal pattern length, we calculated the Akaike Information Criterium (AIC) and compared the predicted attrition with the actual attrition of each model. Based on these indicators, we found that a pattern length of 10 is optimal. Shorter response-time patterns do not perform as well in the models as patterns that were built from 10 waves. Longer response-time patterns do not improve the models.

2. We chose the cut-off point of 14 days because the online respondents had received their second reminder to participate at this point. Choosing a cut-off point was necessary to operationalize the response-time habit. In addition, this operationalization of response time is often applied in the current literature on response times (Olson, 2006).

3. The German high-school system includes lower-level secondary school (Hauptschule), medium-level secondary school (Realschule), and upper-level secondary school (Gymnasium). Low, medium, and high levels of education are related to obtaining the highest educational degree from the aforementioned schools, respectively.

4. Response time and response-time habit are negatively correlated (r = – 0.65). In a bivariate model, response time has an average marginal effect of 0.234 and a standard error of 0.01 on attrition. This effect is statistically significant (p < 0.001). Response-time habit has an average marginal effect of – 0.034 and a standard error of 0.00 on attrition. This effect is also statistically significant (p < 0.001).

Additional information

Notes on contributors

Isabella Minderop

Isabella Minderop is research associate at GESIS - Leibniz Institute for the Social Sciences. Her research interests lie in survey methodology, especially paradata and data quality.

Bernd Weiß

Bernd Weiß is head of the GESIS Panel, a probabilistic mixed-mode access panel. He also serves as Deputy Scientific Director of the Department Survey Design and Methodology at GESIS – Leibniz Institute for the Social Sciences in Mannheim. His research interests range from survey methodology, research synthesis, open science to family sociology and juvenile delinquency.