Can a voluntary web survey be useful beyond explorative research?

ABSTRACT

In this paper, we compare the estimates of earnings determinants based on the non-probabilistic WageIndicator web survey with those based on the widely used, representative EU Study of Income and Living Conditions survey. Using 10 years of Dutch data, we show that there exists an established segment of predominantly junior workers from which the respondents of the WageIndicator survey are disproportionally drawn. In consequence, the composition of WageIndicator sample tends to retain key characteristics over the years, even though it lacks a probabilistic sampling frame. We show that the estimates produced on the basis of an extended Mincerian earnings model using the two data sources are qualitatively similar. In line with much of the literature, however, the two sets of estimates do not pass the formal statistical test of equality. Nonetheless, when we examine only the subsample of junior workers, the statistical testing does not detect a statistically significant difference between the two datasets in many instances. To our knowledge, ours is the first paper showing such statistical evidence for comparability of a web survey based with a widely used representative data source.

KEYWORDS:

• Voluntary survey
• web survey
• data quality
• WageIndicator

Acknowledgement

The authors would like to thank Nikos Askitas, Marcel Das, Jan van Ours, Arthur van Soest, Stepanie Steinmetz, Kea Tijdens, EDUWORKS faculty and fellows, seminar participants at CELSI and the anonymous reviewers for useful feedback that helped us improve the paper. All remaining errors remain our own responsibility.

Notes

1. According to Google Analytics over 95% of the visitors originate from the Netherlands, as opposed to other countries. This is in line with the situation in other countries, suggesting the number of visitors of the individual national website largely reflects the interest in WageIndicator in a given country.

2. For comparison for 15,000 ‘valid’ observations in Netherlands, there were 7500 in Germany and 2500 in Belgium.

3. The general willingness of the public to contribute to socially beneficial activities is often discussed within the framework of the gift exchange reciprocity theory (Falk & Fischbacher, 2006).

4. Equal to the amount of the weekly minimum wage in the Netherlands, adjusted annually, which replaced the original prize of a trip to South Africa after the first three years.

5. See https://www.stata.com/support/faqs/statistics/test-equality-of-coefficients/ for details about the test.

6. Here it is important to mention that our results are heavily sensitive to the use of the income variable. By removing unusually low and higher wages, we can significantly increase the degree of match of estimates generated on the basis of the two samples. Nonetheless, we chose not to eliminate any outlier values to avoid potentially biasing our analysis.