What’s my wage again? Comparing survey and administrative data to validate earning measures

ABSTRACT

One question frequently included in surveys asks about respondents’ earnings. As this information serves, for example, as a basis for evaluating policy interventions, it must be of high quality. This study aims to advance knowledge about possible measurement errors in earnings data and the potential of data linkage to improve substantive conclusions. We use the German sample of the Programme for the International Assessment of Adult Competencies (PIAAC), a subsample of which could be linked to administrative data from the German Integrated Employment Biographies (IEB). We define measurement error as the difference between administrative and survey data. Our results show differences in the ordinary least squares estimates when the administrative and survey measures of earnings were used as the respective dependent variable, which suggests that measurement error causes biased results. Learning more about the size and type of measurement error can help to correct existing biases and improve the quality of survey data.

KEYWORDS:

• Measurement error
• earnings questions
• record linkage
• administrative and survey data

Acknowledgments

We are grateful for feedback and suggestions received at the 10th PhD Workshop “Perspectives on Un-(Employment)” (2018) and the 8th Conference of the European Survey Research Association (2019). Also, we thank two anonymous reviewers for their helpful comments.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Measurement error can refer to both random and systematic error. In the present paper, we focus on systematic error – a consistent direction of response deviations – that has the properties to introduce a bias in regression results. For more details, see the ‘Theoretical framework’.

2. This assumption particularly applies to administrative data from Germany. In fact, employers must expect punishment or sanctions should they provide false information. However, while it seems reasonable to consider administrative records as ‘gold standard’ earnings information in Germany, this might not be the case for other jurisdictions. Administrative records might suffer from different sources that can be summarized as recoding error.

3. It is equally plausible that explanatory variables are measured with an error. Depending on the relationship between this error and the true and observed value of a variable, it might also affect the properties of OLS estimates. Further information can be found in Wooldridge (2010), chapter 4.4.2. However, as earnings are widely used as the dependent variable in empirical applications, we do not consider this case here, but focus on measurement error in the dependent variable.

4. It might equally likely be the case that lower educated individuals overreport their earnings, which would lead to a downward bias in the estimation of the education coefficient.

5. White noise is a noise sequence with zero mean and constant variance. Thus, white noise is a random error.

6. Marginal part-time employment in Germany includes: 1. short-term employment with a maximum duration of three months or a maximum of 70 working days per calendar year; 2. employment with a monthly salary of no more than 450 euros; 3. employment in private households as a special type of marginal part-time employment.

7. To create completely non-overlapping periods, we applied episode splitting. For an example, see Antoni et al. (2016).

8. The assessment ceiling has two functions: first, it defines the maximum amount on which social security contributions are assessed; second, if regular earnings exceed this threshold, one may opt out of the statutory social security system.

9. In addition, we performed two further robustness checks: First, we only repeated our calculations for the respondents who directly provided their monthly earnings in PIAAC. And second, in the step of the data preparation, we averaged the earnings of respondents who had several spells in the IEB data over the latter. Both versions of data and sample preparation do not lead to significant deviations from the findings of our main analyses.

10. These studies found that respondent characteristics, as well as interview and interviewer features, affect consent to linkage and that linked administrative data is likely to suffer from sample composition bias due to non-consent.

Additional information

Funding

This research was supported by the Leibniz Association under grant [SAW-2015-GESIS-2].

Notes on contributors

Britta Gauly

Britta Gauly is a researcher in the Survey Design and Methodology Department at GESIS – Leibniz Institute for the Social Sciences, Mannheim. Her research focuses on adult education and on competence development during the life course.

Jessica Daikeler

Jessica Daikeler is a researcher in the Survey Design and Methodology Department at GESIS – Leibniz Institute for the Social Sciences, Mannheim and at the University of Mannheim. Her research interests are evidence-based survey methodology, measurement error, and mode effects.

Tobias Gummer

Tobias Gummer is a senior researcher and team leader in the Monitoring Society and Social Change Department at GESIS – Leibniz Institute for the Social Sciences, Mannheim. His methodological research interests include survey design, data quality, nonresponse, and correction methods for biases.

Beatrice Rammstedt

Prof. Dr. Beatrice Rammstedt is Professor of Psychological Assessment, Survey Design and Methodology at the University of Mannheim, Vice President of GESIS – Leibniz Institute for the Social Sciences, Mannheim, and Scientific Director of the Survey Design and Methodology Department at GESIS. Her research focus ranges from questionnaire design to the methodology of large-scale cross-cultural comparative studies.