Increase in the quality of methodological documentation of cross-national pan-European multi-wave surveys over the last 40 years – a research note

ABSTRACT

Social research methodologists have postulated that the transparency of survey procedures and data processing is mandatory for assessing the Total Survey Error. Recent analyses of data from cross-national surveys have demonstrated an increase in the quality of documentation reports over time and significant differences in documentation quality between the projects. This research note replicates previous results with an extended set of documentation-related quality indicators describing the degree of completeness of information at the consecutive steps of the survey cycle. It also extends earlier findings by indicating no significant relationship between the quality of the survey documentation and the quality of the survey itself. We analysed a meta-data set of survey characteristics, studying all available up-to-date methodological reports of 1,145 national surveys from four large-scale multi-wave projects: the European Quality of Life Survey (2003–2016), European Social Survey (2002–2018), European Values Study (1981–2017), and International Social Survey Programme (1985–2018).

KEYWORDS:

• Survey documentation
• cross-national surveys
• total survey error
• data quality

Disclosure statement

No potential conflict of interest was reported by the author(s).

Research data and replication files

To facilitate research openness, transparency and reproducibility, replication materials and the datasets for analysis are available in this repository https://osf.io/b9adk/

Additional information

Funding

This work was supported by a grant awarded by the National Science Centre, Poland (no. 2018/31/B/HS6/00403). Principal Investigator: Piotr Jabkowski.

Notes on contributors

Piotr Jabkowski

Piotr Jabkowski is an associate professor at the Faculty of Sociology Adam Mickiewicz University in Poznan, and a member of Sampling and Weighting Experts Panel in the European Social Survey for rounds 10 and 11. His research focuses on the methodology of cross-country comparative surveys, the total survey error paradigm and sampling theory.