Harmonization of Cross-National Survey Projects on Political Behavior: Developing the Analytic Framework of Survey Data Recycling

Abstract

This article describes challenges and solutions to ex post harmonization of survey data in the social sciences based on the big data project “Democratic Values and Protest Behavior: Data Harmonization, Measurement Comparability, and Multi-Level Modeling.” This project engages with the relationship between democracy and protest behavior in comparative perspective by proposing a theoretical model that explains variation in political protest through individual-level characteristics, country-level determinants, and interactions between the two. Testing it requires data with information at both the individual and country levels that vary across space and over time. The project’s team pooled information from 22 well-known international survey projects into a data set of 2.3 million respondents, covering a total of 142 countries and territories, and spanning almost 50 years, to construct common measures of political behavior, social attitudes, and demographics. The integrated data set is appended with country variables from nonsurvey sources. Mapping the methodological complexities this work raised and their solutions became the springboard for the analytic framework of Survey Data Recycling (SDR). SDR facilitates reprocessing information from extant cross-national projects in ways that minimize the “messiness” of data built into original surveys, expand the range of possible comparisons over time and across countries, and improve confidence in substantive results.

Keywords:

• cross-national
• data harmonization
• survey data recycling

ACKNOWLEDGMENTS

We acknowledge comments on the project from Edward Crenshaw, Tadeusz Krauze, Dean Lillard, Malgorzata Mikucka, Francesco Sarracino, and Sandy Marquart-Pyatt.

Notes

We assume that most of these variables could be operationalized in different ways. We use notations similar to those used in popular HLM texts such as Hox (2010), Gellman and Hill (2007), and Raudenbush and Bryk (2002), and adjust the explanation of coefficients to cross-national research where individuals are nested in countries.

We refer to the selected projects as well-known on the basis of publication records and their effect on the social-science disciplines. Data gathered in March 2015 show that the number of publications from projects listed in Table 1 was 2,087 at the Web of Science; 11,746 at the projects’ home pages; and from 20,051 to 116,377 (depending on how strict limitations of the searches were) at Google Scholar. At the same time, the number of citations of the projects in Table 1 was 19,726 (Web of Science).

For practical reasons, we stopped adding new data in the second quarter of 2014.

Because of the thematic coverage criterion, we include only survey waves that contain relevant questions on protest behavior and/or democratic values; hence, not all waves of ISSP, Eurobarometer, and Comparative National Elections Project are in our data.

Some of the international survey projects require agreeing to the “click license” stipulating “not to pass along the data to third parties.” For these projects we are seeking permission to include technical and source variables in the master file that are necessary for a replication of our work.

Many of the national surveys included in our project are prior to 2011 and employ ISCED 1997. Although the 1997 and 2011 ISCED coding differs, the conversion process is possible with some additional information, generally provided in country-specific classifications.

The assumption is that particular values of shorter scales correspond to a range of values on the 0–10 scale. For an n-point scale with k values ranging from 1 to n, k was recoded to 10/(n × 2) + (k – 1) × 10/n.

For an n point scale, for values k ranging from 1 to n, k was recoded to: $$(k-1)\ast \frac{1}{n-1}$$.

For an n-point scale with k values ranging from 1 to n, where X_k is the distribution of the variable, k was recoded to $${\sum }_{i=1}^{k-1}{X}_i+\frac{X_k}{2}$$. The correlation between the two versions of the trust target variable (11-point scale vs. distribution-based variable) is 0.881 for the pooled 1,721 surveys. Within survey, the correlation ranges from 0.914 to 0.99988; the mean correlation is 0.985.

Additional information

Notes on contributors

Irina Tomescu-Dubrow

Irina Tomescu-Dubrow is an associate professor of sociology at the Institute of Philosophy and Sociology, Polish Academy of Sciences (PAN), and program manager for the Cross-National Studies: Interdisciplinary Research and Training program of Ohio State University and PAN. She studies social inequality, stratification, and structural change, and is involved in cross-national methodology research, including ex post survey data harmonization. Her research has appeared in Quality and Quantity, Journal of Urban Affairs, Polish Sociological Review, the International Journal of Sociology, Problems of Post-Communism, and Ask: Research and Methods.

Kazimierz M. Slomczynski

Kazimierz M. Slomczynski is a professor at the Institute of Philosophy and Sociology at the Polish Academy of Sciences (PAN). He is also director of the Cross-National Studies: Interdisciplinary Research and Training program of Ohio State University and PAN. He is principal investigator of the Polish Panel Survey POLPAN, conducted every five years since 1988, and directs other studies, including a project on ex post harmonization of cross-national surveys. He has published numerous articles in various international scientific journals and authored or coauthored several books in Polish and English. His main interest is in social stratification and methodology of social sciences.