Assessing the relative accuracy of gridded population sampling: results from an election survey experiment

ABSTRACT

International survey researchers are increasingly turning to a geospatial sampling approach known as gridded population sampling for finer resolution and a more updated sampling frame than conventional census-based sampling. To date, there have been no direct comparisons of accuracy between survey results derived from the two methods. This experiment fielded two parallel public opinion surveys concurrently before Uruguay’s 2019 first-round election to compare the accuracy of both probabilistic sampling approaches. Holding other factors constant, we found that the gridded population sample survey performed slightly better than the conventionally drawn census sample survey at predicting election results. The quality of the Uruguayan census frame set a high standard for the gridded population sample, and our experimental results demonstrate that it surpassed it. Given gridded population sampling’s additional demonstrated advantages, particularly time savings, these results suggest that it is a promising approach for face-to-face survey sampling in many lower- and middle-income countries.

KEYWORDS:

• Household surveys
• sample design
• gridded population sampling
• GridSample
• elections

Acknowledgments

We thank Dana Thomson, Magaly Rheault, and Stafford Nichols for helpful methodological suggestions during the survey design phase; Santiago Peyrou and Ignacio Zuasnabar for skilled assistance overseeing fieldwork; and Robert Levy, Regina Faranda, Stephen Hornbeck, and two anonymous referees for valuable feedback on earlier drafts. The views expressed in this article are those of the authors and do not necessarily reflect those of the U.S. government. Any errors or omissions are the responsibility of the authors.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Unlike, for example, elections conducted in the United States via the Electoral College, where nationwide opinion polls may not necessarily predict results.

2. Uruguay scored 98 of 100 on the 2019 Global Freedom Index by Freedom House, including a perfect score for political rights (https://freedomhouse.org/country/uruguay/freedom-world/2019). It is also the highest ranked democracy in Latin America and the Caribbean according to the Economist Intelligence Unit’s (EIU) 2020 Democracy Index with a perfect score of 10 on the electoral process and pluralism (https://www.eiu.com/n/campaigns/democracy-index-2020/).

3. See http://www.ine.gub.uy/web/guest/marcos-censales

4. The reduced 2018 version of SEL was used. See https://www.ceismu.org/site/indice-de-nivel-socioeconomico-inse-2018/

5. There was a slight under-allocation in rural areas due to rounding.

6. See https://www.ine.gub.uy/web/guest/censos-2011

7. Our criteria were that there were fewer than 8 points of total Euclidean distance between 2014 vote shares for the top four candidates, and both departments must have selected the same first place candidate in the election.

8. In addition to the quantitative basis for these groupings, we also consulted regional political experts and the local survey firm on the wisdom of these aggregations.

9. It is a common technique in major household surveys (see, e.g. ICF International, 2015) to adjust a proportional to sample size allocation by imposing a minimum sample per stratum to balance out areas with too few respondents with reductions in sample from large urban areas.

10. The survey firm completed 10 interviews for the geospatial survey on the morning of election day, prior to the release of any exit polls. The inclusion of these cases did not have a substantial effect on results.

11. For example, the satellite image showed what appeared to be apartment buildings, but they were part of a university.

12. Weights based on inverse probabilities of selection were raked to the same targets for each survey: a combination of sex and age range, degree of urbanization, education, and collapsed departments (each of the two largest versus all others).

13. Root mean squared error was 5.0 for the conventional survey, compared to 4.4 for the gridded sample.

14. Notably, the gridded sample also outperformed the regular sample on its estimation of vote shares in the prior (2014) Presidential election.

Additional information

Funding

Data collection was funded by the U.S. Department of State.

Notes on contributors

Sarah Staveteig Ford

Sarah Staveteig Ford, Ph.D., is a statistician and demographer at the U.S. Department of State. Her research interests include improving the methodology of international surveys, global demography, mixed methods research, and machine learning. She has published on global health, demography, and survey methodology

Matthew Kirwin

Matthew Kirwin, Ph.D., is a Division Chief at the U.S. Department of State and a Professorial Lecturer at the Elliott School of International Affairs at George Washington University. His research interests include social science research in Africa and Latin America. He has conducted research on migration in Africa, sports and Africa and the drivers of conflict in the Sahel.