The Crowd is the Territory: Assessing Quality in Peer-Produced Spatial Data During Disasters

ABSTRACT

Today, disaster events are mobilizing digital volunteers to meet the data needs of those on the ground. One form of this crowd work is Volunteered Geographic Information. This peer-produced spatial data creates the most up-to-date map of the affected region; maintaining the accuracy of these data is therefore a critical task. Accuracy is one aspect of data quality, a relative concept requiring standards to measure against. The field of Geographic Information Sciences has developed standards for this comparison, achieving widespread acceptance. However, the peer production model of spatial data presents new opportunities—and challenges—to traditional methods of quality assessment. Through analysis of the OpenStreetMap database, we show that temporal editing patterns and contributor characteristics can provide additional means of understanding spatial data quality. Drawing upon experiences from Wikipedia, we offer and evaluate three intrinsic quality metrics of peer-produced spatial data to assess the quality of contributions to OpenStreetMap for crisis response.

Acknowledgements

We thank our reviewers as well as our colleagues and collaborators at the University of Colorado Boulder and Mapbox for their time and valuable feedback.

Notes

¹ For more information on the process of disaster mapping, see (Eckle & Albuquerque, 2015).

² http://taginfo.openstreetmap.org.

³ http://github.com/mapbox/tile-reduce.

⁶ May 2015 HOT mailing list archive https://lists.openstreetmap.org/pipermail/hot/2015-May.txt.gz.

⁷ learnosm.org is an open source project maintained by the HOT and OpenStreetMap communities.

⁸ https://wiki.openstreetmap.org/wiki/2014_West_Africa_Ebola_Response.

⁹ http://wiki.openstreetmap.org/wiki/2015_Nepal_earthquake.

Additional information

Funding

This work is made possible with funding from US NSF Grant IIS-1524806 and a research fellowship between the first author and Mapbox.

Notes on contributors

Jennings Anderson

Jennings Anderson is a PhD Student in Computer Science at CU Boulder. His work explores the production of Volunteered Geographic Information, especially in times of disaster. He uses open source technologies to build interactive, web-based visualization and analysis systems from massive geospatial datasets.

Robert Soden

Robert Soden is a PhD Candidate in Computer Science at CU Boulder. His research examines the various ways in which scientific and engineering understandings of climate change and disaster shape and constrain societal responses to these challenges. Prior to starting his PhD, Robert was a consultant to the World Bank.

Brian Keegan

Brian Keegan is an assistant professor in the Department of Information Science. He uses computational methods to analyze and theorize about how large-scale social systems respond under stress.

Leysia Palen

Leysia Palen is Professor of Computer Science, and Professor and Founding Chair of the Department of Information Science at the University of Colorado Boulder. She was awarded the 2015 ACM Computer Human Interaction (CHI) Social Impact Award for contributions the area of crisis informatics.

Kenneth M. Anderson

Ken Anderson is a Professor of Computer Science and the Associate Dean for Education in the College of Engineering and Applied Science at the University of Colorado Boulder. He received his PhD in Computer Science in 1997 at UC Irvine. His research focuses on the design of data-intensive systems.