Meeting volunteer expectations — a review of volunteer motivations in citizen science and best practices for their retention through implementation of functional features in CS tools

Figures & data

Table 1. Citizen Science terminology.

Term	Definition	Reference
Citizen Science (CS)	The umbrella term of the voluntary involvement of the wider public in scientific research from data collection to research design	(Bonney 1996; Bonney, Cooper, et al. 2009; Hand 2010; Silvertown 2009)
Citizen Observatory (CO)	CO is an information ecosystem where citizens are empowered in community-based environmental monitoring with the capability to collect, report, access and share environmental information and connect and interact with other citizens, scientists and stakeholders to discuss, intervene and make decisions with the help of ICT enabled technologies.	(Degrossi et al. 2014; Grainger 2017; Lanfranchi et al. 2014; Liu et al. 2014; Miorandi et al. 2013; Mominó, Piera, and Jurado 2016; Montargil and Santos 2017a; Tapia et al. 2014; Zaman and Meuter 2015)
Community Remote Sensing (CRS)	Another umbrella term which “combines remote sensing with citizen science, social networks, and crowd-sourcing to enhance the data obtained from traditional sources. It includes the collection, calibration, analysis, communication, or application of remotely sensed information by these community means.”	(“IGARSS” 2010; Williamson 2009)
Community-Based Monitoring (CBM)/ grass root sensing/ civic or community science	Similar to participatory sensing, but a bottom-up process where concerned citizens, in collaboration with scientists, public authorities and further stakeholders monitor, track and respond to issues that arise from common community concern	(Breen et al. 2015; Burke et al. 2006; Haklay 2015; Whitelaw et al. 2003)
Crowdsensing	A sub-category of crowdsourcing in which individuals with sensing and computing devices collectively share data and extract information to measure and map phenomena of common interest	(Ganti, Ye, and Lei 2011; Miorandi et al. 2013)
Crowdsourcing	Outsourcing work to the crowd	(Howe 2006; Howe 2008)
Participatory sensing (PS) (also called citizen-or mobile (crowd)sensing, crowdtasking or participatory mapping)	Volunteers acting alone or in groups observing their environment with smart mobile devices or sensors forming interactive, participatory sensor networks that enable public and professional users to gather, analyze and share local knowledge	(Burke et al. 2006; Haklay 2015; Jennett et al. 2016)
Passive sensing/ opportunistic sensing/ passive crowdsensing/ Ambient Geographic Information (AGI)/ social geographic data (SGD)/ Involuntary volunteered geographic information (iVGI)	Automatic data collection through a volunteer’s resources without the active engagement of volunteers. E.g. social media harvesting. Passive sensing can also refer to a volunteer agreeing to host a sensor device in his/her backyard.	(Dell'Acqua and Vecchi 2017; Haklay 2015; Lane et al. 2010; Loukis and Charalabidis 2015; Stefanidis, Crooks, and Radzikowski 2013)
Public Participation in Scientific Research (PPSR), crowd science, crowd-sourced science, civic science, or networked science	Synonyms for CS	(Bonney, Ballard, et al. 2009; Hand 2010; Liu and Kobernus 2017)
Public Participatory Geographic Information Systems (PPGIS)	The use of geographic information systems (GIS) in community participation in a plethora of social and environmental contexts.	(Craig, Harris, and Weiner 2002; Newman et al. 2010; Sieber 2006)
Volunteer computing	Volunteers share the unused processing power of their computers and device and allow scientists to run complex computer models when the device is not used.	(Haklay 2015; Jennett et al. 2016)
Volunteer thinking	Making use of the volunteers’ cognitive resources in parallel with leisure activities or explicitly on their free time where volunteers, for example, classify or identify species or land cover types through a web application.	(Haklay 2015; Jennett et al. 2016)
Volunteered Geographic Information (VGI)	Spatially referenced data provided voluntarily by individuals	(Goodchild 2007a; Goodchild 2007b; Gómez-Barrón et al. 2016; See, Mooney et al. 2016)

Table 2. Summary of volunteer motivations.

Function	Conceptual example in environmental CS	Reference in CS literature
Values	The wish to contribute to science or society (i.e. civic responsibility) while helping the environment	(Bruyere and Rappe 2007; Dickinson et al. 2012; Grainger 2017; Grossberndt and Liu 2016; Hunt et al. 2015; Jennett et al. 2016; Montargil and Santos 2017b; Robinson et al. 2018; Wright et al. 2015; See, Fritz et al. 2016)
Personal development	Learning opportunities (about the scientific subject area and data collecting and analyzing techniques, environmental awareness, familiar places)	(Bruyere and Rappe 2007; Devictor, Whittaker, and Beltrame 2010; Dickinson et al. 2012; Grainger 2017; Havlik and Schimak 2014; Hunt et al. 2015; Jennett et al. 2016; Montargil and Santos 2017b; Robinson et al. 2018; See, Fritz et al. 2016)
Career and recognition	To gain relevant experience related to their career interests and to gain recognition and other personal benefits for their input	(Grainger 2017; Havlik and Schimak 2014; Hunt et al. 2015; Jennett et al. 2016; Kotovirta et al. 2015; See, Fritz et al. 2016; Wright et al. 2015)
Social	Social interaction and being part of a community of likeminded individuals	(Bruyere and Rappe 2007; Dickinson et al. 2012; Grainger 2017; Montargil and Santos 2017b; See, Fritz et al. 2016; Wright et al. 2015; Jennett et al. 2016)
Recreation	To have fun and to undertake new activities as part of existing recreational activities while commonly being outdoors	(Grainger 2017; Havlik and Schimak 2014; Jennett et al. 2016; See, Fritz et al. 2016; Tserstou et al. 2017; Wright et al. 2015)

Table 3. Summary of approaches to retain volunteers.

Feature	Explanation/Example of a functional feature	Sources
Fun and cool	The project tasks should be cool and tools fun to use while nurturing curiosity	(Castell et al. 2015; Grainger 2017; Havlik and Schimak 2014; Kaufmann, Schulze, and Veit 2011; See, Fritz et al. 2016; Simpson, Page, and De Roure 2014)
Simple, easy, versatile and user-friendly tools	Tools adapted to different user groups according to their skills	(Druschke and Seltzer 2012; Gharesifard and Wehn 2016; Grainger 2017; Havlik and Schimak 2014; Kobernus et al. 2013; Kotovirta et al. 2015; Mazumdar et al. 2017; Newman et al. 2010; Prakash et al. 2004; See, Fritz et al. 2016; Wallace, Snedigar, and Cameron 2015)
Provide relevance and meaning to volunteers’ everyday lives and interests	Address environmental issues, which affect the direct environment of the volunteers	(Ferster and Coops 2013; Grossberndt and Liu 2016; Guillaume et al. 2016; Hunt et al. 2015; Kotovirta et al. 2015; Mominó, Piera, and Jurado 2016; Montargil and Santos 2017b; Williamson 2009)
Rapid feedback on volunteers’ contributions	E.g. visualization of the measurements on a map	(Druschke and Seltzer 2012; Grainger 2017; Gutiérrez et al. 2016; Kotovirta et al. 2015; Liu and Kobernus 2017; Montargil and Santos 2017a; See, Fritz et al. 2016; Zaman and De Meuter 2016)
Communication and recognition of the importance of volunteers’ contribution and scientific outcomes	Certificates of recognition, communication on how results are being used	(Aspuru, Herranz-Pascual, and Santander 2016; Cohn 2008; Dickinson et al. 2012; Druschke and Seltzer 2012; Grainger 2017; Grossberndt and Liu 2016; Havlik and Schimak 2014; Kotovirta et al. 2015; Mackay et al. 2015; See, Fritz et al. 2016; Sullivan et al. 2009; Wright et al. 2015)
Provide access to volunteers’ contributions and the possibility to analyze and share them	Enable volunteers to make discoveries from their data	(Busch et al. 2016; Charvat et al. 2018; de Assis et al. 2018; Gharesifard, Wehn, and van der Zaag 2017; Grainger 2017; Hunt et al. 2015; Jirka, Remke, and Bröring 2013; Liu and Kobernus 2017; Mackay et al. 2015; Marantos et al. 2017; Mazumdar et al. 2017; Mietlicki, Gaudibert, and Vincent 2012; See, Fritz et al. 2016)
Rewarding and gamification	Reputation ranking, competition between volunteers, prices	(Castell et al. 2015; Dickinson et al. 2012; Gutiérrez et al. 2016; Havlik and Schimak 2014; Kotovirta et al. 2015; Mominó, Piera, and Jurado 2016; See, Fritz et al. 2016)
Create a sense of community	E.g. discussion forums, social networking groups	(Cooper et al. 2017; Dickinson et al. 2012; Druschke and Seltzer 2012; Kotovirta et al. 2015; Mackay et al. 2015; Mazumdar et al. 2017; See, Fritz et al. 2016; Sullivan et al. 2009; Williamson 2009)
Enable feeling of co-ownership and control over the scientific process	Actively involve the volunteers in the design, implementation and dissemination of the project	(Breen et al. 2015; Gharesifard, Wehn, and van der Zaag 2017; Grossberndt and Liu 2016; Jones et al. 2010; Liu and Kobernus 2017; Mackay et al. 2015; Mazumdar et al. 2017; See, Fritz et al. 2016)
Actively remind volunteers to contribute	Pop-up features in smartphone apps and using geofencing functions near desirable locations	(Kotovirta et al. 2015; Mazumdar et al. 2018)
Organize training	Adequate information on the data collection procedure either in forms of training or user guides as well as receiving help on demand	(Aspuru, Herranz-Pascual, and Santander 2016; Busch et al. 2016; Degrossi et al. 2014; Druschke and Seltzer 2012; Grainger 2017; Grossberndt and Liu 2016; Hunt et al. 2015; Jones et al. 2010; Kotovirta et al. 2015; Liu and Kobernus 2017; Mietlicki, Gaudibert, and Vincent 2012; Mominó, Piera, and Jurado 2016; See, Fritz et al. 2016; Wehn et al. 2015)

Table 4. Adapting to user needs after Prakash et al. (2004).

User groups	Example of visualization
The general public and young children	Simple visualization of processed data
Decision and policy makers	Retrieve data for different scenarios in order to make decisions
Students	Possibility to change input parameters and explore the data to fulfill curiosity
Web and tool developers	Access to archived and near real-time data for web interface and tool development purposes
Researchers	Access to raw and original data for research purposes

Bonney, Rick. 1996. “Citizen Science: A Lab Tradition.” Living Bird: For the Study and Conservation of Birds 15 (4): 7–15.

 Google Scholar

Bonney, Rick, Caren B. Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, Kenneth V. Rosenberg, and Jennifer Shirk. 2009. “Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy.” BioScience 59 (11): 977–984. doi:10.1525/bio.2009.59.11.9.

 Web of Science ®Google Scholar

Hand, Eric. 2010. “Citizen Science: People Power.” Nature 466 (7307): 685–687. doi:10.1038/466685a.

 PubMed Web of Science ®Google Scholar

Silvertown, Jonathan. 2009. “A New Dawn for Citizen Science.” Trends in Ecology & Evolution 24 (9): 467–471. doi:10.1016/j.tree.2009.03.017.

 PubMed Web of Science ®Google Scholar

Degrossi, Lívia Castro, João Porto de Albuquerque, Maria Clara Fava, and Eduardo Mario Mendiondo. 2014. “Flood Citizen Observatory: A Crowdsourcing-Based Approach for Flood Risk Management in Brazil.” In Proceedings of the International Conference on Software Engineering and Knowledge Engineering,570–576. Vancouver, Canada: SEKE. https://ksiresearchorg.ipage.com/seke/Proceedings/seke/SEKE2014_Proceedings.pdf.

 Google Scholar

Grainger, Alan. 2017. “Citizen Observatories and the New Earth Observation Science.” Remote Sensing 9 (2): 153. doi:10.3390/rs9020153.

 Web of Science ®Google Scholar

Liu, Hai-Ying, Mike Kobernus, David Broday, and Alena Bartonova. 2014. “A Conceptual Approach to a Citizens' Observatory-Supporting Community-Based Environmental Governance.” Environmental Health: A Global Access Science Source 13 (1): 107 doi:10.1186/1476-069X-13-107.

 PubMedGoogle Scholar

Miorandi, D., I. Carreras, E. Gregori, I. Graham, and J. Stewart. 2013. “Measuring Net Neutrality in Mobile Internet: Towards a Crowdsensing-Based Citizen Observatory.” In 2013 IEEE International Conference on Communications Workshops (ICC), 199–203. doi:10.1109/ICCW.2013.6649228.

 Google Scholar

Mominó, Josep M., Jaume Piera, and Elena Jurado. 2016. “Citizen Observatories as Advanced Learning Environments.” In Analyzing the Role of Citizen Science in Modern Research, edited by Luigi Ceccaroni and Jaume Piera, 192–212. Hersehy, PA: IGI Global. https://www.igi-global.com/chapter/citizen-observatories-as-advanced-learning-environments/170190.

 Google Scholar

Montargil, Filipe, and Vitor Santos. 2017a. “Citizen Observatories: Concept, Opportunities and Communication with Citizens in the First EU Experiences.” In Beyond Bureaucracy: Towards Sustainable Governance Informatisation, edited by Alois A. Paulin, Leonidas G. Anthopoulos, and Christopher G. Reddick, 25:167–184. Public Administration and Information Technology. Cham, Switzerland: Springer. doi:10.1007/978-3-319-54142-6_11.

 Google Scholar

Tapia, Andrea H., Nicolas J. LaLone, Elizabeth MacDonald, Reid Priedhorsky, and Michelle Hall. 2014. “Crowdsourcing Rare Events: Using Curiosity to Draw Participants into Science and Early Warning Systems.” In Proceedings of the 11th International Conference on Information Systems for Crisis Response and Management: ISCRAM 2014. ISCRAM. https://www.semanticscholar.org/paper/Crowdsourcing-rare-events%3A-Using-curiosity-to-draw-Tapia-LaLone/9b906befd780460244e9642b9f45ef6e9a86fc55.

 Google Scholar

Zaman, J., and W. De Meuter. 2015. “DisCoPar: Distributed Components for Participatory Campaigning.” In Proceedings of 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), 160–165. St. Louis, MO: IEEE. doi:10.1109/PERCOMW.2015.7134012.

 Google Scholar

“IGARSS.” 2010. Community Remote Sensing. https://www.igarss2010.org/CommunityRemoteSensing.html.

 Google Scholar

Williamson, Ray A. 2009. “Earth Observations and Human and Environmental Security: Opportunities and Challenges.” In Proceedings of the 60th International Astronautical Congress 2009, 2445–2451. Daejeon, Republic of Korea: International Astronautical Federation. https://iafastro.directory/iac/archive/browse/IAC-09/B1/5/4125/.

 Google Scholar

Breen, Jessica, Shanon Dosemagen, Jeffrey Warren, and Mathew Lippincott. 2015. “Mapping Grassroots: Geodata and the Structure of Community-Led Open Environmental Science.” ACME 14: 849–873. https://s3.amazonaws.com/academia.edu.documents/39033566/1236-3646-1-PB.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1555677669&Signature=LjBQ0lvm10Nwx3VR3GG9hFGBuik%3D&response-content-disposition=inline%3B%20filename%3DMapping_Grassroots_Geodata_and_the_struc.pdf.

 Google Scholar

Burke, J.A., D. Estrin, M. Hansen, A. Parker, N. Ramanathan, S. Reddy, and M. B. Srivastava. 2006. Participatory Sensing. Boulder, Colorado, USA: AMC SensSys ’06.

 Google Scholar

Haklay, Muki. 2015. Citizen Science and Policy: A European Perspective. Washington, DC, USA: Woodrow Wilson International Center for Scholars. https://www.wilsoncenter.org/publication/citizen-science-and-policy-european-perspective.

 Google Scholar

Whitelaw, Graham, Hague Vaughan, Brian Craig, and David Atkinson. 2003. “Establishing the Canadian Community Monitoring Network.” Environmental Monitoring and Assessment 88 (1/3): 409–418. doi:10.1023/A:1025545813057.

 PubMed Web of Science ®Google Scholar

Ganti, R. K., F. Ye, and H. Lei. 2011. “Mobile Crowdsensing: Current State and Future Challenges.” IEEE Communications Magazine 49 (11): 32–39. doi:10.1109/MCOM.2011.6069707.

 Web of Science ®Google Scholar

Howe, Jeff. 2006. “The Rise of Crowdsourcing.” Wired. https://www.wired.com/2006/06/crowds/.

 Google Scholar

Howe, Jeff. 2008. Crowdsourcing: How the Power of the Crowd is Driving the Future of Business. London: Random House Business.

 Google Scholar

Jennett, Charlene, Laure Kloetzer, Daniel Schneider, Ioanna Iacovides, Anna Cox, Margaret Gold, Brian Fuchs., et al. 2016. “Motivations, Learning and Creativity in Online Citizen Science.” Journal of Science Communication 15 (3): 1–23. doi:10.22323/2.15030205.

 Web of Science ®Google Scholar

Dell'Acqua, Fabio, and Daniele De Vecchi. 2017. “Potentials of Active and Passive Geospatial Crowdsourcing in Complementing Sentinel Data and Supporting Copernicus Service Portfolio.” Proceedings of the IEEE 105 (10): 1913–1925. doi:10.1109/JPROC.2017.2727284.

 Web of Science ®Google Scholar

Lane, Nicholas, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, and Andrew Campbell. 2010. “A Survey of Mobile Phone Sensing.” IEEE Communications Magazine 48 (9): 140–150. doi:10.1109/MCOM.2010.5560598.

 Web of Science ®Google Scholar

Loukis, Euripidis, and Yannis Charalabidis. 2015. “Active and Passive Crowdsourcing in Government.” In Policy Practice and Digital Science, edited by Marijn Janssen, Maria A. Wimmer, and Ameneh Deljoo, 261–289. Cham, Switzerland: Springer International Publishing. doi:10.1007/978-3-319-12784-2_12.

 Google Scholar

Stefanidis, Anthony, Andrew Crooks, and Jacek Radzikowski. 2013. “Harvesting Ambient Geospatial Information from Social Media Feeds.” GeoJournal 78 (2): 319–338. doi:10.1007/s10708-011-9438-2.

 Google Scholar

Bonney, Rick, Heidi Ballard, Rebecca Jordan, Ellen McCallie, Tina Phillips, Jennifer Shirk, and Candie C. Widelman. 2009. Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education. A CAISE Inquiry Group Report. Washington, DC, USA: Center for Advancement of Informal Science Education (CAISE). http://www.birds.cornell.edu/citscitoolkit/publications/CAISE-PPSR-report-2009.pdf/view.

 Google Scholar

Liu, Hai-Ying, and Mike Kobernus. 2017. “Citizen Science and Its Role in Sustainable Development: Status, Trends, Issues, and Opportunities.” In Analyzing the Role of Citizen Science in Modern Research, edited by Luigi Ceccaroni and Jaume Piera, 147–167. Hershey, PA: IGI Global. https://www.igi-global.com/chapter/citizen-science-and-its-role-in-sustainable-development/170188.

 Google Scholar

Craig, William J., Trevor M. Harris, and Daniel Weiner. 2002. Community Participation and Geographical Information Systems. London: Taylor & Francis.

 Google Scholar

Newman, Greg, Don Zimmerman, Alycia Crall, Melinda Laituri, Jim Graham, and Linda Stapel. 2010. “User-Friendly Web Mapping: Lessons from a Citizen Science Website.” International Journal of Geographical Information Science 24 (12): 1851–1869. doi:10.1080/13658816.2010.490532.

 Web of Science ®Google Scholar

Sieber, Renee. 2006. “Public Participation Geographic Information Systems: A Literature Review and Framework.” Annals of the Association of American Geographers 96 (3): 491–507. doi:10.1111/j.1467-8306.2006.00702.x.

 Web of Science ®Google Scholar

Goodchild, Michael F. 2007a. “Citizens as Voluntary Sensors: Spatial Data Infrastructure in the World of Web 2.0.” International Journal of Spatial Data Infrastructures Research 2 (2): 24–32.

 Google Scholar

Goodchild, Michael F. 2007b. “Citizens as Sensors: The World of Volunteered Geography.” GeoJournal 69 (4): 211–221. doi:10.1007/s10708-007-9111-y.

 Google Scholar

Gómez-Barrón, José-Pablo, Miguel-Ángel Manso-Callejo, Ramón Alcarria, and Teresa Iturrioz. 2016. “Volunteered Geographic Information System Design: Project and Participation Guidelines.” ISPRS International Journal of Geo-Information 5 (7): 108. doi:10.3390/ijgi5070108.

 Web of Science ®Google Scholar

See, L., Peter Mooney, Giles Foody, Lucy Bastin, Alexis Comber, Jacinto Estima, Steffen Fritz, et al. 2016. “Crowdsourcing, Citizen Science or Volunteered Geographic Information? The Current State of Crowdsourced Geographic Information.” ISPRS International Journal of Geo-Information 5 (5): 55. doi:10.3390/ijgi5050055.

 Web of Science ®Google Scholar

Bruyere, Brett, and Silas Rappe. 2007. “Identifying the Motivations of Environmental Volunteers.” Journal of Environmental Planning and Management 50 (4): 503–516. doi:10.1080/09640560701402034.

 Web of Science ®Google Scholar

Dickinson, Janis L., Jennifer Shirk, David Bonter, Rick Bonney, Rhiannon L. Crain, Jason Martin, Tina Phillips, and Karen Purcell. 2012. “The Current State of Citizen Science as a Tool for Ecological Research and Public Engagement.” Frontiers in Ecology and the Environment 10 (6): 291–297. doi:10.1890/110236.

 Web of Science ®Google Scholar

Grossberndt, Sonja, and Hai-Ying Liu. 2016. “Citizen Participation Approaches in Environmental Health.” In Environmental Determinants of Human Health, edited by Jozef M. Pacyna and Elisabeth G. Pacyna, 225–248. Molecular and Integrative Toxicology. Cham: Springer International Publishing. doi:10.1007/978-3-319-43142-0_11.

 Google Scholar

Hunt, Nuala, Michael O’Grady, Conor Muldoon, Barnard Kroon, Jie Wan, and Gregory O’Hare. 2015. “Citizen Science: A Learning Paradigm for the Smart City?” Interaction Design and Architecture(s) 27: 28–43. http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/27_3.pdf.

 Google Scholar

Montargil, Filipe, and Vítor Santos. 2017b. “Communication with Citizens in the First EU Citizen Observatories Experiences.” In The Proceedings of 17th European Conference on Digital Government - ECDG 2017, 96–105. https://repositorio.ipl.pt/handle/10400.21/7830.

 Google Scholar

Robinson, Johanna, David Kocman, Milena Horvat, and Alena Bartonova. 2018. “End-User Feedback on a Low-Cost Portable Air Quality Sensor System: Are We There yet?” Sensors 18 (11): 3768. doi:10.3390/s18113768.

 PubMed Web of Science ®Google Scholar

Wright, Dale R., Les G. Underhill, Matt Keene, and Andrew T. Knight. 2015. “Understanding the Motivations and Satisfactions of Volunteers to Improve the Effectiveness of Citizen Science Programs.” Society & Natural Resources 28 (9): 1013–1029. doi:10.1080/08941920.2015.1054976.

 Web of Science ®Google Scholar

See, Linda, Steffen Fritz, Eduardo Dias, Elise Hendriks, Bas Mijling, Frans Snik, Piet Stammes, et al. 2016. “Supporting Earth-Observation Calibration and Validation: A New Generation of Tools for Crowdsourcing and Citizen Science.” IEEE Geoscience and Remote Sensing Magazine 4 (3): 38–50. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7570340. doi:10.1109/MGRS.2015.2498840.

 Web of Science ®Google Scholar

Devictor, Vincent, Robert J. Whittaker, and Coralie Beltrame. 2010. “Beyond Scarcity: Citizen Science Programmes as Useful Tools for Conservation Biogeography.” Diversity and Distributions 16 (3): 354–362. doi:10.1111/j.1472-4642.2009.00615.x.

 Web of Science ®Google Scholar

Havlik, Denis, and Gerald Schimak. 2014. “State and Trends in Mobile Observation Applications.” In Proceedings of the 7th International Congress on Environmental Modelling and Software, edited by D.P. Ames, N.W.T. Quinn, and A.E. Rizzoli, 294–303. San Diego, CA, USA: International Environmental Modelling and Software Society (iEMSs). https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?referer=https://www.google.com/&httpsredir=1&article=1042&context=iemssconference.

 Google Scholar

Kotovirta, V., T. Toivanen, R. Tergujeff, T. Häme, and M. Molinier. 2015. “Citizen Science for Earth Observation: Applications in Environmental Monitoring and Disaster Response.” In Proceedings of the 36th International Symposium on Remote Sensing of Environment, ISRSE-36, XL-7-W3:1221–1226. Berlin, Germany: International Society for Photogrammetry and Remote Sensing isprs. doi:10.5194/isprsarchives-XL-7-W3-1221-2015.

 Google Scholar

Tserstou, A., A. Jonoski, I. Popescu, T. H. Asumpcao, G. Athanasiou, A. Kallioras, and I. Nichersu. 2017. “SCENT: Citizen Sourced Data in Support of Environmental Monitoring.” In Proceedings of the 21st International Conference on Control Systems and Computer Science (CSCS), 612–616. Bucharest, Romania: IEEE. doi:10.1109/CSCS.2017.93.

 Google Scholar

Castell, Nuria, Mike Kobernus, Hai-Ying Liu, Philipp Schneider, William Lahoz, Arne J. Berre, and Josef Noll. 2015. “Mobile Technologies and Services for Environmental Monitoring: The Citi-Sense-MOB Approach.” Urban Climate 14 (3): 370–382. doi:10.1016/j.uclim.2014.08.002.

 Google Scholar

Kaufmann, Nicolas, Thimo Schulze, and Daniel Veit. 2011. “More than Fun and Money. Worker Motivation in Crowdsourcing: A Study on Mechanical Turk.” In A Renaissance of Information Technology for Sustainability and Global Competitivenes, 11. Detroit, Michigan, USA: AMCIS. https://aisel.aisnet.org/amcis2011_submissions/

 Google Scholar

Simpson, Robert, Kevin R. Page, and David De Roure. 2014. “Zooniverse: Observing the World’s Largest Citizen Science Platform.” In Proceedings of the 23rd International Conference on World Wide Web, 1049–1054. WWW ’14 Companion. Seoul, South Korea: ACM. doi:10.1145/2567948.2579215.

 Google Scholar

Druschke, Caroline Gottschalk, and Carrie E. Seltzer. 2012. “Failures of Engagement: Lessons Learned from a Citizen Science Pilot Study.” Applied Environmental Education & Communication 11 (3-4): 178–188. Routledge: doi:10.1080/1533015X.2012.777224.

 Google Scholar

Gharesifard, Mohammad, and Uta Wehn. 2016. “To Share or Not to Share: Drivers and Barriers for Sharing Data via Online Amateur Weather Networks.” Journal of Hydrology 535 (April): 181–190. doi:10.1016/j.jhydrol.2016.01.036.

 Google Scholar

Kobernus, Mike, Arne J. Berre, M. Gonzalez, Hai-Ying Liu, Mirjam Fredriksen, Richard Rombouts, and Alena Bartonova. 2013. “A Practical Approach to an Integrated Citizens’ Observatory: The CITI-SENSE Framework.” In Proceedings of the Workshop "Environmental Information Systems and Services - Infrastructures and Platforms 2013, 1322:15. Neusiedl am See, Austria: CEUR-Workshop Proceedings. http://ceur-ws.org/Vol-1322/paper_1.pdf.

 Google Scholar

Mazumdar, Suvodeep, Stuart Wrigley, Fabio Ciravegna, Suvodeep Mazumdar, Stuart Wrigley, and Fabio Ciravegna. 2017. “Citizen Science and Crowdsourcing for Earth Observations: An Analysis of Stakeholder Opinions on the Present and Future.” Remote Sensing 9 (1): 87. doi:10.3390/rs9010087.

 Web of Science ®Google Scholar

Prakash, A., R. Gens, J. Kelley, V. Alexander, L. Johnson, and G. Yanow. 2004. “Space-Based Observations in the International Polar Year: Educational Opportunities to Strengthen the STEM Pipeline.” In Proceedings of the IGARSS 2004. 2004IEEE International Geoscience and Remote Sensing Symposium, 7:1972–1975. Anchorage, AK: IEEE. doi:10.1109/IGARSS.2004.1370733.

 Google Scholar

Wallace, Kristi, Seth Snedigar, and Cheryl Cameron. 2015. “Is Ash Falling?’ An Online Ashfall Reporting Tool in Support of Improved Ashfall Warnings and Investigations of Ashfall Processes.” Journal of Applied Volcanology 4 (1): 8. doi:10.1186/s13617-014-0022-6.

 Google Scholar

Ferster, Colin J., and Nicholas C. Coops. 2013. “A Review of Earth Observation Using Mobile Personal Communication Devices.” Computers & Geosciences 51 (February): 339–349. doi:10.1016/j.cageo.2012.09.009.

 Web of Science ®Google Scholar

Guillaume, Gwenaël, Arnaud Can, Gwendall Petit, Nicolas Fortin, Sylvain Palominos, Benoit Gauvreau, Erwan Bocher, and Judicaël Picaut. 2016. “Noise Mapping Based on Participative Measurements.” Noise Mapping 3 (1): 140–156. doi:10.1515/noise-2016-0011.

 Web of Science ®Google Scholar

Gutiérrez, Verónica, Evangelos Theodoridis, Georgios Mylonas, Fengrui Shi, Usman Adeel, Luis Diez, Dimitrios Amaxilatis, et al. 2016. “Co-Creating the Cities of the Future.” Sensors 16 (11): 1971. doi:10.3390/s16111971.

 Web of Science ®Google Scholar

Zaman, J., and W. De Meuter. 2016. “Crowd Sensing Applications: A Distributed Flow-Based Programming Approach.” In Proceedings of the 2016 IEEE International Conference on Mobile Services (MS), 79–86. San Francisco, CA: IEEE. doi:10.1109/MobServ.2016.22.

 Google Scholar

Aspuru, Itziar, Karmele Herranz-Pascual, and Alvaro Santander. 2016. “Empowering People on the Assessment of the Acoustic Comfort of Urban Places: CITI-SENSE Project.” In Proceedings of the INTER-NOISE 2016 - 45th International Congress and Exposition on Noise Control Engineering: Towards a Quieter Future, 4534–4543. Hamburg, Germany: Institute of Noise Control Engineering. https://www.ingentaconnect.com/contentone/ince/incecp/2016/00000253/00000004/art00075.

 Google Scholar

Cohn, Jeffrey P. 2008. “Citizen Science: Can Volunteers Do Real Research?” BioScience 58 (3): 192–197. doi:10.1641/B580303.

 Web of Science ®Google Scholar

Mackay, E. B., M. E. Wilkinson, C. J. A. Macleod, K. Beven, B. J. Percy, M. G. Macklin, P. F. Quinn, M. Stutter, and P. M. Haygarth. 2015. “Digital Catchment Observatories: A Platform for Engagement and Knowledge Exchange between Catchment Scientists, Policy Makers, and Local Communities.” Water Resources Research 51 (6): 4815–4822. doi:10.1002/2014WR016824.

 Web of Science ®Google Scholar

Sullivan, Brian L., Christopher L. Wood, Marshall J. Iliff, Rick E. Bonney, Daniel Fink, and Steve Kelling. 2009. “EBird: A Citizen-Based Bird Observation Network in the Biological Sciences.” Biological Conservation 142 (10): 2282–2292. doi:10.1016/j.biocon.2009.05.006.

 Web of Science ®Google Scholar

Busch, Julia, Raul Bardaji, Luigi Ceccaroni, Anna Friedrichs, Jaume Piera, Carine Simon, Peter Thijsse., et al. 2016. “Citizen Bio-Optical Observations from Coast- and Ocean and Their Compatibility with Ocean Colour Satellite Measurements.” Remote Sensing 8 (11): 879. doi:10.3390/rs8110879.

 Web of Science ®Google Scholar

Charvat, Karel, Bente Lilja Bye, Tomas Mildorf, Arne J. Berre, and Karel Jedlicka. 2018. “Open Data, VGI and Citizen Observatories INSPIRE Hackathon.” International Journal of Spatial Data Infrastructures Research 13 (0): 109–130. doi:10.2902/ijsdir.v13i0.481.

 Google Scholar

de Assis, Luiz F.G., E.A. Flávio, Edison de Freitas Horita, Jó Ueyama, João de Albuquerque, Luiz Fernando F. G. de Assis, Flávio E. A. Horita, Edison P. de Freitas, Jó Ueyama, and João Porto de Albuquerque. 2018. “A Service-Oriented Middleware for Integrated Management of Crowdsourced and Sensor Data Streams in Disaster Management.” Sensors 18 (6): 1689. doi:10.3390/s18061689.

 Web of Science ®Google Scholar

Gharesifard, Mohammad, Uta Wehn, and Pieter van der Zaag. 2017. “Towards Benchmarking Citizen Observatories: Features and Functioning of Online Amateur Weather Networks.” Journal of Environmental Management 193 (May): 381–393. doi:10.1016/j.jenvman.2017.02.003.

 PubMedGoogle Scholar

Jirka, Dr Simon, Dr Albert Remke, and Arne Bröring. 2013. “EnviroCar: Crowd Sourced Traffic and Environment Data for Sustainable Mobility.” In Proceedings of the Workshop “Environmental Information Systems and Services - Infrastructures and Platforms 2013 - with Citizens Observatories, Linked Open Data and SEIS/SDI Best Practices,” 1322:7. Neusiedl am See, Austria: CEUR-Workshop Proceedings. http://ceur-ws.org/Vol-1322/.

 Google Scholar

Marantos, C., I. S. Paraskevas, K. Siozios, J. Mothe, C. Menou, and D. Soudris. 2017. “FabSpace 2.0: A Platform for Application and Service Development Based on Earth Observation Data.” In 2017 6th International Conference on Modern Circuits and Systems Technologies (MOCAST), 1–4. doi:10.1109/MOCAST.2017.7937657.

 Google Scholar

Mietlicki, Fanny, Piotr Gaudibert, and Bruno Vincent. 2012. “HARMONICA Project (HARMOnised Noise Information for Citizens and Authorities).” In Proceedings of the 41st International Congress and Exposition on Noise Control Engineering 2012 (INTER-NOISE 2012), edited by Courtney Burroughs and Steve Conlon, 6160–6172. New York City, New York, USA: Curran. http://toc.proceedings.com/18441webtoc.pdf.

 Google Scholar

Cooper, Caren, Lincoln Larson, Kathleen Krafte Holland, Rebecca Gibson, David Farnham, Diana Hsueh, Patricia Culligan, and Wade McGillis. 2017. “Contrasting the Views and Actions of Data Collectors and Data Consumers in a Volunteer Water Quality Monitoring Project: Implications for Project Design and Management.” Citizen Science: Theory and Practice 2 (1): 8. doi:10.5334/cstp.82.

 Google Scholar

Jones, K.B., H. Bogena, H. Vereecken, and J. F. Weltzin. 2010 “Design and Importance of Multi-Tiered Ecological Monitoring Networks.” In Long-Term Ecological Research: Between Theory and Application, edited by Felix Müller, Cornelia Baessler, Hendrik Schubert, and Stefan Klotz, 355–374. Dordrecht: Springer. www.springer.com/gp/book/9789048187812.

 Google Scholar

Mazumdar, Suvodeep, Stuart N. Wrigley, Neil Ireson, and Fabio Ciravegna. 2018. “Harnessing Location-Based Services for Effective Citizen Observatories.” International Journal of Spatial Data Infrastructures Research 13 (0): 101–108. doi:10.2902/ijsdir.v13i0.479.

 Google Scholar

Wehn, Uta, Simon McCarthy, Vita Lanfranchi, and Sue Tapsell. 2015. “Citizen Observatories as Facilitators of Change in Water Governance? Experiences from Three European Cases.” Environmental Engineering and Management Journal 14 (9): 2073–2086. http://www.eemj.icpm.tuiasi.ro/issues/vol14/vol14no9.htm. doi:10.30638/eemj.2015.222.

 Web of Science ®Google Scholar