Meeting volunteer expectations — a review of volunteer motivations in citizen science and best practices for their retention through implementation of functional features in CS tools

References

• Aspuru, Itziar, Igone García, Karmele Herranz, and Alvaro Santander. 2016. “CITI-SENSE: Methods and Tools for Empowering Citizens to Observe Acoustic Comfort in Outdoor Public Spaces.” Noise Mapping 3 (1): 37–48. doi:10.1515/noise-2016-0003.
   Web of Science ®Google Scholar
• Aspuru, Itziar, Karmele Herranz-Pascual, and Alvaro Santander. 2016. “Empowering People on the Assessment of the Acoustic Comfort of Urban Places: CITI-SENSE Project.” In Proceedings of the INTER-NOISE 2016 - 45th International Congress and Exposition on Noise Control Engineering: Towards a Quieter Future, 4534–4543. Hamburg, Germany: Institute of Noise Control Engineering. https://www.ingentaconnect.com/contentone/ince/incecp/2016/00000253/00000004/art00075.
   Google Scholar
• Barcellos, Christovam, Emmanuel Roux, Pietro Ceccato, Pierre Gosselin, Antonio Miguel Monteiro, Vanderlei Pascoal de Matos, and Diego Ricardo Xavier. 2016. “An Observatory to Gather and Disseminate Information on the Health-Related Effects of Environmental and Climate Change.” Revista Panamericana de Salud Publica = Pan American Journal of Public Health 40 (3): 167–173.
   PubMed Web of Science ®Google Scholar
• Bonney, Rick. 1996. “Citizen Science: A Lab Tradition.” Living Bird: For the Study and Conservation of Birds 15 (4): 7–15.
   Google Scholar
• Bonney, Rick, Heidi Ballard, Rebecca Jordan, Ellen McCallie, Tina Phillips, Jennifer Shirk, and Candie C. Widelman. 2009. Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education. A CAISE Inquiry Group Report. Washington, DC, USA: Center for Advancement of Informal Science Education (CAISE). http://www.birds.cornell.edu/citscitoolkit/publications/CAISE-PPSR-report-2009.pdf/view.
   Google Scholar
• Bonney, Rick, Caren B. Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, Kenneth V. Rosenberg, and Jennifer Shirk. 2009. “Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy.” BioScience 59 (11): 977–984. doi:10.1525/bio.2009.59.11.9.
   Web of Science ®Google Scholar
• Botteldooren, Dick, Timothy Van Renterghem, Damiano Oldoni, Dauwe Samuel, Luc Dekoninck, Pieter Thomas, Weigang Wei, et al. 2013. “The Internet of Sound Observatories.” Proceedings of Meetings on Acoustics 19 (1): 040140–040140. doi:10.1121/1.4799869.
   Google Scholar
• Breen, Jessica, Shanon Dosemagen, Jeffrey Warren, and Mathew Lippincott. 2015. “Mapping Grassroots: Geodata and the Structure of Community-Led Open Environmental Science.” ACME 14: 849–873. https://s3.amazonaws.com/academia.edu.documents/39033566/1236-3646-1-PB.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1555677669&Signature=LjBQ0lvm10Nwx3VR3GG9hFGBuik%3D&response-content-disposition=inline%3B%20filename%3DMapping_Grassroots_Geodata_and_the_struc.pdf.
   Google Scholar
• Bruyere, Brett, and Silas Rappe. 2007. “Identifying the Motivations of Environmental Volunteers.” Journal of Environmental Planning and Management 50 (4): 503–516. doi:10.1080/09640560701402034.
   Web of Science ®Google Scholar
• Burke, J.A., D. Estrin, M. Hansen, A. Parker, N. Ramanathan, S. Reddy, and M. B. Srivastava. 2006. Participatory Sensing. Boulder, Colorado, USA: AMC SensSys ’06.
   Google Scholar
• Busch, Julia, Raul Bardaji, Luigi Ceccaroni, Anna Friedrichs, Jaume Piera, Carine Simon, Peter Thijsse., et al. 2016. “Citizen Bio-Optical Observations from Coast- and Ocean and Their Compatibility with Ocean Colour Satellite Measurements.” Remote Sensing 8 (11): 879. doi:10.3390/rs8110879.
   Web of Science ®Google Scholar
• Castell, Nuria, Mike Kobernus, Hai-Ying Liu, Philipp Schneider, William Lahoz, Arne J. Berre, and Josef Noll. 2015. “Mobile Technologies and Services for Environmental Monitoring: The Citi-Sense-MOB Approach.” Urban Climate 14 (3): 370–382. doi:10.1016/j.uclim.2014.08.002.
   Google Scholar
• Charvat, Karel, Bente Lilja Bye, Tomas Mildorf, Arne J. Berre, and Karel Jedlicka. 2018. “Open Data, VGI and Citizen Observatories INSPIRE Hackathon.” International Journal of Spatial Data Infrastructures Research 13 (0): 109–130. doi:10.2902/ijsdir.v13i0.481.
   Google Scholar
• Clary, E. Gil, and Mark Snyder. 1999. “The Motivations to Volunteer: Theoretical and Practical Considerations.” Current Directions in Psychological Science 8 (5): 156–159. doi:10.1111/1467-8721.00037.
   Web of Science ®Google Scholar
• Cohn, Jeffrey P. 2008. “Citizen Science: Can Volunteers Do Real Research?” BioScience 58 (3): 192–197. doi:10.1641/B580303.
   Web of Science ®Google Scholar
• Cooper, Caren, Lincoln Larson, Kathleen Krafte Holland, Rebecca Gibson, David Farnham, Diana Hsueh, Patricia Culligan, and Wade McGillis. 2017. “Contrasting the Views and Actions of Data Collectors and Data Consumers in a Volunteer Water Quality Monitoring Project: Implications for Project Design and Management.” Citizen Science: Theory and Practice 2 (1): 8. doi:10.5334/cstp.82.
   Google Scholar
• Couvet, Denis, and Anne-Caroline Prevot. 2015. “Citizen-Science Programs: Towards Transformative Biodiversity Governance.” Environmental Development 13 (January): 39–45. doi:10.1016/j.envdev.2014.11.003.
   Google Scholar
• Craig, William J., Trevor M. Harris, and Daniel Weiner. 2002. Community Participation and Geographical Information Systems. London: Taylor & Francis.
   Google Scholar
• Crall, Alycia W., Gregory J. Newman, Thomas J. Stohlgren, Kirstin A. Holfelder, Jim Graham, and Donald M. Waller. 2011. “Assessing Citizen Science Data Quality: An Invasive Species Case Study.” Conservation Letters 4 (6): 433–442. doi:10.1111/j.1755-263X.2011.00196.x.
   Web of Science ®Google Scholar
• de Assis, Luiz F.G., E.A. Flávio, Edison de Freitas Horita, Jó Ueyama, João de Albuquerque, Luiz Fernando F. G. de Assis, Flávio E. A. Horita, Edison P. de Freitas, Jó Ueyama, and João Porto de Albuquerque. 2018. “A Service-Oriented Middleware for Integrated Management of Crowdsourced and Sensor Data Streams in Disaster Management.” Sensors 18 (6): 1689. doi:10.3390/s18061689.
   Web of Science ®Google Scholar
• Degrossi, Lívia Castro, João Porto de Albuquerque, Maria Clara Fava, and Eduardo Mario Mendiondo. 2014. “Flood Citizen Observatory: A Crowdsourcing-Based Approach for Flood Risk Management in Brazil.” In Proceedings of the International Conference on Software Engineering and Knowledge Engineering,570–576. Vancouver, Canada: SEKE. https://ksiresearchorg.ipage.com/seke/Proceedings/seke/SEKE2014_Proceedings.pdf.
   Google Scholar
• Dell'Acqua, Fabio, and Daniele De Vecchi. 2017. “Potentials of Active and Passive Geospatial Crowdsourcing in Complementing Sentinel Data and Supporting Copernicus Service Portfolio.” Proceedings of the IEEE 105 (10): 1913–1925. doi:10.1109/JPROC.2017.2727284.
   Web of Science ®Google Scholar
• Devictor, Vincent, Robert J. Whittaker, and Coralie Beltrame. 2010. “Beyond Scarcity: Citizen Science Programmes as Useful Tools for Conservation Biogeography.” Diversity and Distributions 16 (3): 354–362. doi:10.1111/j.1472-4642.2009.00615.x.
   Web of Science ®Google Scholar
• Dickinson, Janis L., Jennifer Shirk, David Bonter, Rick Bonney, Rhiannon L. Crain, Jason Martin, Tina Phillips, and Karen Purcell. 2012. “The Current State of Citizen Science as a Tool for Ecological Research and Public Engagement.” Frontiers in Ecology and the Environment 10 (6): 291–297. doi:10.1890/110236.
   Web of Science ®Google Scholar
• Druschke, Caroline Gottschalk, and Carrie E. Seltzer. 2012. “Failures of Engagement: Lessons Learned from a Citizen Science Pilot Study.” Applied Environmental Education & Communication 11 (3-4): 178–188. Routledge: doi:10.1080/1533015X.2012.777224.
   Google Scholar
• Ferster, Colin J., and Nicholas C. Coops. 2013. “A Review of Earth Observation Using Mobile Personal Communication Devices.” Computers & Geosciences 51 (February): 339–349. doi:10.1016/j.cageo.2012.09.009.
   Web of Science ®Google Scholar
• Fraisl, Dilek, Jillian Campbell, Linda See, Uta Wehn, Jessica Wardlaw, Margaret Gold, Inian Moorthy., et al. 2020. “Mapping Citizen Science Contributions to the UN Sustainable Development Goals.” Sustainability Science 15 (6): 1735–1751. July. doi:10.1007/s11625-020-00833-7.
   Web of Science ®Google Scholar
• Freitag, Amy, and Max J. Pfeffer. 2013. “Process, Not Product: Investigating Recommendations for Improving Citizen Science ‘Success’.” PLoS ONE 8 (5): e64079 doi:10.1371/journal.pone.0064079.
   PubMed Web of Science ®Google Scholar
• Fritz, Steffen, Linda See, Tyler Carlson, Mordechai (Muki) Haklay, Jessie L. Oliver, Dilek Fraisl, Rosy Mondardini., et al. 2019. “Citizen Science and the United Nations Sustainable Development Goals.” Nature Sustainability 2 (10): 922–930. doi:10.1038/s41893-019-0390-3.
   Web of Science ®Google Scholar
• Ganti, R. K., F. Ye, and H. Lei. 2011. “Mobile Crowdsensing: Current State and Future Challenges.” IEEE Communications Magazine 49 (11): 32–39. doi:10.1109/MCOM.2011.6069707.
   Web of Science ®Google Scholar
• Gharesifard, Mohammad, and Uta Wehn. 2016. “To Share or Not to Share: Drivers and Barriers for Sharing Data via Online Amateur Weather Networks.” Journal of Hydrology 535 (April): 181–190. doi:10.1016/j.jhydrol.2016.01.036.
   Google Scholar
• Gharesifard, Mohammad, Uta Wehn, and Pieter van der Zaag. 2017. “Towards Benchmarking Citizen Observatories: Features and Functioning of Online Amateur Weather Networks.” Journal of Environmental Management 193 (May): 381–393. doi:10.1016/j.jenvman.2017.02.003.
   PubMedGoogle Scholar
• Golumbic, Yaela N., Barak Fishbain, and Ayelet Baram-Tsabari. 2019. “User Centered Design of a Citizen Science Air-Quality Monitoring Project.” International Journal of Science Education, Part B 9 (3): 195–119. doi:10.1080/21548455.2019.1597314.
   Web of Science ®Google Scholar
• Gómez-Barrón, José-Pablo, Miguel-Ángel Manso-Callejo, Ramón Alcarria, and Teresa Iturrioz. 2016. “Volunteered Geographic Information System Design: Project and Participation Guidelines.” ISPRS International Journal of Geo-Information 5 (7): 108. doi:10.3390/ijgi5070108.
   Web of Science ®Google Scholar
• Goodchild, Michael F. 2007a. “Citizens as Voluntary Sensors: Spatial Data Infrastructure in the World of Web 2.0.” International Journal of Spatial Data Infrastructures Research 2 (2): 24–32.
   Google Scholar
• Goodchild, Michael F. 2007b. “Citizens as Sensors: The World of Volunteered Geography.” GeoJournal 69 (4): 211–221. doi:10.1007/s10708-007-9111-y.
   Google Scholar
• Grainger, Alan. 2017. “Citizen Observatories and the New Earth Observation Science.” Remote Sensing 9 (2): 153. doi:10.3390/rs9020153.
   Web of Science ®Google Scholar
• Grossberndt, Sonja, and Hai-Ying Liu. 2016. “Citizen Participation Approaches in Environmental Health.” In Environmental Determinants of Human Health, edited by Jozef M. Pacyna and Elisabeth G. Pacyna, 225–248. Molecular and Integrative Toxicology. Cham: Springer International Publishing. doi:10.1007/978-3-319-43142-0_11.
   Google Scholar
• Guillaume, Gwenaël, Arnaud Can, Gwendall Petit, Nicolas Fortin, Sylvain Palominos, Benoit Gauvreau, Erwan Bocher, and Judicaël Picaut. 2016. “Noise Mapping Based on Participative Measurements.” Noise Mapping 3 (1): 140–156. doi:10.1515/noise-2016-0011.
   Web of Science ®Google Scholar
• Gutiérrez, Verónica, Evangelos Theodoridis, Georgios Mylonas, Fengrui Shi, Usman Adeel, Luis Diez, Dimitrios Amaxilatis, et al. 2016. “Co-Creating the Cities of the Future.” Sensors 16 (11): 1971. doi:10.3390/s16111971.
   Web of Science ®Google Scholar
• Haklay, Muki. 2015. Citizen Science and Policy: A European Perspective. Washington, DC, USA: Woodrow Wilson International Center for Scholars. https://www.wilsoncenter.org/publication/citizen-science-and-policy-european-perspective.
   Google Scholar
• Haklay, Muki. 2016. “Why is Participation Inequality Important?” In European Handbook of Crowdsourced Geographic Information, edited by Cristina Capineri, Muki Haklay, Haosheng Huang, Vyron Antoniou, Juhani Kettunen, Frank Ostermann, and Ross Purves, 35–44. London, UK: Ubiquity Press. https://www.ubiquitypress.com/site/chapters/10.5334/bax.c/.
   Google Scholar
• Hand, Eric. 2010. “Citizen Science: People Power.” Nature 466 (7307): 685–687. doi:10.1038/466685a.
   PubMed Web of Science ®Google Scholar
• Havlik, Denis, and Gerald Schimak. 2014. “State and Trends in Mobile Observation Applications.” In Proceedings of the 7th International Congress on Environmental Modelling and Software, edited by D.P. Ames, N.W.T. Quinn, and A.E. Rizzoli, 294–303. San Diego, CA, USA: International Environmental Modelling and Software Society (iEMSs). https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?referer=https://www.google.com/&httpsredir=1&article=1042&context=iemssconference.
   Google Scholar
• Houghton, Robert, James Sprinks, Jessica Wardlaw, Steven Bamford, and Stuart Marsh. 2019. “A Sociotechnical System Approach to Virtual Citizen Science: An Application of BS ISO 27500:2016.” Journal of Science Communication 18 (01): A01. doi:10.22323/2.18010201.
   Google Scholar
• Howe, Jeff. 2006. “The Rise of Crowdsourcing.” Wired. https://www.wired.com/2006/06/crowds/.
   Google Scholar
• Howe, Jeff. 2008. Crowdsourcing: How the Power of the Crowd is Driving the Future of Business. London: Random House Business.
   Google Scholar
• Hunt, Nuala, Michael O’Grady, Conor Muldoon, Barnard Kroon, Jie Wan, and Gregory O’Hare. 2015. “Citizen Science: A Learning Paradigm for the Smart City?” Interaction Design and Architecture(s) 27: 28–43. http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/27_3.pdf.
   Google Scholar
• “IGARSS.” 2010. Community Remote Sensing. https://www.igarss2010.org/CommunityRemoteSensing.html.
   Google Scholar
• Jennett, Charlene, and Anna L. Cox. 2014. “Eight Guidelines for Designing Virtual Citizen Science Projects.” In Second AAAI Conference on Human Computation and Crowdsourcing, 16–17. Pittsburgh, PA: AAAI. https://www.aaai.org/ocs/index.php/HCOMP/HCOMP14/paper/view/9261.
   Google Scholar
• Jennett, Charlene, Laure Kloetzer, Daniel Schneider, Ioanna Iacovides, Anna Cox, Margaret Gold, Brian Fuchs., et al. 2016. “Motivations, Learning and Creativity in Online Citizen Science.” Journal of Science Communication 15 (3): 1–23. doi:10.22323/2.15030205.
   Web of Science ®Google Scholar
• Jirka, Dr Simon, Dr Albert Remke, and Arne Bröring. 2013. “EnviroCar: Crowd Sourced Traffic and Environment Data for Sustainable Mobility.” In Proceedings of the Workshop “Environmental Information Systems and Services - Infrastructures and Platforms 2013 - with Citizens Observatories, Linked Open Data and SEIS/SDI Best Practices,” 1322:7. Neusiedl am See, Austria: CEUR-Workshop Proceedings. http://ceur-ws.org/Vol-1322/.
   Google Scholar
• Jones, K.B., H. Bogena, H. Vereecken, and J. F. Weltzin. 2010 “Design and Importance of Multi-Tiered Ecological Monitoring Networks.” In Long-Term Ecological Research: Between Theory and Application, edited by Felix Müller, Cornelia Baessler, Hendrik Schubert, and Stefan Klotz, 355–374. Dordrecht: Springer. www.springer.com/gp/book/9789048187812.
   Google Scholar
• Kaufmann, Nicolas, Thimo Schulze, and Daniel Veit. 2011. “More than Fun and Money. Worker Motivation in Crowdsourcing: A Study on Mechanical Turk.” In A Renaissance of Information Technology for Sustainability and Global Competitivenes, 11. Detroit, Michigan, USA: AMCIS. https://aisel.aisnet.org/amcis2011_submissions/
   Google Scholar
• Kobernus, Mike, Arne J. Berre, M. Gonzalez, Hai-Ying Liu, Mirjam Fredriksen, Richard Rombouts, and Alena Bartonova. 2013. “A Practical Approach to an Integrated Citizens’ Observatory: The CITI-SENSE Framework.” In Proceedings of the Workshop "Environmental Information Systems and Services - Infrastructures and Platforms 2013, 1322:15. Neusiedl am See, Austria: CEUR-Workshop Proceedings. http://ceur-ws.org/Vol-1322/paper_1.pdf.
   Google Scholar
• Kotovirta, V., T. Toivanen, R. Tergujeff, T. Häme, and M. Molinier. 2015. “Citizen Science for Earth Observation: Applications in Environmental Monitoring and Disaster Response.” In Proceedings of the 36th International Symposium on Remote Sensing of Environment, ISRSE-36, XL-7-W3:1221–1226. Berlin, Germany: International Society for Photogrammetry and Remote Sensing isprs. doi:10.5194/isprsarchives-XL-7-W3-1221-2015.
   Google Scholar
• Lai, P. C. 2017. “The Literature Review of Technology Adoption Models and Theories for the Novelty Technology.” Journal of Information Systems and Technology Management 14 (1): 21–38. doi:10.4301/S1807-17752017000100002.
   Google Scholar
• Lane, Nicholas, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, and Andrew Campbell. 2010. “A Survey of Mobile Phone Sensing.” IEEE Communications Magazine 48 (9): 140–150. doi:10.1109/MCOM.2010.5560598.
   Web of Science ®Google Scholar
• Lanfranchi, Vitaveska, Neil Ireson, Uta Wehn, Stuart N. Wrigley, and Fabio Ciravegna. 2014. “Citizens’ Observatories for Situation Awareness in Flooding.” In Proceedings of the 11th International Conference on Information Systems for Crisis Response and Management, edited by S.R. Hiltz, M.S. Pfaff, L. Plotnick, and P.C. Shih, 145–154. Pennsylvania: The Pennsylvania State University. http://www.iscram.org/legacy/ISCRAM2014/ISCRAM2014_proceedings.pdf.
   Google Scholar
• Lawton, Brandon, Bonnie Eisenhamer, Barbara J. Mattson, and Jordan M. Raddick. 2011. “Bringing the Virtual Astronomical Observatory to the Education Community.” In Proceedings of a National Conference on Science Education and Public Outreach: Connecting People to Science, edited by Joseph B. Jensen, James G. Manning, Michael G. Gibbs, and Doris Daou, 457:283–287. Baltimore, MA: Astronomical Society of the Pacific. http://www.aspbooks.org/a/volumes/article_details/?paper_id=34202.
   Google Scholar
• Liu, Hai-Ying, and Mike Kobernus. 2017. “Citizen Science and Its Role in Sustainable Development: Status, Trends, Issues, and Opportunities.” In Analyzing the Role of Citizen Science in Modern Research, edited by Luigi Ceccaroni and Jaume Piera, 147–167. Hershey, PA: IGI Global. https://www.igi-global.com/chapter/citizen-science-and-its-role-in-sustainable-development/170188.
   Google Scholar
• Liu, Hai-Ying, Mike Kobernus, David Broday, and Alena Bartonova. 2014. “A Conceptual Approach to a Citizens' Observatory-Supporting Community-Based Environmental Governance.” Environmental Health: A Global Access Science Source 13 (1): 107 doi:10.1186/1476-069X-13-107.
   PubMedGoogle Scholar
• Loukis, Euripidis, and Yannis Charalabidis. 2015. “Active and Passive Crowdsourcing in Government.” In Policy Practice and Digital Science, edited by Marijn Janssen, Maria A. Wimmer, and Ameneh Deljoo, 261–289. Cham, Switzerland: Springer International Publishing. doi:10.1007/978-3-319-12784-2_12.
   Google Scholar
• Mackay, E. B., M. E. Wilkinson, C. J. A. Macleod, K. Beven, B. J. Percy, M. G. Macklin, P. F. Quinn, M. Stutter, and P. M. Haygarth. 2015. “Digital Catchment Observatories: A Platform for Engagement and Knowledge Exchange between Catchment Scientists, Policy Makers, and Local Communities.” Water Resources Research 51 (6): 4815–4822. doi:10.1002/2014WR016824.
   Web of Science ®Google Scholar
• Mahmood, Mo A., Janice M. Burn, Leopoldo A. Gemoets, and Carmen Jacquez. 2000. “Variables Affecting Information Technology End-User Satisfaction: A Meta-Analysis of the Empirical Literature.” International Journal of Human-Computer Studies 52 (4): 751–771. doi:10.1006/ijhc.1999.0353.
   Web of Science ®Google Scholar
• Marantos, C., I. S. Paraskevas, K. Siozios, J. Mothe, C. Menou, and D. Soudris. 2017. “FabSpace 2.0: A Platform for Application and Service Development Based on Earth Observation Data.” In 2017 6th International Conference on Modern Circuits and Systems Technologies (MOCAST), 1–4. doi:10.1109/MOCAST.2017.7937657.
   Google Scholar
• Mazumdar, Suvodeep, Stuart Wrigley, Fabio Ciravegna, Suvodeep Mazumdar, Stuart Wrigley, and Fabio Ciravegna. 2017. “Citizen Science and Crowdsourcing for Earth Observations: An Analysis of Stakeholder Opinions on the Present and Future.” Remote Sensing 9 (1): 87. doi:10.3390/rs9010087.
   Web of Science ®Google Scholar
• Mazumdar, Suvodeep, Stuart N. Wrigley, Neil Ireson, and Fabio Ciravegna. 2018. “Harnessing Location-Based Services for Effective Citizen Observatories.” International Journal of Spatial Data Infrastructures Research 13 (0): 101–108. doi:10.2902/ijsdir.v13i0.479.
   Google Scholar
• Mietlicki, Fanny, Piotr Gaudibert, and Bruno Vincent. 2012. “HARMONICA Project (HARMOnised Noise Information for Citizens and Authorities).” In Proceedings of the 41st International Congress and Exposition on Noise Control Engineering 2012 (INTER-NOISE 2012), edited by Courtney Burroughs and Steve Conlon, 6160–6172. New York City, New York, USA: Curran. http://toc.proceedings.com/18441webtoc.pdf.
   Google Scholar
• Miorandi, D., I. Carreras, E. Gregori, I. Graham, and J. Stewart. 2013. “Measuring Net Neutrality in Mobile Internet: Towards a Crowdsensing-Based Citizen Observatory.” In 2013 IEEE International Conference on Communications Workshops (ICC), 199–203. doi:10.1109/ICCW.2013.6649228.
   Google Scholar
• Mominó, Josep M., Jaume Piera, and Elena Jurado. 2016. “Citizen Observatories as Advanced Learning Environments.” In Analyzing the Role of Citizen Science in Modern Research, edited by Luigi Ceccaroni and Jaume Piera, 192–212. Hersehy, PA: IGI Global. https://www.igi-global.com/chapter/citizen-observatories-as-advanced-learning-environments/170190.
   Google Scholar
• Montargil, Filipe, and Vitor Santos. 2017a. “Citizen Observatories: Concept, Opportunities and Communication with Citizens in the First EU Experiences.” In Beyond Bureaucracy: Towards Sustainable Governance Informatisation, edited by Alois A. Paulin, Leonidas G. Anthopoulos, and Christopher G. Reddick, 25:167–184. Public Administration and Information Technology. Cham, Switzerland: Springer. doi:10.1007/978-3-319-54142-6_11.
   Google Scholar
• Montargil, Filipe, and Vítor Santos. 2017b. “Communication with Citizens in the First EU Citizen Observatories Experiences.” In The Proceedings of 17th European Conference on Digital Government - ECDG 2017, 96–105. https://repositorio.ipl.pt/handle/10400.21/7830.
   Google Scholar
• Newman, Greg, Don Zimmerman, Alycia Crall, Melinda Laituri, Jim Graham, and Linda Stapel. 2010. “User-Friendly Web Mapping: Lessons from a Citizen Science Website.” International Journal of Geographical Information Science 24 (12): 1851–1869. doi:10.1080/13658816.2010.490532.
   Web of Science ®Google Scholar
• Prakash, A., R. Gens, J. Kelley, V. Alexander, L. Johnson, and G. Yanow. 2004. “Space-Based Observations in the International Polar Year: Educational Opportunities to Strengthen the STEM Pipeline.” In Proceedings of the IGARSS 2004. 2004IEEE International Geoscience and Remote Sensing Symposium, 7:1972–1975. Anchorage, AK: IEEE. doi:10.1109/IGARSS.2004.1370733.
   Google Scholar
• Preece, Jennifer. 2016. “Citizen Science: New Research Challenges for Human–Computer Interaction.” International Journal of Human-Computer Interaction 32 (8): 585–612. doi:10.1080/10447318.2016.1194153.
   Web of Science ®Google Scholar
• Robinson, Johanna. 2019. “Deliverable D3.5 Citizen Observatories (COs) Implementation.” SMURBS. https://drive.google.com/file/d/1kaCt4t10V6HXAD_U093q7PxPxRv_yXzi/view.
   Google Scholar
• Robinson, Johanna, David Kocman, Milena Horvat, and Alena Bartonova. 2018. “End-User Feedback on a Low-Cost Portable Air Quality Sensor System: Are We There yet?” Sensors 18 (11): 3768. doi:10.3390/s18113768.
   PubMed Web of Science ®Google Scholar
• Sanz, Serrano, Teresa, Holocher-Ertl, Barbara Kieslinger, Francisco Sanz García, and Cândida Silva. 2014. White Paper on Citizen Science for Europe. Socientize Consortium. https://ec.europa.eu/futurium/en/content/white-paper-citizen-science.
   Google Scholar
• Sauermann, Henry, and Chiara Franzoni. 2015. “Crowd Science User Contribution Patterns and Their Implications.” Proceedings of the National Academy of Sciences of the United States of America 112 (3): 679–684. doi:10.1073/pnas.1408907112.
   PubMed Web of Science ®Google Scholar
• See, Linda, Steffen Fritz, Eduardo Dias, Elise Hendriks, Bas Mijling, Frans Snik, Piet Stammes, et al. 2016. “Supporting Earth-Observation Calibration and Validation: A New Generation of Tools for Crowdsourcing and Citizen Science.” IEEE Geoscience and Remote Sensing Magazine 4 (3): 38–50. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7570340. doi:10.1109/MGRS.2015.2498840.
   Web of Science ®Google Scholar
• See, L., Peter Mooney, Giles Foody, Lucy Bastin, Alexis Comber, Jacinto Estima, Steffen Fritz, et al. 2016. “Crowdsourcing, Citizen Science or Volunteered Geographic Information? The Current State of Crowdsourced Geographic Information.” ISPRS International Journal of Geo-Information 5 (5): 55. doi:10.3390/ijgi5050055.
   Web of Science ®Google Scholar
• Shirk, Jennifer, Heidi Ballard, Candie Wilderman, Tina Phillips, Andrea Wiggins, Rebecca Jordan, Ellen McCallie, et al. 2012. “Public Participation in Scientific Research: A Framework for Deliberate Design.” Ecology and Society 17 (2): 29–48 doi:10.5751/ES-04705-170229.
   Web of Science ®Google Scholar
• Sieber, Renee. 2006. “Public Participation Geographic Information Systems: A Literature Review and Framework.” Annals of the Association of American Geographers 96 (3): 491–507. doi:10.1111/j.1467-8306.2006.00702.x.
   Web of Science ®Google Scholar
• Silvertown, Jonathan. 2009. “A New Dawn for Citizen Science.” Trends in Ecology & Evolution 24 (9): 467–471. doi:10.1016/j.tree.2009.03.017.
   PubMed Web of Science ®Google Scholar
• Simperl, Elena, Neal Reeves, Chris Phethean, Todd Lynes, and Ramine Tinati. 2018. “Is Virtual Citizen Science a Game?” ACM Transactions on Social Computing 1 (2): 1–39. doi:10.1145/3209960.
   Google Scholar
• Simpson, Robert, Kevin R. Page, and David De Roure. 2014. “Zooniverse: Observing the World’s Largest Citizen Science Platform.” In Proceedings of the 23rd International Conference on World Wide Web, 1049–1054. WWW ’14 Companion. Seoul, South Korea: ACM. doi:10.1145/2567948.2579215.
   Google Scholar
• Skarlatidou, Artemis, Alexandra Hamilton, Michalis Vitos, and Muki Haklay. 2019. “What Do Volunteers Want from Citizen Science Technologies? A Systematic Literature Review and Best Practice Guidelines.” Journal of Science Communication 18 (01): A02. doi:10.22323/2.18010202.
   Google Scholar
• Soleri, Daniela, Jonathan Long, Mónica Ramirez-Andreotta, Rose Eitemiller, and Rajul Pandya. 2016. “Finding Pathways to More Equitable and Meaningful Public-Scientist Partnerships.” Citizen Science: Theory and Practice 1 (1): 9. doi:10.5334/cstp.46.
   Google Scholar
• Stefanidis, Anthony, Andrew Crooks, and Jacek Radzikowski. 2013. “Harvesting Ambient Geospatial Information from Social Media Feeds.” GeoJournal 78 (2): 319–338. doi:10.1007/s10708-011-9438-2.
   Google Scholar
• Sullivan, Brian L., Christopher L. Wood, Marshall J. Iliff, Rick E. Bonney, Daniel Fink, and Steve Kelling. 2009. “EBird: A Citizen-Based Bird Observation Network in the Biological Sciences.” Biological Conservation 142 (10): 2282–2292. doi:10.1016/j.biocon.2009.05.006.
   Web of Science ®Google Scholar
• Tapia, Andrea H., Nicolas J. LaLone, Elizabeth MacDonald, Reid Priedhorsky, and Michelle Hall. 2014. “Crowdsourcing Rare Events: Using Curiosity to Draw Participants into Science and Early Warning Systems.” In Proceedings of the 11th International Conference on Information Systems for Crisis Response and Management: ISCRAM 2014. ISCRAM. https://www.semanticscholar.org/paper/Crowdsourcing-rare-events%3A-Using-curiosity-to-draw-Tapia-LaLone/9b906befd780460244e9642b9f45ef6e9a86fc55.
   Google Scholar
• Thompson, Jonathan E. 2016. “Crowd-Sourced Air Quality Studies: A Review of the Literature and Portable Sensors.” Trends in Environmental Analytical Chemistry 11 (July): 23–34. doi:10.1016/j.teac.2016.06.001.
   Google Scholar
• Tserstou, A., A. Jonoski, I. Popescu, T. H. Asumpcao, G. Athanasiou, A. Kallioras, and I. Nichersu. 2017. “SCENT: Citizen Sourced Data in Support of Environmental Monitoring.” In Proceedings of the 21st International Conference on Control Systems and Computer Science (CSCS), 612–616. Bucharest, Romania: IEEE. doi:10.1109/CSCS.2017.93.
   Google Scholar
• Uhrner, Ulrich, Gioavanna Grosso, Anne-Claude Romain, Virginie Hutsemekers, Julien Delva, Arnaud De Groof, Philippe Valoggia, L. Johannsen, Bernard Stevenot, and Philippe Ledent. 2014. “Development of an Environmental Information System for Odour Using Citizen and Technology Innovative Sensors and Advanced Modelling.” In Proceedings of the Workshop “Environmental Information Systems and Services - Infrastructures and Platforms 2013 - with Citizens Observatories, Linked Open Data and SEIS/SDI Best Practices, 1322:10. Neusiedl am See, Austria: CEUR-Workshop Proceedings. http://ceur-ws.org/Vol-1322/paper_4.pdf.
   Google Scholar
• Vincent, Bruno, Arnaud Cristini, Julie Vallet, Céline Sales, Hélène Poimboeuf, Claire Sorrentini, and Céline Anselme. 2011. “An Urban Noise Observatory: Scientific, Technical, Strategic and Political Challenges: A Systemic Complementary Approach to the New Requirements of the European Directives.” In Proceedings of the 40th International Congress and Exposition on Noise Control Engineering 2011, INTER-NOISE 2011, 5. Osaka, Japan: INTER-NOISE. https://www.acoucite.org/IMG/pdf/ARTICLE-internoise2011_ENG.pdf.
   Google Scholar
• Wald, Dara M., Justin Longo, and A. R. Dobell. 2016. “Design Principles for Engaging and Retaining Virtual Citizen Scientists.” Conservation Biology: The Journal of the Society for Conservation Biology 30 (3): 562–570. doi:10.1111/cobi.12627.
   PubMed Web of Science ®Google Scholar
• Wallace, Kristi, Seth Snedigar, and Cheryl Cameron. 2015. “Is Ash Falling?’ An Online Ashfall Reporting Tool in Support of Improved Ashfall Warnings and Investigations of Ashfall Processes.” Journal of Applied Volcanology 4 (1): 8. doi:10.1186/s13617-014-0022-6.
   Google Scholar
• Wehn, Uta, Kevin Collins, Kim Anema, Laura Basco-Carrera, and Alix Lerebours. 2018. “Stakeholder Engagement in Water Governance as Social Learning: Lessons from Practice.” Water International 43 (1): 34–59. doi:10.1080/02508060.2018.1403083.
   Web of Science ®Google Scholar
• Wehn, Uta, and Jaap Evers. 2015. “The Social Innovation Potential of ICT-Enabled Citizen Observatories to Increase EParticipation in Local Flood Risk Management.” Technology in Society 42 (August): 187–198. doi:10.1016/j.techsoc.2015.05.002.
   Google Scholar
• Wehn, Uta, Simon McCarthy, Vita Lanfranchi, and Sue Tapsell. 2015. “Citizen Observatories as Facilitators of Change in Water Governance? Experiences from Three European Cases.” Environmental Engineering and Management Journal 14 (9): 2073–2086. http://www.eemj.icpm.tuiasi.ro/issues/vol14/vol14no9.htm. doi:10.30638/eemj.2015.222.
   Web of Science ®Google Scholar
• Whitelaw, Graham, Hague Vaughan, Brian Craig, and David Atkinson. 2003. “Establishing the Canadian Community Monitoring Network.” Environmental Monitoring and Assessment 88 (1/3): 409–418. doi:10.1023/A:1025545813057.
   PubMed Web of Science ®Google Scholar
• Wiggins, Andrea. 2013. “Free as in Puppies: Compensating for ICT Constraints in Citizen Science.” In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, 1469–1480. CSCW ’13.New York, NY, USA: ACM. doi:10.1145/2441776.2441942.
   Google Scholar
• Wiggins, Andrea, and Kevin Crowston. 2011. “From Conservation to Crowdsourcing: A Typology of Citizen Science.” In Proceedings of the 2011 44th Hawaii International Conference on System Sciences, 1–10. HICSS ’11.Washington, DC: IEEE Computer Society. doi:10.1109/HICSS.2011.207.
   Google Scholar
• Williamson, Ray A. 2009. “Earth Observations and Human and Environmental Security: Opportunities and Challenges.” In Proceedings of the 60th International Astronautical Congress 2009, 2445–2451. Daejeon, Republic of Korea: International Astronautical Federation. https://iafastro.directory/iac/archive/browse/IAC-09/B1/5/4125/.
   Google Scholar
• Wright, Dale R., Les G. Underhill, Matt Keene, and Andrew T. Knight. 2015. “Understanding the Motivations and Satisfactions of Volunteers to Improve the Effectiveness of Citizen Science Programs.” Society & Natural Resources 28 (9): 1013–1029. doi:10.1080/08941920.2015.1054976.
   Web of Science ®Google Scholar
• Zaman, J., and W. De Meuter. 2015. “DisCoPar: Distributed Components for Participatory Campaigning.” In Proceedings of 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), 160–165. St. Louis, MO: IEEE. doi:10.1109/PERCOMW.2015.7134012.
   Google Scholar
• Zaman, J., and W. De Meuter. 2016. “Crowd Sensing Applications: A Distributed Flow-Based Programming Approach.” In Proceedings of the 2016 IEEE International Conference on Mobile Services (MS), 79–86. San Francisco, CA: IEEE. doi:10.1109/MobServ.2016.22.
   Google Scholar