Crowds, citizens, and science: a multi-dimensional framework and agenda for future research

References

• Aceves‐Bueno, E., A. S. Adeleye, M. Feraud, Y. Huang, M. Tao, Y. Yang, S. E. Anderson. et al. 2017. “The Accuracy of Citizen Science Data: A Quantitative Review.” The Bulletin of the Ecological Society of America 98 (4): 278–290. doi:https://doi.org/10.1002/bes2.1336.
   Google Scholar
• Afuah, A. 2018. “Crowdsourcing: A Primer and Research Framework.” In Creating & Capturing Value through Crowdsourcing, edited by C. L. Tucci, A. Afuah, and G. Viscusi. 11-38. Oxford, UK: Oxford University Press.
   Google Scholar
• Afuah, A., and C. L. Tucci. 2012. “Crowdsourcing as a Solution to Distant Search.” Academy of Management Review 37 (3): 355–375. doi:https://doi.org/10.5465/amr.2010.0146.
   Web of Science ®Google Scholar
• Arnstein, S. R. 1969. “A Ladder of Citizen Participation.” Journal of the American Institute of Planners 35 (4): 216–224. doi:https://doi.org/10.1080/01944366908977225.
   Web of Science ®Google Scholar
• Beck, S., C. Bergenholtz, M. Bogers, T.-M. Brasseur, M. L. Conradsen, D. Di Marco. et al. 2021. “The Open Innovation in Science Research Field: A Collaborative Conceptualisation Approach.” Industry and Innovation 1–50. https://www.tandfonline.com/doi/full/10.1080/13662716.2020.1792274
   Web of Science ®Google Scholar
• Beck, S., T.-M. Brasseur, M. K. Poetz, and H. Sauermann. 2020. “What Is the Problem? Crowdsourcing Research Questions in Science.” Working Paper.
   Google Scholar
• Benchoufi, M., O. De Fresnoye, K. Levy-Heidmann, E. Mariani, and O. Tauvel-Mocquet. 2017. “Epidemium - White Paper.” Accessed September 12, 2021. https://www.roche.fr/fr/medias/presse/livre-blanc-epidemium.html
   Google Scholar
• Black, R. C., and T. R. Johnson. 2018. “Behind the Velvet Curtain: Understanding Supreme Court Conference Discussions through Justices’ Personal Conference Notes.” Journal of Appellate Practice and Process 19: 223.
   Google Scholar
• Bonney, R. 1996. “Citizen Science: A Lab Tradition.” Living Biard 15 (4): 7–15.
   Google Scholar
• Bonney, R., C. B. Cooper, J. Dickinson, S. Kelling, T. Phillips, K. V. Rosenberg, J. Shirk. 2009. “Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy.” BioScience 59 (11): 977–984. doi:https://doi.org/10.1525/bio.2009.59.11.9.
   Web of Science ®Google Scholar
• Bonney, R., J. L. Shirk, T. B. Phillips, A. Wiggins, H. L. Ballard, A. J. Miller-Rushing, J. K. Parrish. et al. 2014. “Next Steps for Citizen Science.” Science 343 (6178): 1436–1437. doi:https://doi.org/10.1126/science.1251554.
   PubMed Web of Science ®Google Scholar
• Bonney, R., T. B. Phillips, H. L. Ballard, and J. W. Enck. 2016. “Can Citizen Science Enhance Public Understanding of Science?” Public Understanding of Science 25 (1): 2–16. doi:https://doi.org/10.1177/0963662515607406.
   PubMed Web of Science ®Google Scholar
• Boudreau, K. J., and K. R. Lakhani. 2013. “Using the Crowd as an Innovation Partner.” Harvard Business Review 91 (4): 60–140.
   PubMed Web of Science ®Google Scholar
• Boudreau, K. J., N. Lacetera, and K. R. Lakhani. 2011. “Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis.” Management Science 57 (5): 843–863. doi:https://doi.org/10.1287/mnsc.1110.1322.
   Web of Science ®Google Scholar
• Brabham, D. C. 2008. “Crowdsourcing as a Model for Problem Solving: An Introduction and Cases.” Convergence: The International Journal of Research into New Media Technologies 14 (1): 75–90. doi:https://doi.org/10.1177/1354856507084420.
   Google Scholar
• Brossard, D., B. Lewenstein, and R. Bonney. 2005. “Scientific Knowledge and Attitude Change: The Impact of a Citizen Science Project.” International Journal of Science Education 27 (9): 1099–1121. doi:https://doi.org/10.1080/09500690500069483.
   Web of Science ®Google Scholar
• Brynjolfsson, E., and T. Mitchell. 2017. “What Can Machine Learning Do? Workforce Implications.” Science 358 (6370): 1530–1534. doi:https://doi.org/10.1126/science.aap8062.
   PubMed Web of Science ®Google Scholar
• Burgess, H., L. DeBey, H. Froehlich, N. Schmidt, E. Theobald, A. Ettinger, J. HilleRisLambers. et al. 2017. “The Science of Citizen Science: Exploring Barriers to Use as a Primary Research Tool”. Biological Conservation 208: 113–120. doi:https://doi.org/10.1016/j.biocon.2016.05.014.
   Web of Science ®Google Scholar
• CASRAI. 2018. “Credit - Contributor Roles Taxonomy.” Accessed 12 September 2021. https://casrai.org/credit/
   Google Scholar
• Ceccaroni, L., J. Bibby, E. Roger, P. Flemons, K. Michael, L. Fagan. J. L. Oliver. 2019. “Opportunities and Risks for Citizen Science in the Age of Artificial Intelligence.” Citizen Science: Theory and Practice 4:1.
   Google Scholar
• Chen, M. K., P. E. Rossi, J. A. Chevalier, and E. Oehlsen. 2019. “The Value of Flexible Work: Evidence from Uber Drivers.” Journal of Political Economy 127 (6): 2735–2794. doi:https://doi.org/10.1086/702171.
   Web of Science ®Google Scholar
• Cohen, W. M., H. Sauermann, and P. Stephan. 2020. “Not in the Job Description: The Commercial Activities of Academic Scientists and Engineers.” Management Science 66 (9): 4108–4117. doi:https://doi.org/10.1287/mnsc.2019.3535.
   Web of Science ®Google Scholar
• Congress, U. S. 2016. “Crowdsourcing and Citizen Science Act.”
   Google Scholar
• Cooper, S., F. Khatib, A. Treuille, J. Barbero, J. Lee, M. Beenen, A. Leaver-Fay. et al. 2010. “Predicting Protein Structures with a Multiplayer Online Game.” Nature 466 (7307): 756–760. doi:https://doi.org/10.1038/nature09304.
   PubMed Web of Science ®Google Scholar
• Crowston, K., C. Østerlund, T. K. Lee, C. Jackson, M. Harandi, S. Allen, S. Bahaadini. et al. 2019. “Knowledge Tracing to Model Learning in Online Citizen Science Projects.” IEEE Transactions on Learning Technologies 13 (1): 123–134. doi:https://doi.org/10.1109/TLT.2019.2936480.
   Web of Science ®Google Scholar
• Crowston, K., E. Mitchell, and C. Østerlund. 2019. “Coordinating Advanced Crowd Work: Extending Citizen Science.” Citizen Science: Theory and Practice 4 (1): 1–2.
   Google Scholar
• Dahlander, L., and S. O’Mahony. 2010. “Progressing to the Center: Coordinating Project Work.” Organization Science 22 (4): 961–979. doi:https://doi.org/10.1287/orsc.1100.0571.
   Web of Science ®Google Scholar
• Dasgupta, P., and P. A. David. 1994. “Toward a New Economics of Science.” Research Policy 23 (5): 487–521. doi:https://doi.org/10.1016/0048-7333(94)01002-1.
   Web of Science ®Google Scholar
• Davis, M., and K. Laas. 2014. ““Broader Impacts” or “Responsible Research and Innovation”? A Comparison of Two Criteria for Funding Research in Science and Engineering.” Science and Engineering Ethics 20 (4): 963–983. doi:https://doi.org/10.1007/s11948-013-9480-1.
   PubMed Web of Science ®Google Scholar
• De Winter, J. C., and D. Dodou. 2014. “Why the Fitts List Has Persisted Throughout the History of Function Allocation.” Cognition, Technology & Work 16 (1): 1–11. doi:https://doi.org/10.1007/s10111-011-0188-1.
   Web of Science ®Google Scholar
• Del Savio, L., B. Prainsack, and A. Buyx. 2016. “Crowdsourcing the Human Gut. Is Crowdsourcing Also ‘Citizen Science’?” Journal of Science Communication 15 (3): 1–16. doi:https://doi.org/10.22323/2.15030203.
   Web of Science ®Google Scholar
• Druschke, C. G., and C. E. Seltzer. 2012. “Failures of Engagement: Lessons Learned from a Citizen Science Pilot Study.” Applied Environmental Education & Communication 11 (3–4): 178–188. doi:https://doi.org/10.1080/1533015X.2012.777224.
   Google Scholar
• ECSA. 2020. “ECSA’s Characteristics of Citizen Science.”Accessed 12 September 2021. https://doi.org/http://doi.org/10.5281/zenodo.3758668.
   Google Scholar
• European Citizen Science Association. 2015. “ECSA Strategy.” Accessed 12 September 2021. https://ecsa.citizen-science.net/wp-content/uploads/2020/02/ecsa_strategy.pdf
   Google Scholar
• European Commission. 2016. “8 ambitions of the EU's open science policy.” Accessed 12 September 2021. https://ec.europa.eu/info/research-and-innovation/strategy/strategy-2020-2024/our-digital-future/open-science_en
   Google Scholar
• ExCiteS. 2019. “Citizen Science and Botanic Knowledge among Herders and Farmers in Kenya.” Accessed 12 September 2021. https://uclexcites.blog/2019/05/29/citizen-science-and-botanic-knowledge-among-herders-and-farmers-in-kenya/
   Google Scholar
• Felin, T., and T. R. Zenger. 2014. “Closed or Open Innovation? Problem Solving and the Governance Choice.” Research Policy 43 (5): 914–925. doi:https://doi.org/10.1016/j.respol.2013.09.006.
   Web of Science ®Google Scholar
• Franzen, M., L. Kloetzer, M. Ponti, J. Trojan, and J. Vicens. 2021. “Machine Learning in Citizen Science: Promises and Implications.” In The Science of Citizen Science, edited by K. Vohland, A. Land-Zandstra, L. Ceccaroni, J. Rob Lemmens, P. M. Ponti, R. Samson, and K. Wagenknecht, 183–198, Cham, Switzerland: Springer.
   Google Scholar
• Franzoni, C., and H. Sauermann. 2014. “Crowd Science: The Organization of Scientific Research in Open Collaborative Projects.” Research Policy 43 (1): 1–20. doi:https://doi.org/10.1016/j.respol.2013.07.005.
   Web of Science ®Google Scholar
• Franzoni, C., H. Sauermann, and D. Di Marco. 2021. “When Citizens Judge Science: Evaluations of Social Impact and Support for Research.” Working Paper.
   Google Scholar
• Freeman, R., E. Weinstein, E. Marincola, J. Rosenbaum, and F. Solomon. 2001. “Competition and Careers in Biosciences.” Science 294 (5550): 2293–2294. doi:https://doi.org/10.1126/science.1067477.
   PubMed Web of Science ®Google Scholar
• Furman, J. L., and S. Stern. 2011. “Climbing Atop the Shoulders of Giants: The Impact of Institutions on Cumulative Research.” American Economic Review 101 (5): 1933–1963. doi:https://doi.org/10.1257/aer.101.5.1933.
   Web of Science ®Google Scholar
• Gander, A. J., M. Ponti, and A. Seredko. 2021. “What Can Machine Learning Do? Implications for Citizen Scientists.” Zendo Working Paper.
   Google Scholar
• Geoghegan, H., A. Dyke, R. Pateman, S. West, and G. Everett. 2016. “Understanding Motivations for Citizen Science.” In UK Environmental Observation Framework. https://www.ukeof.org.uk/resources/citizen-science-resources/MotivationsforCSREPORTFINALMay2016.pdf 
   Google Scholar
• Green, S. E., J. P. Rees, P. A. Stephens, R. A. Hill, and A. J. Giordano. 2020. “Innovations in Camera Trapping Technology and Approaches: The Integration of Citizen Science and Artificial Intelligence.” Animals 10 (1): 132. doi:https://doi.org/10.3390/ani10010132.
   Web of Science ®Google Scholar
• Gruber, M., D. Harhoff, and K. Hoisl. 2013. “Knowledge Recombination across Technological Boundaries: Scientists Vs. Engineers.” Management Science 59 (4): 837–851. doi:https://doi.org/10.1287/mnsc.1120.1572.
   Web of Science ®Google Scholar
• Guerrini, C. J., M. A. Majumder, M. J. Lewellyn, and A. L. McGuire. 2018. “Citizen Science, Public Policy.” Science 361 (6398): 134–136. doi:https://doi.org/10.1126/science.aar8379.
   PubMed Web of Science ®Google Scholar
• Guinan, E., K. J. Boudreau, and K. R. Lakhani. 2013. “Experiments in Open Innovation at Harvard Medical School.” MIT Sloan Management Review 54 (3): 45–52.
   PubMed Web of Science ®Google Scholar
• Haeussler, C., and H. Sauermann. 2020. “Division of Labor in Collaborative Knowledge Production: The Role of Team Size and Interdisciplinarity.” Research Policy 49 (6): 103987. doi:https://doi.org/10.1016/j.respol.2020.103987.
   Web of Science ®Google Scholar
• Haklay, M. 2013. “Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation.” In Crowdsourcing Geographic Knowledge, edited by D. Sui, S. Elwood, and M. Goodchild, 105–122, Dordrecht, Netherlands: Springer.
   Google Scholar
• Haklay, M., D. Fraisl, B. Greshake Tzovaras, S. Hecker, M. Gold, G. Hager, et al. 2021. “Contours of Citizen Science: A Vignette Study.” Accessed 12 September 2021. https://osf.io/preprints/socarxiv/6u2ky/.
   Google Scholar
• Haklay, M. M., D. Dörler, F. Heigl, M. Manzoni, S. Hecker, and K. Vohland. 2021. “What Is Citizen Science? The Challenges of Definition.” In The Science of Citizen Science, edited by K. Vohland, A. Land-Zandstra, L. Ceccaroni, J. Rob Lemmens, P. M. Ponti, R. Samson, and K. Wagenknecht, 13–33, Cham, Switzerland: Springer.
   Google Scholar
• Hamari, J., J. Koivisto, and H. Sarsa. 2014. Does Gamification Work? A Literature Review of Empirical Studies on Gamification. Paper presented at the System Sciences (HICSS), 2014 47th Hawaii International Conference on System Sciences, Waikoloa, Hawaii, USA.
   Google Scholar
• Hamid, E., and N. Bonnie. 2014. “Heteromation and Its (Dis)contents: The Invisible Division of Labor between Humans and Machines.” First Monday 19: 6. doi:https://doi.org/10.5210/fm.v19i6.5331.
   Google Scholar
• Harrison, J. R., and G. R. Carroll. 1991. “Keeping the Faith: A Model of Cultural Transmission in Formal Organizations.” Administrative Science Quarterly 36 (4): 552–582. doi:https://doi.org/10.2307/2393274.
   Web of Science ®Google Scholar
• Heaven, D. 2019. “Why Deep-Learning Ais are so Easy to Fool.” Nature 574 (7777): 163–166. doi:https://doi.org/10.1038/d41586-019-03013-5.
   PubMedGoogle Scholar
• Hopkin, M. 2007. “See New Galaxies—without Leaving Your Chair.” Nature Publishing Group. Accessed 12 September 2021. https://www.nature.com/articles/news070709-7#article-info
   Google Scholar
• ICMJE. 2020. “Defining the Role of Authors and Contributors.” Accessed 12 September 2021. http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html
   Google Scholar
• Irwin, A. 1995. “Citizen Science: A Study of People, Expertise and Sustainable Development.” London and New York: Routledge. https://www.routledge.com/Citizen-Science-A-Study-of-People-Expertise-and-Sustainable-Development/Irwin/p/book/9780415130103
   Google Scholar
• Irwin, A. 2001. “Constructing the Scientific Citizen: Science and Democracy in the Biosciences.” Public Understanding of Science 10 (1): 1–18. doi:https://doi.org/10.1088/0963-6625/10/1/301.
   Web of Science ®Google Scholar
• Irwin, A. 2018. “Citizen Science Comes of Age.” Nature 562 (7728): 480–482. doi:https://doi.org/10.1038/d41586-018-07106-5.
   PubMedGoogle Scholar
• Jensen, J. H. M., M. Gajdacz, S. Z. Ahmed, J. H. Czarkowski, C. Weidner, J. Rafner, J. J. Sørensen. et al. 2021. “Crowdsourcing Human Common Sense for Quantum Control.” Physical Review Research 3 (1): 013057. doi:https://doi.org/10.1103/PhysRevResearch.3.013057.
   Google Scholar
• Jeppesen, L. B., and K. Lakhani. 2010. “Marginality and Problem-Solving Effectiveness in Broadcast Search.” Organization Science 21 (5): 1016–1033. doi:https://doi.org/10.1287/orsc.1090.0491.
   Web of Science ®Google Scholar
• Joshi, S., N. Randall, S. Chiplunkar, T. Wattimena, and K. Stavrianakis. 2018. ‘We’-a Robotic System to Extend Social Impact of Community Gardens. Paper presented at the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA.
   Google Scholar
• Kamar, E., A. Kapoor, and E. Horvitz. 2015. Identifying and Accounting for Task-Dependent Bias in Crowdsourcing. Paper presented at the Third AAAI Conference on Human Computation and Crowdsourcing, San Diego, California, USA.
   Google Scholar
• Kamar, E., S. Hacker, and E. Horvitz. 2012. Combining Human and Machine Intelligence in Large-Scale Crowdsourcing. Paper presented at the 11th International Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain.
   Google Scholar
• Katrak-Adefowora, R., J. L. Blickley, and A. J. Zellmer. 2020. “Just-in-Time Training Improves Accuracy of Citizen Scientist Wildlife Identifications from Camera Trap Photos.” Citizen Science: Theory and Practice 5: 1.
   PubMedGoogle Scholar
• Kelling, S., J. Gerbracht, D. Fink, C. Lagoze, W.-K. Wong, J. Yu, T. Damoulas, C. Gomes. 2012. Ebird: A Human/Computer Learning Network to Improve Biodiversity Conservation and Research. Paper presented at the Twenty-Fourth Innovative Applications of Artificial Intelligence Conference, Toronto, Ontario, Canada.
   Google Scholar
• Khatib, F., F. DiMaio, S. Cooper, M. Kazmierczyk, M. Gilski, S. Krzywda, H. Zabranska. et al. 2011. “Crystal Structure of a Monomeric Retroviral Protease Solved by Protein Folding Game Players”. Nature Structural & Molecular Biology 18(10): 1175–1177. doi:https://doi.org/10.1038/nsmb.2119.
   PubMed Web of Science ®Google Scholar
• Kittur, A., J. V. Nickerson, M. Bernstein, E. Gerber, A. Shaw, J. Zimmerman, M. Lease, J. Horton. 2013. The Future of Crowd Work. Paper presented at the Proceedings of the 2013 conference on Computer supported cooperative work, San Antonio, Texas, USA.
   Google Scholar
• Kittur, A., L. Yu, T. Hope, J. Chan, H. Lifshitz-Assaf, K. Gilon, F. Ng. et al. 2019. “Scaling up Analogical Innovation with Crowds and AI.” Proceedings of the National Academy of Sciences 116 (6): 1870–1877. doi:https://doi.org/10.1073/pnas.1807185116.
   PubMed Web of Science ®Google Scholar
• Kokshagina, O. forthcoming. “Open Covid‐19: Organizing an Extreme Crowdsourcing Campaign to Tackle Grand Challenges.” R&D Management.
   Web of Science ®Google Scholar
• Kuminski, E., and L. Shamir. 2016. “A Computer-Generated Visual Morphology Catalog Of 3,000,000 SDSS Galaxies.” The Astrophysical Journal Supplement Series 223 (2): 20. doi:https://doi.org/10.3847/0067-0049/223/2/20.
   Web of Science ®Google Scholar
• Lakhani, K. 2011. “Innocentive.Com.” HBS Case #608170. Cambridge, MA, USA: Harvard Business School Publishing.
   Google Scholar
• Latour, B., and S. Woolgar. 1979. Laboratory Life: The Social Construction of Scientific Facts. Beverly Hills: Sage.
   Google Scholar
• Lintott, C. J., K. Schawinski, W. Keel, H. Van Arkel, N. Bennert, E. Edmondson, D. Thomas. et al. 2009. “Galaxy Zoo: ‘Hanny’s Voorwerp’, a Quasar Light Echo?” Monthly Notices of the Royal Astronomical Society 399 (1): 129–140. doi:https://doi.org/10.1111/j.1365-2966.2009.15299.x.
   Web of Science ®Google Scholar
• Lyons, E., and L. Zhang. 2019. “Trade-Offs in Motivating Volunteer Effort: Experimental Evidence on Voluntary Contributions to Science.” PLOS ONE 14 (11): 11. doi:https://doi.org/10.1371/journal.pone.0224946.
   Web of Science ®Google Scholar
• Majchrzak, A., A. Malhotra, and M. A. Zaggl. 2021. “How Open Crowds Self-Organize.” Academy of Management Discoveries 7 (1): 104–129. doi:https://doi.org/10.5465/amd.2018.0087.
   Web of Science ®Google Scholar
• Martins, L. L., L. L. Gilson, and M. T. Maynard. 2004. “Virtual Teams: What Do We Know and Where Do We Go from Here?” Journal of Management 30 (6): 805–835. doi:https://doi.org/10.1016/j.jm.2004.05.002.
   Web of Science ®Google Scholar
• McDonald, D., E. Hyde, J. W. Debelius, J. T. Morton, A. Gonzalez, G. Ackermann, A. A. Aksenov. et al. 2018. “American Gut: An Open Platform for Citizen Science Microbiome Research.” Msystems 3 (3): e00031–18. doi:https://doi.org/10.1128/mSystems.00031-18.
   PubMed Web of Science ®Google Scholar
• Merton, R. K. 1973. The Sociology of Science: Theoretical and Empirical Investigations. Chicago: University of Chicago Press.
   Google Scholar
• National Academies. 2018. “The Next Generation of Biomedical and Behavioral Sciences Researchers.” Washington, DC: The National Academies Press. doi:https://doi.org/10.17226/25008.
   Google Scholar
• Newman, G., A. Wiggins, A. Crall, E. Graham, S. Newman, and K. Crowston. 2012. “The Future of Citizen Science: Emerging Technologies and Shifting Paradigms.” Frontiers in Ecology and the Environment 10 (6): 298–304. doi:https://doi.org/10.1890/110294.
   Web of Science ®Google Scholar
• Okhuysen, G. A., and B. A. Bechky. 2009. “Coordination in Organizations: An Integrative Perspective.” Academy of Management Annals 3 (1): 463–502. doi:https://doi.org/10.5465/19416520903047533.
   Web of Science ®Google Scholar
• Ottinger, G. 2010. “Buckets of Resistance: Standards and the Effectiveness of Citizen Science.” Science, Technology, & Human Values 35 (2): 244–270. doi:https://doi.org/10.1177/0162243909337121.
   Web of Science ®Google Scholar
• Ouchi, W. G. 1979. “A Conceptual Framework for the Design of Organizational Control Mechanisms.” Management Science 25 (9): 833–848. doi:https://doi.org/10.1287/mnsc.25.9.833.
   Web of Science ®Google Scholar
• Owen-Smith, J. 2001. “Managing Laboratory Work through Skepticism: Processes of Evaluation and Control.” American Sociological Review 66 (3): 427–452. doi:https://doi.org/10.2307/3088887.
   Web of Science ®Google Scholar
• Petridis, P., M. Fischer-Kowalski, S. J. Singh, and D. Noll. 2017. “The Role of Science in Sustainability Transitions: Citizen Science, Transformative Research, and Experiences from Samothraki Island, Greece.” Island Studies Journal 12 (1): 115–134. doi:https://doi.org/10.24043/isj.8.
   Web of Science ®Google Scholar
• Poetz, M. K., and M. Schreier. 2012. “The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas?” Journal of Product Innovation Management 29 (2): 245–256. doi:https://doi.org/10.1111/j.1540-5885.2011.00893.x.
   Web of Science ®Google Scholar
• Polymath, D. H. J. 2012. “A New Proof of the Density Hales-Jewett Theorem.” Annals of Mathematics 175 (3): 1283–1327. doi:https://doi.org/10.4007/annals.2012.175.3.6.
   Web of Science ®Google Scholar
• Ponciano, L., F. Brasileiro, R. Simpson, and A. Smith. 2014. “Volunteers‘ Engagement in Human Computation for Astronomy Projects.” Computing in Science & Engineering 16 (6): 52–59. doi:https://doi.org/10.1109/MCSE.2014.4.
   Web of Science ®Google Scholar
• Ponti, M., I. Stankovic, W. Barendregt, B. Kestemont, and L. Bain. 2018. “Chefs Know More than Just Recipes: Professional Vision in a Citizen Science Game.” Human Computation 5: 1–12. doi:https://doi.org/10.15346/hc.v5i1.1.
   Google Scholar
• Puranam, P., O. Alexy, and M. Reitzig. 2014. “What’s “New” about New Forms of Organizing?” Academy of Management Review 39 (2): 162–180. doi:https://doi.org/10.5465/amr.2011.0436.
   Web of Science ®Google Scholar
• Raddick, M. J., G. Bracey, P. L. Gay, C. Lintott, C. Cardamone, P. Murray, K. Schawinski, A. S. Szalay, J, Vandenberg. 2013. “Galaxy Zoo: Motivations of Citizen Scientists.” Astronomy Education Review 12:1.
   Google Scholar
• Rafner, J., M. Gajdacz, G. Kragh, A. Hjorth, A. Gander, B. Palfi. A. Berditchevskaia, et al. 2021. Revisiting Citizen Science through the Lens of Hybrid Intelligence. arXiv 2104.14961. https://arxiv.org/abs/2104.14961
   Google Scholar
• Raisch, S., and S. Krakowski. 2021. “Artificial Intelligence and Management: The Automation–Augmentation Paradox.” Academy of Management Review 46 (1): 192–210. doi:https://doi.org/10.5465/amr.2018.0072.
   Web of Science ®Google Scholar
• Rasmussen, L. M. 2019. “Confronting Research Misconduct in Citizen Science.” Citizen Science: Theory and Practice 4 (1): 1–11.
   Google Scholar
• Rohden, F. 2018. “Support, Connect, and Organize. Exploring Moderator Roles in Citizen Science Discussion Forums.” Accessed 12 September 2021. https://gupea.ub.gu.se/bitstream/2077/57001/1/gupea_2077_57001_1.pdf
   Google Scholar
• Santolini, M. 2020. “Covid-19 the Rise of a Global Collective Intelligence.” Accessed 12 September 2021. https://theconversation.com/covid-19-the-rise-of-a-global-collective-intelligence-135738
   Google Scholar
• Sauermann, H., and C. Franzoni. 2013. “Participation Dynamics in Crowd-Based Knowledge Production: The Scope and Sustainability of Interest-Based Motivation.” SSRN Working Paper.
   Google Scholar
• Sauermann, H., and C. Franzoni. 2015. “Crowd Science User Contribution Patterns and Their Implications.” Proceedings of the National Academy of Sciences 112 (3): 679–684. doi:https://doi.org/10.1073/pnas.1408907112.
   PubMed Web of Science ®Google Scholar
• Sauermann, H., and C. Haeussler. 2017. “Authorship and Contribution Disclosures.” Science Advances 3 (11): e1700404. doi:https://doi.org/10.1126/sciadv.1700404.
   PubMed Web of Science ®Google Scholar
• Sauermann, H., K. Shafi, and C. Franzoni. 2019. “Crowdfunding Scientific Research: Descriptive Insights and Correlates of Funding Success.” Plos One14 (1): e0208384. doi:https://doi.org/10.1371/journal.pone.0208384.
   Google Scholar
• Sauermann, H., K. Vohland, V. Antoniou, B. Balaz, C. Goebel, K. Karatzas, P. Mooney. et al. 2020. “Citizen Science and Sustainability Transitions.” Research Policy 49 (5): 103978. doi:https://doi.org/10.1016/j.respol.2020.103978.
   Web of Science ®Google Scholar
• Sauermann, H., and P. Stephan. 2013. “Conflicting Logics? A Multidimensional View of Industrial and Academic Science.” Organization Science 24 (3): 889–909. doi:https://doi.org/10.1287/orsc.1120.0769.
   Web of Science ®Google Scholar
• Scheliga, K., S. Friesike, C. Puschmann, and B. Fecher. 2018. “Setting up Crowd Science Projects.” Public Understanding of Science 27 (5): 515–534. doi:https://doi.org/10.1177/0963662516678514.
   PubMed Web of Science ®Google Scholar
• Senior, A. W., R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, T. Green, C. Qin. et al. 2020. “Improved Protein Structure Prediction Using Potentials from Deep Learning.” Nature 577 (7792): 706–710. doi:https://doi.org/10.1038/s41586-019-1923-7.
   PubMed Web of Science ®Google Scholar
• Shah, S. K. 2006. “Motivation, Governance, and the Viability of Hybrid Forms in Open Source Software Development.” Management Science 52 (7): 1000–1014. doi:https://doi.org/10.1287/mnsc.1060.0553.
   Web of Science ®Google Scholar
• Shapin, S. 2008. The Scientific Life: A Moral History of a Late Modern Vocation. Chicago, IL, USA: University of Chicago Press. https://press.uchicago.edu/ucp/books/book/chicago/S/bo5747715.html
   Google Scholar
• Shi, Y., P. L. Prieto, T. Zepel, S. Grunert, and J. E. Hein. 2021. “Automated Experimentation Powers Data Science in Chemistry.” Accounts of Chemical Research 54 (3): 546–555. doi:https://doi.org/10.1021/acs.accounts.0c00736.
   PubMed Web of Science ®Google Scholar
• Shibayama, S., Y. Baba, and J. P. Walsh. 2015. “Organizational Design of University Laboratories: Task Allocation and Lab Performance in Japanese Bioscience Laboratories.” Research Policy 44 (3): 610–622. doi:https://doi.org/10.1016/j.respol.2014.12.003.
   Web of Science ®Google Scholar
• Shirk, J. L., H. L. Ballard, C. C. Wilderman, T. Phillips, A. Wiggins, R. Jordan, E. McCallie. et al. 2012. “Public Participation in Scientific Research: A Framework for Deliberate Design.” Ecology and Society 17 (2): A29. Ddoi:https://doi.org/10.5751/ES-04705-170229.
   Web of Science ®Google Scholar
• Simpson, R., K. R. Page, and D. De Roure. 2014. Zooniverse: Observing the World’s Largest Citizen Science Platform. Paper presented at the Proceedings of the 23rd International Conference on World Wide Web, Seoul, Korea.
   Google Scholar
• Singh, J., and L. Fleming. 2010. “Lone Inventors as Sources of Breakthroughs: Myth or Reality?” Management Science 56 (1): 41–56. doi:https://doi.org/10.1287/mnsc.1090.1072.
   Web of Science ®Google Scholar
• Socientize Consortium. 2013. “Citizen Science for Europe. Towards a Better Society of Empowered Citizens and Enhanced Research.” Accessed 12 September 2021. https://ec.europa.eu/digital-single-market/en/news/green-paper-citizen-science-europe-towards-society-empowered-citizens-and-enhanced-research
   Google Scholar
• Sparkes, A., W. Aubrey, E. Byrne, A. Clare, M. N. Khan, M. Liakata, M. Markham. et al. 2010. “Towards Robot Scientists for Autonomous Scientific Discovery.” Automated Experimentation 2 (1): A1. doi:https://doi.org/10.1186/1759-4499-2-1.
   PubMedGoogle Scholar
• Stephan, P. 2012. How Economics Shapes Science. Cambridge, MA, USA: Harvard University Press.
   Google Scholar
• Strasser, B., J. Baudry, D. Mahr, G. Sanchez, and E. Tancoigne. 2019. ““Citizen Science”? Rethinking Science and Public Participation.” Science & Technology Studies 32: 52–76.
   Web of Science ®Google Scholar
• Sullivan, B. L., C. L. Wood, M. J. Iliff, R. E. Bonney, D. Fink, and S. Kelling. 2009. “Ebird: A Citizen-Based Bird Observation Network in the Biological Sciences.” Biological Conservation 142 (10): 2282–2292. doi:https://doi.org/10.1016/j.biocon.2009.05.006.
   Web of Science ®Google Scholar
• Sullivan, D. P., C. F. Winsnes, L. Åkesson, M. Hjelmare, M. Wiking, R. Schutten, L. Campbell. et al. 2018. “Deep Learning Is Combined with Massive-Scale Citizen Science to Improve Large-Scale Image Classification.” Nature Biotechnology 36 (9): 820–828. doi:https://doi.org/10.1038/nbt.4225.
   PubMed Web of Science ®Google Scholar
• SwafS. 2017. Citizen Science Policies in the European Commission. Science with and for Society Policy Brief. Accessed 12 September 2021. https://www.sisnetwork.eu/media/sisnet/Policy_brief_Citizen_Science_SiSnet.pdf
   Google Scholar
• Swanson, A., M. Kosmala, C. Lintott, R. Simpson, A. Smith, and C. Packer. 2015. “Snapshot Serengeti, High-Frequency Annotated Camera Trap Images of 40 Mammalian Species in an African Savanna.” Scientific Data 2 (1): 1–14. doi:https://doi.org/10.1038/sdata.2015.26.
   Google Scholar
• Teodoridis, F., M. Bikard, and K. Vakili. 2019. “Creativity at the Knowledge Frontier: The Impact of Specialization in Fast- and Slow-paced Domains.” Administrative Science Quarterly 64 (4): 894–927. doi:https://doi.org/10.1177/0001839218793384.
   Web of Science ®Google Scholar
• Trouille, L., C. J. Lintott, and L. F. Fortson. 2019. “Citizen Science Frontiers: Efficiency, Engagement, and Serendipitous Discovery with Human–Machine Systems.” Proceedings of the National Academy of Sciences 116 (6): 1902–1909. doi:https://doi.org/10.1073/pnas.1807190116.
   PubMed Web of Science ®Google Scholar
• Tucci, C. L., A. Afuah, and G. Viscusi. 2018. Creating and Capturing Value through Crowdsourcing. Oxford, United Kingdom: Oxford University Press.
   Google Scholar
• Turrini, T., D. Dörler, A. Richter, F. Heigl, and A. Bonn. 2018. “The Threefold Potential of Environmental Citizen science - Generating Knowledge, Creating Learning Opportunities and Enabling Civic Participation.” Biological Conservation 225: 176–186. doi:https://doi.org/10.1016/j.biocon.2018.03.024.
   Web of Science ®Google Scholar
• Van Brussel, S., and H. Huyse. 2018. “Citizen Science on Speed? Realising the Triple Objective of Scientific Rigour, Policy Influence and Deep Citizen Engagement in a Large-Scale Citizen Science Project on Ambient Air Quality in Antwerp.” Journal of Environmental Planning and Management 62 (3): 1–18.
   Web of Science ®Google Scholar
• Von Hippel, E. 1994. ““Sticky Information” and the Locus of Problem Solving: Implications for Innovation.” Management Science 40 (4): 429–439. doi:https://doi.org/10.1287/mnsc.40.4.429.
   Web of Science ®Google Scholar
• Waldispühl, J., A. Szantner, R. Knight, S. Caisse, and R. Pitchford. 2020. “Leveling up Citizen Science.” Nature Biotechnology 38 (10): 1124–1126. doi:https://doi.org/10.1038/s41587-020-0694-x.
   PubMed Web of Science ®Google Scholar
• Walmsley, M., C. Lintott, T. Geron, S. Kruk, C. Krawczyk, K. W. Willett, S. Bamford, et al. 2021. Galaxy Zoo Decals: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxies. arXiv:2102.08414. https://arxiv.org/abs/2102.08414
   Google Scholar
• Walsh, J. P., and Y.-N. Lee. 2015. “The Bureaucratization of Science.” Research Policy 44 (8): 1584–1600. doi:https://doi.org/10.1016/j.respol.2015.04.010.
   Web of Science ®Google Scholar
• Weichenthal, S., E. Dons, K. Y. Hong, P. O. Pinheiro, and F. J. Meysman. 2021. “Combining Citizen Science and Deep Learning for Large-Scale Estimation of Outdoor Nitrogen Dioxide Concentrations.” Environmental Research 196: 110389. doi:https://doi.org/10.1016/j.envres.2020.110389.
   PubMed Web of Science ®Google Scholar
• Weinstein, B. G. 2018. “A Computer Vision for Animal Ecology.” Journal of Animal Ecology 87 (3): 533–545. doi:https://doi.org/10.1111/1365-2656.12780.
   PubMed Web of Science ®Google Scholar
• West, J., and J. Sims. 2018. “How Firms Leverage Crowds and Communities for Open Innovation.” SSRN Working Paper.
   Google Scholar
• Wiggins, A., and J. Wilbanks. 2019. “The Rise of Citizen Science in Health and Biomedical Research.” The American Journal of Bioethics 19 (8): 3–14. doi:https://doi.org/10.1080/15265161.2019.1619859.
   PubMed Web of Science ®Google Scholar
• Wiggins, A., and K. Crowston. 2011. From Conservation to Crowdsourcing: A Typology of Citizen Science. Paper presented at the 44th Hawaii International Conference on Systems Sciences (HICSS), Kauai, Hawaii, USA.
   Google Scholar
• Willett, K. W., C. J. Lintott, S. P. Bamford, K. L. Masters, B. D. Simmons, K. R. Casteels, E. M. Edmondson. et al. 2013. “Galaxy Zoo 2: Detailed Morphological Classifications for 304 122 Galaxies from the Sloan Digital Sky Survey.” Monthly Notices of the Royal Astronomical Society 435 (4): 2835–2860. doi:https://doi.org/10.1093/mnras/stt1458.
   Web of Science ®Google Scholar
• Williams, A. C., J. F. Wallin, H. Yu, M. Perale, H. D. Carroll, A.-F. Lamblin, L. Fortson, D. Obbink, C. Lintott, James H. Brusuelas. 2014. A Computational Pipeline for Crowdsourced Transcriptions of Ancient Greek Papyrus Fragments. Paper presented at the 2014 IEEE International Conference on Big Data, Washington DC, USA.
   Google Scholar
• Wood, A. J., M. Graham, V. Lehdonvirta, and I. Hjorth. 2019. “Good Gig, Bad Gig: Autonomy and Algorithmic Control in the Global Gig Economy.” Work, Employment and Society 33 (1): 56–75. doi:https://doi.org/10.1177/0950017018785616.
   PubMed Web of Science ®Google Scholar
• Woolley, J. P., M. L. McGowan, H. J. A. Teare, V. Coathup, J. R. Fishman, R. A. Settersten, S. Sterckx. et al. 2016. “Citizen Science or Scientific Citizenship? Disentangling the Uses of Public Engagement Rhetoric in National Research Initiatives.” BMC Medical Ethics 17 (1): 33. doi:https://doi.org/10.1186/s12910-016-0117-1.
   PubMedGoogle Scholar
• Wuchty, S., B. Jones, and B. Uzzi. 2007. “The Increasing Dominance of Teams in Production of Knowledge.” Science 316 (5827): 1036–1039. doi:https://doi.org/10.1126/science.1136099.
   PubMed Web of Science ®Google Scholar
• Yang, J., I. Anishchenko, H. Park, Z. Peng, S. Ovchinnikov, and D. Baker. 2020. “Improved Protein Structure Prediction Using Predicted Interresidue Orientations.” Proceedings of the National Academy of Sciences 117 (3): 1496–1503. doi:https://doi.org/10.1073/pnas.1914677117.
   PubMed Web of Science ®Google Scholar
• Zaken, D. B., K. Gal, G. Shani, A. Segal, and D. Cavalier. 2021. Intelligent Recommendations for Citizen Science. Paper presented at the AAAI-21 Conference on Artificial Intelligence. https://ojs.aaai.org/index.php/AAAI/article/view/17726
   Google Scholar