Citizen science projects require agreement: a Delphi study to identify which knowledge on urban ecology is considered relevant from scientists’ and citizens’ perspectives

References

• Aikenhead, G. S. (2003). Review of research on humanistic perspectives in science curricula. European Science Education Research Association (ESERA) conference, Nordwijkerhout, Netherlands, 19–23 August 2003. Noordwijkerhout, Netherlands.
   Google Scholar
• Allum, N., Sturgis, P., Tabourazi, D., & Brunton-Smith, I. (2008). Science knowledge and attitudes across cultures: A meta-analysis. Public Understanding of Science, 17(1), 35–54. https://doi.org/https://doi.org/10.1177/0963662506070159
   Web of Science ®Google Scholar
• Aristeidou, M., & Herodotou, C. (2020). Online citizen science: A systematic review of effects on learning and scientific literacy. Citizen Science: Theory and Practice, 5(1), 69. https://doi.org/https://doi.org/10.5334/cstp.224
   Google Scholar
• Baram-Tsabari, A., Sethi, R. J., Bry, L., & Yarden, A. (2006). Using questions sent to an ask-a-scientist site to identify children’s interests in science. Science Education, 90(6), 1050–1072. https://doi.org/https://doi.org/10.1002/sce.20163
   Web of Science ®Google Scholar
• Baram-Tsabari, A., Sethi, R. J., Bry, L., & Yarden, A. (2009). Asking scientists: A decade of questions analyzed by age, gender, and country. Science Education, 93(1), 131–160. https://doi.org/https://doi.org/10.1002/sce.20284
   Web of Science ®Google Scholar
• Bela, G., Peltola, T., Young, J. C., Balázs, B., Arpin, I., Pataki, G., Hauck, J., Kelemen, E., Kopperoinen, L., van Herzele, A., Keune, H., Hecker, S., Suškevičs, M., Roy, H. E., Itkonen, P., Külvik, M., László, M., Basnou, C., Pino, J., & Bonn, A. (2016). Learning and the transformative potential of citizen science. Conservation Biology, 30(5), 990–999. https://doi.org/https://doi.org/10.1111/cobi.12762
   PubMed Web of Science ®Google Scholar
• Besley, J. C. (2014). Science and technology: Public attitudes and understanding. In National Science Board (Ed.), Science and engineering indicators 2014 (7-1-7-53). National Science Foundation (NSB 14-01).
   Google Scholar
• Besley, J. C., Dudo, A., & Storksdieck, M. (2015). Scientists’ views about communication training. Journal of Research in Science Teaching, 52(2), 199–220. https://doi.org/https://doi.org/10.1002/tea.21186
   Web of Science ®Google Scholar
• Blanco-López, Á, España-Ramos, E., González-García, F. J., & Franco-Mariscal, A. J. (2015). Key aspects of scientific competence for citizenship: A delphi study of the expert community in Spain. Journal of Research in Science Teaching, 52(2), 164–198. https://doi.org/https://doi.org/10.1002/tea.21188
   Web of Science ®Google Scholar
• Bonney, R. E., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T. B., Rosenberg, K. V., & Shirk, J. (2009). Citizen science: A developing tool for expanding science knowledge and scientific literacy. BioScience, 59(11), 977–984. https://doi.org/https://doi.org/10.1525/bio.2009.59.11.9
   Web of Science ®Google Scholar
• Bonney, R. E., Phillips, T. B., Ballard, H. L., & Enck, J. W. (2016). Can citizen science enhance public understanding of science? Public Understanding of Science, 25(1), 2–16. https://doi.org/https://doi.org/10.1177/0963662515607406
   PubMed Web of Science ®Google Scholar
• Bromme, R., & Goldman, S. R. (2014). The public’s bounded understanding of science. Educational Psychologist, 49(2), 59–69. https://doi.org/https://doi.org/10.1080/00461520.2014.921572
   Web of Science ®Google Scholar
• Bromme, R., & Kienhues, D. (2017). Gewissheit und Skepsis: Wissenschaftskommunikation als Forschungsthema der Psychologie. Psychologische Rundschau, 68(3), 167–171. https://doi.org/https://doi.org/10.1026/0033-3042/a000359
   Google Scholar
• Bromme, R., Rambow, R., & Nückles, M. (2001). Expertise and estimating what other people know: The influence of professional experience and type of knowledge. Journal of Experimental Psychology: Applied, 7(4), 317–330. https://doi.org/https://doi.org/10.1037/1076-898X.7.4.317
   PubMed Web of Science ®Google Scholar
• Brossard, D., Lewenstein, B. V., & Bonney, R. E. (2005). Scientific knowledge and attitude change: The impact of a citizen science project. International Journal of Science Education, 27(9), 1099–1121. https://doi.org/https://doi.org/10.1080/09500690500069483
   Web of Science ®Google Scholar
• Bruckermann, T., Greving, H., Schumann, A., Stillfried, M., Börner, K., Kimmig, S. E., Hagen, R., Brandt, M., & Harms, U. (2021). To know about science is to love it? Unraveling cause–effect relationships between knowledge and attitudes toward science in citizen science on urban wildlife ecology. Journal of Research in Science Teaching, 58(8), 1179–1202. https://doi.org/https://doi.org/10.1002/tea.21697
   Web of Science ®Google Scholar
• Carr, A. E., Grand, A., & Sullivan, M. (2017). Knowing me, knowing you. Science Communication, 39(6), 771–781. https://doi.org/https://doi.org/10.1177/1075547017736891
   Web of Science ®Google Scholar
• Clayton, M. J. (1997). Delphi: A technique to harness expert opinion for critical decision-making tasks in education. Educational Psychology, 17(4), 373–386. https://doi.org/https://doi.org/10.1080/0144341970170401
   Google Scholar
• de Groot, J. I. M., Steg, L., & Poortinga, W. (2013). Values, perceived risks and benefits, and acceptability of nuclear energy. Risk Analysis, 33(2), 307–317. https://doi.org/https://doi.org/10.1111/j.1539-6924.2012.01845.x
   PubMed Web of Science ®Google Scholar
• Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014). Defining consensus: A systematic review recommends methodologic criteria for reporting of delphi studies. Journal of Clinical Epidemiology, 67(4), 401–409. https://doi.org/https://doi.org/10.1016/j.jclinepi.2013.12.002
   PubMed Web of Science ®Google Scholar
• Falk, J. H., Storksdieck, M., & Dierking, L. D. (2007). Investigating public science interest and understanding: Evidence for the importance of free-choice learning. Public Understanding of Science, 16(4), 455–469. https://doi.org/https://doi.org/10.1177/0963662506064240
   Web of Science ®Google Scholar
• Flick, U. (2009). An introduction to qualitative research (4. Ed., rev., expanded and updated.). Sage.
   Google Scholar
• Follett, R., & Strezov, V. (2015). An analysis of citizen science based research: Usage and publication patterns. PloS One, 10(11), e0143687. https://doi.org/https://doi.org/10.1371/journal.pone.0143687
   PubMed Web of Science ®Google Scholar
• Groulx, M., Brisbois, M. C., Lemieux, C. J., Winegardner, A., & Fishback, L. (2017). A role for nature-based citizen science in promoting individual and collective climate change action? A systematic review of learning outcomes. Science Communication, 39(1), 45–76. https://doi.org/https://doi.org/10.1177/1075547016688324
   Web of Science ®Google Scholar
• Haywood, B. K. (2016). Beyond data points and research contributions: The personal meaning and value associated with public participation in scientific research. International Journal of Science Education, Part B, 6(3), 239–262. https://doi.org/https://doi.org/10.1080/21548455.2015.1043659
   Web of Science ®Google Scholar
• Haywood, B. K., & Besley, J. C. (2014). Education, outreach, and inclusive engagement: Towards integrated indicators of successful program outcomes in participatory science. Public Understanding of Science, 23(1), 92–106. https://doi.org/https://doi.org/10.1177/0963662513494560
   PubMed Web of Science ®Google Scholar
• Hendriks, F., Kienhues, D., & Bromme, R. (2016). Trust in science and the science of trust. In B. Blöbaum (Ed.), Trust and communication in a digitized world: Models and concepts of trust research (pp. 143–159). Springer International Publishing. https://doi.org/https://doi.org/10.1007/978-3-319-28059-2_8
   Google Scholar
• Jordan, R. C., Ballard, H. L., & Phillips, T. B. (2012). Key issues and new approaches for evaluating citizen-science learning outcomes. Frontiers in Ecology and the Environment, 10(6), 307–309. https://doi.org/https://doi.org/10.1890/110280
   Google Scholar
• Jordan, R. C., Gray, S. A., Howe, D. V., Brooks, W. R., & Ehrenfeld, J. G. (2011). Knowledge gain and behavioral change in citizen-science programs. Conservation Biology, 25(6), 1148–1154. https://doi.org/https://doi.org/10.1111/j.1523-1739.2011.01745.x
   PubMed Web of Science ®Google Scholar
• Krippendorff, K. (2004). Reliability in content analysis. Human Communication Research, 30(3), 411–433. https://doi.org/https://doi.org/10.1111/j.1468-2958.2004.tb00738.x
   Web of Science ®Google Scholar
• Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/https://doi.org/10.2307/2529310
   PubMed Web of Science ®Google Scholar
• Lewenstein, B. V., & Bonney, R. E. (2004). Different ways of looking at public understanding of research. In D. Chittenden, G. Farmelo, & B. V. Lewenstein (Eds.), Creating connections: Museums and the public understanding of current research (pp. 63–71). Altamira Press.
   Google Scholar
• Lu, H., McComas, K. A., Buttke, D. E., Roh, S., & Wild, M. A. (2016). A one health message about bats increases intentions to follow public health guidance on bat rabies. PloS One, 11(5), e0156205. https://doi.org/https://doi.org/10.1371/journal.pone.0156205
   PubMed Web of Science ®Google Scholar
• Mayoh, K., & Knutton, S. (1997). Using out-of-school experience in science lessons: Reality or rhetoric? International Journal of Science Education, 19(7), 849–867. https://doi.org/https://doi.org/10.1080/0950069970190708
   Web of Science ®Google Scholar
• Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2). http://www.qualitative-research.net/index.php/fqs/article/download/1089/2384
   Google Scholar
• McDonald, R. I., Mansur, A. V., Ascensão, F., Colbert, M., Crossman, K., Elmqvist, T., Gonzalez, A., Güneralp, B., Haase, D., Hamann, M., Hillel, O., Huang, K., Kahnt, B., Maddox, D., Pacheco, A., Pereira, H. M., Seto, K. C., Simkin, R., Walsh, B., … Ziter, C. (2020). Research gaps in knowledge of the impact of urban growth on biodiversity. Nature Sustainability, 3(1), 16–24. https://doi.org/https://doi.org/10.1038/s41893-019-0436-6
   Web of Science ®Google Scholar
• Mukherjee, N., Hugé, J., Sutherland, W. J., McNeill, J., van Opstal, M., Dahdouh-Guebas, F., Koedam, N., & Anderson, B. (2015). The delphi technique in ecology and biological conservation: Applications and guidelines. Methods in Ecology and Evolution, 6(9), 1097–1109. https://doi.org/https://doi.org/10.1111/2041-210X.12387
   Web of Science ®Google Scholar
• Osborne, J., Collins, S., Ratcliffe, M., Millar, R., & Duschl, R. (2003). What “Ideas-about-science" should be taught in school science? A Delphi study of the expert community. Journal of Research in Science Teaching, 40(7), 692–720. https://doi.org/https://doi.org/10.1002/tea.10105
   Web of Science ®Google Scholar
• Peter, M., Diekötter, T., & Kremer, K. (2019). Participant outcomes of biodiversity citizen science projects: A systematic literature review. Sustainability, 11(10), 2780. https://doi.org/https://doi.org/10.3390/su11102780
   Web of Science ®Google Scholar
• Peterman, K., & Young, D. (2015). Mystery shopping: An innovative method for observing interactions with scientists during public science events. Visitor Studies, 18(1), 83–102. https://doi.org/https://doi.org/10.1080/10645578.2015.1016369
   Web of Science ®Google Scholar
• Phillips, T. B., Porticella, N., Constas, M., & Bonney, R. E. (2018). A framework for articulating and measuring individual learning outcomesfrom participation in citizen science. Citizen Science: Theory and Practice, 3(2), 3. https://doi.org/https://doi.org/10.5334/cstp.126
   Google Scholar
• Ryder, J. (2001). Identifying science understanding for functional scientific literacy. Studies in Science Education, 36(1), 1–44. https://doi.org/https://doi.org/10.1080/03057260108560166
   Google Scholar
• Schuttler, S. G., Sorensen, A. E., Jordan, R. C., Cooper, C., & Shwartz, A. (2018). Bridging the nature gap: Can citizen science reverse the extinction of experience? Frontiers in Ecology and the Environment, 16(7), 405–411. https://doi.org/https://doi.org/10.1002/fee.1826
   Web of Science ®Google Scholar
• Seakins, A. (2015). Revealing questions: What are learners asking about? In D. Corrigan, C. Buntting, J. Dillon, A. Jones, & R. Gunstone (Eds.), The future in learning science: What’s in it for the learner? (1st ed., pp. 245–262). Springer. https://doi.org/https://doi.org/10.1007/978-3-319-16543-1_13
   Google Scholar
• Seakins, A., & Dillon, J. (2013). Exploring research themes in public engagement within a natural history museum: A modified delphi approach. International Journal of Science Education, Part B, 3(1), 52–76. https://doi.org/https://doi.org/10.1080/21548455.2012.753168
   Google Scholar
• Sickler, J., Cherry, T. M., Allee, L., Smyth, R. R., & Losey, J. (2014). Scientific value and educational goals: Balancing priorities and increasing adult engagement in a citizen science project. Applied Environmental Education & Communication, 13(2), 109–119. https://doi.org/https://doi.org/10.1080/1533015X.2014.947051
   Google Scholar
• Stillfried, M., Gras, P., Börner, K., Göritz, F., Painer, J., Röllig, K., Wenzler, M., Hofer, H., Ortmann, S., & Kramer-Schadt, S. (2017). Secrets of success in a landscape of fear: Urban wild boar adjust risk perception and tolerate disturbance. Frontiers in Ecology and Evolution, 5, 683. https://doi.org/https://doi.org/10.3389/fevo.2017.00157
   Web of Science ®Google Scholar
• Stocklmayer, S. M., & Bryant, C. (2012). Science and the public—what should people know? International Journal of Science Education, Part B, 2(1), 81–101. https://doi.org/https://doi.org/10.1080/09500693.2010.543186
   Google Scholar
• Stocklmayer, S. M., Rennie, L. J., & Gilbert, J. K. (2010). The roles of the formal and informal sectors in the provision of effective science education. Studies in Science Education, 46(1), 1–44. https://doi.org/https://doi.org/10.1080/03057260903562284
   Web of Science ®Google Scholar
• Stuckey, M., Hofstein, A., Mamlok-Naaman, R., & Eilks, I. (2013). The meaning of ‘relevance’ in science education and its implications for the science curriculum. Studies in Science Education, 49(1), 1–34. https://doi.org/https://doi.org/10.1080/03057267.2013.802463
   Web of Science ®Google Scholar
• Trench, B. (2008). Towards an analytical framework of science communication models. In D. Cheng, M. Claessens, T. Gascoigne, J. Metcalfe, B. Schiele, & S. Shi (Eds.), Communicating science in social contexts: New models, new practices (pp. 119–135). Springer. https://doi.org/https://doi.org/10.1007/978-1-4020-8598-7_7
   Google Scholar
• Wals, A. E. J., Brody, M., Dillon, J., & Stevenson, R. B. (2014). Convergence between science and environmental education. Science, 344(6184), 583–584. https://doi.org/https://doi.org/10.1126/science.1250515
   PubMed Web of Science ®Google Scholar