A Five-Star Guide for Achieving Replicability and Reproducibility When Working with GIS Software and Algorithms

References

• Anselin, L., S. J. Rey, and W. Li. 2014. Metadata and provenance for spatial analysis: The case of spatial weights. International Journal of Geographical Information Science 28 (11):2261–80. doi: https://doi.org/10.1080/13658816.2014.917313.
   Web of Science ®Google Scholar
• Atkins, D. E., K. K. Droegemeier, S. I. Feldman, H. Garcia-Molina, M. L. Klein, D. G. Messerschmitt, P. Messina, J. P. Ostriker, and M. H. Wright. 2003. Revolutionizing science and engineering through cyberinfrastructure: Report of the National Science Foundation blue-ribbon advisory panel on cyberinfrastructure. Accessed October 19, 2019. https://www.nsf.gov/cise/sci/reports/atkins.pdf.
   Google Scholar
• Berners-Lee, T. 2009. Linked data: Design issues. Accessed October 11, 2019. https://www.w3.org/DesignIssues/LinkedData.html.
   Google Scholar
• Buchanan, B. P., M. Fleming, R. L. Schneider, B. K. Richards, J. Archibald, Z. Qiu, and M. T. Walter. 2014. Evaluating topographic wetness indices across central New York agricultural landscapes. Hydrology and Earth System Sciences 18 (8):3279–99. doi: https://doi.org/10.5194/hess-18-3279-2014.
   Web of Science ®Google Scholar
• Costello, M. J., W. K. Michener, M. Gahegan, Z.-Q. Zhang, and P. E. Bourne. 2013. Biodiversity data should be published, cited, and peer reviewed. Trends in Ecology & Evolution 28 (8):454–60. doi: https://doi.org/10.1016/j.tree.2013.05.002.
   PubMed Web of Science ®Google Scholar
• Deelman, E., D. Gannon, M. Shields, and I. Taylor. 2009. Workflows and e-Science: An overview of workflow system features and capabilities. Future Generation Computer Systems 25 (5):528–40. doi: https://doi.org/10.1016/j.future.2008.06.012.
   Web of Science ®Google Scholar
• Gallagher, J., J. Orcutt, P. Simpson, D. Wright, J. Pearlman, and L. Raymond. 2015. Facilitating open exchange of data and information. Earth Science Informatics 8 (4):721–39. doi: https://doi.org/10.1007/s12145-014-0202-2.
   Web of Science ®Google Scholar
• Garijo, D., P. Alper, K. Belhajjame, O. Corcho, Y. Gil, and C. Goble. 2014. Common motifs in scientific workflows: An empirical analysis. Future Generation Computer Systems 36:338–51. doi: https://doi.org/10.1016/j.future.2013.09.018.
   Web of Science ®Google Scholar
• Goble, C. A., J. Bhagat, S. Aleksejevs, D. Cruickshank, D. Michaelides, D. Newman, M. Borkum, S. Bechhofer, M. Roos, P. Li et al. 2010. myExperiment: A repository and social network for the sharing of bioinformatics workflows. Nucleic Acids Research 38 (Web Server issue):W677–W682. doi: https://doi.org/10.1093/nar/gkq429.
   PubMed Web of Science ®Google Scholar
• Goodchild, M. F. 2007. Beyond metadata: Towards user-centric description of data quality. Paper presented at the 5th International Symposium on Spatial Data Quality, Enschede, The Netherlands, June 13–15.
   Google Scholar
• Hey, T., S. Tansley, and K. Tolie, eds. 2009. The fourth paradigm: Data-intensive scientific discovery. Redmond, WA: Microsoft Research.
   Google Scholar
• Hutchinson, M. F., and J. C. Gallant. 2011. A differential equation for specific catchment area. Water Resources Research 47 (5):95335. doi: https://doi.org/10.1029/2009WR008540.
   Google Scholar
• Li, W., M. Song, and Y. Tian. 2019. An ontology-driven cyberinfrastructure for intelligent spatiotemporal question answering and open knowledge discovery. ISPRS International Journal of Geo-Information 8 (11):496. doi: https://doi.org/10.3390/ijgi8110496.
   Web of Science ®Google Scholar
• McPhillips, T., S. Bowers, D. Zinn, and B. Ludäscher. 2009. Scientific workflow design for mere mortals. Future Generation Computer Systems 25 (5):541–51. doi: https://doi.org/10.1016/j.future.2008.06.013.
   Web of Science ®Google Scholar
• Missier, P., K. Belhajjame, and J. Cheney. 2013. The W3C PROV family of specifications for modelling provenance metadata. In Proceedings of the 16th International Conference on Extending Database Technology, ed. N. W. Paton, G. Guerrini, B. Catania, M. Castellanos, P. Atzeni, P. Fraternali, and A. Gounaris, 773–76. New York: ACM. doi: https://doi.org/10.1145/2452376.2452478.
   Google Scholar
• Nüst, D., and E. Pebesma. 2020. Practical reproducibility in geography and geosciences. Annals of the American Association of Geographers. doi:https://doi.org/10.1080/24694452.2020.1806028.
   Google Scholar
• Pilesjö, P., and A. Hasan. 2014. A triangular form-based multiple flow algorithm to estimate overland flow distribution and accumulation in a digital elevation model. Transactions in GIS 18 (1):108–24. doi: https://doi.org/10.1111/tgis.12015.
   Web of Science ®Google Scholar
• Qin, C.-Z., B.-B. Ai, A.-X. Zhu, and J.-Z. Liu. 2017. An efficient method for applying a differential equation to deriving the spatial distribution of specific catchment area from gridded digital elevation models. Computers & Geosciences 100:94–102. doi: https://doi.org/10.1016/j.cageo.2016.12.009.
   Web of Science ®Google Scholar
• Qin, C.-Z., X.-W. Wu, J.-C. Jiang, and A.-X. Zhu. 2016. Case-based formalization and reasoning method for knowledge in digital terrain analysis: Application to extracting drainage networks. Hydrology and Earth System Sciences 20 (8):3379–92. doi: https://doi.org/10.5194/hess-20-3379-2016.
   Web of Science ®Google Scholar
• Tullis, A. A., and B. Kar. 2020. Where is the provenance? Ethical replicability and reproducibility in GIScience and its critical applications. Annals of the American Association of Geographers. doi: https://doi.org/10.1080/24694452.2020.1806029.
   Google Scholar
• Wang, S. 2010. A CyberGIS framework for the synthesis of cyberinfrastructure, GIS, and spatial analysis. Annals of the Association of American Geographers 100 (3):535–57. doi: https://doi.org/10.1080/00045601003791243.
   Web of Science ®Google Scholar
• Waters, N. 2020. Motivations and methods for replication in geography: Working with data streams. Annals of the American Association of Geographers. doi:https://doi.org/10.1080/24694452.2020.1806027.
   Google Scholar
• Wilson, J. P. 2018. Environmental applications of digital terrain modelling. Oxford, UK: Wiley-Blackwell.
   Google Scholar
• Wilson, J. P., and P. A. Burrough. 1999. Dynamic modeling, geostatistics, and fuzzy classification: New sneakers for a new geography? Annals of the Association of American Geographers 89 (4):736–46. doi: https://doi.org/10.1111/0004-5608.00173.
   Web of Science ®Google Scholar
• Wilson, J. P., and J. C. Gallant. 2000. Secondary topographic attributes. In Terrain analysis: Principles and applications, ed. J. P. Wilson and J. C. Gallant, 51–85. New York: Wiley.
   Google Scholar
• Wilson, J. P., C. S. Lam, and X. Y. Deng. 2007. Comparison of performance of flow routing algorithms used in geographic information systems. Hydrological Processes 21 (8):1026–44. doi: https://doi.org/10.1002/hyp.6277.
   Web of Science ®Google Scholar
• Zhou, Q., and X. Liu. 2002. Error assessment of grid-based flow routing algorithms used in hydrological models. International Journal of Geographical Information Science 16 (8):819–42. doi: https://doi.org/10.1080/13658810210149425.
   Web of Science ®Google Scholar