A Ceph-based storage strategy for big gridded remote sensing data

References

• Aghayev, A., Weil, S., Kuchnik, M., Nelson, M., Ganger, G. R., & Amvrosiadis, G. (2019). File systems unfit as distributed storage backends: Lessons from 10 years of ceph evolution. In Proceedings of the 27th acm symposium on operating systems principles, New York, NY, USA, (pp. 353–369).
   Google Scholar
• Arafa, Y., Barai, A., Zheng, M., & Badawy, A.-H. A. (2018). Evaluating the fault tolerance performance of hdfs and ceph. In Proceedings of the practice and experience on advanced research computing, New York, NY, USA, (pp. 1–3).
   Google Scholar
• Barclay, T., Gray, J., & Slutz, D. (2000). Microsoft terraserver: A spatial data warehouse. In Proceedings of the 2000 acm sigmod international conference on management of data, New York, NY, USA, (pp. 307–318).
   Google Scholar
• Benet, J. (2014). Ipfs-content addressed, versioned, p2p file system. arXiv Preprint, arXiv, 1407.3561.
   Google Scholar
• Borthakur, D. (2008). HDFS architecture guide. Hadoop Apache Project, 53(1–13), 2.
   Google Scholar
• Carstoiu, D., Cernian, A., & Olteanu, A. (2010). Hadoop hbase-0.20. 2 performance evaluation. In 4th international conference on new trends in information science and service science, Gyeongju, Korea (South), (pp. 84–87).
   Google Scholar
• Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., … Gruber, R. E. (2008). Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS), 26(2), 1–26.
   Web of Science ®Google Scholar
• Cheng, L., Kotoulas, S., Ward, T. E., & Theodoropoulos, G. (2014). Robust and efficient large-large table outer joins on distributed infrastructures. In European conference on parallel processing, Switzerland, (pp. 258–269).
   Google Scholar
• Davies, A., & Fisk, H. (2006). MySQL clustering. Indianapolis, Indiana, USA: Sams Publishing.
   Google Scholar
• Galić, Z., Mešković, E., & Osmanović, D. (2017). Distributed processing of big mobility data as spatio-temporal data streams. Geoinformatica, 21(2), 263–291.
   Web of Science ®Google Scholar
• Hajjaji, Y., & Farah, I. R. (2018). Performance investigation of selected NoSQL databases for massive remote sensing image data storage. In 2018 4th international conference on advanced technologies for signal and image processing (atsip), Sousse, Tunisia, (pp. 1–6).
   Google Scholar
• Hu, F., Yang, C., Jiang, Y., Li, Y., Song, W., Duffy, D. Q., … Lee, T. (2020). A hierarchical indexing strategy for optimizing apache spark with hdfs to efficiently query big geospatial raster data. International Journal of Digital Earth, 13(3), 410–428.
   Web of Science ®Google Scholar
• Jhummarwala, A., Potdar, M., & Chauhan, P. (2014). Parallel and distributed GIS for processing geo-data: An overview. International Journal of Computer Applications, 106(16), 18602–19881.
   Google Scholar
• Jing, W., & Tian, D. (2018). An improved distributed storage and query for remote sensing data. Procedia Computer Science, 129, 238–247.
   Google Scholar
• Li, J., & Narayanan, R. M. (2004). Integrated information mining and image retrieval in remote sensing. Recent Advances in Hyperspectral Signal and Image Processing, 1, 449–478.
   Google Scholar
• Li, Q., Lu, Y., Gong, X., & Zhang, J. (2014). Optimizational method of hbase multidimensional data query based on hilbert space-filling curve. In 2014 ninth international conference on p2p, parallel, grid, cloud and internet computing, Guangdong, China, (pp. 469–474).
   Google Scholar
• Liu, X., Hao, L., & Yang, W. (2019). Bigeo: A foundational paas framework for efficient storage, visualization, management, analysis, service, and migration of geospatial big dataa case study of Sichuan province, China. ISPRS International Journal of Geo-Information, 8(10), 449.
   Web of Science ®Google Scholar
• Lv, Z., Li, X., Lv, H., & Xiu, W. (2019). Bim big data storage in webvrgis. IEEE Transactions on Industrial Informatics, 16(4), 2566–2573.
   Web of Science ®Google Scholar
• Ma, Y., Li, G., Yao, X., Cao, Q., Zhao, L., Wang, S., & Zhang, L. (2021). A precision evaluation index system for remote sensing data sampling based on hexagonal discrete grids. ISPRS International Journal of Geo-Information, 10(3), 194.
   Web of Science ®Google Scholar
• Martinho, N., Almeida, J.-P. D., Simões, N. E., & Sá-Marques, A. (2020). Urbanwater: Integrating epanet 2 in a postgresql/postgis-based geospatial database management system. ISPRS International Journal of Geo-Information, 9(11), 613.
   Web of Science ®Google Scholar
• Nishimura, S., Das, S., Agrawal, D., & El Abbadi, A. (2011). Md-hbase: A scalable multidimensional data infrastructure for location aware services. In 2011 ieee 12th international conference on mobile data management, Lulea, Sweden, (Vol.1, pp. 7–16).
   Google Scholar
• Rajak, R., Raveendran, D., Bh, M. C., & Medasani, S. S. (2015). High resolution satellite image processing using Hadoop framework. In 2015 ieee international conference on cloud computing in emerging markets (ccem), Bangalore, India, (pp. 16–21).
   Google Scholar
• Salanio, K. A. E., Santos, C., Magturo, R., Quevedo, G. P., Virtucio, K., Langga, K., … Paringit, E. (2015). Development of data archiving and distribution system for the Philippines’ lidar program using object storage systems. In Free and open source software for geospatial (foss4g) conference proceedings, Seoul, South Korea, (Vol. 15, p. 53).
   Google Scholar
• Wang, L., Cheng, C., Wu, S., Wu, F., & Teng, W. (2015). Massive remote sensing image data management based on hbase and geosot. In 2015 ieee international geoscience and remote sensing symposium (igarss) (pp. 4558–4561).
   Google Scholar
• Wang, S., Zhong, Y., & Wang, E. (2019). An integrated GIS platform architecture for spatiotemporal big data. Future Generation Computer Systems, 94, 160–172.
   Web of Science ®Google Scholar
• Wang, X., Wang, R., Zhan, W., Yang, B., Li, L., Chen, F., & Meng, L. (2020). A storage method for remote sensing images based on google s2. IEEE Access, 8, 74943–74956.
   Web of Science ®Google Scholar
• Wang, X., Zhang, H., Zhao, J., Lin, Q., Zhou, Y., & Li, J. (2015). An interactive web-based analysis framework for remote sensing cloud computing. In ISPRS annals of photogrammetry, remote sensing and spatial information sciences, Milan, Italy, (pp. 43–50). IEEE.
   Google Scholar
• Wang, Y., & Wang, S. (2010). Research and implementation on spatial data storage and operation based on Hadoop platform. In 2010 second iita international conference on geoscience and remote sensing, Qingdao, China, (Vol.2, pp. 275–278).
   Google Scholar
• Weil, S., Brandt, S., Miller, E., & Maltzahn, C. (2006). Crush: Controlled, scalable, decentralized placement of replicated data. In Sc’06: Proceedings of the 2006 acm/ieee conference on supercomputing, Tampa, FL, USA, (pp. 31–31).
   Google Scholar
• Weil, S. A., Brandt, S. A., Miller, E. L., Long, D. D., & Maltzahn, C. (2006). Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7th symposium on operating systems design and implementation, Berkeley, CA, USA, (pp. 307–320).
   Google Scholar
• Weipeng, J., Dongxue, T., Guangsheng, C., & Yiyuan, L. (2018). Research on improved method of storage and query of large-scale remote sensing images. Journal of Database Management (JDM), 29(3), 1–16.
   Web of Science ®Google Scholar
• Xu, C., Du, X., Yan, Z., & Fan, X. (2020). Scienceearth: A big data platform for remote sensing data processing. Remote Sensing, 12(4), 607.
   Web of Science ®Google Scholar
• Yan, S., Yao, X., Zhu, D., Liu, D., Zhang, L., Yu, G., … Yun, W. (2021). Large-scale crop mapping from multi-source optical satellite imageries using machine learning with discrete grids. International Journal of Applied Earth Observation and Geoinformation, 103, 102485.
   Web of Science ®Google Scholar
• Yao, X., Li, G., Xia, J., Ben, J., Cao, Q., Zhao, L., … Zhu, D. (2020). Enabling the big earth observation data via cloud computing and dggs: Opportunities and challenges. Remote Sensing, 12(1), 62.
   Web of Science ®Google Scholar
• Zeiler, M. (1999). Modeling our world: The esri guide to geodatabase design, Redlands, California, USA, (Vol. 40). ESRI.
   Google Scholar
• Zhang, L., Sun, J., Su, S., Liu, Q., & Liu, J. (2020). Uncertainty modeling of object-oriented biomedical information in hbase. IEEE Access, 8, 51219–51229.
   Web of Science ®Google Scholar
• Zhang, X., Gaddam, S., & Chronopoulos, A. (2015). Ceph distributed file system benchmarks on an openstack cloud. In 2015 ieee international conference on cloud computing in emerging markets (ccem), Bangalore, India, (pp. 113–120).
   Google Scholar
• Zhou, H., Chen, X., Wu, X., Dai, W., He, F., & Zhou, Z. (2019, CN110491478-A 22 Nov 2019 G16H-030/20 201998 Pages: 8 Chinese). Ceph based image file distributed storage system has metadata storage module which is configured to store related information of image file, including unique identification and metadata information of image file. (IDS Number: 2019A0226F)
   Google Scholar