Big spatial vector data management: a review

Figures & data

Table 1. New thinking about GIS in the era of big data.

New concepts and ideas	Keywords or highlights	Contributors
Big geo data	The quality of big (geo) data and three methods for quality improvement and assurance	Goodchild (2013)
Big data GIS	Big data GIS is from offline analysis/ model-driven/ simple visualization to dynamic/ data-driven/ visual analysis in real time	Li and Li (2014)
Generalized GIS	The characteristics of generalized GIS and technical challenges from data collection and cleaning, management and integration, data analysis, and computing	Lu and Zhang (2014)
Open GIS	Eight dimensions of open GIS in big data era	Sui, Ye, and Gan (2014)
Big spatial data/Spatial big data	Big data technologies for systems; traditional /emerging spatial big data with different data types; metrics for spatial big data benchmarks	Shekhar et al. (2014), Eldawy and Mokbel (2015a)
Urban computing	A process of acquisition, integration, and analysis of big and heterogeneous data generated by a diversity of sources in urban spaces to tackle the major issues that cities face	Zheng (2015, 2017)
Social sensing	A big geo-data-based approach to understand our socioeconomic environments	Liu et al. (2015)
Data-driven geography	From a data-scarce to a data-rich environment and use big data to inform both geographic knowledge-discovery and spatial modeling	Miller and Goodchild (2015)
Geospatial big data	Geospatial big data refers to spatial data-sets exceeding capacity of current computing systems	Lee and Kang (2015)
New definition for GIS	A multiple discipline science and technology, integrating data acquisition, extraction, management, knowledge discovering, spatial sensing and recognition, as well as intelligent location-based services of any physical objects and human activities around the earth and its environment	Li (2016)
Big earth data	Big Earth Data from space can be defined as big data obtained via the methodologies of Earth observation, such as data from space borne, airborne, and ground sensors	Guo, Wang, and Liang (2016), Guo (2017)
Spatio-temporal big data/Big spatio-temporal data	Big data is big spatio-temporal data essentially	Alarabi and Mokbel (2017), Wang (2017)
	Beside the huge achieved volume of the data, space and time are two fundamental characteristics that raise the demand for processing spatio-temporal data
Remote sensing big data	Data infrastructure for remote sensing big data: integration, management and on-demand service. The RS “Big Data” not merely refers to the volume and velocity of data that outstrip the storage and computing capacity, but also the variety and complexity of the RS data	Li and Huang (2017), Ma et al. (2015)

Figure 1. The classification diagram of big spatial data.

Figure 2. Big spatial vector data management.

Table 2. Existing frameworks or systems for big spatial vector data.

Name	Architecture	Spatial index	Spatial query	Contributors
MD-HBase	HBase	Quad Tree; K-d tree	Range Query; KNN	Nishimura et al. (2011)
CloST	Hadoop	R-tree	Range Query; Spatial Join; KNN	Tan, Luo, and Ni (2012)
Hadoop-GIS	Hadoop	Grid	Range Query; Spatial Join; KNN	Aji et al. (2013)
VegaSTDE	Hadoop HBase	Quad Tree	ST-Range Query	Zhong, Fang, and Zhao (2013)
SpatialHadoop	Hadoop	R-tree; Quad-tree; Grid; K-d tree	Range Query; Spatial Join; KNN	Eldawy and Mokbel (2013)
Parallel SECONDO	Parallel DB	−	Range Query; Spatial Join	Lu and Guting (2014)
ScalaGiST	Hadoop	B^+-tree; R-tree	Range Query; KNN	Lu et al. (2014)
GeoMesa	Hbase	GeoHash	Range Query	Hughes et al. (2015)
GeoSpark	Spark	R-tree; Quad-tree	Range Query; Spatial Join; KNN	Yu, Wu, and Sarwat (2015)
Sphinx	Parallel DB	R-tree; Quad tree	Range Query; Spatial Join	Eldawy, Elganainy, et al. (2015)
Simba	Spark	R-tree	Range Query; Spatial Join; KNN	Xie et al. (2016)
ST-Hadoop	Hadoop	ST-index	ST-range query; ST-join	Alarabi and Mokbel (2017)
LandQ^v1	Parallel DB	Grid	Range Query	Yao, Yang, et al. (2017)

Figure 3. Big spatial vector data model.

Figure 4. The structure of big spatial vector data, GeoCSV.

Figure 5. Distributed spatial index for big spatial vector data.

Figure 6. Data processing and analysis of BSVD.

Figure 7. Tile pyramid model for map visualization.

Goodchild, M. F. (2013). The quality of big (geo)data. Dialogues in Human Geography , 3 (3), 280–284. doi:10.1177/2043820613513392

 Google Scholar

Li, Q. , & Li, D. (2014). Big data GIS. Wuhan Daxue Xuebao (Xinxi Kexue Ban)/Geomatics and Information Science of Wuhan University , 39 (6), 641–646. doi:10.13203/j.whugis20140150

 Google Scholar

Lu, F. , & Zhang, H. (2014). Big data and generalized GIS. Wuhan Daxue Xuebao (Xinxi Kexue Ban)/Geomatics and Information Science of Wuhan University , 39 (6), 645–654. doi:10.13203/j.whugis20140148

 Google Scholar

Sui, D. , Ye, X. , & Gan, T. (2014). Open GIS for big data: Opportunities and impediments. Progress in Geography , 33 (6), 723–737. doi:10.11820/dlkxjz.2014.06.001

 Google Scholar

Shekhar, S. , Evans, M. R. , Gunturi, V. , Yang, Y. , & Cugler, D. C. (2014). Benchmarking spatial big data. Paper presented at the 2nd Workshop on Specifying Big Data Benchmarks, Pune, India, December 17–18.

 Google Scholar

Eldawy, A. , & Mokbel, M. F. (2015a). The era of big spatial data. Paper presented at the 31st IEEE International Conference on Data Engineering Workshops, Seoul, South Korea, April 13–17.

 Google Scholar

Liu, Y. , Liu, X. , Gao, S. , Gong, L. , Kang, C. , Zhi, Y. , … Shi, L. (2015). Social sensing: A new approach to understanding our socioeconomic environments. Annals of the Association of American Geographers , 105 (3), 512–530. doi:10.1080/00045608.2015.1018773

 Web of Science ®Google Scholar

Miller, H. J. , & Goodchild, M. F. (2015). Data-driven geography. GeoJournal , 80 (4), 449–461. doi:10.1007/s10708-014-9602-6

 Web of Science ®Google Scholar

Lee, J.-G. , & Kang, M. (2015). Geospatial big data: Challenges and opportunities. Big Data Research , 2 (2), 74–81. doi:10.1016/j.bdr.2015.01.003

 Web of Science ®Google Scholar

Li, D. (2016). Towards geo-spatial information science in big data era. Cehui Xuebao/Acta Geodaetica et Cartographica Sinica , 45 (4), 379–384. doi:10.11947/j.AGCS.2016.20160057

 Google Scholar

Guo, H. , Wang, L. , & Liang, D. (2016). Big Earth Data from space: A new engine for Earth science. Science Bulletin , 61 (7), 505–513. doi:10.1007/s11434-016-1041-y

 Web of Science ®Google Scholar

Guo, H. (2017). Big Earth data: A new frontier in Earth and information sciences. Big Earth Data , 1 (1–2), 4–20. doi:10.1080/20964471.2017.1403062

 Google Scholar

Alarabi, L. , & Mokbel, M. F. (2017). A demonstration of ST-hadoop: A MapReduce framework for big spatio-temporal data. Proceedings of the VLDB Endowment , 10 (12), 1961–1964. doi:10.14778/3137765.3137819

 Web of Science ®Google Scholar

Wang, J. (2017). Cartography in the age of spatio-temporal big data. Cehui Xuebao/Acta Geodaetica et Cartographica Sinica , 46 (10), 1226–1237. doi:10.11947/j.AGCS.2017.20170308

 Google Scholar

Li, G. , & Huang, Z. (2017). Data infrastructure for remote sensing big data: Integration, management and on-demand service. Jisuanji Yanjiu yu Fazhan/Computer Research and Development , 54 (2), 267–283. doi:10.7544/issn1000-1239.2017.20160837

 Google Scholar

Ma, Y. , Wu, H. , Wang, L. , Huang, B. , Ranjan, R. , Zomaya, A. , & Jie, W. (2015). Remote sensing big data computing: Challenges and opportunities. Future Generation Computer Systems , 51 , 47–60. doi:10.1016/j.future.2014.10.029

 Web of Science ®Google Scholar

Nishimura, S. , Das, S. , Agrawal, D. , & El Abbadi, A. (2011). MD-HBase: A scalable multi-dimensional data infrastructure for location aware services. Paper presented at the 12th IEEE International Conference on Mobile Data Management, Lulea, Sweden, June 6–9.

 Google Scholar

Tan, H. , Luo, W. , & Ni, L. M. (2012). CloST: A hadoop-based storage system for big spatio-temporal data analytics. Paper presented at the Proceedings of the 21st ACM international conference on Information and knowledge management, Maui, Hawaii, USA, October 29–November 2.

 Google Scholar

Aji, A. , Wang, F. , Vo, H. , Lee, R. , Liu, Q. , Zhang, X. , & Saltz, J. (2013). Hadoop-GIS: A high performance spatial data warehousing system over MapReduce. Proceedings of the VLDB Endowment , 6 (11), 1009–1020.10.14778/2536222

 Google Scholar

Zhong, Y. , Fang, J. , & Zhao, X. (2013). VegaIndexer: A distributed composite index scheme for big spatio-temporal sensor data on cloud. Paper presented at the 33rd IEEE International Geoscience and Remote Sensing Symposium, IGARSS, Melbourne, VIC, Australia, July 21–26.

 Google Scholar

Eldawy, A. , & Mokbel, M. F. (2013). A demonstration of spatialhadoop: An efficient mapreduce framework for spatial data. Proceedings of the VLDB Endowment , 6 (12), 1230–1233. doi:10.14778/2536274.2536283

 Google Scholar

Lu, J. , & Guting, R. H. (2014). Parallel SECONDO: A practical system for large-scale processing of moving objects. Paper presented at the 30th IEEE International Conference on Data Engineering , Chicago, IL, USA. March 31–April 04.

 Google Scholar

Lu, P , Chen, G. , Ooi, B. C. , Vo, H. T. , & Wu, S. (2014). ScalaGiST: Scalable generalized search trees for mapreduce systems [innovative systems paper]. Proceedings of the VLDB Endowment , 7 (14), 1797–1808. doi:10.14778/2733085.2733087

 Google Scholar

Hughes, J. N. , Annex, A. , Eichelberger, C. N. , Fox, A. , Hulbert, A. , & Ronquest, M. (2015). GeoMesa: A distributed architecture for spatio-temporal fusion. Paper presented at the Geospatial Informatics, Fusion, and Motion Video Analytics V, Baltimore, MD, USA, April 20–21.

 Google Scholar

Yu, J. , Wu, J. , & Sarwat, M. (2015). GeoSpark: A cluster computing framework for processing large-scale spatial data. Paper presented at the Proceedings of the 23rd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, Washington, November 3–6.

 Google Scholar

Eldawy, A. , Elganainy, M. , Bakeer, A. , Abdelmotaleb, A. , & Mokbel, M. F. (2015). Sphinx: Distributed execution of interactive SQL queries on big spatial data. Paper presented at the 23rd ACM Sigspatial International Conference on Advances In Geographic Information Systems, Seattle, WA, November 3–6.

 Google Scholar

Xie, D. , Li, F. , Yao, B. , Li, G. , Chen, Z. , Zhou, L. , & Guo, M. (2016). Simba: Spatial in-memory big data analysis. Paper presented at the 24th ACM SIGSPATIAL International Conference on Advances In Geographic Information Systems, San Francisco, CA, USA, October 31–November 3.

 Google Scholar

Yao, X. , Yang, J. , Li, L. , Yun, W. , Zhao, Z. , Ye, S. , & Zhu, D. (2017). LandQv1: A GIS cluster-based management information system for arable land quality big data. Paper presented at the 6th International Conference on Agro-Geoinformatics, Fairfax, VA, USA, August 7–10.

 Google Scholar

Data availability statement

Data sharing is not applicable to this article as no new data were created or analysed in this study.