Modelling and implementing big data warehouses for decision support

Abstract

In the era of Big Data, many NoSQL databases emerged for the storage and later processing of vast volumes of data, using data structures that can follow columnar, key-value, document or graph formats. For analytical contexts, requiring a Big Data Warehouse, Hive is used as the driving force, allowing the analysis of vast amounts of data. Data models in Hive are usually defined taking into consideration the queries that need to be answered. In this work, a set of rules is presented for the transformation of multidimensional data models into Hive tables, making available data at different levels of detail. These several levels are suited for answering different queries, depending on the analytical needs. After the identification of the Hive tables, this paper summarizes a demonstration case in which the implementation of a specific Big Data architecture shows how the evolution from a traditional Data Warehouse to a Big Data Warehouse is possible.

Keywords:

• big data
• data model
• data warehouse
• hive
• NoSQL

Acknowledgements

A component of this work was previously presented at Confenis’2016 held in Vienna, Austria, co-organized by Austrian Computer Society. Some of the figures in this paper use icons made by Freepik, from www.flaticon.com.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT (Fundação para a Ciência e Tecnologia) within the Project Scope: UID/CEC/00319/2013. This work has been funded by the SusCity project (MITP-TB/CS/0026/2013) and by Portugal Incentive System for Research and Technological Development, Project in co-promotion no 002814/2015 (iFACTORY 2015-2018).