An Analysis of Distributed Programming Models and Frameworks for Large-scale Graph Processing

ABSTRACT

In recent years, processing and analysing large graphs has become a major need in many research areas. Distributed graph processing programming models and frameworks arised as a natural solution to process linked data of large volumes, such as data originating from social media. These solutions are distributed by design and help developers to perform operations on the graph, sometimes reaching almost real-time performance even on huge graphs. Some of the available graph processing frameworks exploit generic data processing models, like MapReduce, while others were specifically built for graph processing, introducing techniques such as vertex or edge partitioning and graph-oriented programming models. In this work, we analyse the properties of recent and widely popular frameworks – from the perspective of the adopted programming model – designed to process large-scale graphs with the goal of assisting software developers/designers in choosing the most adequate tool.

Keywords:

• Distributed graph-processing frameworks
• Large-scale graphs
• Programming models

DISCLOSURE STATEMENT

No potential conflict of interest was reported by the authors.

Notes

1 Hadoop Web Page https://hadoop.apache.org/

2 Graph spectral analysis is the analysis of graph properties based on the spectrum of its defining matrices, such as eigenvectors obtained from the adjacency matrix.

3 Apache Giraph Web Page, http://giraph.apache.org/

4 jPregel Web Page, http://kowshik.github.com/JPregel/

5 GoldenOrb Web Page, http://www.goldenorbos.org/

6 Phoebus Web Page, https://github.com/xslogic/phoebus

Additional information

Notes on contributors

Alejandro Corbellini

Alejandro Corbellini received his PhD in computer science in 2016 at UNICEN University. He is also a graduate teaching assistant at UNICEN. His main research interests are recommender systems, distributed programming and large-scale data mining. Email: [email protected]

Daniela Godoy

Daniela Godoy is a full-time professor in the Computer Science Department at UNICEN, member of ISISTAN Research Institute and researcher at CONICET. Her research interests include recommender systems, social networksand text mining. Email: [email protected]

Cristian Mateos

Cristian Mateos is a full-time professor in the Computer Science Department at UNICEN, member of ISISTAN Research Institute and researcher at CONICET. His main research interest are parallel/distributed programming, distributed middlewares and service-oriented computing.

Silvia Schiaffino

Silvia Schiaffino is a full-time professor in the Computer Science Department at UNICEN, member of ISISTAN Research Institute and researcher at CONICET. Her main research interests are intelligent agents, personalisation, recommender systems, and data mining. Email: [email protected]

Alejandro Zunino

Alejandro Zunino is a full-time professor in the Computer Science Department at UNICEN, member of ISISTAN Research Institute and researcher at CONICET.. His research areas include grid computing, service-oriented computing, semantic web services, and mobile computing. Email: [email protected]