Performance evaluation of K-means clustering on Hadoop infrastructure

Abstract

Today we are living with the extensive volume of information which is developing at a very fast pace. Clustering is the process of group similar kinds of information. The serial k-means clustering method takes a large amount of computational time when it is applied to enormous data sets which are large in size of tera and petabyte. In this paper, we evaluated the performance of the K-means algorithm in different ways like K-mean simple (using java codes on MapReduce), K-means (using java codes on MapReduce) with IEC (Initial Equidistant Centres), K-mean on Mahout (using Machine learning library) and K-mean on Spark (Machine learning library) on static IP address data sets over the MapReduce framework. In addition to this we also compare the K-mean simple and K-mean (IEC) on different iteration level. Outcome shows on the better performance on the basis of time on different infrastructure and it also defines the behaviour of K-means algorithms on the basis of centroids and different iteration levels.

Subject Classification:

• Big Data analytics

Keywords:

• MapReduce
• Hadoop
• HDFS
• K-means
• Clustering
• Big Data
• Spark
• Mahout