A framework for big data pre-processing and search optimization using HMGA-ACO: a hierarchical optimization approach

ABSTRACT

The huge potential associated with big data has prompted a developing research field that has rapidly attracted huge enthusiasm from differing sectors. Unlike traditional databases, advanced for quick access and summarization of structured data and all around characterized inquiries, big data is accepted to fill in as a raw material for the creation of new knowledge. We look at the complexity placed by big search spaces, dominated by the number of variables and domain of each variable, in search and optimization problems. While an extensive, even unbounded, search area disables the effectiveness and efficiency of search, a complex structure of constraints additionally increases the difficulty in that the search space becomes highly unpredictable. In order to overcome the above issues we propose a novel Hierarchical Manipulated Genetic Algorithm with Ant Colony Optimization with data pre-processing which can possibly improve the data and recover the data with more accuracy and precision. The proposed hierarchical optimization can help to boost the speed of search, and the exertion of search is reduced with the utilization of pre-processing.

KEYWORDS:

• Big data
• pre-processing
• hierarchical optimization
• Manipulated Genetic Algorithm with Ant Colony Optimization (MGA-ACO)

Notes on contributors

K. V. Rama Satish received his bachelor of Science degree in Computer Science from Bangalore University, master degree in Computer Applications (MCA) from IGNOU, New Delhi, and MPhil degree in Computer Science from Annamalai University. He has been working as assistant professor, RNSIT, Bangalore. His area of interests include Cloud Computing, Data Sciences, Web Technologies, and Networking Security.

N. P. Kavya received her bachelor of Engineering degree in Computer Science from Bangalore University, MS degree in Software Systems from BITS, Pilani, Rajastan, and PhD degree in Data Mining from VTU, Belgaum, Karnataka. She has been working as a professor and Head of Department of Master of Computer Application, RNSIT, Bangalore. While guiding the students, her main research interests include Data Sciences, Image processing, Advanced Algorithms, Software Engineering, and Architecture.

Disclosure statement

No potential conflict of interest was reported by the authors.