Research on Global BloomFilter-Based Data Routing Strategy of Deduplication in Cloud Environment

Abstract

The application of data deduplication technology reduces the demand for data storage and improves resource utilization. Compared with limited storage capacity and computing capacity of a single node, cluster data deduplication technology has great advantages. However, the cluster data duplication technology also brings new issues on deduplication rate reduction and load balancing of storage nodes. The application of data routing strategy can well balance the problem of deduplication rate and load balancing. The paper introducesa global BloomFilter routing strategy. In order to avoid the communication overhead caused by sending the fingerprints to the data storage node and inquiring about the BloomFilter maintained in the memory, a BloomFilter array is maintained in the memory of the client-server. Each row of the array corresponds to a storage node; before sending the Superchunk to the storage node, it will inquire the BloomFilter array and storage capacity information to get the optimal node. The theoretical analysis and experimental results prove the feasibility of the strategies proposed by this paper.

KEYWORDS:

• Cluster
• Data routing
• Deduplication
• Load balancing

DISCLOSURE STATEMENT

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work is supported by the Industrial field of general projects of science and Technology Department of Shaanxi Province (2023-YBGY-203); Industrialization Project of Shaanxi Provincial Department of Education (21JC017); “Thirteenth Five-Year” National Key R&D Program Project (Project Number: 2019YFD1100901).

Notes on contributors

Qinlu He

Qinlu He is an associate professor at Xi’an University of Architecture and Technology. He received PhD degree in computer science from Northwestern Polytechnical University. His current research interests include data deduplication, cloud storage, and distributed file systems. He has more than 20 publications in journals and international conferences. He is a member of the IEEE and China Computer Federation. Corresponding author. Email: [email protected]

Zhen Li

Zhen Li is a senior engineer of Shaanxi Institute of Metrology. Research direction: reliability test of electronic and electrical products, EMC electromagnetic compatibility, computer information network inspection and testing. More than 10 papers of various types have been published in journals and international conferences. He is member of Intelligent Building and Building Automation Professional Committee of Shaanxi Institute of Automation. Email: [email protected]

Chen Chen

Chen Chen was born in Xi’an, PR China. He is an enigineer in the Network Information Department at The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an, P.R.China. His scope of work includes hospital informatization construction, artificial intelligence and medical data analysis. E-mail: [email protected]

Hao Feng

Feng hao is a senior engineer of SHAAN XI Big Data Group Co, Ltd. His main research interests include network resilience, complex system reliability and big data techonolgy. E-mail: [email protected]