Distributed Data Load Balancing for Scalable Key-Value Cache Systems
In recent years,in-memory key-value cache systems have become increasingly popular in tackling real-time and interactive data processing tasks.Caching systems are often used to help with the temporary storage and processing of data.Due to skewed and dynamic workload patterns,e.g.data increase/decrease or request changes in read/write ratio,it can cause load imbalance and degrade performance of caching systems.Migrating data is often essential for balancing load in distributed storage systems.However,it can be difficult to determine when to move data,where to move data,and how much data to move.This depends on the resources required,e.g.CPU,memory and bandwidth,as well as polices on data movement.Since frequent and global rebalance of systems may affect the QoS of applications utilizing caching systems,it is necessary to minimize system imbalances whilst considering the total migration cost.We propose a novel distributed load balancing method for the mainstream Cloud-based data framework(Redis Cluster).We show how distributed graph clustering through load balancing can be used to exploit varying rebalancing scenarios comprising local and global needs.During the rebalancing process,three phrases are adopted — random walk matching load balancing,local round-robin migration and data migration between the trigger node and new added servers.Our experiments show that the proposed approach can reduce migration time compared with other approach by 30s and load imbalance degree can be reduced by 4X when the locality degree reaches 50%whilst achieving high throughput.
Key-value cache Load balancing Random locality
Shanshan Chen Xudong Zhou Guiping Zhou Richard O.Sinnott
School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing,China;Departme School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing,China School of Computing and Information Systems,University of Melbourne,Melbourne,Australia
国际会议
the 12th Conference on Advanced Computer Architecture?(ACA 2018)(2018年全国计算机体系结构学术年会)
辽宁营口
英文
181-194
2018-08-10（万方平台首次上网日期，不代表论文的发表时间）