An Optimization Strategy of Shard on Elasticsearch
With the development of big data application,there are more demands on big data storage and retrieval.Thus,Elasticsearch,a distributed full-text search engine,has appeared,which can well meet these demands.However,Elasticsearch has the following disadvantages: First,shard settings are based on user experience and may degrade retrieval performance due to human factors.Second,the factors considered in the distribution strategy of shards are incomplete.And the last,without effective processing of concurrent access to hot index data,the average performance of each node in the Elasticsearch cluster varies greatly.In this paper,we propose a shard optimization strategy of Elasticsearch,through data and performance analysis,to obtain reasonable shard settings.After that,the shards are placed in nodes with better performance which are evaluated by the linear weighting method.Then,the optimized load balancing strategy will migrate hot shards caused by hot data to make the cluster load balanced.The experimental results show that the proposed shard optimization strategy can achieve better index retrieval performance and better cluster load.
ElasticSearch Index Shard Load balancing Mathematical modeling Linear weighting method
Zhanglong Wang Yang Pi
School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing,400065,China;Chongqing Engineering Research Center of Mobile Internet Data Application,Chongqing,400065,China
国际会议
重庆
英文
13-21
2019-05-30（万方平台首次上网日期，不代表论文的发表时间）