Towards Model Based Approach to Hadoop Deployment and Configuration

摘要：

　　Hadoop is an open source software framework of distributed processing of big data.There are many kinds of services in Hadoop ecosystem, such as HDFS, Map-Reduce,HBase, Hive, Yarn, Flume, Spark, Storm, Zookeeper, and so on, which increase the complexity of deployment and configuration.It takes plenty of time to construct a Hadoop cluster.Although there are some management tools which help administrators deploy and configure Hadoop clusters automatically, they usually provide a fixed solution.So administrators couldnt construct their Hadoop clusters according to different management requirements by the tools.Software architecture acts as a bridge between requirements and implementations.It has been used to reduce the complexity and cost mainly resulted from the difficulties faced by understanding the large-scale and complex software system.This paper proposes a model based approach to Hadoop deployment and configuration which help administrators construct Hadoop clusters in a simple but powerful enough manner.First, we provide the unified models of Hadoop software architecture, according to the domain knowledge of current Hadoop deployment and configuration.Second, we provide a framework with a set of definable rules for domain experts to describe their solutions to deploy and configure Hadoop clusters.Thus, administrators can use various custom solutions to automatically deploy and configure their Hadoop clusters according to different management requirements.In addition, a real-world experiment demonstrates the feasibility,effectiveness and benefits of the new approach to Hadoop deployment and configuration.

关键词： Hadoop Deployment and Configuration Software Architecture Models at Runtime

作者: Yicheng Huang Xingtu Lan Xing Chen Wenzhong Guo

作者单位: College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350108, China;Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing

会议类型: 国际会议

会议名称: The 12th Web Information System and Application Conference第十二届全国Web信息系统及其应用学术会议(WISA2015)、全国第十次语义Web 与本体论学术研讨会(SWON2015)、全国第九次电子政务技术及应用学术研讨会(EGTA2015)

会议地点: 济南

会议语种:英文

页码: 79-84

在线出版日期: 2015-09-11（万方平台首次上网日期，不代表论文的发表时间）