An Intelligent ETL Workflow Framework based on data Partition
ETL tool is an important part to build a data warehouse and data centers,. For massive data processing, this paper presents an intelligent ETL workflow framework based on the distributed computing servers, adding an intelligent manipulative module, acquiring the data of the system efficiency and resources, operating data, dynamically adjusting the ETL strategy, and doing corresponding data segmentation for larger jobs, realizing workflow optimization for multi-machine parallel execution, improving operational efficiency, and facilitating error recovery. Intelligent control module is composed of the monitor, knowledge base, and the selector. The source data horizontal partition is the basis and difficulty to achieve multi-machine parallel.
Distributed Computation Intelligence ETL data partition multi-agent
Yingying Tu Chaozhen Guo
The college of Mathematics and Computer Science Fuzhou University Fuzhou, China
国际会议
厦门
英文
358-363
2010-10-29（万方平台首次上网日期，不代表论文的发表时间）