Log Data Real Time Analysis Using Big Data Analytic Framework with Storm and Hadoop

摘要：

　　The log data real-time processing platform which is built using Storm On YARN integrated MapReduce and Storm that use MapReduce to complete large-scale off-line data global knowledge extraction,sudden knowledge extraction of small-scale data in Kafka buffers through Storm,and continuous real-time calculation of streaming data in combination with global knowledge.We tested our technique with the well-known KDD99 CUP data set.The experimentation results prove the system to be effective and efficient.

作者: LV Jia-Ke LI Yang WANG Xuan

作者单位: College of computer and information science,Southwest University,Chongqing,China;The Key laboratory College of computer and information science,Southwest University,Chongqing,China

会议类型: 国际会议

会议名称: 2018 International Symposium on Water System Operations(ISWRSO 2018)(2018年水资源系统及调度国际研讨会)

会议地点: 北京

会议语种:英文

页码: 1-5

在线出版日期: 2018-10-12（万方平台首次上网日期，不代表论文的发表时间）