Research on the Page Replacement Model in Search Engine Collector

摘要：

The method of repeat URL filtering for the existing search engine collector is analyzed, and some shortcomings are pointed out.On the base of virtual memory page replacement algorithm in operation system,a page replacement model is introduced the search engine collector.And then the disk page structure and memory page structure are designed respectively.Finally,a fingerprint search algorithm is given.Through the practical application of the models and technologies in our projects,we find they can solve the speed problem for filtering tens of thousands of URL in the small-capacity memory.

关键词： search engine collector page-replacement filter finger mark URL

作者: Meiren Zhang Yongfeng Li Yongfeng Li

作者单位: School of Mathematics and Information engineering,Taizhou University,Linhai,Zhejiang 317000,China School of Computer Science and Techenology,WuHan University of Technology ,Wuhan,Hubei 430070,China

会议类型: 国际会议

会议名称: 2008年国际电子商务、工程及科学领域的分布式计算和应用学术研讨会(2008 International Symposium on Distributed Computing and Applications for Business Engineering and Science)

会议地点: 大连

会议语种:英文

页码: 1187-1192

在线出版日期: 2008-07-27（万方平台首次上网日期，不代表论文的发表时间）