Mining Contents in Web Page Using Cosine Similarity
Web pages typically contain a large amount of information that is not part of the main contents of the pages, e.g.; banner ads, navigation bars, copy right and privacy notices, advertisements which are not related to the main content (relevant information). In this paper, an algorithm is proposed that extract the main content from the web documents. The algorithm based on Content Structure Tree (CST). Firstly, the proposed system use HTML Parser to construct DOM (Document Object Model) tree from which construct Content Structure Tree (CST) which can easily separate the main content blocks from the other blocks. The proposed system then introduce cosine similarity measure to evaluate which parts of the CST tree represent the less important and which parts represent the more important of the page. The proposed system can define the ranking of the documents using similarity values and also extracts the top ranked documents as more relevant to the query.
DOM tree CST tree Cosine Similarity
Swe Swe Nyein
University of Computer Studies, Mandalay Mandalay, Myanmar
国际会议
上海
英文
472-475
2011-03-11（万方平台首次上网日期，不代表论文的发表时间）