Managing the Google Web 1T 5-gram Data Set

摘要：

This paper describes how the Google Web 1T 5- gram data set, contributed by Google Inc., can be stored so that it can be used eciently with respect to time. We present an ecient way of accessing all the 5-grams for a specic word of interest from the stored les. We measure the maximum access and processing eciency achievable for any word of interest. We also compare results (access time and memory requirements) on the task of accessing all the 5-grams for a list of words, on both the processed and the original organization of the data set.

关键词： Google web 1T n-gram 5-grams

作者: Aminul ISLAM Diana INKPEN

作者单位: Department of Computer Science, SITE University of Ottawa Ottawa, ON, Canada

会议类型: 国际会议

会议名称: International Conference on Natural Language Processing and Knowledge Engineering(IEEE自然语言处理与知识工程国际会议 IEEE NLP-KE 2009)

会议地点: 大连

会议语种:英文

页码: 1-5

在线出版日期: 2009-09-24（万方平台首次上网日期，不代表论文的发表时间）