关键词: 主题爬虫, 子空间, 语义分析, 支持向量机

Abstract:

Massive web and its rapid growth make it difficult for generalpurpose search engines to provide satisfactory results for the theme or areaoriented queries. This paper studies the subject of gathering information relevant to the subject, to significantly reduce the amount of web pages dealing. By assessing the degree of Web pages, it gives priority to the crawling pages related to a higher degree. Using a subspacebased semantic analysis technique, combined with the Bayesian mechanism and support vector machine, we design and implement an efficient topic crawler. Experiments show that our algorithm has good accuracy and efficiency.

Key words: topic crawler;subspace;semantic analysis;support vector machine