关键词: 搜索引擎 网络蜘蛛 搜索策略 主题提取

Abstract:

Based on our in-depth research on the search strategy in topic-driven engines and topic dependency judgement algorithms, this article presents a desig n model of the topic-oriented web spider and analyzes its structure in detail. As the key component of the search strategy for the topic-oriented web sp  ider,the topic dependency judgement algorithms ensure a focused web crawling process of the spider. In the process of the dependency judgement between URLs and topics, a novel URL pruning algorithm called EPR is presented based on an analysis of the anchor text and the related properties.

Key words: search engine;Web spider, search strategy, topic distillation