• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

J4 ›› 2013, Vol. 35 ›› Issue (1): 82-87.

• 论文 • 上一篇    下一篇

基于云计算的定向搜索监控研究

屈振新,朱文昌   

  1. (中南财经政法大学信息与安全工程学院,湖北 武汉 430073)
  • 收稿日期:2011-01-18 修回日期:2011-04-17 出版日期:2013-01-25 发布日期:2013-01-25
  • 作者简介:屈振新(1972),男,湖北房县人,博士,副教授,研究方向为语义Web、云计算、网络和数据库。

Research of directed searching and monitoring based on cloud computing   

QU Zhenxin,ZHU Wenchang   

  1. (School of Information and Safety Engineering,
    Zhongnan University of Economics and Law,Wuhan 430073,China)
  • Received:2011-01-18 Revised:2011-04-17 Online:2013-01-25 Published:2013-01-25

摘要:

传统的搜索引擎不能代替用户实行实时监控,为了解决这个问题,提出了定向搜索监控技术,用户可以根据自己的需求定制任务,包括指定搜索范围和搜索主题,系统按用户定义周期监控,并将结果及时主动地反馈给用户。以Google云平台Google App Engine作为开发平台,利用其提供的多项云服务,有效地解决了计划任务管理、多任务触发以及高并发等问题。重写了通用网络爬虫,通过算法改进提出了定向网络爬虫模型,定向网络爬虫与云端强大的服务器相结合,极大地缩短了爬行时间,提高了搜索监控效率。云平台和搜索监控技术的结合是平台即服务思想的一次成功实验。

关键词: Google云平台, 定向, 搜索, 监控, 计划任务管理, 定向网络爬虫

Abstract:

Traditional search engines cannot replaces users to support realtime monitoring. To solve this problem, this paper proposes the initiative directed searching and monitoring technology. Users can customize their own tasks, including search websites and search theme. The system monitors at the userdefined period, and the results are returned to the user immediately. The Google App Engine (GAE) is used as the development platform, its several cloud computing services are used to solve the problems such as the planned task management, multitasking and high concurrency. We rewrite the web crawler and propose the directed web crawler. Combining the directed crawler and the cloud server, the crawling time is shorten and the monitoring efficiency is increased. It is a successful experiment on Platform as a Service (PaaS) that combining the cloud platform and the searching and monitoring technology.

 

Key words: Google’s cloud platform;directed;search;monitor;scheduled tasks management;directed web crawler