即时定向新闻采集技术研究

J4 ›› 2012, Vol. 34 ›› Issue (9): 180-183.

即时定向新闻采集技术研究

王辛1，黄穗1，龙舜1,2

(1.暨南大学计算机科学系,广东广州 510632;
2.广东省公共网络安全风险评价与预警应急技术研究中心,广东广州 510632)

收稿日期:2012-04-13 修回日期:2012-06-25 出版日期:2012-09-25 发布日期:2012-09-25

An Efficient Approach to JustInTime Focused News Acquisition

WANG Xin1,HUANG Sui1,LONG Shun1,2

(1.Department of Computer Science,Jinan University,Guangzhou 510632;
2.Emergency Technology Research Center of Risk Evaluation and Prewarning on
Public Network Security,Guangzhou 510632,China)

Received:2012-04-13 Revised:2012-06-25 Online:2012-09-25 Published:2012-09-25

摘要/Abstract

摘要：

互联网的迅速发展带动了信息量的爆炸性增加。如何更快地采集所需信息一直是国内外研究和开发的热点。近年来，不断增长的对特定信息(例如特定领域的新闻)的需求要求有针对性地从指定的网站即时采集相关信息。这些新闻一般具有不可预见性、更新频率较快、时效性强等特点。这要求我们必须能针对这些特点实现即时定向的采集。本文提出了一种有效抓取网页并进行分析的方法，实践表明取得了满意的效果。

关键词: 新闻采集, 爬虫, 即时

Abstract:

The rapid development of the Internet leads to the explosive increase in the amount of information.How to collect the required information quickly has been a hot topic in both industry and research areas.In recent years, the growing demand for specific information (such as news of specific topics) information should be acquired from some specified sites in a justintime manner.However,they are generally unpredictable,of quicker update frequency,more timesensitive,and therefore more difficult to acquire justintime.This paper proposes a novel approach to tackle this problem,whose efficiency has been demonstrated in practice.

Key words: news acquisition;crawler;justintime

王辛1，黄穗1，龙舜1,2. 即时定向新闻采集技术研究[J]. J4, 2012, 34(9): 180-183.

WANG Xin1,HUANG Sui1,LONG Shun1,2. An Efficient Approach to JustInTime Focused News Acquisition[J]. J4, 2012, 34(9): 180-183.

参考文献

［1］Nelson M L,Smith J A,del Campo I G.Efficient,Automatic Web Resource Harvesting［C］∥Proc of the 8th Annual ACM International Workshop on Web Information and Data Management.November,2006:4350.
［2］Mali S,Ninoriya S,Meshram B B.Freshness Tuning in Focused Crawler［C］∥Proc of the International Conference & Workshop on Emerging Trends in Technology,2011:548552.
［3］Thanh T T, David H, Nick C, et al. Focused Crawling for Both Topical Relevance and Quality of Medical Information［C］∥Proc of the 14th ACM International Conference on Information and Knowledge Management,2005:147154.
［4］Kumar M,Vig R.Design of CORE:Context Ontology Rule Enhanced Focused Web Crawler［C］∥Proc of the International Conference on Advances in Computing,Communication and Control,2009:494497.
［5］Mali S,Meshram B B.Focused Web Crawler with Revisit Policy［C］∥Proc of the International Conference & Workshop on Emerging Trends in Technology, 2011:474479.
［6］Cho J,GarciaMolina H.Synchronizing a Database to Improve Freshness［C］∥Proc of the 2000 ACM SIGMOD International Conference on Management of Data,2000:117128.
［7］Kharazmi S, Nejad A F, Abolhassani H. Freshness of Web Search Engines:Improving Performance of Web Search Engines Using Data Mining Techniques［C］∥Proc of Internet Technology and Secured Transactions,2009:17.
［8］Cho J,GarciaMolina H.Effective Page Refresh Policies for Web Crawler［J］.ACM Transactions on Database Systems,2003,28(4):390426.
［9］Brewington B E,Cybenko G.Keeping up with the Changing Web［J］.Journal Computer, 2000,33(5):5258.
［10］Cho J,Ntoulas A.Effective Change Detection Using Sampling［C］∥Proc of the 28th International Conference on Very Large Data Bases,2002:514525.
［11］Altingovde I S,Demir E,Can F,et al.Incremental Clusterbased Retrieval Using Compressed Clusterskipping Inverted Files［J］.ACM Transactions on Information Systems,2008,26(3):artide 15.
［12］Tan Q,Mitra P.ClusteringBased Incremental Web Crawling［J］.ACM Transactions on Information Systems,2010,28(4):artide 17.

[1]	莫舒恒, 卢圣有, 黄聃, 卢宇彤. 基于即时编译的GNU Octave性能优化[J]. 计算机工程与科学, 2022, 44(12): 2091-2101.
[2]	王淑芬1,高军礼1,邹普1，宋海涛2. 基于Hadoop的广域网分布式主题爬虫系统框架[J]. J4, 2015, 37(04): 670-675.
[3]	于娟，刘强. 主题网络爬虫研究综述[J]. J4, 2015, 37(02): 231-237.
[4]	黄克颖1,3，高玥2，李国庆1. 非合约对地观测数据源的动态汇聚[J]. J4, 2013, 35(11): 68-75.
[5]	屈振新，朱文昌. 基于云计算的定向搜索监控研究[J]. J4, 2013, 35(1): 82-87.
[6]	白玉昭，梁久祯. 基于概率模型的主题爬虫的研究和实现[J]. J4, 2013, 35(1): 160-165.
[7]	王振宇1，唐远华1，郭力2. 面向分层结构的网页分类与抓取[J]. J4, 2012, 34(11): 1-6.
[8]	彭冬，蔡皖东. 面向Web论坛的网络信息获取技术及系统实现[J]. J4, 2011, 33(1): 157-160.
[9]	蒋宗礼，田晓燕，赵旭. 一种基于语义分析的主题爬虫算法[J]. J4, 2010, 32(9): 145-147.
[10]	范会联1，李献礼2，曾广朴1. 基于改进遗传算法的聚焦爬虫设计[J]. J4, 2010, 32(5): 126-129.
[11]	程菲汪建海罗键. 增量更新Crawler进行Web收集方法研究[J]. J4, 2006, 28(12): 28-30.