用有向图法解决网页爬行中循环链接问题

用有向图法解决网页爬行中循环链接问题

赫枫龄, 左万利

吉林大学计算机科学与技术学院, 长春 130012

收稿日期:2003-12-17 修回日期:1900-01-01 出版日期:2004-07-26 发布日期:2004-07-26
通讯作者: 赫枫龄

Solve the cycle links problem in Internet crawlingby directe d graph

HE Feng-ling, ZUO Wan-Li

College of Computer Science and Technology, Jilin University, Chan gchun 130012, China

Received:2003-12-17 Revised:1900-01-01 Online:2004-07-26 Published:2004-07-26
Contact: HE Feng-ling

摘要/Abstract

摘要： 提出网页构成的有向回路问题, 描述了由网页构成有向图的形式定义, 并给出了用有向图法发现网页构成的有向回路算法. 所给定的算法能使网页爬行器避免掉入由已爬行过的网页构成的有向回路陷阱.

关键词: 爬行器, 网络搜索引擎, 超链接, 有向图

Abstract: The present paper deals with the technique how to solve the problem of cycle links in internet crawling by directed graph. First, the problem is proposed. Then, the formal definition of cycle links in internet crawling is described. Finally, the algorithm to solve the problem by directed graph is given. The key problem to a crawler is how to find directed loops effectively in web pages crawled by the crawler. The algorithm described in this paper can make the crawler avoid dropping in the pitfall created by cycle links.

Key words: crawler, internet search engine, hyperlink, directed graph

中图分类号:

TP393.09

赫枫龄, 左万利. 用有向图法解决网页爬行中循环链接问题[J]. J4, 2004, 42(03): 402-404.

HE Feng-ling, ZUO Wan-Li. Solve the cycle links problem in Internet crawlingby directe d graph[J]. J4, 2004, 42(03): 402-404.

[1]	梁浩, 徐俊明. 有向图最小圈长不大于4的一个充分条件[J]. J4, 2013, 51(02): 241-243.
[2]	汪荣, 邵燕灵, 高玉斌. 一类双色有向图本原指数的上界[J]. J4, 2008, 46(04): 601-606.
[3]	赫枫龄, 左万利, 张雪松. 高性能网页索引器JU_Indexer的实现[J]. J4, 2006, 44(01): 50-56.
[4]	赫枫龄，陶文学，李凯，周力，左万利. 新一代网络搜索引擎系统CHINA_VIVI的实现[J]. J4, 2003, 41(02): 192-195.