Abstract: The present paper deals with the technique how to solve the problem of cycle links in internet crawling by directed graph. First, the problem is proposed. Then, the formal definition of cycle links in internet crawling is described. Finally, the algorithm to solve the problem by directed graph is given. The key problem to a crawler is how to find directed loops effectively in web pages crawled by the crawler. The algorithm described in this paper can make the crawler avoid dropping in the pitfall created by cycle links.

Key words: crawler, internet search engine, hyperlink, directed graph