J4

• 计算机科学 • 上一篇    下一篇

基于改进后缀树算法中英文聚类引擎的实现

胡海龙1, 孙 晨2, 赫枫龄1, 左万利1   

  1. 1. 吉林大学 计算机科学与技术学院, 长春 130012; 2. 吉林大学 通讯工程学院, 长春 130012
  • 收稿日期:2008-06-13 修回日期:1900-01-01 出版日期:2009-03-26 发布日期:2009-03-26
  • 通讯作者: 赫枫龄

Implementation of Chinese and English Clustering EngineBased on Improved Suffix Tree Algorithm

HU Hailong1, SUN Chen2, HE Fengling1, ZUO Wanli1   

  1. 1. College of Computer Science and Technology, Jilin University, Changchun 130012, China;2. College of Communication Engineering, Jilin University, Changchun 130012, China
  • Received:2008-06-13 Revised:1900-01-01 Online:2009-03-26 Published:2009-03-26
  • Contact: HE Fengling

摘要: 提出一种基于改进后缀树与交互聚类思想相结合的算法ISTC算法, 通过改造传统后缀树结构实现了对文档标题和摘要的层次化聚类, 同时用交互聚类的方式替代了传统的递归算法. ISTC算法具有语言无关性, 不仅适用于基于单词的西方文字, 而且可以在不引入词典分词技术的情况下有效地处理基于单字的中文字符. 在此算法基础上, 设计并实现了基于改进后缀树算法的交互聚类引擎, 在不同的网络环境下对其 进行了系统测试, 并与其他元搜索引擎进行了对比. 实验结果表明, 使用改进后缀树算法进 行实时交互式聚类是可行的.

关键词: 后缀树, 文本聚类, 元搜索引擎

Abstract: This paper presents an algorithm based on the improved suffix tree and interactclustering idea. Hierarchical clustering for document title and summary is implemented by improved traditional suffix tree structure. Meanwhile, the interactive clustering is employed instead of traditional recursive algorithm. The algorithm is not related with language. Not only is it applicable to wordbased English, but also it can deal effectively with characterbased Chinese without dictionarybased Chinese word segmentation. Furthermore, the interactive clustering engine was realized on the basis of the algorithm, the system was tested in different network environments, and the performance of the system was compared with other metasearch engines. The experimemnt demonstrates that it is feasible effectively to conduct realtime interactive cluste ring by using the improved suffix tree algorithm.

Key words: suffix tree, text clustering, meta search engine

中图分类号: 

  • TP31