Abstract:

Chinese word segmentation is a very important task in information processing. The present Chinese word segmentation technology mainly relies on commonword dictionary. But the dictionary has no recognition capability for unknown words. The authors brought forth a method of using doubledictionary to recognize unknown words. The process is to build a commonword dictionary and a singleword dictionary, then combine  them for  segmentation, solving the inefficiency in recognizing unknown words. As a result, the accuracy rate can reach above 90%.

Key words: singleword dictionary, unknown words, Chinese word segmentation, doubledictionary