Abstract:

Marginal probability has a greater effect on traditional mutual information feature selection method, which may leads to evaluation of rare words bigger than common words, resulting in selecting low frequency words. In order to improve these insufficiencies, we analyze several traditional feature extraction methods, associates the mutual information method with characteristics of word frequency by introducing disparity and average frequency, and increases the accuracy of mutual information classification Experiment shows that this method makes better classification results.

Key words: text classification, feature selection, mutual information