Abstract: In feature selection, with the increasing number of selected features, the relevance between candidate features and class labels is dynamically changed. This paper presents a new definition of relevance, called Conditional Relevance (CR). That is, we give a new definition of the relevance between candidate features and class labels when each selected feature is given. Consequently, we propose a novel Conditional Relevance Feature Selection (CRFS) method based on information theory. First, the superiority of the CR is verified in theory. Then, the new feature selection algorithm is compared with seven feature selection algorithms on two different classifies and on 10 real data sets. The results show the highest accuracy and the average highest accuracy of the 10 data sets on two classifiers. Experimental results show that the new algorithm can effectively improve the classification performance.

Key words: artificial intelligence, feature selection, information theory, conditional relevance, classification