Journal of Jilin University(Engineering and Technology Edition) ›› 2019, Vol. 49 ›› Issue (4): 1293-1300.doi: 10.13229/j.cnki.jdxbgxb20180478

Previous Articles    

Nonlinear feature selection method based on dynamic change of selected features

Wan-fu GAO1(),Ping ZHANG2,Liang HU1()   

  1. 1. College of Computer Science and Technology，Jilin University，Changchun 130012, China
    2. College of Software，Jilin University，Changchun 130012，China
  • Received:2018-05-15 Online:2019-07-01 Published:2019-07-16
  • Contact: Liang HU E-mail:gaowf16@mails.jlu.edu.cn;hul@jlu.edu.cn

RICH HTML

 12   

PDF (PC)

136

Abstract:

Information theory is widely used in feature selection methods. Traditional feature selection methods employ cumulative summation to select features. Different from previous feature selection methods, this paper proposes a nonlinear feature selection method that also considers dynamic change of selected features. The experimental results demonstrate that the proposed method achieves the best classification performance in terms of average classification accuracy and highest classification accuracy. To verify the effectiveness of the proposed method, the proposed method is compared with seven very competitive feature selection methods on three different classifiers on eight real-world data sets.The experimental results shows that this algorithm has strong classification superiorities.

Key words: artificial intelligence, feature selection, information theory, dynamic change, nonlinear methods, classification

CLC Number: 

  • TP301

Table 1

Description of data sets"

数据集 样本个数 特征数目 类别数目 类型
Semeion 1593 256 10 离散
Movement_libras 360 90 15 连续
WarpPIE10P 210 2420 10 连续
TOX_171 171 5748 4 连续
SMK_CAN_187 187 19993 2 连续
Isolet 1560 617 26 连续
USPS 9298 256 10 连续
RELATHE 1427 4322 2 离散

Table 2

Average accuracy (mean±std) with statistical significance on NB"

Data sets CIFE CMIM DISR mRMR IWFS MRI JMIM NDCSF
Semeion

44.98±

7.48(+)

51.32±

10.54(=)

44.94±

7.15(+)

46.94±

8.07(+)

44.28±

7.81(+)

48.8±

9.01(+)

46.68±

8.99(+)

51.95±

10.52

Movement_libras

45.9±

8.02(+)

52.9±

11.06(=)

46.31±

7.93(+)

49.9±

9.52(+)

48.67±

8.24(+)

52.71±

10.91(=)

51.79±

10.38(=)

52.23±

11.38

WarpPIE10P

58.23±

7.78(+)

66.53±

13.11(+)

55.15±

10.71(+)

62.85±

10.96(+)

60.89±

9.59(+)

59.85±

10.14(+)

60.81±

11.67(+)

71.63±

14.37

TOX_171

57.42±

1.86(=)

61.87±

2.86(-)

53.76±

3.44(+)

54.97±

1.78(+)

50.99±

1.96(+)

59.37±

3.31(-)

55.11±

1.56(+)

57.76±

3.07
SMK_CAN_187

65.69±

1.79(+)

62.37±

1.72(+)

65.04±

2.35(+)

62.00±

1.45(+)

63.37±

3.81(+)

64.22±

1.7(+)

62.00±

1.68(+)

67.66±

2.40

Isolet

55.53±

13.47(+)

58.26±

14.33(+)

44.46±

10.38(+)

46.87±

10.6(+)

49.37±

10.55(+)

56.55±

13.67(+)

51.57±

11.24(+)

59.43±

17.51

USPS

66.4±

8.8(+)

76.21±

12.55(+)

67.4±

9.95(+)

71.72±

10.55(+)

67.63±

9.02(+)

70.93±

10.41(+)

70.75±

10.22(+)

76.11±

12.43

RELATHE

60.84±

0.98(+)

65.89±

3.76(+)

63.56±

3.01(+)

65.85±

3.67(+)

67.71±

2.82(+)

65.47±

2.64(=)

66.66±

2.5(+)

67.59±

4.6

Average 56.87 61.92 55.08 57.64 56.61 59.74 58.17 63.05

Table 3

Average accuracy (mean±std) with statistical significance on SVM"

Data sets CIFE CMIM DISR mRMR IWFS MRI JMIM NDCSF
Semeion

60.67±

11.8(+)

67.1±

15.89(+)

56.26±

11.36(+)

60.3±

12.32(+)

58.9±

11.59(+)

61.5±

12.93(+)

61.96±

13.59(+)

67.97±

16.84

Movement_libras

66.06±

15.4(+)

70.81±

17.81(+)

60.1±

12.54(+)

66.99±

16.03(+)

67.77±

15.53(+)

70.13±

17.35(+)

70.87±

17.73(+)

72.59±

18.12

WarpPIE10P

81.02±

14.99(+)

84.86±

17.04(+)

76.1±

12.86(+)

81.61±

15.81(+)

83.71±

16.96(+)

80.29±

14.94(+)

81.52±

16.53(+)

87.18±

18.24

TOX_171

59.6±

5.26(+)

65.15±

5.04(-)

58.36±

3.02(+)

59.87±

3.07(+)

58.16±

3.87(+)

57.83±

2.74(+)

56.35±

2.48(+)

63.01±

5.91
SMK_CAN_187

62.51±

2.32(+)

61.48±

1.29(+)

64.46±

1.19(+)

62.69±

1.61(+)

63.56±

2.87(+)

61.77±

1.96(+)

61.89±

1.68(+)

67.39±

1.81

Isolet

59.41±

14.09(+)

68.07±

18.22(-)

60.3±

14.89(+)

60.45±

14.78(+)

58.28±

13.81(+)

65.46±

16.63(=)

60.06±

14.07(+)

65.41±

18.65
USPS

80.12±

13.7(+)

84.12±

15.24(+)

78.74±

13.77(+)

81.35±

13.98(+)

80.29±

13.35(+)

81.55±

14.1(+)

81.92±

14.41(+)

85.84±

15.9

RELATHE

69.66±

1.55(+)

73.25±

2.89(+)

72.74±

3.22(+)

75.14±

3.75(=)

74.67±

2.88(+)

71.03±

1.85(+)

72.19±

2.06(+)

74.65±

3.84

Average 67.38 71.86 65.88 68.55 68.17 68.7 68.35 73.01

Table 4

Average accuracy (mean±std) with statistical significance on 3NN"

Data sets CIFE CMIM DISR mRMR IWFS MRI JMIM NDCSF
Semeion

55.99±

13.38(+)

62.06±

18.63(=)

49.39±

14.08(+)

54.81±

14.32(+)

53.76±

12.87(+)

56.11±

14.21(+)

55.88±

15.46(+)

62.56±

19.21

Movement_libras

61.27±

15.1(+)

64.97±

16.88(+)

56.46±

12.54(+)

62.38±

15.56(+)

62.64±

13.92(+)

65.00±

17.22(+)

64.01±

16.32(+)

66.7±

17.23

WarpPIE10P

77.29±

14.62(+)

81.95±

17.77(+)

73.97±

13.91(+)

78.62±

16.94(+)

77.63±

16.12(+)

76.68±

15.42(+)

78.78±

17.68(+)

84.58±

18.81

TOX_171

48.92±

3.01(+)

61.27±

6.79(-)

57.11±

5.6(=)

54.93±

4.89(+)

53.43±

3.97(+)

60.05±

6.37(-)

59.1±

5.66(-)

57.12±

5.86
SMK_CAN_187

60.76±

2.81(+)

59.76±

2.25(+)

62.39±

1.9(+)

62.71±

2.29(+)

60.53±

2.54(+)

59.72±

2.62(+)

59.4±

2.79(+)

67.32±

2.44

Isolet

46.42±

10.66(+)

60.37±

17.53(-)

53.62±

14.75(+)

53.09±

13.97(+)

45.79±

10.54(+)

58.08±

16.23(-)

52.36±

13.5(+)

55.66±

16.86
USPS

75.53±

15.16(+)

80.56±

17.25(+)

74.36±

15.06(+)

76.97±

15.96(+)

75.88±

14.68(+)

77.48±

15.76(+)

77.97±

16.06(+)

82.41±

17.91

RELATHE

62.94±

3.19(+)

64.95±

4.54(=)

62.05±

3.57(+)

65.27±

4.67(=)

57.89±

2.5(+)

63.46±

3.86(+)

63.23±

4.01(+)

65.06±

5.47

Average 61.14 66.99 61.17 63.60 60.94 64.57 63.84 67.68

Fig.1

Accuracy of classifier achieved with lsolet"

Fig.2

Accuracy of classifier achieved with movement_libras"

Fig.3

Accuracy of classifier achieved with RELATHE"

Fig.4

Accuracy of classifier achieved with Semeion"

Fig.5

Accuracy of classifier achieved with SMK_CAN_187"

Fig.6

Accuracy of classifier achieved with TOX_171"

Fig.7

Accuracy of classifier achieved with USPS"

Fig.8

Accuracy of classifier achieved with WarpPIE10P"

Table 5

Highest average accuracies of three classifiers with eight methods"

Data sets CIFE CMIM DISR mRMR IWFS MRI JMIM NDCSF
Semeion 64.81 75.75 63.28 65.20 63.47 67.53 69.60 78.01
Movement_libras 66.33 71.89 61.89 69.33 67.81 71.85 71.52 72.89
WarpPIE10P 80.33 89.78 80.06 85.11 85.67 83.17 87.06 93.39
TOX_171 59.93 68.15 60.54 59.69 57.63 62.86 59.64 63.98
SMK_CAN_187 66.04 64.61 65.94 65.11 65.36 65.34 64.61 71.32
Isolet 65.24 75.90 64.53 64.96 60.04 73.82 63.35 76.39
USPS 84.46 90.54 83.63 85.96 84.23 85.79 87.46 91.65
RELATHE 66.08 72.76 70.24 73.58 68.91 69.42 70.22 75.46
Average 69.15 76.17 68.76 71.12 69.14 72.47 71.68 77.89
1 Gao W , Hu L , Zhang P . Class-specific mutual information variation for feature selection[J]. Pattern Recognition, 2018, 79:328-339.
2 Hu L , Gao W , Zhao K , et al . Feature selection considering two types of feature relevancy and feature interdependency[J]. Expert Systems with Applications, 2018, 93:423-434.
3 石峰，莫忠息 .信息论基础[M].3版.武汉:武汉大学出版社，2014:14-52.
4 赵晓群 .信息论基础及应用[M].北京:机械工业出版社，2015:27-53.
5 Lewis D D . Feature selection and feature extraction for text categorization[C]⫽The Workshop on Speech & Natural Language, Association for Computational Linguistics,Harriman, New York, 1992:212-217.
6 Battiti R . Using mutual information for selecting features in supervised neural net learning[J]. IEEE Transactions on Neural Networks, 1994, 5(4):537-550.
7 Peng H , Long F , Ding C . Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2005, 27(8):1226-1238.
8 Lin D , Tang X . Conditional infomax learning: an integrated framework for feature extraction and fusion[C]⫽European Conference on Computer Vision, Graz, 2006: 68-82.
9 Salueña C , Avalos J B . Information-theoretic feature selection in microarray data using variable complementarity[J]. IEEE Journal of Selected Topics in Signal Processing, 2008, 2(3):261-274.
10 Zeng Z , Zhang H , Zhang R , et al . A novel feature selection method considering feature interaction[J]. Pattern Recognition, 2015, 48(8):2656-2666.
11 Fleuret F . Fast Binary Feature selection with conditional mutual information[J]. Journal of Machine Learning Research, 2004, 5(3):1531-1555.
12 Bennasar M , Hicks Y , Setchi R . Feature selection using joint mutual information maximisation[J]. Expert Systems with Applications, 2015, 42(22):8520-8532.
13 Wang J , Wei J M , Yang Z , et al . Feature selection by maximizing independent classification information[J]. IEEE Transactions on Knowledge & Data Engineering, 2017, 29(4):828-841.
14 Che J , Yang Y , Li L , et al . Maximum relevance minimum common redundancy feature selection for nonlinear data[J]. Information Sciences, 2017, 409:68-86.
15 Li J , Cheng K , Wang S , et al . Feature selection: a data perspective[J]. Acm Computing Surveys, 2016, 50(6).
16 Lichman M . UCI machine learning repository[DB/OL].[2018-05-15]..
[1] Dan⁃tong OUYANG,Jun XIAO,Yu⁃xin YE. Distant supervision for relation extraction with weakconstraints of entity pairs [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 912-919.
[2] DONG Sa, LIU Da-you, OUYANG Ruo-chuan, ZHU Yun-gang, LI Li-na. Logistic regression classification in networked data with heterophily based on second-order Markov assumption [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(5): 1571-1577.
[3] GU Hai-jun, TIAN Ya-qian, CUI Ying. Intelligent interactive agent for home service [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(5): 1578-1585.
[4] WANG Xu, OUYANG Ji-hong, CHEN Gui-fen. Measurement of graph similarity based on vertical dimension sequence dynamic time warping method [J]. 吉林大学学报(工学版), 2018, 48(4): 1199-1205.
[5] ZHANG Hao, ZHAN Meng-ping, GUO Liu-xiang, LI Zhi, LIU Yuan-ning, ZHANG Chun-he, CHANG Hao-wu, WANG Zhi-qiang. Human exogenous plant miRNA cross-kingdom regulatory modeling based on high-throughout data [J]. 吉林大学学报(工学版), 2018, 48(4): 1206-1213.
[6] LI Xiong-fei, FENG Ting-ting, LUO Shi, ZHANG Xiao-li. Automatic music composition algorithm based on recurrent neural network [J]. 吉林大学学报(工学版), 2018, 48(3): 866-873.
[7] HUANG Lan, JI Lin-ying, YAO Gang, ZHAI Rui-feng, BAI Tian. Construction of disease-symptom semantic net for misdiagnosis prompt [J]. 吉林大学学报(工学版), 2018, 48(3): 859-865.
[8] LIU Jie, ZHANG Ping, GAO Wan-fu. Feature selection method based on conditional relevance [J]. 吉林大学学报(工学版), 2018, 48(3): 874-881.
[9] CHEN Tao, CUI Yue-han, GUO Li-min. Improved algorithm of multiple signal classification for single snapshot [J]. 吉林大学学报(工学版), 2018, 48(3): 952-956.
[10] CHEN Mian-shu, SU Yue, SANG Ai-jun, LI Pei-peng. Image classification methods based on space vector model [J]. 吉林大学学报(工学版), 2018, 48(3): 943-951.
[11] LIU Xue-juan, YUAN Jia-bin, XU Juan, DUAN Bo-jia. Quantum k-means algorithm [J]. 吉林大学学报(工学版), 2018, 48(2): 539-544.
[12] WANG Xu, OUYANG Ji-hong, CHEN Gui-fen. Heuristic algorithm of all common subsequences of multiple sequences for measuring multiple graphs similarity [J]. 吉林大学学报(工学版), 2018, 48(2): 526-532.
[13] YANG Xin, XIA Si-jun, LIU Dong-xue, FEI Shu-min, HU Yin-ji. Target tracking based on improved accelerated gradient under tracking-learning-detection framework [J]. 吉林大学学报(工学版), 2018, 48(2): 533-538.
[14] YANG Hong-yu, XU Jin. An Android malware static detection model [J]. 吉林大学学报(工学版), 2018, 48(2): 564-570.
[15] LI Jia-fei, SUN Xiao-yu. Clustering method for uncertain data based on spectral decomposition [J]. 吉林大学学报(工学版), 2017, 47(5): 1604-1611.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd