Abstract:

In imbalanced data classification， the majority of class samples have an advantage in terms of quantity， and their distribution will have a significant "pulling" effect on the clustering results. However， the minority class samples， due to their small quantity， have relatively unclear features in the entire dataset， resulting in drift problems in the data stream and affecting the classification performance of the data stream. To address this issue， research is conducted on an imbalanced drift big data stream classification algorithm based on spectral clustering undersampling. By using undersampling techniques to reduce the redundant amount of majority class data in imbalanced drift big data streams， balance the amount of majority class data and minority class data， and alleviate the problem of data drift caused by clustering "pulling"； select the core points of the balanced big data stream to form a core point set， and use spectral clustering algorithm to cluster this core point set. Based on the clustering structure obtained from spectral clustering and the selected core points， realize the classification of imbalanced drift big data streams. The experimental results show that the algorithm can achieve balanced processing of imbalanced drift big data streams， and the average imbalance degree after processing can be reduced to 1.024， almost approaching the equilibrium state； it can achieve the selection and effective grouping of core points for different attribute big data streams， providing guarantees for the subsequent effective application of such big data streams.

Key words: spectral clustering, undersampling, out of balance, drift big data stream, core point set, group division