基于知识蒸馏和评论时间的文本情感分类新方法

doi:10.13229/j.cnki.jdxbgxb.20230845

Abstract

Abstract:

Aiming at the problem that the existing sentiment classification methods generally did not fully consider the user's personalized characteristics and ignore the influence of time factor on the sentiment classification results， a new method for text sentiment classification based on knowledge distillation and comment time is proposed. Firstly， in order to solve the problem of less labeled data with high quality， the RoFormer-Sim generative model is used to augment the training text data. Then， a sentiment score prediction model of comment text based on multi-feature fusion is proposed by introducing comment time attribute to extract users' personalized information from user's historical comments. Finally， in order to improve the generalization performance for cold start users， the knowledge distillation theory is introduced， and SKEP model is used to enhance the versatility of the sentiment classification model based on multi-feature fusion. The experimental results on the real dataset crawled from the Chinese stock page show that compared with typical methods such as SKEP and ELECTRA， the accuracy of the proposed method is improved by 3.1% and 0.9%， and the F₁ value is increased by 2.7% and 1.0%， respectively， which verifies its effectiveness in improving the performance of sentiment classification.

Key words: computer application, sentiment classification, knowledge distillation, data augmentation, historical comment

CLC Number:

TP391.1

You-wei WANG,Ao LIU,Li-zhou FENG. New method for text sentiment classification based on knowledge distillation and comment time[J].Journal of Jilin University(Engineering and Technology Edition), 2025, 55(5): 1664-1674.

Figures/Tables 11

Table 1

Main symbols and meanings of this article"

符号	含义
word $k i j$	将U $i j P$ 分词、去停用词等操作后的第k个词
h $i j k W$ ∈R ^m	word $k i j$ 对应的词向量
h $i j S$ ∈R ^m	U $i j P$ 的句向量
h $i L$ ∈R ^m	用户i的历史评论向量
h $i F$ ∈R ^m+n	用户i的特征向量
h $i A$ ∈R ⁿ	用户i的属性向量
h $i j C$ ∈R ^m+n	用户当前评论U $i j C$ 的句向量
h $i R$ ∈R ^m+n	用户i的关系向量
N	用户总数
m	词向量的维数

Table 1

Fig.1

Table 2

Table 3

Fig.2

Fig.3

Fig.4

Fig.5

Table 4

Table 5

Fig.6

References 25

[1]	Umer M, Sadiq S, Nappi M, et al. ETCNN: extra tree and convolutional neural network-based ensemble model for COVID-19 tweets sentiment classification[J]. Pattern Recognition Letters, 2022, 164: 224-231.
[2]	Lan Z, Chen M, Goodman S, et al. ALBERT: a lite bert for self-supervised learning of language representations[C]∥International Conference on Learning Representations, Addis Ababa, Ethiopia, 2020: 1-17.
[3]	Liu Z, Huang D, Huang K, et al. Finbert: a pre-trained financial language representation model for financial text mining[C]∥Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, Yokohama, Japan, 2021: 4513-4519.
[4]	赵亚欧, 张家重, 李贻斌, 等. 基于ELMo和Transformer混合模型的情感分析[J]. 中文信息学报, 2021, 35(3): 115-124.
	Zhao Ya-ou, Zhang Jia-chong, Li Yi-bin, et al. Sentiment analysis based on hybird model of elmo and transformer[J]. Journal of Chinese Information Processing, 2021, 35(3): 115-124.
[5]	Yang J, Zou X, Zhang W, et al. Microblog sentiment analysis via embedding social contexts into an attentive LSTM[J]. Engineering Applications of Artificial Intelligence, 2021, 97: 104048.
[6]	蒋宗礼, 张静. 融合用户和产品信息的多头注意力情感分类模型[J]. 计算机系统应用, 2020, 29(7): 131-138.
	Jiang Zong-li, Zhang Jing. Multi-head attention model with user and product information for sentiment classification[J]. Computer Systems & Applications, 2020, 29(7): 131-138.
[7]	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. Computer Science, 2015, 14(7): 38-39.
[8]	邵仁荣, 刘宇昂, 张伟, 等. 深度学习中知识蒸馏研究综述[J]. 计算机学报, 2022, 45(8): 1638-1673.
	Shao Ren-rong, Liu Yu-ang, Zhang Wei, et al. A survey of knowledge distillation in deep learning[J]. Chinese Journal of Computers, 2022, 45(8): 1638-1673.
[9]	Tian H, Gao C, Xiao X, et al. SKEP: sentiment knowledge enhanced pre-training for sentiment analysis[J/OL]. [2023-07-25].arXiv Preprint arXiv: 2005. 05635v2.
[10]	Turney P D. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[J]. Association for Computational Linguistics, 2002, 6: 417-424.
[11]	Xie W. Entity linking based on roformer-sim for chinese short texts[J]. Frontiers in Computing and Intelligent Systems, 2023, 4(1): 46-50.
[12]	Zhao Y, Liu S, Zhang Q, et al. Test case classification via few-shot learning[J]. Information and Software Technology, 2023, 160:107228.
[13]	Lewis M, Liu Y, Goyal N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]∥Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020: 7871-7880.
[14]	Ebbinghaus H. Memory: a contribution to experimental psychology[J]. Annals of Neurosciences, 2013, 20(4): 2004155.
[15]	Grover A, Leskovec J. Node2vec: scalable feature learning for networks[C]∥Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, USA, 2016: 855-864.
[16]	乔百友, 武彤, 杨璐, 等. 一种基于BiGRU和胶囊网络的文本情感分析方法[J]. 吉林大学学报: 工学版, 2024,54(7): 2026-2037.
	Qiao Bai-you, Wu Tong, Yang Lu, et al. A text sentiment analysis method based on BiGRU and capsule network[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(7): 2026-2037.
[17]	王友卫, 童爽, 凤丽洲, 等. 基于图卷积网络的归纳式微博谣言检测新方法[J].浙江大学学报: 工学版, 2022, 56(5): 956-996.
	Wang You-wei, Tong Shuang, Feng Li-zhou, et al. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(5): 956-996.
[18]	陈洁, 王思雨, 赵姝, 等. 基于多粒度用户偏好的文档级情感分析[J]. 中文信息学报, 2023, 37(7): 122-130.
	Chen Jie, Wang Si-yu, Zhao Shu, et al. Multi-granular user preferences for document-level sentiment analysis[J]. Journal of Chinese Information Processing, 2023, 37(7): 122-130.
[19]	Li Y, Ni P, Li G, et al. Inter-personal relation extraction model based on bidirectional GRU and attention mechanism[C]∥IEEE 5th International Conference on Computer and Communications (ICCC), Harbin, China, 2019: 1867-1871.
[20]	Guo B, Zhang C, Liu J, et al. Improving text classification with weighted word embeddings via a multi-channel textcnn model[J]. Neurocomputing, 2019, 363: 366-374.
[21]	Clark K, Luong M T, Le Q V, et al. Electra: pre-training text encoders as discriminators rather than generators[J/OL]. [2023-07-26]. arXiv Preprint arXiv: 2003. 10555.
[22]	Kamal A, Abulaish M. Cat-bigru: convolution and attention with bi-directional gated recurrent unit for self-deprecating sarcasm detection[J]. Cognitive Computation, 2022, 14: 91-109.
[23]	Ahmad W, Wang B, Martin P, et al. Enhanced sentiment analysis regarding COVID-19 news from global channels[J]. Journal of Computational Social Science, 2023, 6: 19-57.
[24]	Gao Z, Li Z, Luo J, et al. Short text aspect-based sentiment analysis based on CNN + BiGRU[J]. Applied Sciences, 2022, 12(5): 12052707.
[25]	Aslam N, Rustam F, Lee E, et al. Sentiment analysis and emotion detection on cryptocurrency related tweets using ensemble LSTM-GRU model[J]. IEEE Access, 2022, 10: 39313-39324.

Related Articles 15

[1]	Yue HOU,Jin-song GUO,Wei LIN,Di ZHANG,Yue WU,Xin ZHANG. Multi-view video speed extraction method that can be segmented across lane demarcation lines [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(5): 1692-1704.
[2]	Hong-wei ZHAO,Ming-zhu ZHOU,Ping-ping LIU,Qiu-zhan ZHOU. Medical image segmentation based on confident learning and collaborative training [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(5): 1675-1681.
[3]	Zi-hao SHEN,Yong-sheng GAO,Hui WANG,Pei-qian LIU,Kun LIU. Deep deterministic policy gradient caching method for privacy protection in Internet of Vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(5): 1638-1647.
[4]	Jun WANG,Chang-fu SI,Kai-peng WANG,Qiang FU. Intrusion detection method based on ensemble learning and feature selection by PSO-GA [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(4): 1396-1405.
[5]	Tao XU,Shuai-di KONG,Cai-hua LIU,Shi LI. Overview of heterogeneous confidential computing [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(3): 755-770.
[6]	Meng-xue ZHAO,Xiang-jiu CHE,Huan XU,Quan-le LIU. A method for generating proposals of medical image based on prior knowledge optimization [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 722-730.
[7]	Xiao-dong CAI,Qing-song ZHOU,Yan-yan ZHANG,Yun XUE. Social recommendation based on global capture of dynamic， static and relational features [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 700-708.
[8]	Xiang-jiu CHE,Yu-ning WU,Quan-le LIU. A weighted isomorphic graph classification algorithm based on causal feature learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 681-686.
[9]	Xiao-ran GUO,Tie-jun WANG,Yue YAN. Entity relationship extraction method based on local attention and local remote supervision [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 307-315.
[10]	Hao WANG,Bin ZHAO,Guo-hua LIU. Temporal and motion enhancement for video action recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 339-346.
[11]	Yuan-ning LIU,Zi-nan ZANG,Hao ZHANG,Zhen LIU. Deep learning-based method for ribonucleic acid secondary structure prediction [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 297-306.
[12]	Lu Li,Jun-qi Song,Ming Zhu,He-qun Tan,Yu-fan Zhou,Chao-qi Sun,Cheng-yu Zhou. Object extraction of yellow catfish based on RGHS image enhancement and improved YOLOv5 network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(9): 2638-2645.
[13]	Hong-wei ZHAO,Hong WU,Ke MA,Hai LI. Image classification framework based on knowledge distillation [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2307-2312.
[14]	Yun-zuo ZHANG,Yu-xin ZHENG,Cun-yu WU,Tian ZHANG. Accurate lane detection of complex environment based on double feature extraction network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 1894-1902.
[15]	Ming-hui SUN,Hao XUE,Yu-bo JIN,Wei-dong QU,Gui-he QIN. Video saliency prediction with collective spatio-temporal attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1767-1776.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	Peng Ya-ping1，Guo Ying-nan, Huang Wei-jun，Tan Man-zhi，Dong Lei,Wang Zhi-wei . Cyclebycycle variation of ethanol homogeneous charge compression ignition combustion[J]. 吉林大学学报(工学版), 2007, 37(02): 301 -0306 .
[2]	Gao Feng;Huang He;Ren Lu-quan. Erosive wear resistance of Laudakin stoliczkana's ternary coupling and bionic experiments[J]. 吉林大学学报(工学版), 2008, 38(03): 586 -0590 .
[3]	Kou Shu-qing，Qiao Jian，Pi Wen-hao，Yang Shen-hua . Analysis of factors influencing knurling connected hollow camshaft assembling process [J]. 吉林大学学报(工学版), 2008, 38(02): 323 -0328 .
[4]	Xu Yun-fei，Jin Zhen-hua，Lu Qing-chun，Yan Dong-lin . System design and performance test of energy hybrid fuel cell city bus [J]. 吉林大学学报(工学版), 2007, 37(02): 296 -0300 .
[5]	ZHOU Li-Jun, WANG Dian-Hai, LI Wei-Jing. [J]. 吉林大学学报(工学版), 2009, 39(04): 896 -899 .
[6]	WANG Qi,HU Jian-ming,WANG Yi-zhi,Zhang Yi. Method of collecting traffic information based on inter vehicle communication[J]. 吉林大学学报(工学版), 2009, 39(增刊2): 7 -0012 .
[7]	Jin Lei，Wang Bing，Li Chunyuan，Zhang Ping. EVM measurement algorithm for TDSCDMA UE conformance test based on iterative process[J]. 吉林大学学报(工学版), 2006, 36(02): 253 -0258 .
[8]	LI Lu，GUO Qing，GUAN Ming-xiang，JING Qing-feng . Crosslayer design for satellite MAC protocol[J]. 吉林大学学报(工学版), 2008, 38(06): 1458 -1462 .
[9]	YE Yu-ling,SAN Ye . Hybrid optimization algorithm and its performance[J]. 吉林大学学报(工学版), 2009, 39(01): 131 -136 .
[10]	ZHANG Li-Bin, SU Jian, SHAN Hong-Ying, TIAN Yong-Jun, LIU Su-Chao. Error compensation method of piezoelectric detection by tire print of vehicle wheelbase difference[J]. 吉林大学学报(工学版), 2011, 41(02): 327 -0332 .

用户名	标题+正文	情感标记	发表时间
股友3663E5w329	储能最差的股！盛洪是储能板块最差的，前面走过几倍了，是不可能拉升	0	2021/09/17
芝加哥夜鹰	这个股价是不可能回头了，唉，郁闷呢。大帝一出手，股市抖三抖	0	2021/09/11
未来可期30721	涨停无疑了，连板股，吃大肉［微笑］	1	2021/05/26
漫步冥王星	塔牌这一跌，跌得没头没脑。另一方面来看，很好的进货机会	1	2021/04/28
松松雪7856	已经大举加仓。锁仓！关电脑	1	2021/04/2
喉喉喉喉369	这货踏踏实实跌，主力底下等着吶，黑庄，这股就是笑话	0	2021/02/1

分类方法	准确率
BiGRU^［16］	BiGRU层数num_bigru=1
BiGRU+Attention^［19］	BiGRU维度dim_bigru=100
TextCNN^［20］	卷积核尺寸filter_size=3，4，5 卷积核数量num_kernel=100
ALBERT^［2］	albert_small_zh_google模型
ELECTRA^［21］	electra_180g_small模型
SKEP^［9］	ernie_1.0_skep_large_ch模型
CAT-BiGRU^［22］	词向量维数word2vec_size=200
CAT-BiGRU^［22］	句子最大长度max_len=20
Cov-Att-Bilstm^［23］	BiLSTM层数num_bilstm=2
CNN+BiGRU^［24］	卷积核尺寸filter_size=2，3，4，5
CNN+BiGRU^［24］	卷积核数量num_kernel=100
LSTM-GRU^［25］	词向量维数word2vec_size=200
MFMS	温度系数T=3，权重因子a=0.7

分类方法	准确率	F₁	RMSE
BiGRU	0.830	0.881	0.412
BiGRU+Attention	0.834	0.882	0.407
TextCNN	0.826	0.882	0.418
ALBERT	0.841	0.885	0.397
ELECTRA	0.857	0.898	0.379
SKEP	0.835	0.881	0.406
CAT-BiGRU	0.812	0.868	0.433
Cov-Att-BiLSTM	0.824	0.878	0.419
CNN+BiGRU	0.841	0.890	0.398
LSTM-GRU	0.799	0.856	0.449
MFM	0.855	0.900	0.380
MFMS	0.866	0.908	0.366

分类方法	准确率	F₁
SKEP	0.835	0.881
MFMS-NRel	0.858	0.901
MFMS-Attr	0.862	0.903
MFMS-DW	0.860	0.903
MFM	0.850	0.896
MFMS	0.863	0.905

New method for text sentiment classification based on knowledge distillation and comment time

RICH HTML

PDF (PC)