基于知识蒸馏和评论时间的文本情感分类新方法

doi:10.13229/j.cnki.jdxbgxb.20230845

摘要/Abstract

摘要：

针对现有的情感分类方法普遍未能充分考虑用户个性化特征且忽略时间因素对情感分类结果的影响的问题，提出一种基于知识蒸馏和评论时间的文本情感分类新方法。首先，为解决数据集中高质量标注数据较少的问题，采用RoFormer-Sim生成模型对训练文本数据增强；然后，引入评论时间属性，从用户历史评论中提取用户的个性化信息，提出基于多特征融合的评论文本情感得分预测模型；最后，为提高针对冷启动用户的泛化性能，引入知识蒸馏理论，利用SKEP模型对基于多特征融合的情感分类模型进行通用性增强。在从中文股吧爬取的真实数据集上的实验结果表明：与SKEP、ELECTRA等典型方法相比，本文方法在准确率上分别提高了3.1%和0.9%，在F₁值上分别提高了2.7%和1.0%，验证了其在改善情感分类表现方面的有效性。

关键词: 计算机应用, 情感分类, 知识蒸馏, 数据增强, 历史评论

Abstract:

Aiming at the problem that the existing sentiment classification methods generally did not fully consider the user's personalized characteristics and ignore the influence of time factor on the sentiment classification results， a new method for text sentiment classification based on knowledge distillation and comment time is proposed. Firstly， in order to solve the problem of less labeled data with high quality， the RoFormer-Sim generative model is used to augment the training text data. Then， a sentiment score prediction model of comment text based on multi-feature fusion is proposed by introducing comment time attribute to extract users' personalized information from user's historical comments. Finally， in order to improve the generalization performance for cold start users， the knowledge distillation theory is introduced， and SKEP model is used to enhance the versatility of the sentiment classification model based on multi-feature fusion. The experimental results on the real dataset crawled from the Chinese stock page show that compared with typical methods such as SKEP and ELECTRA， the accuracy of the proposed method is improved by 3.1% and 0.9%， and the F₁ value is increased by 2.7% and 1.0%， respectively， which verifies its effectiveness in improving the performance of sentiment classification.

Key words: computer application, sentiment classification, knowledge distillation, data augmentation, historical comment

中图分类号:

TP391.1

王友卫,刘奥,凤丽洲. 基于知识蒸馏和评论时间的文本情感分类新方法[J]. 吉林大学学报(工学版), 2025, 55(5): 1664-1674.

You-wei WANG,Ao LIU,Li-zhou FENG. New method for text sentiment classification based on knowledge distillation and comment time[J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(5): 1664-1674.

图/表 11

表1

本文主要符号及含义"

符号	含义
word $k i j$	将U $i j P$ 分词、去停用词等操作后的第k个词
h $i j k W$ ∈R ^m	word $k i j$ 对应的词向量
h $i j S$ ∈R ^m	U $i j P$ 的句向量
h $i L$ ∈R ^m	用户i的历史评论向量
h $i F$ ∈R ^m+n	用户i的特征向量
h $i A$ ∈R ⁿ	用户i的属性向量
h $i j C$ ∈R ^m+n	用户当前评论U $i j C$ 的句向量
h $i R$ ∈R ^m+n	用户i的关系向量
N	用户总数
m	词向量的维数

表1

图1

表2

表3

图2

图3

图4

图5

表4

表5

图6

参考文献 25

[1]	Umer M, Sadiq S, Nappi M, et al. ETCNN: extra tree and convolutional neural network-based ensemble model for COVID-19 tweets sentiment classification[J]. Pattern Recognition Letters, 2022, 164: 224-231.
[2]	Lan Z, Chen M, Goodman S, et al. ALBERT: a lite bert for self-supervised learning of language representations[C]∥International Conference on Learning Representations, Addis Ababa, Ethiopia, 2020: 1-17.
[3]	Liu Z, Huang D, Huang K, et al. Finbert: a pre-trained financial language representation model for financial text mining[C]∥Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, Yokohama, Japan, 2021: 4513-4519.
[4]	赵亚欧, 张家重, 李贻斌, 等. 基于ELMo和Transformer混合模型的情感分析[J]. 中文信息学报, 2021, 35(3): 115-124.
	Zhao Ya-ou, Zhang Jia-chong, Li Yi-bin, et al. Sentiment analysis based on hybird model of elmo and transformer[J]. Journal of Chinese Information Processing, 2021, 35(3): 115-124.
[5]	Yang J, Zou X, Zhang W, et al. Microblog sentiment analysis via embedding social contexts into an attentive LSTM[J]. Engineering Applications of Artificial Intelligence, 2021, 97: 104048.
[6]	蒋宗礼, 张静. 融合用户和产品信息的多头注意力情感分类模型[J]. 计算机系统应用, 2020, 29(7): 131-138.
	Jiang Zong-li, Zhang Jing. Multi-head attention model with user and product information for sentiment classification[J]. Computer Systems & Applications, 2020, 29(7): 131-138.
[7]	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. Computer Science, 2015, 14(7): 38-39.
[8]	邵仁荣, 刘宇昂, 张伟, 等. 深度学习中知识蒸馏研究综述[J]. 计算机学报, 2022, 45(8): 1638-1673.
	Shao Ren-rong, Liu Yu-ang, Zhang Wei, et al. A survey of knowledge distillation in deep learning[J]. Chinese Journal of Computers, 2022, 45(8): 1638-1673.
[9]	Tian H, Gao C, Xiao X, et al. SKEP: sentiment knowledge enhanced pre-training for sentiment analysis[J/OL]. [2023-07-25].arXiv Preprint arXiv: 2005. 05635v2.
[10]	Turney P D. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[J]. Association for Computational Linguistics, 2002, 6: 417-424.
[11]	Xie W. Entity linking based on roformer-sim for chinese short texts[J]. Frontiers in Computing and Intelligent Systems, 2023, 4(1): 46-50.
[12]	Zhao Y, Liu S, Zhang Q, et al. Test case classification via few-shot learning[J]. Information and Software Technology, 2023, 160:107228.
[13]	Lewis M, Liu Y, Goyal N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]∥Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020: 7871-7880.
[14]	Ebbinghaus H. Memory: a contribution to experimental psychology[J]. Annals of Neurosciences, 2013, 20(4): 2004155.
[15]	Grover A, Leskovec J. Node2vec: scalable feature learning for networks[C]∥Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, USA, 2016: 855-864.
[16]	乔百友, 武彤, 杨璐, 等. 一种基于BiGRU和胶囊网络的文本情感分析方法[J]. 吉林大学学报: 工学版, 2024,54(7): 2026-2037.
	Qiao Bai-you, Wu Tong, Yang Lu, et al. A text sentiment analysis method based on BiGRU and capsule network[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(7): 2026-2037.
[17]	王友卫, 童爽, 凤丽洲, 等. 基于图卷积网络的归纳式微博谣言检测新方法[J].浙江大学学报: 工学版, 2022, 56(5): 956-996.
	Wang You-wei, Tong Shuang, Feng Li-zhou, et al. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(5): 956-996.
[18]	陈洁, 王思雨, 赵姝, 等. 基于多粒度用户偏好的文档级情感分析[J]. 中文信息学报, 2023, 37(7): 122-130.
	Chen Jie, Wang Si-yu, Zhao Shu, et al. Multi-granular user preferences for document-level sentiment analysis[J]. Journal of Chinese Information Processing, 2023, 37(7): 122-130.
[19]	Li Y, Ni P, Li G, et al. Inter-personal relation extraction model based on bidirectional GRU and attention mechanism[C]∥IEEE 5th International Conference on Computer and Communications (ICCC), Harbin, China, 2019: 1867-1871.
[20]	Guo B, Zhang C, Liu J, et al. Improving text classification with weighted word embeddings via a multi-channel textcnn model[J]. Neurocomputing, 2019, 363: 366-374.
[21]	Clark K, Luong M T, Le Q V, et al. Electra: pre-training text encoders as discriminators rather than generators[J/OL]. [2023-07-26]. arXiv Preprint arXiv: 2003. 10555.
[22]	Kamal A, Abulaish M. Cat-bigru: convolution and attention with bi-directional gated recurrent unit for self-deprecating sarcasm detection[J]. Cognitive Computation, 2022, 14: 91-109.
[23]	Ahmad W, Wang B, Martin P, et al. Enhanced sentiment analysis regarding COVID-19 news from global channels[J]. Journal of Computational Social Science, 2023, 6: 19-57.
[24]	Gao Z, Li Z, Luo J, et al. Short text aspect-based sentiment analysis based on CNN + BiGRU[J]. Applied Sciences, 2022, 12(5): 12052707.
[25]	Aslam N, Rustam F, Lee E, et al. Sentiment analysis and emotion detection on cryptocurrency related tweets using ensemble LSTM-GRU model[J]. IEEE Access, 2022, 10: 39313-39324.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

用户名	标题+正文	情感标记	发表时间
股友3663E5w329	储能最差的股！盛洪是储能板块最差的，前面走过几倍了，是不可能拉升	0	2021/09/17
芝加哥夜鹰	这个股价是不可能回头了，唉，郁闷呢。大帝一出手，股市抖三抖	0	2021/09/11
未来可期30721	涨停无疑了，连板股，吃大肉［微笑］	1	2021/05/26
漫步冥王星	塔牌这一跌，跌得没头没脑。另一方面来看，很好的进货机会	1	2021/04/28
松松雪7856	已经大举加仓。锁仓！关电脑	1	2021/04/2
喉喉喉喉369	这货踏踏实实跌，主力底下等着吶，黑庄，这股就是笑话	0	2021/02/1

分类方法	准确率
BiGRU^［16］	BiGRU层数num_bigru=1
BiGRU+Attention^［19］	BiGRU维度dim_bigru=100
TextCNN^［20］	卷积核尺寸filter_size=3，4，5 卷积核数量num_kernel=100
ALBERT^［2］	albert_small_zh_google模型
ELECTRA^［21］	electra_180g_small模型
SKEP^［9］	ernie_1.0_skep_large_ch模型
CAT-BiGRU^［22］	词向量维数word2vec_size=200
CAT-BiGRU^［22］	句子最大长度max_len=20
Cov-Att-Bilstm^［23］	BiLSTM层数num_bilstm=2
CNN+BiGRU^［24］	卷积核尺寸filter_size=2，3，4，5
CNN+BiGRU^［24］	卷积核数量num_kernel=100
LSTM-GRU^［25］	词向量维数word2vec_size=200
MFMS	温度系数T=3，权重因子a=0.7

分类方法	准确率	F₁	RMSE
BiGRU	0.830	0.881	0.412
BiGRU+Attention	0.834	0.882	0.407
TextCNN	0.826	0.882	0.418
ALBERT	0.841	0.885	0.397
ELECTRA	0.857	0.898	0.379
SKEP	0.835	0.881	0.406
CAT-BiGRU	0.812	0.868	0.433
Cov-Att-BiLSTM	0.824	0.878	0.419
CNN+BiGRU	0.841	0.890	0.398
LSTM-GRU	0.799	0.856	0.449
MFM	0.855	0.900	0.380
MFMS	0.866	0.908	0.366

分类方法	准确率	F₁
SKEP	0.835	0.881
MFMS-NRel	0.858	0.901
MFMS-Attr	0.862	0.903
MFMS-DW	0.860	0.903
MFM	0.850	0.896
MFMS	0.863	0.905