基于Transformer编码器的中文命名实体识别

doi:10.13229/j.cnki.jdxbgxb20200640

吉林大学学报(工学版) ›› 2021, Vol. 51 ›› Issue (3): 989-995.doi: 10.13229/j.cnki.jdxbgxb20200640

基于Transformer编码器的中文命名实体识别

郭晓然¹(),罗平²,王维兰³

^1.西北民族大学数学与计算机科学学院，兰州 730030
^2.兰州交通大学电子与信息工程学院，兰州 730070
^3.西北民族大学中国民族语言文字信息技术教育部重点实验室，兰州 730030

收稿日期:2020-08-20 出版日期:2021-05-01 发布日期:2021-05-07
作者简介:郭晓然（1981-），女，副教授，博士研究生. 研究方向：自然语言处理,知识图谱,知识抽取.E-mail：guoxiaoran369@163.com
基金资助:
国家自然科学基金项目(61862057);国家民委创新团队计划项目(〔2018〕98号);中央高校国家民委专项项目(1001160448);中央高校基本科研业务费项目(31920210090)

Chinese named entity recognition based on Transformer encoder

Xiao-ran GUO¹(),Ping LUO²,Wei-lan WANG³

^1.School of Mathematics and Computer Science，Northwest Minzu University，Lanzhou 730030，China
^2.School of Electronic and Information Engineering，Lanzhou Jiaotong University，Lanzhou 730070，China
^3.Key Laboratory of China's Ethnic Languages and Information Technology，Ministry of Education，Northwest Minzu University，Lanzhou 730030，China

Received:2020-08-20 Online:2021-05-01 Published:2021-05-07

摘要/Abstract

摘要：

提出了一种基于Transformer编码器和BiLSTM的字级别中文命名实体识别方法，将字向量与位置编码向量拼接成联合向量作为字表示层，避免了字向量信息的损失和位置信息的丢失；利用BiLSTM为联合向量融入方向性信息，引入Transformer编码器进一步抽取字间关系特征。实验结果表明，该方法在MSRA数据集和唐卡数据集上的F1值分别达到了81.39%和86.99%，有效提升了中文命名实体识别的效果。

关键词: 命名实体识别, Transformer编码器, BiLSTM, 位置编码

Abstract:

This paper proposes a Chinese named entity recognition method based on Transformer encoder and BiLSTM. This method uses a joint vector as the word representation layer by combining the word embedding and the position coding vector to avoid the losses of the word embedding information and the position information. The directional information is integrated into the joint vector using BiLSTM. The Transformer encoder is introduced to further extract the word relationship features. The experimental results show that the F value of this method on the general MSRA and Thangka domain data sets reaches 81.39% and 86.99% respectively， which effectively improve the effect of Chinese named entity recognition.

Key words: named entity recognition, Transformer encoder, BiLSTM, position coding

中图分类号:

TP391

郭晓然,罗平,王维兰. 基于Transformer编码器的中文命名实体识别[J]. 吉林大学学报(工学版), 2021, 51(3): 989-995.

Xiao-ran GUO,Ping LUO,Wei-lan WANG. Chinese named entity recognition based on Transformer encoder[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 989-995.

图/表 6

图1

图2

图3

表1

表2

表3

参考文献 22

1	张晓艳, 王挺, 陈火旺. 命名实体识别研究[J]. 计算机科学, 2005, 32(4):44-48.
	Zhang Xiao-yan, Wang Ting, Chen Huo-wang. Research on named entity recognition[J]. Computer Science, 2005, 32(4):44-48.
2	刘浏, 王东波. 命名实体识别研究综述[J]. 情报学报, 2018, 37(3): 329-340.
	Liu Liu, Wang Dong-bo. A review on named entity recognition[J]. Journal of the China Society for Scientific and Technical Information, 2018, 37(3):329-340.
3	张玥杰, 徐智婷, 薛向阳. 融合多特征的最大熵汉语命名实体识别模型[J]. 计算机研究与发展, 2008, 45(6):1004-1010.
	Zhang Yue-jie, Xu Zhi-ting, Xue Xiang-yang. Fusion of multiple features for Chinese named entity recognition based on maximum entropy model[J]. Journal of Computer Research and Development, 2008, 45(6):1004-1010.
4	Morwal S, Jahan N, Chopra D. Named entity recognition using hidden Markov model[J]. International Journal on Natural Language Computing,2012, 1(4):15-23.
5	Ju Zhen-fei, Wang Jian, Zhu Fei. Named entity recognition from biomedical text using SVM[C]∥International Conference on Bioinformatics & Biomedical Engineering, Wuhan,China, 2011:1-4.
6	王路路, 艾山·吾买尔, 买合木提·买买提,等. 基于CRF和半监督学习的维吾尔文命名实体识别[J]. 中文信息学报, 2018, 32(11):16-26, 33.
	Wang Lu-lu, Wumaier Aishan, Maimaiti Maihemuti, et al. A semi-supervised approach to uyghur named entity recognition based on CRF[J]. Journal of Chinese Information Processing, 2018, 32(11):16-26, 33.
7	Maryam Habibi, Leon Weber, Mariana Neves, et al. Deep learning with word embeddings improves biomedical named entity recognition[J]. Bioinformatics, 2017, 33(14):37-48.
8	Lei J, Tang B, Lu X, et al. Research and applications: a comprehensive study of named entity recognition in Chinese clinical text[J]. Journal of the American Medical Informatics Association, 2014, 21(5):808-814.
9	Ji Y, Tong C, Liang J, et al. A deep learning method for named entity recognition in bidding document[J]. Journal of Physics:Conference Series, 2019, 1168(3):032076.
10	Levy O, Goldberg Y. Neural word embedding asimplicit matrix factorization[J/OL].[2020-08-12].
11	Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
12	Graves A, Jurgen S. Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J]. Neural Networks, 2005, 18(5/6):602-610.
13	Huang Z H, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging[J/OL].[2020-08-15].
14	李明浩, 刘忠, 姚远哲. 基于LSTM-CRF的中医医案症状术语识别[J]. 计算机应用, 2018, 38(2):42-46.
	Li Ming-hao, Liu Zhong, Yao Yuan-zhe. LSTM-CRF based symptom term recognition on traditional Chinese medical case[J]. Journal of Computer Applications, 2018, 38(2):42-46.
15	Ma X Z, Hovy E. End-to-end sequence labeling via bidirectional LSTM-CNNs-CRF[J/OL]. [2020-08-15].
16	韩鑫鑫,贲可荣,张献. 军用软件测试领域的命名实体识别技术研究[J]. 计算机科学与探索, 2020, 14(5):740-748.
	Han Xin-xin, Ke-rong Ben, Zhang Xian. Research on named entity recognition technology in military software testing[J]. Journal of Frontiers of Computer Science & Technology, 2020, 14(5):740-748.
17	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Proceedings of Advances in Neural Information Processing Systems, 2017(12):6000-6010.
18	Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J/OL].[2020-08-17].
19	Rei M, Crichton G, Pyysalo S. Attending to characters in neural sequence labeling models[C]∥International Conference on Computational Linguistics, Osaka, Japan, 2016: 309-318.
20	李明扬, 孔芳. 融入自注意力机制的社交媒体命名实体识别[J]. 清华大学学报:自然科学版, 2019, 59(6):461-467.
	Li Ming-yang, Kong Fang. Combined self-attention mechanism for named entity recognition in social media[J]. Journal of Tsinghua University(Science and Technology), 2019, 59(6): 461-467.
21	Yan H, Deng B, Li X N, et al. TENER: adapting transformer encoder for named entity recognition[J]. Computation and Language, 2019, 1:04474.
22	Li X Y, Meng Y X, Sun X F, et al. Is word segmentation necessary for deep learning of Chinese representations?[C]∥Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019:3242-3252.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

语料	类别	训练集	验证集	测试集
MSRA	句子	32 421	8 105	4 631
	PER	11 274	2 818	1 973
	LOC	23 372	5 842	2 877
	ORG	13 166	3 291	1 331
唐卡	句子	1 591	397	692
唐卡	NSL	3 154	788	1 056

Coups	Model	P	R	F1
MSRA	CRF	52.96	31.89	39.81
	BiLSTM-CRF	81.24	76.36	78.73
	BiLSTM-Attention-CRF	83.97	74.62	79.02
	Transformer-CRF	66.70	60.33	63.35
	Transformer*-CRF	67.31	61.35	64.19
	Ours*	81.83	79.01	80.40
	Ours	87.05	76.43	81.39
唐卡	CRF	80.33	68.90	74.18
	BiLSTM-CRF	87.56	80.02	83.63
	BiLSTM-Attention-CRF	87.67	80.78	84.09
	Transformer-CRF	80.36	63.92	71.20
	Transformer*-CRF	90.15	81.44	85.57
	Ours*	90.15	80.68	85.15
	Ours	93.86	81.06	86.99

	命名实体长度					总计
	1	2	3~5	6~10	>10	总计
测试集实体个数	0	55	930	69	2	1056
Model1识别实体个数	54	46	821	52	0	973
Model1识别正确实体个数	0	28	774	51	0	853
Model1正确率/%	0	60.8	94.2	98.1	-	87.7
Model2识别实体个数	11	51	797	52	1	912
Model2识别正确实体个数	0	33	773	50	0	856
Model2正确率/%	0	64.7	96.9	96.1	0	93.9

基于Transformer编码器的中文命名实体识别

Chinese named entity recognition based on Transformer encoder

RICH HTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 22

相关文章 1

Metrics

本文评价

推荐阅读 0