生物医学命名实体识别的两阶段学习算法

doi:10.13229/j.cnki.jdxbgxb.20211156

Abstract

Abstract:

In order to solve the problem of high cost of labeling named entity data and difficulty in obtaining large amounts of labeled data in the biomedical field，this article proposes a two-stage learning framework to realize BioNER under low resources. In the first stage， Word2Vec and BERT are used as the basic model to pre-train and fine-tune to obtain the word embedding representation in a specific field； In the second stage， the generated word embedding representations are input to the neural network composed of BiLSTM and CRF and then used for the training of the final task. This paper conducts experiments on the Yidu-S4k dataset， and even in the case of a small number of labels， the results show that the algorithm in this paper achieves an accuracy of 80.94% and has great performance.

Key words: computer application, natural language processing, named entity recognition, convolutional neural network, text representation, pre-training

CLC Number:

TP391.1

Xiang-jiu CHE,Huan XU,Ming-yang PAN,Quan-le LIU. Two-stage learning algorithm for biomedical named entity recognition[J].Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2380-2387.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Table 1

Fig.4

Fig.5

Table 2

Table 3

Table 4

References 25

1	何玉洁, 杜方, 史英杰, 等. 基于深度学习的命名实体识别研究综述[J]. 计算机工程与应用, 2021,7(11): 21-36.
	He Yu-jie, Du Fang, Shi Ying-jie, et al. Review of named entity recognition based on deep learning[J]. Computer Engineering and Application, 2021,7(11): 21-36.
2	Campos D, Matos S, Oliveira J L. Biomedical named entity recognition: a survey of machine-learning tools[J]. Theory and Applications for Advanced Text Mining, 2012, 11: 175-195.
3	Shen J, Wang X, Li S, et al. Exploiting rich features for Chinese named entity recognition[C]∥IEEE International Conference on Intelligent Systems and Knowledge Engineering, Hangzhou, China, 2010: 278-282.
4	Soomro P D, Kumar S, Banbhrani A A S, et al. Bio-NER: biomedical named entity recognition using rulebased and statistical learners[J]. Int.J. Adv. Comput. Sci. Appl, 2017, 8: 163-170.
5	Durbin R, Eddy S R, Krogh A,et al. Biological sequence analysis: multiple sequence alignment methods[J/OL]. (2021-05-21).
6	Zhang Y, Wang X, Hou Z, et al. Clinical named entity recognition from Chinese electronic health records via machine learning methods (Preprint)[J]. JMIR Medical Informatics, 2018, 6(4): 30559093.
7	燕杨, 文敦伟, 王云吉, 等. 基于层叠条件随机场的中文病历命名实体识别[J]. 吉林大学学报: 工学版, 2014, 44(6): 1843-1848.
	Yan Yang, Wen Dun-wei, Wang Yun-ji, et al. Named entity recognition in Chinese medical records based on cascaded conditional random field[J]. Journal of Jilin University(Engineering and Technology Edition), 2014, 44(6): 1843-1848.
8	Leaman R, Lu Z. TaggerOne: joint named entity recognition and normalization with semi-Markov models[J]. Bioinformatics, 2016, 32(18): 2839-46.
9	潘国巍, 吉久明, 李楠, 等. 基于两类统计机器学习模型的中文化学物质名称识别研究[J]. 现代情报, 2011, 31(11): 163-165.
	Pan Guo-wei, Ji Jiu-ming, Li Nan, et al. Research on Chinese chemical substance name recognition based on two types of statistical machine learning models[J]. Modern Information, 2011, 31(11): 163-165.
10	Cotterell R, Duh K. Low-resource named entity recognition with cross-lingual, character-level neural conditional random fields[C]//Proceedings of the Eighth International Joint Conference on Natural Language Processing, 2017: 91-96.
11	Feng X, Feng X, Qin B, et al. Improving low resource named entity recognition using cross-lingual knowledge transfer[C]∥Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence Main track, Stockholm, Sweden, 2018: 4071-4077.
12	Yu H, Mao X L, Chi Z, et al. A robust and domain-adaptive approach for low-resource named entity recognition[C]∥IEEE International Conference on Knowledge Graph, 2020: 297-304.
13	Fries J, Wu S, Ratner A, et al. Swellshark: a generative model for biomedical named entity recognition without labeled data[J/OL]. arXiv Preprint arXiv:, 2017.
14	Liu A L, Du J F, Stoyanov V. Knowledge-augmented language model and its application to unsupervised namedentity recognition[J/OL]. arXiv preprint arXiv:, 2019.
15	Shang J B, Liu L Y, Ren X, et al. Learning named entity tagger using domain-specific dictionary[J/OL]. arXiv preprint arXiv:, 2018.
16	Xiao Z F. Towards a two-phase unsupervised system for cybersecurity concepts extraction[C]//13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, Guilin, China, 2017: 2161-2168.
17	Ji B, Li S S, Yu J, et al. Research on Chinese medical named entity recognition based on collaborative cooperation of multiple neural network models[J]. Journal of Biomedical Informatics, 2020, 104: 103395.
18	Zhao Q, Wang D, Li J Q, et al. Exploiting the concept level feature for enhanced name entity recognition in Chinese EMRs[J]. The Journal of Supercomputing, 2020, 76: 63991-6420.
19	Wang Q, Zhou Y M, Ruan T, et al. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition[J]. Journal of Biomedical Informatics, 2019, 92: 103133.
20	郭晓然, 罗平, 王维兰. 基于Transformer编码器的中文命名实体识别[J]. 吉林大学学报: 工学版, 2021, 51(3): 989-995.
	Guo Xiao-ran, Luo Ping, Wang Wei-lan. Chinese named entity recognition based on Transformer encoder[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 989-995.
21	Devlin J, Chang M W, Lee K, et al. Bert: pre-training of deep bidirectional transformers for language understanding[OL]. arXiv Preprint arXiv:, 2018.
22	Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space[OL]. arXiv Preprint arXiv:, 2013.
23	Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition[J]. arXiv Preprint arXiv:, 2016.
24	Matthew E P, Mark N, Mohit I, et al. Deep contextualized word representations[OL]. arXiv Preprint arXiv:, 2018.
25	Zhang S, Zhang X, Wang H, et al. Multi-scale attentive interaction networks for chinese medical question answer selection[J]. IEEE Access, 2018, 6: 74061-74071.

Related Articles 15

[1]	Ya-hui ZHAO,Fei-yu LI,Rong-yi CUI,Guo-zhe JIN,Zhen-guo ZHANG,De LI,Xiao-feng JIN. Korean⁃Chinese translation quality estimation based on cross⁃lingual pretraining model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2371-2379.
[2]	Lian-ming WANG,Xin WU. Method for 3D motion parameter measurement based on pose estimation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2099-2108.
[3]	Ze-qiang ZHANG,Wei LIANG,Meng-ke XIE,Hong-bin ZHENG. Elite differential evolution algorithm for mixed⁃model two⁃side disassembly line balancing problem [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1297-1304.
[4]	Jian LI,Qi XIONG,Ya-ting HU,Kong-yu LIU. Chinese named entity recognition method based on Transformer and hidden Markov model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1427-1434.
[5]	Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
[6]	Pei-yong LIU,Jie DONG,Luo-feng XIE,Yang-yang ZHU,Guo-fu YIN. Surface defect detection algorithm of magnetic tiles based on multi⁃branch convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1449-1457.
[7]	Yu JIANG,Jia-zheng PAN,He-huai CHEN,Ling-zhi FU,Hong QI. Segmentation-based detector for traditional Chinese newspaper [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1146-1154.
[8]	Peng YU,Yan PIAO. New method for extracting person re-identification attributes based on multi-scale features [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1155-1162.
[9]	Hong-yang PAN,Zhao LIU,Bo YANG,Geng SUN,Yan-heng LIU. Overview of swarm intelligence methods for unmanned aerial vehicle systems based on new⁃generation information technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 629-642.
[10]	Ying HE,Jun-song FAN,Wei WANG,Geng SUN,Yan-heng LIU. Joint optimization of secure communication and trajectory planning in unmanned aerial vehicle air⁃to⁃ground [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 913-922.
[11]	Zhen-yu WU,Xiao-fei LIU,Yi-pu WANG. Trajectory planning of unmanned system based on DKRRT^*⁃APF algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 781-791.
[12]	Bo TAO,Fu-wu YAN,Zhi-shuai YIN,Dong-mei WU. 3D object detection based on high⁃precision map enhancement [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 802-809.
[13]	Shan XUE,Ya-liang ZHANG,Qiong-ying LYU,Guo-hua CAO. Anti⁃unmanned aerial vehicle system object detection algorithm under complex background [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 891-901.
[14]	Xian-yu QI,Wei WANG,Lin WANG,Yu-fei ZHAO,Yan-peng DONG. Semantic topological map building with object semantic grid map [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 569-575.
[15]	Xiao-hu SHI,Jia-qi WU,Chun-guo WU,Shi CHENG,Xiao-hui WENG,Zhi-yong CHANG. Residual network based curve enhanced lane detection method [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 584-592.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

实体类型	训练集	测试集
疾病和诊断	4 206	1 323
影像检查	972	348
实验室检查	1 195	590
手术	1 492	203
药物	1 931	597
解剖部位	8 426	3 094

模型	T.L．	V.L.	P	R	F₁	最佳F₁
CRF^［16］	61.55	58.57	0.4972	0.1788	0.2630	0.3079
W2V+BiLSTM+CRF^［20］	21.27	84.22	0.6396	0.5425	0.5871	0.6186
BERT+BiLSTM+CRF^［6］	6.07	57.09	0.7725	0.7767	0.7746	0.7802
W2V（FT）+BiLSTM+CRF	1.85	63.36	0.3900	0.3325	0.3551	0.4122
BERT（FT）+BiLSTM+CRF	4.72	57.52	0.7779	0.7821	0.7800	0.7853
W2V（FT）+BERT（FT）+BiLSTM+CRF	3.89	57.81	0.7773	0.7794	0.7783	0.7908
W2V（FTv2）+BERT（FTv2）+BiLSTM+CRF	3.24	36.44	0.8094	0.7881	0.7986	0.8014

模型	T.L．	V.L.	P	R	F₁	最佳F₁
Word2Vec+BERT（FT）+BiLSTM+CRF	4.17	62.93	0.7881	0.7880	0.7830	0.7853
Word2Vec（FT）+BERT（FT）+BiLSTM+CRF	3.89	57.81	0.7773	0.7794	0.7783	0.7908

模型	T.L．	V.L.	P	R	F₁	最佳F₁
Word2Vec（FT）+BERT+BiLSTM+CRF	4.11	58.21	0.7758	0.7821	0.7789	0.7854
Word2Vec（FT）+BERT（FT）+BiLSTM+CRF	3.89	57.81	0.7800	0.7794	0.7783	0.7908

Two-stage learning algorithm for biomedical named entity recognition

RICH HTML

PDF (PC)