关键词: 古汉语, 命名实体识别, SikuBERT 模型, 多头注意力机制

Abstract:

Aiming at the problem that the traditional named entity recognition method can not fully learn the complex sentence structure information of ancient Chinese and it is easy to cause information loss in the process of long sequence feature extraction, an ancient Chinese fusion of SikuBERT ( Siku Bidirectional Encoder Representation from Transformers) model and MHA (Multi-Head Attention) is proposed. First, the SikuBERT model is used to pre-train the ancient Chinese corpus, the information vector obtained from the training into the BiLSTM (Bidirectional Long Short-Term Memory) network is input to extract features, and then the output features of the BiLSTM layer are assigned different weights through MHA to reduce the information loss problem of long sequences. And finally the predicted sequence labels are obtained through CRF (Conditional Random Field) decoding. Experiments show that compared with commonly used BiLSTM-CRF, BERT-BiLSTM-CRF and other models, the F1 value of this method has been significantly improved, which verifies that this method can effectively improve the effect of ancient Chinese named entity recognition.

Key words: ancient Chinese, named entity recognition, siku bidirectional encoder representation from transformers(SikuBERT) model, multi-head attention mechanism