关键词: 中文分词, 专业领域, XLNet预训练模型, 多头注意力机制, 控制工程

Abstract: Aiming at the problem of low accuracy of methods in Chinese word segmentation in professional domains due to the mismatch of cross-domain data distribution and the limitation of a large number of unregistered professional words, a professional domain word segmentation method based on XLBMC(XLNet- BiGRU-Multi-head Self-attention-Conditional Random Field) is proposed. Firs, the dynamic word vectors containing contextual semantic information is generated through an improved XLNet pre-training model, enabling the model to better utilize the boundary and semantic knowledge. Then the acquired word vectors are input into BiGRU for feature extraction to obtain the hidden state representation of each character. On the basis of BiGRU coding, a sparsified MHSA(Multi-head Self-Attention) mechanism is introduced to weight the representation of each character, which improves the prediction accuracy of the model for fine-grained and strongly long-term dependent time series under restricted memory budget. Finally, the CRF(Conditional Random Field) decodes the dependencies between neighboring tags and outputs the optimal segmentation sequence. Segmentation experiments are conducted on a self鄄constructed control engineering corpus. The results show that the accuracy of the proposed segmentation model is 94. 27%, the recall is 93. 24%, and the F1 value is 95.52%, which proves the reliability of the model in Chinese segmentation tasks in the professional domain.

Key words: Chinese word segmentation, domain specific, XLNet, multi-head self-attention, control engineering