Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (8): 2371-2379.doi: 10.13229/j.cnki.jdxbgxb.20220005

Previous Articles    

Korean⁃Chinese translation quality estimation based on cross⁃lingual pretraining model

Ya-hui ZHAO(),Fei-yu LI,Rong-yi CUI,Guo-zhe JIN,Zhen-guo ZHANG,De LI,Xiao-feng JIN   

  1. Department of Computer Science & Technology，Yanbian University，Yanji 133002，China
  • Received:2022-01-04 Online:2023-08-01 Published:2023-08-21

RICH HTML

 13   

PDF (PC)

346

Abstract:

On low-resource corpus， the mainstream translation quality estimation models have poor performance. Meanwhile， the sentence embedding strategy is naive. In view of reasons mentioned above， a Korean-Chinese translation quality estimation based on cross-lingual pretraining model is proposed. Firstly， a cross-lingual sentence embedding method is proposed by drawing on the idea of attention. The method can effectively fuse the cross-layer information and token positions of the pre-trained model. Second， a cross-lingual pretraining model is introduced to the task as a way to alleviate the few-shot caused by the low-resource of Korean. Finally， the regression is performed on the sentence embedding vectors， so that the Korean-Chinese translation quality estimation can be completed. Experimental results show that the method can effectively improve the performance of the Korean-Chinese translation quality estimation task. Compared with QuEst++， Bilingual Expert， and TransQuest， the dominant models for quality estimation tasks， Pearson correlation coefficients improved by 0.226， 0.156， and 0.034， and Spearman correlation coefficients improved by 0.123， 0.038， and 0.026， respectively.

Key words: computer application technology, translation quality estimation, cross-lingual pretraining model, sentence embedding

CLC Number: 

  • TP391.1

Fig.1

Schematic diagram of sentence embedding"

Fig.2

Schematic diagram of linguistic attention"

Fig.3

Schematic diagram of token attention"

Fig.4

Architecture of Korean-Chinese translation quality estimation"

Table 2

Correlation coefficient of each model"

模型PearsonSpearman
QuEst++0.3970.471
Bilingual Expert0.4760.556
TransQuest0.5890.568
本文0.6230.594

Table 3

Pearson correlation coefficient in WMT2020"

模型Ro-EnEt-EnNe-EnSi-En
TransQuest0.8980.7750.7910.652
本文0.9020.8200.8270.705

Table 4

Performance of different sentence embedding"

句子嵌入方法Pearson↑Spearman↓MAE↓RMSE↓
Last［CLS］0.5890.5680.1600.204
Last+GRU0.5970.5710.1450.180
Conv（0-24）0.5540.5440.1530.191
Conv（20-24）0.6050.5820.1500.188
Attention（1-24）0.6090.5910.1460.185
Attention（0-24）0.6230.5940.1440.183

Table 5

Performance of different attention order"

注意力方法PearsonSpearmanMAERMSE
LangAtten0.5920.5700.1590.190
TokenAtten0.6000.5720.1450.189
TokenAtten+LangAtten0.6130.5920.1490.187
LangAtten+TokenAtten0.6230.5940.1440.183

Fig.5

Examples of result"

1 Kim H, Lee J. Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation[J]. ACM Transactions on Asian and Low-resource Language Information Processing, 2017, 2: 562-568.
2 Takahashi K, Sudoh K, Nakamura S. Automatic machine translation evaluation using source language inputs and cross-lingual language model[C]∥Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, Washington, United States, 2020: 3553-3558.
3 Ranasinghe T, Orasan C, Mitkov R. TransQuest: translation quality estimation with cross-lingual transformers[C]∥Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain, 2020: 5070-5081.
4 Jawahar G, Sagot B, Eddah D. What does BERT learn about the structure of language? [C]∥Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019: 3651-3657.
5 Pires T, Schlinger E, Garrette D. How multilingual is multilingual BERT? [C]∥Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019: 4996-5001.
6 Specia L, Shah K, Desouza J, et al. QuEst: a translation quality estimation framework[C]∥Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Sofia, Bulgaria, 2013: 79-84.
7 Fan K, Wang J Y, Li B, et al. "Bilingual Expert" can find translation errors[C]∥Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Phoenix, United States, 2019: 6367-6374.
8 Liu Y H, Ott M, Goyal N, et al. RoBERTa: a robustly optimized bert pretraining approach[C]∥Proceedings of the 2020 International Conference on Learning Representations, Addis Ababa, Ethiopia, 2020: 1-15.
9 Devlin J, Chang M, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]∥Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, United States, 2019: 4171-4186.
10 Conneau A, Lample G. Cross-lingual language model pretraining[J]. Advances in Neural Information Processing Systems, 2019, 32: 7059-7069.
11 Conneau A, Khandelwal K, Goyal N, et al. Unsupervised cross-lingual representation learning at scale[C]∥Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, Washington, United States, 2020: 8840-8451.
12 Arora S, Liang Y Y, Ma T Y. A simple but tough-to-beat baseline for sentence embeddings[C]∥Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 2017: 1-16.
13 Conneau A, Kiela D, Schwenk H, et al. Supervised learning of universal sentence representations from natural language inference data[C]∥Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017: 670-680.
14 Logeswaran L, Lee H. An efficient framework for learning sentence representations[C]∥Proceedings of the 6th International Conference on Learning Representations, Pennsylvania, United States, 2018: 1-16.
15 Reimers N, Gurevych I, Reimers N, et al. Sentence-BERT: sentence embeddings using siamese BERT-networks[C]∥Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China, 2019: 3982-3992.
16 Wang B, Kuo J. SBERT-WK: a sentence embedding method by dissecting BERT-based word models[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020, 28: 2146-2157.
17 Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, United States, 2018: 7132-7141.
18 Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]∥Proceedings of the European Conference on Computer Vision, Antibes, France, 2018: 3-19.
19 赵亚慧, 杨飞扬, 张振国, 等. 基于强化学习和注意力机制的朝鲜语文本结构发现[J]. 吉林大学学报: 工学版, 2021, 51(4): 1387-1395.
Zhao Ya-hui, Yang Fei-yang, Zhang Zhen-guo, et al. Korean text structure discovery based on reinforcement learning and attention mechanism[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1387-1395.
20 李健, 熊琦, 胡雅婷, 等. 基于Transformer和隐马尔科夫模型的中文命名实体识别方法[J]. 吉林大学学报: 工学版,2023, 53(5): 1427-1434.
Li Jian, Xiong Qi, Hu Ya-ting, et al. Chinese named entity recognition method based on transformer and hidden markov model[J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1427-1434.
[1] Shan XUE,Ya-liang ZHANG,Qiong-ying LYU,Guo-hua CAO. Anti⁃unmanned aerial vehicle system object detection algorithm under complex background [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 891-901.
[2] Jun-jie WANG,Yuan-jun NONG,Li-te ZHANG,Pei-chen ZHAI. Visual relationship detection method based on construction scene [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(1): 226-233.
[3] Bing ZHU,Zi-wei LI,Qi LI. Building segmentation method of remote sensing image based on improved SegNet [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(1): 248-254.
[4] Gui-he QIN,Jun-feng HUANG,Ming-hui SUN. Text input based on two⁃handed keyboard in virtual environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1881-1888.
[5] Tian BAI,Ming-wei XU,Si-ming LIU,Ji-an ZHANG,Zhe WANG. Dispute focus identification of pleading text based on deep neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1872-1880.
[6] Fu-heng QU,Tian-yu DING,Yang LU,Yong YANG,Ya-ting HU. Fast image codeword search algorithm based on neighborhood similarity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1865-1871.
[7] Ming LIU,Yu-hang YANG,Song-lin ZOU,Zhi-cheng XIAO,Yong-gang ZHANG. Application of enhanced edge detection image algorithm in multi-book recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 891-896.
[8] Shi-min FANG. Multiple source data selective integration algorithm based on frequent pattern tree [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 885-890.
[9] Sheng-sheng WANG,Chen-xu LI,Xiang-yu WANG,Zhi-lin YAO,Yi-shen LIU,Jia-qian WU,Qing-ran YANG. Brain tumor image classification based on improved residual capsule network and sparrow search [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2653-2661.
[10] Xiang-jiu CHE,He-yuan CHEN. Muti⁃Object dishes detection algorithm based on improved YOLOv4 [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2662-2668.
[11] Sheng-sheng WANG,Jing-yu CHEN,Yi-nan LU. COVID⁃19 chest CT image segmentation based on federated learning and blockchain [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2164-2173.
[12] Hong-wei ZHAO,Zi-jian ZHANG,Jiao LI,Yuan ZHANG,Huang-shui HU,Xue-bai ZANG. Bi⁃direction segmented anti⁃collision algorithm based on query tree [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1830-1837.
[13] Jie CAO,Xue QU,Xiao-xu LI. Few⁃shot image classification method based on sliding feature vectors [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1785-1791.
[14] Chun-bo WANG,Xiao-qiang DI. Cloud storage integrity verification audit scheme based on label classification [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1364-1369.
[15] Rong QIAN,Ru ZHANG,Ke-jun ZHANG,Xin JIN,Shi-liang GE,Sheng JIANG. Capsule graph neural network based on global and local features fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1048-1054.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd