关键词: 混合机器学习模型, 短文本, 文本分词, 语义相似性, 卡方检验, 相似性度量

Abstract: In order to improve the accuracy of short text semantic similarity measurement, we designed a short text semantic similarity measurement algorithm based on a hybrid machine learning model. Firstly, we preprocessed the short text, constructed a word vector model of the short text based on the hybrid machine learning model, and extended the  features of the short text. Secondly, we  combined the various metric features of the short text, implemented dimensional reduction on the various metric features. Finally, we constructed an ensemble learning  model to calculate the semantic similarity results and achieve the  semantic similarity measurement. We tested the performance of the method by using the “Quora Question Pairs” competition dataset, the test results show that the accuracy of the  method is high, the logarithmic loss, and the measurement mean square error are both low, indicating that the similarity measurement accuracy of the method is high.

Key words: hybrid machine learning model, short text, text segmentation, semantic similarity, Chi-square test, similarity measurement