Abstract: Aiming at  the problem of susceptibility to outlier interference and unstable optimization during training process of  multi-index speech enhancement algorithms, we proposed an adaptive speech enhancement algorithm based on a multi-head attention mechanism. Firstly, by  introducing a multi-head attention structure into the intermediate layer of the discriminator network, we  enhanced the joint modeling ability of the  model for local features and overall structure of speech spectrum, and  combined it with an online knowledge distillation strategy to achieve information sharing among multiple generators, thereby improving the collaborative optimization effect under multi-index conditions. Secondly, in order to reduce the impact of outliers on the training process, we replaced the loss function with a logarithmic mean-squared error form to improve  stability and robustness of the model. Experimental results on the publicly available speech dataset VoiceBank-DEMAND show that this method outperforms existing multi-index speech enhancement models in terms of speech quality, background noise suppression, and speech intelligibility metrics. Therefore,  introducing an attention mechanism and a stabilizing loss function can significantly improve the overall performance of multi-index speech enhancement algorithms.

Key words: speech enhancement, frequency domain, multi-head attention mechanism, online knowledge distillation, logarithmic mean-squared error loss