Journal of Jilin University Science Edition ›› 2024, Vol. 62 ›› Issue (2): 391-0398.

Previous Articles     Next Articles

Deep Neural Network Image Restoration Method Based on Multimodal Fusion 

LI Weiwei1, WANG Liyan2, FU Bo2, WANG Juan1, HUANG Hong1   

  1. 1. School of Information Engineering, Shandong Youth University of Political Science, Jinan 250103, China;2. School of Computer and Artificial Intelligence, Liaoning Normal University, Dalian 116081, Liaoning Province, China
  • Received:2022-08-07 Online:2024-03-26 Published:2024-03-26

Abstract: Aiming at the problems of the complicated underwater image imaging environment resulted in the subsequent image analysis often being affected by color bias and other factors, we proposed a deep convolutional neural network image restoration method based on multi-scale features and triple attention multimodal fusion. Firstly, the deep convolutional neural network introduced the image multi-scale transformation feature on the basis of extracting the image spatial feature. Secondly, by using channel attention, supervised attention and non-local attention, the scale correlation and feature correlation of image features were mined. Finally, by designing a multimodal feature fusion mechanism, the above two types of features could be effectively fused. The proposed method was tested on the open underwater image test set and compared with the current mainstream methods. The results show that this method is superior to the comparison method in quantitative comparison such as peak signal-to-noise ratio and structural similarity, as well as qualitative comparison such as color and details.

Key words: multimodal fusion, deep neural network, triple attention, image restoration

CLC Number: 

  • TP391