基于多尺度注意力融合和卷积神经网络的水下图像恢复

doi:10.13229/j.cnki.jdxbgxb20200431

Abstract

Abstract:

Due to the absorption and scattering of light by suspended particles in water, the original underwater image has a low definition, fuzzy details, and color distortion. To solve these problems, an underwater image recovery method based on multi-scale attention fusion and Convolutional Neural Network (CNN) is proposed. First,the Space Channel (SC) module is constructed by using the attention mechanism. Then, by adding the SC module into the multi-scale feature extraction, the information in the image can be extracted effectively, and the image sharpness and color correction can be realized. Finally, the Laplacian operator is used to construct the multiple loss functions to further enhance the detail features of the image, so that the recovered image quality can be significantly improved. The method in this paper is compared with other methods qualitatively and quantitatively on the two test sets. The experimental results show that this method is superior to other methods in image sharpness, detail enhancement, and color correction.

Key words: information processing technology, underwater image recovery, attention mechanism, Laplacian operator, convolutional neural network

CLC Number:

TP391.4

De-xing WANG,Ruo-you WU,Hong-chun YUAN,Peng GONG,Yue WANG. Underwater image restoration based on multi-scale attention fusion and convolutional neural network[J].Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1396-1404.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Table 1

PSNR and SSIM values of MSCANet（without LDDL） and MSCANet（with LDDL） on test set A"

方法	PSNR	SSIM
MSCANet（无 $L D D L$ ）	26.2937	0.9252
MSCANet（有 $L D D L$ ）	26.3381	0.9278

Table 1

Table 2

Visible edge fractions and information entropy of MSCANet（without LDDL） and MSCANet（with LDDL） on test set B"

方法	$r ˉ$	$σ$	IE
MSCANet（无 $L D D L$ ）	1.8237	0.0078	7.3965
MSCANet（有 $L D D L$ ）	2.0350	0.0043	7.4115

Table 2

Table 3

Table 4

Fig.4

Table 5

Fig.5

Table 6

References 30

1	Meline A, Triboulet J, Jouvencel B. Comparative study of two 3D reconstruction methods for underwater archaeology[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, 2012: 740-745.
2	Skarlatos D, Agrafifiotis P, Menna F, et al. Ground control networks for underwater photogrammetry in archaeological excavations[C]∥Proceedings of the 3rd IMEKO International Conference on Metrology for Archaeology and Cultural Heritage,Lecce, Italy, 2017: 23-25.
3	Chuang M C, Hwang J N, Williams K. A feature learning and object recognition framework for underwater fish images[J]. IEEE Transactions on Image Processing, 2016, 25(4): 1862-1872.
4	Shkurti F, Xu A, Meghjani M, et al. Multi-domain monitoring of marine environments using a heterogeneous robot team[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems,Vilamoura, Portugal, 2012: 1747-1753.
5	Wang X, Li Q, Yin J, et al. An adaptive denoising and detection approach for underwater sonar image[J]. Remote Sensing, 2019, 11(4): No.396.
6	Zeng L, Sun B, Zhang W, et al. Underwater Image Target Detection with Cascade Classifier and Image Preprocessing Method[C]∥International Conference on Intelligent Robotics and Applications, Shenyang, China, 2019: 195-205.
7	Güraksin G E, Köse U, Deperlioğlu Ö. Underwater image enhancement based on contrast adjustment via differential evolution algorithm[C]∥International Symposium on Innovations in Intelligent Systems and Applications, Sinaia, Romania, 2016: 1-5.
8	Henke B, Vahl M, Zhou Z. Removing color cast of underwater images through non-constant color constancy hypothesis[C]∥The 8th International Symposium on Image and Signal Processing and Analysis, Trieste, Italy, 2013: 20-24.
9	Ancuti C O, Ancuti C, de Vleeschouwer C, et al. Locally adaptive color correction for underwater image dehazing and matching[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, USA, 2017: 1-9.
10	Zhang W, Li G, Ying Z. A New Underwater Image Enhancing Method via Color Correction and Illumination Adjustment[C]∥IEEE Visual Communications and Image Processing, USA: IEEE, 2017: 1-4.
11	Zou W, Wang X, Li K, et al. Self-tuning underwater image fusion method based on dark channel prior[C]∥IEEE International Conference on Robotics and Biomimetics, Qingdao, China, 2016: 788-793.
12	Chiang J Y, Chen Y C. Underwater image enhancement by wavelength compensation and dehazing[J]. IEEE Transactions on Image Processing, 2011, 21(4): 1756-1769.
13	Akkaynak D, Treibitz T. A revised underwater image formation model[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6723-6732.
14	Li C Y, Guo J C, Cong R M, et al. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior[J]. IEEE Transactions on Image Processing, 2016, 25(12): 5664-5677.
15	Berman D, Levy D, Avidan S, et al. Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset [EB/OL]∥[2020-06-13]. .
16	Li C, Anwar S, Porikli F. Underwater scenes prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038.
17	Li C, Guo C, Ren W, et al. An Underwater Image Enhancement Benchmark Dataset and Beyond [EB/OL]. [2020-06-13]. .
18	徐岩, 孙美双. 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报:工学版, 2018, 48(6): 1895-1903.
	Xu Yan, Sun Mei-shuang. Underwater image enhancement method based on convolutional neural network[J]. Journal of Jilin University (Engineering and Technology Edition), 2018, 48(6): 1895-1903.
19	Li J, Skinner K A, Eustice R M, et al. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2017, 3(1): 387-394.
20	Lu J, Li N, Zhang S, et al. Multi-scale adversarial network for underwater image restoration[J]. Optics & Laser Technology, 2019, 110: 105-113.
21	Islam M J, Xia Y, Sattar J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227-3234.
22	Zhang B, Jin S, Xia Y, et al. Attention Mechanism Enhanced Kernel Prediction Networks for Denoising of Burst Images[C]∥IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020: 2083-2087.
23	Chen D, He Z, Cao Y, et al. Deep neural network for fast and accurate single image super-resolution via channel-attention-based fusion of orientation-aware features [EB/OL].[2020-06-13]. .
24	Li H, Qiu K, Chen L, et al. SCAttNet: semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2020,18(5):905-909.
25	Woo S, Park J, Lee J Y, et al. Cbam: Convolutional block attention module[C]∥Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 3-19.
26	Hautière N, Tarel J P, Aubert D, et al. Blind contrast enhancement assessment by gradient ratioing at visible edges[J]. Image Analysis & Stereology, 2008, 27(2): 87-95.
27	Liu L, Liu B, Huang H, et al. No-reference image quality assessment based on spatial and spectral entropies[J]. Signal Processing: Image Communication, 2014, 29(8): 856-863.
28	Mittal A, Soundararajan R, Bovik A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters, 2012, 20(3): 209-212.
29	Liu L, Hua Y, Zhao Q, et al. Blind image quality assessment by relative gradient statistics and adaboosting neural network[J]. Signal Processing: Image Communication, 2016, 40: 1-15.
30	Panetta K, Gao C, Agaian S. Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2015, 41(3): 1-11.

Related Articles 15

[1]	Ya-hui ZHAO,Fei-yang YANG,Zhen-guo ZHANG,Rong-yi CUI. Korean text structure discovery based on reinforcement learning and attention mechanism [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1387-1395.
[2]	Hou⁃jie LI,Fa⁃sheng WANG,Jian⁃jun HE,Yu ZHOU,Wei LI,Yu⁃xuan DOU. Pseudo sample regularization Faster R⁃CNN for traffic sign detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1251-1260.
[3]	Hua-wei JIANG,Zhen YANG,Xin ZHANG,Qian-lin DONG. Research progress of image dehazing algorithms [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1169-1181.
[4]	Jing JIN,Jian-wu DANG,Yang-ping WANG,Dong SHEN. Multi⁃cue particle filter tracking based on fuzzy statistical texture features [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1111-1120.
[5]	Yuan-ning LIU,Di WU,Xiao-dong ZHU,Qi-xian ZHANG,Shuang-shuang LI,Shu-jun GUO,Chao WANG. User interface components detection algorithm based on improved YOLOv3 [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1026-1033.
[6]	Hai-ying ZHAO,Wei ZHOU,Xiao-gang HOU,Xiao-li ZHANG. Double-layer annotation of traditional costume images based on multi-task learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 293-302.
[7]	Hong-wei ZHAO,Xiao-han LIU,Yuan ZHANG,Li-li FAN,Man-li LONG,Xue-bai ZANG. Clothing classification algorithm based on landmark attention and channel attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1765-1770.
[8]	Guo-hua LIU,Wen-bin ZHOU. Pulse wave signal classification algorithm based on time⁃frequency domain feature aliasing using convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1818-1825.
[9]	Ke-yan WANG,Di WANG,Xi ZHAO,Jing-yi CHEN,Yun-song LI. Image dehazing based on joint estimation via convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1771-1777.
[10]	Zai-feng SHI,Jin-zhuo LI,Qing-jie CAO,Hui-long LI,Qi-xing HU. Low⁃dose spectral computer⁃tomography imaging denoising method via a generative adversarial network [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1755-1764.
[11]	Xiang-jiu CHE,You-zheng DONG. Improved image recognition algorithm based on multi⁃scale information fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1747-1754.
[12]	Hua CHEN,Wei GUO,Jing-wen YAN,Wen-hao ZHUO,Liang-bin WU. A new deep learning method for roads recognition from SAR images [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1778-1787.
[13]	Wei ZHANG,Yong HAN,Ming JIN,Xiao-lin QIAO. Toeplitz matrices reconstruction based DOA estimation for coherent signals [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(2): 703-710.
[14]	Yan-fen CHENG,Li-juan YAO,Qiao YUAN,Xian-qiao CHEN. Clearing strategy of underwater video image [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(2): 668-677.
[15]	Xiao-hui YU,Zhi-cheng ZHANG,Xin-bo LI,Xiao-dong SUN. Parameter estimation of exponentially damped sinusoidsbased on state⁃space model [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2083-2088.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Metrics	PSNR	SSIM	Metrics	PSNR	SSIM
CLAHE	16.4785	0.6227	文献［16］	16.2842	0.7079
文献［10］	19.1530	0.8096	文献［17］	22.3531	0.8366
文献［14］	18.0882	0.7351	文献［21］	25.6645	0.8814
文献［15］	17.9068	0.7330	本文	26.3381	0.9278

Metrics	SSEQ	NIQE	OG?IQA	UIQM
CLAHE	28.6748	6.1307	0.7916	4.2140
文献［10］	22.6463	5.2626	0.8025	5.0043
文献［14］	21.1102	5.1531	0.8392	5.0128
文献［15］	21.3022	5.3124	0.8268	4.5545
文献［16］	32.2132	4.9541	0.6130	5.1146
文献［17］	26.3040	7.1562	0.8016	4.7900
文献［21］	26.5895	5.2195	0.7421	4.8943
本文	21.0329	5.1419	0.8415	5.1436

Underwater image restoration based on multi-scale attention fusion and convolutional neural network

RICH HTML

PDF (PC)