隐低秩结合低秩表示的多聚焦图像融合

doi:10.13229/j.cnki.jdxbgxb20180805

Abstract

Abstract:

In order to achieve that the fusion results can maintain the global structure, preserve the local details and be robust to the unregistered source images at the same time, this paper proposes a multi-focus image fusion algorithm based on latent low-rank representation combining low-rank representation. First, the algorithm decomposes images into low-rank part (global structure) and saliency part (local structure) by latent low-rank representation. Then it extracts low-rank coefficients from each part by low-rank representation. Third, the algorithm fuses the low-rank coefficients of each part. Fourth, the fusion images of two parts are obtained by inverse transformation. Finally the two parts are added together as the final fusion image. Besides, guided filtering is used to enhance spatial continuity. The dictionaries of low-rank representation are consist of several sub-dictionaries, respectively. The source images are divided into blocks with sliding window technology, then four kinds of Sobel operators are used to classify the blocks into four categories. Learning sub-dictionaries from each category can enhance the ability of detail retention. We select three groups of data, including two groups of perfectly registered images and one group of unregistered images for verifying the validity of the proposed algorithm. Experimental results show that this algorithm outperforms the current mainstream multi-focus image fusion algorithms from both subjective visual effect analysis and objective quality assessment analysis.

Key words: computer application, image fusion, multi-focus, latent low-rank representation, dictionary learning, low-rank representation

CLC Number:

TP399

Man CHEN,Yong ZHONG,Zhen-dong LI. Multi-focus image fusion based on latent low⁃rank representation combining low⁃rank representation[J].Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 297-305.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Table 1

Comparison of fusion quality of 5 algorithms A"

指标	IF? DSIFT	IF? GF	IF? NSCT	IF? MSR	本文算法
$Q N M I$	1.046 2	1.014 0	0.823 4	1.094 7	1.100 7
$Q G$	0.688 1	0.709 6	0.664 9	0.7093	0.712 3
$Q S$	0.937 8	0.937 9	0.925 6	0.937 0	0.938 0
$Q C B$	0.782 0	0.805 6	0.741 8	0.812 7	0.820 6
$Q N M I$	1.016 5	1.021 5	0.763 6	1.140 6	1.129 0
$Q G$	0.648 4	0.682 8	0.616 9	0.683 1	0.683 8
$Q S$	0.914 5	0.920 0	0.914 1	0.917 1	0.927 9
$Q C B$	0.781 9	0.815 8	0.775 9	0.818 8	0.818 9

Table 1

Fig.9

Fig.10

Table 2

Comparison of fusion quality of 5 algorithms B"

指标	IF? DSIFT	IF? GF	IF? NSCT	IF? MSR	本文算法
$Q N M I$	1.064 9	1.133 2	1.007 2	1.201 8	1.219 0
$Q G$	0.632 1	0.704 4	0.664 3	0.709 3	0.710 0
$Q S$	0.947 7	0.956 0	0.948 2	0.951 8	0.952 5
$Q C B$	0.671 7	0.692 1	0.678 8	0.704 9	0.714 7

Table 2

Table 3

References 27

1	James A P, Dasarathy B V. Medical image fusion: a survey of the state of the art[J]. Information Fusion, 2014, 19(1): 4-19.
2	Liu Y, Chen X, Wang X S, et al. Deep learning for pixel-level image fusion: recent advances and future prospects[J]. Information Fusion, 2018, 42(3): 158-173.
3	Jin X, Jiang Q, Yao S W, et al. A survey of infrared and visual image fusion methods[J]. Infrared Physics and Technology, 2017, 85: 478-501.
4	Ghassemian H. A review of remote sensing image fusion methods[J]. Information Fusion, 2016, 32: 75-89.
5	Wang W C, Chang F L. A multi-focus image fusion method based on Laplacian pyramid[J]. Journal of Computers, 2011, 6(12): 2559-2566.
6	Yang Y. A novel DWT based multi-focus image fusion method[J]. Procedia Engineering, 2011, 24(1): 177-181.
7	Zhang Q, Guo B L. Multifocus image fusion using the nonsubsampled contourlet transform[J]. Signal Processing, 2009, 89(7):1334-1346.
8	Yang Wei, Chai Qi, Wang Li-ming. Multi-focus image fusion method based on dual-tree complex wavelet transform[J]. Computer Engineering and Applications, 2007, 43(28): 12-14.
9	杨扬, 戴明, 周箩鱼, 等. 基于非下采样Bandelet变换的多聚焦图像融合[J]. 吉林大学学报: 工学版, 2014, 44(2): 525-530.
	Yang Yang, Dai Ming, Zhou Luo-yu, et al. Multifocus image fusion based on nonsubsampled Bandelet transform[J]. Journal of Jilin University (Engineering and Technology Edition), 2014, 44(2): 525-530.
10	Li S T, Kang X D, Hu J W, et al. Image matting for fusion of multi-focus images in dynamic scenes[J]. Information Fusion, 2013, 14(2): 147-162.
11	Wang Z B, Ma Y D, Gu J. Multi-focus image fusion using PCNN[J]. Pattern Recognition, 2010, 43(6): 2003-2016.
12	Li S T, Kang X D, Hu J W. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875.
13	Liu S P, Liu Y, Wang Z F. Multi-focus image fusion with dense SIFT[J]. Information Fusion, 2015, 23: 139-155.
14	Chen L, Li J B, Philip C C L. Regional multifocus image fusion using sparse representation[J]. Optics Express, 2013, 21(4): 5182-5197.
15	尹明, 战荫伟, 裴海龙. 基于稀疏补算子学习的图像融合方法[J]. 吉林大学学报: 工学版, 2016, 46(6): 2052-2058.
	Yin Ming, Zhan Yin-wei, Pei Hai-long, Co-sparse analysis operator learning for image fusion[J]. Journal of Jilin University (Engineering and Technology Edition), 2016, 46(6): 2052-2058.
16	Liu Y, Chen X, Ward R K, et al. Image fusion with convolutional sparse representation[J]. IEEE Signal Processing Letters, 2016, 23(12): 1882-1886.
17	首照宇, 胡蓉, 欧阳宁, 等. 基于多尺度稀疏表示的图像融合方法[J]. 计算机工程与设计, 2015, 36(1): 232-235.
	Shou Zhao-yu, Hu Rong, Ouyang Ning, et al. Image fusion based on multi-scale sparse representation[J]. Computer Engineering and Design, 2015, 36(1): 232-235.
18	Li H, Wu X J. Multi-focus image fusion using dictionary learning and low-rank representation[J]. LNCS, 2017:675- 686.
19	Zhong J Y, Yang B, Li Y H, et al. Image fusion and super-resolution with convolutional neural network[C]∥ Chinese Conference on Pattern Recognition. Singapore: Springer, 2016, 663:78-88.
20	Liu Y, Chen X, Peng H, et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 2017, 36: 191-207.
21	Du C B, Gao S S. Image segmentation-based multi-focus image fusion through multi-scale convolutional neural network[J]. IEEE Access, 2017, 5(99):15750-15761.
22	Liu G, Yan S. Latent low-rank representation for subspace segmentation and feature extraction[C]∥International Conference on Computer Vision. Barcelona, Spain, 2011: 1615-1622.
23	Lin Z, Chen M, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices[DB/OL]. [2010.9-29]. http:∥export.arxiv.org/abs/1009.5055.
24	Liu G C, Lin Z C, Yan S C, et al. Robust recovery of subspace structures by low-rank representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 171-184.
25	He K M, Sun J, Tang X O. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409.
26	Aharon M, Elad M, Bruckstein A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322.
27	Liu Z, Blasch E, Xue Z, et al. Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(1): 94-109.

Related Articles 15

[1]	Xiao-dong ZHANG,Xiao-jun XIA,Hai-feng LYU,Xu-chao GONG,Meng-jia LIAN. Dynamic load balancing of physiological data flow in big data network parallel computing environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 247-254.
[2]	Xiao-hui WANG,Lu-shen WU,Hua-wei CHEN. Denoising of scattered point cloud data based on normal vector distance classification [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 278-288.
[3]	Jun-yi DENG,Yan-heng LIU,Shi FENG,Rong-cun ZHAO,Jian WANG. GSPN⁃based model to evaluate the performance and securi tytradeoff in Ad-hoc network [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 255-261.
[4]	Tie-jun WANG,Wei-lan WANG. Thangka image annotation based on ontology [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 289-296.
[5]	Xiong-fei LI,Jing WANG,Xiao-li ZHANG,Tie-hu FAN. Multi-focus image fusion based on support vector machines and window gradient [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 227-236.
[6]	Hong-yan WANG,He-lei QIU,Jia ZHENG,Bing-nan PEI. Visual tracking method based on low⁃rank sparse representation under illumination change [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 268-277.
[7]	Xiang-jiu CHE,Hua-luo LIU,Qing-bin SHAO. Fabric defect recognition algorithm based onimproved Fast RCNN [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2038-2044.
[8]	Bing-hai ZHOU,Qiong WU. Balancing and optimization of robotic assemble lines withtool and space constraint [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2069-2075.
[9]	Hong-wei ZHAO,Peng WANG,Li-li FAN,Huang-shui HU,Ping-ping LIU. Similarity retention instance retrieval method [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2045-2050.
[10]	Jun SHEN,Xiao ZHOU,Zu-qin JI. Implementation of service dynamic extended network and its node system model [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2058-2068.
[11]	You ZHOU,Sen YANG,Da-lin LI,Chun-guo WU,Yan WANG,Kang-ping WANG. Acceleration platform for face detection and recognition based on field⁃programmable gate array [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2051-2057.
[12]	Bin LI,Xu ZHOU,Fang MEI,Shuai-ning PAN. Location recommendation algorithm based on K-means and matrix factorization [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1653-1660.
[13]	Xiong-fei LI,Lu SONG,Xiao-li ZHANG. Remote sensing image fusion based on cooperative empirical wavelet transform [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1307-1319.
[14]	Yuan-ning LIU,Shuai LIU,Xiao-dong ZHU,Guang HUO,Tong DING,Kuo ZHANG,Xue JIANG,Shu-jun GUO,Qi-xian ZHANG. Iris secondary recognition based on decision particle swarm optimization and stable texture [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1329-1338.
[15]	Bin LI,Guo⁃jun SHEN,Geng SUN,Ting⁃ting ZHENG. Improved chicken swarm optimization algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1339-1344.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Multi-focus image fusion based on latent low⁃rank representation combining low⁃rank representation

RICH HTML

PDF (PC)