基于响应值中心加权卷积特征的图像检索算法

doi:10.13229/j.cnki.jdxbgxb20220043

Abstract

Abstract:

A novel cross-entropy-based loss is proposed， which is applied to a classical convolutional neural network to obtain a better embedding space. A convolutional feature aggregation algorithm based on the weighting of the center of the response value is designed to process the three-dimensional convolutional features obtained by the neural network. The algorithm obtains the spatial weighting coefficient of the feature map by calculating the current position response value and the Gaussian center. The algorithm reduces the loss of information when the 3D convolution feature map is reduced to a 1D image feature descriptor and realizes the enhancement of the target area. Finally， the obtained image feature descriptors are used for retrieval tasks. On the CUB-200-2011 dataset， the effectiveness of the loss function and the response value center weighting algorithm are respectively verified by ablation experiments. Compared with the current retrieval methods， higher precision and recall rates have been obtained on the four datasets of Paris6k， Oxford5k， CUB-200-2011 and CARS196.

Key words: computer applications, image retrieval, deep learning, convolutional feature aggregation

CLC Number:

TP391

Xiao-ning LI,Hong-wei ZHAO,Dan-yang ZHANG,Yuan ZHANG. Image retrieval algorithm based on response value center weighted convolution feature[J].Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2669-2675.

Figures/Tables 9

Fig.1

Fig.2

Table 1

Fusion parameter μ parameter test"

方法	CUB-200-2011 K值
方法	1	2	4	8
$μ$ =0.2	68.9	79.2	86.6	92.2
$μ$ =0.4	69.4	80.0	87.7	92.7
$μ$ =0.6	68.9	79.0	86.7	92.3
$μ$ =0.8	68.1	78.7	86.6	92.0

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

References 23

1	Tolias G, Sicre R, Jégou H. Particular object retrieval with integral max-pooling of CNN activations[J/OL]. [2021-11-18].
2	Chen W, Liu Y, Wang W, et al. Deep learning for instance retrieval: a survey[J/OL]. [2021-01-27].
3	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
4	Bell S, Bala K. Learning visual similarity for product design with convolutional neural networks[J]. Transactions on Graphics, 2015, 34(4): 2766959.
5	Babenko A, Lempitsky V. Aggregating local deep features for image retrieval[C]∥IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1269-1277.
6	Razavian A S, Sullivan J, Carlsson S, et al. Visual instance retrieval with deep convolutional networks[J]. ITE Transactions on Media Technology and Applications, 2016, 4(3): 251-258.
7	Anwaar M U, Labintcev E, Kleinsteuber M. Compositional learning of image-text query for image retrieval[C]∥IEEE Winter Conference on Applications of Computer Vision, Waikoloa, USA, 2015: 1140-1149.
8	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[J/OL]. [2021-12-10].
9	Krause J, Stark M, Deng J, et al. 3D object representations for fine-grained categorization[C]∥In Proceedings of Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, Australia, 2013: 554-561.
10	Wah C, Branson S, Welinder P, et al. The caltech-UCSD birds-200-2011 dataset[J]. California Institute of Technology, 2011(1): 20111026.
11	Philbin J, Chum O, Isard M, et al. Object retrieval with large vocabularies and fast spatial matching[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, 2007: 1-8.
12	Philbin J, Chum O, Isard M, et al. Lost in quantization: Improving particular object retrieval in large scale image databases[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 1-8.
13	Dubey A, Gupta O, Raskar R, et al. Maximum-entropy fine grained classification[C]∥32nd Conference on Neural Information Processing Systems,Montreal, Canada, 2018: 637-647.
14	Zhang X F, Zhou F, Lin Y Q, et al. Embedding label structures for fine-grained feature representation[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1114-1123.
15	Wei X S, Luo J H, Wu J X, et al. Selective convolutional descriptor aggregation for fine-grained image retrieval[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2868-2881.
16	Zhang X, Zhou F, Lin Y, et al. Embedding label structures for fine-grained feature representation[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1114-1123.
17	Song H O, Yu X, Jegelka S, et al. Deep metric learning via lifted structured feature embedding[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las VegasLas,USA,2016: 4004-4012.
18	Song H O, Jegelka S, Rathod V, et al. Deep metric learning via facility location[C]∥30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5382-5390.
19	Sohn K. Improved deep metric learning with multi-class n-pair loss objective[C]∥Advances in Neural Information Processing Systems 29, Barcelona, Spain, 2016: 1857-1865.
20	Chen H, Chen C L, Tang X O. Local similarity-aware deep feature embedding[J/OL]. [2021-10-27].
21	Wei X S, Luo J H, Wu J, et al. Selective convolutional descriptor aggregation for fine-grained image retrieval[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2868-2881.
22	Zheng X W, Ji R R, Sun X S, et al. Centralized ranking loss with weakly supervised localization for fine-grained object retrieval[C]∥27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018: 1226-1233.
23	Zheng X W, Ji R R, Sun X X, et al. Towards optimal fine grained retrieval via decorrelated centralized loss with normalize-scale layer[C]∥In Proceedings of Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, USA, 2019: 9291-9298.

Related Articles 15

[1]	Jin-wu GAO,Zhi-huan JIA,Xiang-yang WANG,Hao XING. Degradation trend prediction of proton exchange membrane fuel cell based on PSO⁃LSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(9): 2192-2202.
[2]	Xiao-ying LI,Ming YANG,Rui QUAN,Bao-hua TAN. Unbalanced text classification method based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1889-1895.
[3]	Xuan-jing SHEN,Xue-feng ZHANG,Yu WANG,Yu-bo JIN. Multi⁃focus image fusion algorithm based on pixel⁃level convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1857-1864.
[4]	Dan HU,Xin MENG. Vessel search method by earth observation satellite based on time⁃varying grid [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1896-1903.
[5]	Ming-hua GAO,Can YANG. Traffic target detection method based on improved convolution neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1353-1361.
[6]	Ji-hong OUYANG,Ze-qi GUO,Si-guang LIU. Dual⁃branch hybrid attention decision net for diabetic retinopathy classification [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(3): 648-656.
[7]	Lin SONG,Li-ping WANG,Jun WU,Li-wen GUAN,Zhi-gui LIU. Reliability analysis based on cyber⁃physical system and digital twin [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(2): 439-449.
[8]	Jie CAO,Jia-lin MA,Dai-lin HUANG,Ping YU. A fault diagnosis method based on multi Markov transition field [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(2): 491-496.
[9]	Shuai-na HUANG,Yu-xiang LI,Yue-heng MAO,Ai-ying BAN,Zhi-yong ZHANG. Black-box transferable adversarial attacks based on ensemble advGAN [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(10): 2391-2398.
[10]	You QU,Wen-hui LI. Single-stage rotated object detection network based on anchor transformation [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 162-173.
[11]	Gui-xia LIU,Zhi-yao PEI,Jia-zhi SONG. Prediction of protein-ATP binding site based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 187-194.
[12]	Jie ZHANG,Wen JING,Fu CHEN. Vulnerability detection of instant messaging network protocol based on passive clustering algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2253-2258.
[13]	Li-li REN,Zhi-jun WANG,Dong-mei YAN. Improved multi⁃verse algorithm with combined slime mould foraging behavior [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2190-2197.
[14]	Li-li DONG,Dan YANG,Xiang ZHANG. Large⁃scale semantic text overlapping region retrieval based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1817-1822.
[15]	Li-sheng JIN,Bai-cang GUO,Fang-rong WANG,Jian SHI. Dynamic multiple object detection algorithm for vehicle forward based on improved YOLOv3 [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1427-1436.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

方法	CUB-200-2011 K值
方法	1	2	4	8	16
基础网络^［13］	66.4	77.2	85.4	91.1	94.6
基础网络+ 本文损失	67.0	77.7	85.9	91.3	95.1
改进网络+响应值中心加权算法	69.4	80.0	87.7	92.7	95.9

损失	mAP
三元组^［14］	58.96	61.52
交叉熵^［13］	81.73	67.64
本文	82.30	69.01

损失	CUB-200-2011 K值
损失	1	2	4	8	16
三元组^［14］	38.0	50.3	62.2	74.1	84.6
交叉熵^［13］	68.6	78.8	86.6	92.2	95.2
本文	69.4	80.0	87.7	92.7	95.9

方法	mAP
最大池化^［6］	50.98	54.62
求和池化^［5］	51.93	54.38
多尺度最大池化^［1］	60.59	61.83
卷积描述子选择算法^［15］	78.92	67.81
响应值中心加权算法	82.30	69.01

方法	CUB-200-2011 K值
方法	1	2	4	8	16
最大池化^［6］	67.0	77.7	85.9	91.3	95.1
求和池化^［5］	48.6	61.7	72.9	82.9	90.2
多尺度最大池化^［1］	53.0	65.5	76.4	85.9	92.3
响应值中心加权算法	69.4	80.0	87.7	92.7	95.9

Image retrieval algorithm based on response value center weighted convolution feature

RICH HTML

PDF (PC)