基于语义耦合和身份一致性的跨模态行人重识别方法

doi:10.13229/j.cnki.jdxbgxb20210481

Abstract

Abstract:

To solve the problem of the large inter-modality discrepancy and the intra-class variations in cross modality person re-Identification （CM-Reid）, a novel CM-Reid framework based on semantic coupling and identity-consistence constraint was proposed. In the semantic level， the semantic representations bi-directionally and fuse the semantic information between different modalities were coupled to alleviate the inter-modality discrepancy. In the identity level， the cross-modality triplet loss and identity loss to maintain the identity consistence were optimized to alleviate the intra-class variations. The experimental results show that the proposed method can effectively improve the performance of CM-Reid. Compared with the baseline method, the accuracy of Top-1 and mAP indicators is improved by more than 10%.

Key words: computer application, cross-modality person re-identification, deep learning, semantic coupling, identity-consistence constrain

CLC Number:

TP391

Chun-ping HOU,Qing-yuan YANG,Mei-yan HUANG,Zhi-peng WANG. Cross⁃modality person re⁃identification based on semantic coupling and identity⁃consistence constraint[J].Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2954-2963.

Figures/Tables 8

Fig.1

Fig.2

Table 1

Table 2

Table 3

Table 4

Fig.3

Fig.4

References 25

1	Ye Mang, Shen Jian-bing, Lin Gao-jie, et al. Deep learning for person re-identification: a survey and outlook[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 2872-2893.
2	Zhong Zhun, Zheng Liang, Luo Zhi-ming, et al. Learning to adapt invariance in memory for person re-identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(8): 2723-2738.
3	Qian Xue-lin, Fu Yan-wei, Tao Xiang, et al. Leader-based multi-scale attention deep architecture for person re-identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 42(2): 371-385.
4	马淼, 王云涛, 潘海鹏. 基于注意力机制和空间几何约束的行人重识别方法[J]. 吉林大学学报: 工学版, 2022, 52(5): 1079-1087.
	Ma Miao, Wang Yun-tao, Pan Hai-peng. Person re-identification method based on attention mechanism and spatial geometric constraint[J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(5): 1079-1087.
5	Chen Guang-yi, Lu Ji-wen, Yang Ming, et al. Learning recurrent 3D attention for video-based person re-identification[J]. IEEE Transactions on Image Processing, 2020, 29: 6963-6976.
6	Wang K, Ding C, Maybank S J, et al. CDPM: convolutional deformable part models for semantically aligned person re-identification[J]. IEEE Transactions on Image Processing, 2019, 29: 3416-3428.
7	Wu A, Zheng W S, Yu H X, et al. RGB-infrared cross-modality person re-identification[C]∥Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017: 5380-5389.
8	Ye Mang, Wang Zheng, Lan Xiang-yuan, et al. Visible thermal person re-identification via dual-constrained top-ranking[C]∥International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018:1092-1099.
9	Hao Yi, Wang Nan-nan, Li Jie, et al. HSME: hypersphere manifold embedding for visible thermal person re-identification[C]∥Proceedings of the AAAI Conference on Artificial Intelligence, Hawaii, USA,2019: 8385-8392.
10	Wang Guan-an, Zhang Tian-zhu, Cheng Jian, et al. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 2019: 3623-3632.
11	Wang Zhi-xiang, Wang Zheng, Zheng Yin-qiang, et al. Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 618-626.
12	Deng J, Dong W, Socher R, et al. ImageNet: a large-scale hierarchical image database[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 248-255.
13	He Kai-ming, Zhang Xiang-yu, Ren Shao-qing, et al. Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770-778.
14	Hermans A, Beyer L, Leibe B. In defense of the triplet loss for person re-identification[DB/OL]. [2020-12-03].
15	Nguyen D T, Hong H G, Kim K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors, 2017, 17(3): No.605.
16	Kansal K, Subramanyam A V, Wang Z, et al. SDL: spectrum-disentangled representation learning for visible-infrared person re-identification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(10): 3422-3432.
17	Feng Zhan-xiang, Lai Jian-hua, Xie Xiao-hua. Learning modality-specific representations for visible-infrared person re-identification[J]. IEEE Transactions on Image Processing, 2019, 29: 579-590.
18	Ye Mang, Lan Xiang-yuan, Leng Qing-ming, et al. Cross-modality person re-identification via modality-aware collaborative ensemble learning[J]. IEEE Transactions on Image Processing, 2020, 29: 9387-9399.
19	Ye Mang, Lan Xiang-hua, Wang Zheng, et al. Bi-directional center-constrained top-ranking for visible thermal person re-identification[J]. IEEE Transactions on Information Forensics and Security, 2019, 15: 407-419.
20	Ye Han-rong, Liu Hong, Meng Fan-yang, et al. Bi-directional exponential angular triplet loss for RGB-infrared person re-identification[J]. IEEE Transactions on Image Processing, 2021, 30: 1583-1595.
21	Wu An-cong, Zheng Wei-shi, Gong Shao-gang, et al. RGB-IR person re-identification by cross-modality similarity preservation[J]. International Journal of Computer Vision, 2020, 128(6): 1765-1785.
22	Dai P Y, Ji R R, Wang H B, et al. Cross-modality person re-identification with generative adversarial training[C]∥Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden: 2018: 677-683.
23	Wang G A, Zhang T, Yang Y, et al. Cross-modality paired-images generation for RGB-infrared person re-identification[C]∥Proceedings of the AAAI Conference on Artificial Intelligence, New York, USA, 2020: 12144-12151.
24	胡静,陶洋.基于RPCA的群稀疏表示人脸识别方法[J]. 重庆邮电大学学报: 自然科学版, 2020, 32(3): 459-468.
	Hu Jing, Tao Yang. Group sparse representation face recognition method based on RPCA[J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2020, 32(3): 459-468.
25	李新春,马红艳,林森.基于局部邻域四值模式的掌纹掌脉融合识别[J].重庆邮电大学学报: 自然科学版, 2020, 32(4): 630-638.
	Li Xin-chun, Ma Hong-yan, Lin Sen. Palmprint and palm vein fusion recognition based on local neighbor quaternary pattern[J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2021, 47(5): 285-291, 300.

Related Articles 15

[1]	Xian-yu QI,Wei WANG,Lin WANG,Yu-fei ZHAO,Yan-peng DONG. Semantic topological map building with object semantic grid map [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 569-575.
[2]	Xiao-hu SHI,Jia-qi WU,Chun-guo WU,Shi CHENG,Xiao-hui WENG,Zhi-yong CHANG. Residual network based curve enhanced lane detection method [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 584-592.
[3]	Peng GUO,Wen-chao ZHAO,Kun LEI. Dual⁃resource constrained flexible job shop optimal scheduling based on an improved Jaya algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 480-487.
[4]	Jin-Zhen Liu,Guo-Hui Gao,Hui Xiong. Multi⁃scale attention network for brain tissue segmentation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 576-583.
[5]	Jin-wu GAO,Zhi-huan JIA,Xiang-yang WANG,Hao XING. Degradation trend prediction of proton exchange membrane fuel cell based on PSO⁃LSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(9): 2192-2202.
[6]	Feng-feng ZHOU,Hai-yang ZHU. SEE： sense EEG⁃based emotion algorithm via three⁃step feature selection strategy [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1834-1841.
[7]	Xuan-jing SHEN,Xue-feng ZHANG,Yu WANG,Yu-bo JIN. Multi⁃focus image fusion algorithm based on pixel⁃level convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1857-1864.
[8]	Fu-heng QU,Tian-yu DING,Yang LU,Yong YANG,Ya-ting HU. Fast image codeword search algorithm based on neighborhood similarity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1865-1871.
[9]	Tian BAI,Ming-wei XU,Si-ming LIU,Ji-an ZHANG,Zhe WANG. Dispute focus identification of pleading text based on deep neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1872-1880.
[10]	Xiao-ying LI,Ming YANG,Rui QUAN,Bao-hua TAN. Unbalanced text classification method based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1889-1895.
[11]	Gui-he QIN,Jun-feng HUANG,Ming-hui SUN. Text input based on two⁃handed keyboard in virtual environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1881-1888.
[12]	Dan HU,Xin MENG. Vessel search method by earth observation satellite based on time⁃varying grid [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1896-1903.
[13]	Jun WANG,Yan-hui XU,Li LI. Data fusion privacy protection method with low energy consumption and integrity verification [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1657-1665.
[14]	Feng-feng ZHOU,Yi-chi ZHANG. Unsupervised feature engineering algorithm BioSAE based on sparse autoencoder [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1645-1656.
[15]	Ming-hua GAO,Can YANG. Traffic target detection method based on improved convolution neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1353-1361.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	AN Shi, LI Jing, CUI Na . Simulation of commuter day-to-day dynamic route switching behavior under ATIS information[J]. 吉林大学学报(工学版), 2009, 39(03): 587 -0592 .
[2]	DUAN Bin, SUN Tong-jing,LI Zhen-hua, HUANG Chang-wei,ZHANG Guang-xian. IIR Butterworth digital filter for full digital inverter[J]. 吉林大学学报(工学版), 2009, 39(增刊2): 311 -0314 .
[3]	JIN Jing-Fu, MA Yi, LIU Yu-Rong, CONG Qian. Aerodynamic analysis of the family of airfoil of owl[J]. 吉林大学学报(工学版), 2010, 40(增刊): 278 -0281 .
[4]	Jin Lisheng, Wang Rongben, Gao Long, Guo Lie. Shadow path mark segmentation method based on regiongrowing for intelligent vehicle[J]. 吉林大学学报(工学版), 2006, 36(增刊1): 132 -0135 .
[5]	WANG Qiang,DAI Jing-min,HE Xiao-wa. Effect of time delay on thermal conductivity measurement with transient planar heat source technique[J]. 吉林大学学报(工学版), 2011, 41(03): 711 -715 .
[6]	MA Kai, GUAN Hsin, PANG Shu-yi, ZHAN Jun. Interval control method on consistency of suspension kinematics[J]. 吉林大学学报(工学版), 2011, 41(4): 910 -914 .
[7]	. [J]. 吉林大学学报(工学版), 2007, 37(06): 0 .
[8]	Wei Hai-bin, Liu Han-bing, Gao Yi-ping, Li Chang-yu, Fang Ying . Effect of freezethaw cycles on dynamic strength of fly ash soils [J]. 吉林大学学报(工学版), 2007, 37(02): 329 -0333 .
[9]	Meng Guang-wei，Li Feng, Zhao Yun-liang. Reliability analysis of fatigue and fracture based on stochastic finite element method[J]. 吉林大学学报(工学版), 2006, 36(增刊1): 16 -0019 .
[10]	GUO Zi-Zheng, CHEN Chong-Shuang, WANG Xin, CHEN Ya-Qing, TAN Yong-Gang. Driving behavior risk status identification \|model based on FCM[J]. 吉林大学学报(工学版), 2010, 40(02): 427 -0430 .

序号	方法	All Search		Indoor Search
序号	方法	Top-1	mAP	Top-1	mAP
1	Baseline	45.81	44.85	47.06	46.13
2	Baseline+SCM_block1	46.01	45.15	47.98	46.05
3	Baseline+SCM_block2	48.97	48.00	50.10	49.25
4	Baseline+SCM_block3	54.10	52.61	55.83	52.96
5	Baseline+SCM_block4	57.85	55.33	58.15	56.19

方法	All Search		Indoor Search
方法	Top-1	mAP	Top-1	mAP
1	55.92	54.20	56.85	54.92
2	57.85	55.33	58.15	56.19

方法	All Search				Indoor Search
方法	Top-1	Top-10	Top-20	mAP	Top-1	Top-10	Top-20	mAP
One-stream^［7］	12.04	49.68	66.74	13.67	16.94	63.55	82.10	22.95
Two-stream^［7］	11.65	47.99	65.50	12.85	15.60	61.18	81.02	21.49
Zero-Padding^［7］	14.80	54.12	71.33	15.95	20.58	68.38	85.79	26.92
SDL^［16］	28.12	70.23	83.67	29.01	32.56	80.45	90.67	39.56
MSR^［17］	37.35	83.40	93.34	38.11	39.64	89.29	97.66	50.88
MACE^［18］	51.64	87.25	94.44	50.11	57.35	93.02	97.47	64.79
HSM^［9］	20.68	32.74	77.95	23.21	-	-	-	-
BDTR^［19］	27.32	66.96	81.07	27.32	31.92	77.18	89.28	41.86
eBDTR^［19］	27.82	67.34	81.34	28.42	32.46	77.42	89.62	42.46
expAT^［20］	38.57	76.64	86.39	38.61	44.71	69.82	77.87	32.20
CMSP^［21］	43.56	86.25	-	44.98	48.62	89.50	-	57.50
cmGAN^［22］	26.97	67.51	80.56	31.49	31.63	77.23	89.18	42.19
D2RL^［11］	28.90	70.60	82.40	29.20	-	-	-	-
JSIA^［23］	38.10	80.70	89.90	36.90	43.80	86.20	94.20	52.90
AliGAN^［10］	42.40	85.00	93.70	40.70	45.90	87.60	94.40	54.30
本文	57.85	89.25	94.45	55.33	58.15	90.02	95.63	56.19

方法	RGB→Thermal				Thermal→RGB
方法	Top-1	Top-10	Top-20	mAP	Top-1	Top-10	Top-20	mAP
Zero-Padding^［7］	17.75	34.21	44.35	18.90	16.63	34.68	44.25	17.82
SDL^［16］	26.47	51.34	61.22	23.58	25.74	50.23	59.66	22.89
MSR^［17］	48.43	70.32	79.95	48.67	-	-	-	-
MACE^［18］	72.37	88.40	93.59	69.09	72.12	88.07	93.07	68.57
BDTR^［19］	33.56	58.61	67.43	32.76	32.92	58.46	68.43	31.96
eBDTR^［19］	34.62	58.96	68.72	33.46	34.21	58.74	68.64	32.49
HSME^［9］	50.85	73.36	81.66	47.00	50.15	72.40	81.07	46.16
expAT^［20］	67.45	-	-	66.51	66.48	-	-	67.31
CMSP^［21］	65.07	83.71	-	64.50	-	-	-	-
D2RL^［11］	43.40	66.10	76.30	44.10	-	-	-	-
JSIA^［23］	48.50	-	-	48.90	-	-	-	-
AliGAN^［10］	57.90	-	-	53.60	56.30	-	-	53.40
本文	75.12	90.56	96.09	74.51	74.96	91.02	95.76	76.10

Cross⁃modality person re⁃identification based on semantic coupling and identity⁃consistence constraint

RICH HTML

PDF (PC)