基于多任务联合学习的多目标跟踪方法

doi:10.13229/j.cnki.jdxbgxb.20211357

Abstract

Abstract:

In order to improve the efficiency of the multi-object tracking method， a joint detection-apparence network based on anchor aligned convolutional feature， called AAC-JDAN， was proposed. On the basis of the object detection network YOLOv3， an anchor transformation network and an anchor aligned convolutional operation was introduced， so that the network can detect the rotated objects， while alleviating the problem of the weak correlation between the apparence feature extracted by the exsiting joint network and the rotated objects； by adding an apparence feature extraction branch in the detection network， two subtasks of object detection and object apparence feature extraction were combined in a multi-task joint learning manner to realize the sharing of the low-level feature， and the apparence feature vectors can be extracted along with the corresponding detected objects， which improves the overall efficiency of the tracking algorithm. A fast online data association method was proposed to realize the efficient tracking of multiple rotated objects in the video. The similarity matrix between the incoming detections and the trajectories was calculated with the object apparence feature extracted by AAC-JDAN and the motion prediction result given by the Kalman filter， and the matching was done by the KM algorithm. When tested on two public datasets and a custom dataset， the TPR， MOTA， and IDF-1 reached 80.4%， 71.3%， and 69.5%， respectively， and the framerate reached 20 frames per second， this showed that the proposed method achieves a better balance in the speed and accuracy of tracking.

Key words: computer application, multi-object tracking, rotated object tracking, multi-task learning, deep learning

CLC Number:

TP391

You QU,Wen-hui LI. Multiple object tracking method based on multi-task joint learning[J].Journal of Jilin University(Engineering and Technology Edition), 2023, 53(10): 2932-2941.

Figures/Tables 7

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

References 35

1	Brown M, Funke J, Erlien S, et al. Safe driving envelopes for path tracking in autonomous vehicles[J]. Control Engineering Practice, 2017, 61: 307-316.
2	Tian B, Yao Q, Gu Y, et al. Video processing techniques for traffic flow monitoring: a survey[C]∥The 14th International IEEE Conference on Intelligent Transportation Systems, Washington DC,USA,2011: 1103-1108.
3	Sivanantham S, Paul N N, Iyer R S. Object tracking algorithm implementation for security applications[J]. Far East Journal of Electronics and Communications, 2016, 16(1): 1-13.
4	Onate J M B, Chipantasi D J M, Erazo Nd R V. Tracking objects using artificial neural networks and wireless connection for robotics[J]. Journal of Telecommunication, Electronic and Computer Engineering, 2017, 9(1/3): 161-164.
5	Aggarwal J K, Xia L. Human activity recognition from 3d data: a review[J]. Pattern Recognition Letters, 2014, 48: 70-80.
6	Pfister T, Charles J, Zisserman A. Flowing ConvNets for human pose estimation in videos[C]∥IEEE International Conference of Computer Vision, Santiago,Chile, 2015: 1913-1921.
7	Choi W, Savarese S. A unified framework for multi-target tracking and collective activity recognition[C]∥European Conference on Computer Vision,Florence, Italy, 2012: 215-230.
8	Hu W, Tan T, Wang L, et al. A survey on visual surveillance of object motion and behaviors[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2004, 34(3): 334-352.
9	Ciaparrone G, Sánchez F L, Tabik S, et al. Deep learning in video multi-object tracking: a survey[J]. Neurocomputing, 2019, 381: 61-88.
10	Yu F, Li W, Li Q, et al. POI: multiple object tracking with high performance detection and appearance feature[C]∥European Conference on Computer Vision, Amsterdam, Netherlands, 2016: 36-42.
11	Fang K, Xiang Y, Li X, et al. Recurrent autoregressive networks for online multi-object tracking[C]∥IEEE Winter Conference on Applications of Computer Vision,Santa Rosa, USA, 2017: 466-475.
12	Zhou Z, Xing J, Zhang M, et al. Online Multi-target tracking with tensor-based high-order graph matching[C]∥The 24th International Conference on Pattern Recognition, Beijing, China, 2018: 1809-1814.
13	Mahmoudi N, Ahadi S M, Rahmati M. Multi-target tracking using CNN-based features: CNNMTT[J]. Multimedia Tools and Applications, 2019, 78(6): 7077-7096.
14	Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
15	Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]∥European Conference on Computer Vision,Amsterdam, Netherlands,2016: 21-37.
16	Redmon J, Farhadi A. YOLOv3: an incremental improvement[J/OL].[2018-04-21]. arXiv preprint arXiv:.
17	Voigtlaender P, Krause M, Osep A, et al. MOTS: multi-object tracking and segmentation[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Los Angeles, USA, 2019: 7934-7943.
18	Wang Z, Zheng L, Liu Y, et al. Towards real-time multi-object tracking[C]∥European Conference on Computer Vision, Online, 2020: 107-122.
19	Zhang Y, Wang C, Wang X, et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2020, 129: 3069-3087.
20	Arulampalam M S, Maskell S, Gordon N, et al. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[J]. IEEE Transactions on Signal Processing, 2002, 50(2): 174-188.
21	Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564-577.
22	Magee D R. Tracking multiple vehicles using foreground, background and motion models[J]. Image and Vision Computing, 2004, 22(2): 143-155.
23	Indu S, Gupta M, Bhattacharyya A. Vehicle tracking and speed estimation using optical flow method[J]. International Journal of Engineering Science and Technology, 2011, 3(1): 429-434.
24	Niknejad H T, Takeuchi A, Mita S, et al. On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation[J]. IEEE Transactions on Intelligent Transportation Systems, 2012, 13(2): 748-758.
25	王林, 胥中南.改进的KCF算法在车辆跟踪中的应用[J]. 计算机测量与控制, 2019, 27(7): 195-199.
	Wang Lin, Xu Zhong-nan. Application of improved KCF algorithm in vehicle tracking[J]. Computer Measurement and Control, 2019, 27(7): 195-199.
26	Yang X, Yang J, Yan J, et al. SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects[C]∥IEEE International Conference on Computer Vision, Seoul, South Korea, 2019: 8231-8240.
27	Yang X, Sun H, Fu K, et al. Automatic ship detection of remote sensing images from google earth in complex scenes based on multi-scale rotation dense feature pyramid networks[J]. Remote Sensing, 2018, 10(1): 132.
28	Sohn K. Improved deep metric learning with multi-class N-pair loss objective[C]∥Proceedings of the 30th International Conference on Neural Information Processing Systems,Honolulu, USA, 2016: 1857-1865.
29	Kumbasar T. Revisiting KM algorithms: a linear programming approach[C]∥IEEE International Conference on Fuzzy Systems,Zhangjiajie, China,2015:1-6.
30	Fan H, Du D, Wen L, et al. VisDrone-MOT2020: the vision meets drone multiple object tracking challenge results[C]∥European Conference on Computer Vision, Online, 2020: 713-727.
31	Yu H, Li G, Zhang W, et al. The unmanned aerial vehicle benchmark: object detection, tracking and baseline[J]. International Journal of Computer Vision, 2020, 128(5): 1141-1159.
32	Bernardin K, Stiefelhagen R. Evaluating multiple object tracking performance: the CLEAR MOT metrics[J]. EURASIP Journal on Image and Video Processing, 2008: No.246309.
33	Ristani E, Solera F, Zou R, et al. Performance measures and a data set for multi-target, multi-camera tracking[C]∥European Conference on Computer Vision, Amsterdam, Netherlands, 2016: 17-35.
34	Wojke N, Bewley A, Paulus D. Simple online and realtime tracking with a deep association metric[C]∥IEEE International Conference on Image Processing,Beijing, China, 2017: 3645-3649.
35	Yu F, Li W, Li Q, et al. POI: multiple object tracking with high performance detection and appearance feature[C]∥European Conference on Computer Vision, Amsterdam, Netherlands, 2016: 36-42.

Related Articles 15

[1]	Guang HUO,Da-wei LIN,Yuan-ning LIU,Xiao-dong ZHU,Meng YUAN,Di GAI. Lightweight iris segmentation model based on multiscale feature and attention mechanism [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2591-2600.
[2]	Ying HE,Zhuo-ran WANG,Xu ZHOU,Yan-heng LIU. Point of interest recommendation algorithm integrating social geographical information based on weighted matrix factorization [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2632-2639.
[3]	Yun-zuo ZHANG,Xu DONG,Zhao-quan CAI. Multi view gait cycle detection by fitting geometric features of lower limbs [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2611-2619.
[4]	Ming-yao XIAO,Xiong-fei LI,Rui ZHU. Medical image fusion based on pixel correlation analysis in NSST domain [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2640-2648.
[5]	Ya-hui ZHAO,Fei-yu LI,Rong-yi CUI,Guo-zhe JIN,Zhen-guo ZHANG,De LI,Xiao-feng JIN. Korean⁃Chinese translation quality estimation based on cross⁃lingual pretraining model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2371-2379.
[6]	Xiao-jun JIN,Yan-xia SUN,Jia-lin YU,Yong CHEN. Weed recognition in vegetable at seedling stage based on deep learning and image processing [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2421-2429.
[7]	Xiang-jiu CHE,Huan XU,Ming-yang PAN,Quan-le LIU. Two-stage learning algorithm for biomedical named entity recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2380-2387.
[8]	Qing-tian GENG,Zhi LIU,Qing-liang LI,Fan-hua YU,Xiao-ning LI. Prediction of soil moisture based on a deep learning model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2430-2436.
[9]	Lian-ming WANG,Xin WU. Method for 3D motion parameter measurement based on pose estimation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2099-2108.
[10]	Wei-tiao WU,Kun ZENG,Wei ZHOU,Peng LI,Wen-zhou JIN. Deep learning method for bus passenger flow prediction based on multi-source data and surrogate-based optimization [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2001-2015.
[11]	Pei-yong LIU,Jie DONG,Luo-feng XIE,Yang-yang ZHU,Guo-fu YIN. Surface defect detection algorithm of magnetic tiles based on multi⁃branch convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1449-1457.
[12]	Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
[13]	Ze-qiang ZHANG,Wei LIANG,Meng-ke XIE,Hong-bin ZHENG. Elite differential evolution algorithm for mixed⁃model two⁃side disassembly line balancing problem [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1297-1304.
[14]	Wen-li JI,Zhong TIAN,Jing CHAI,Ding-ding ZHANG,Bin WANG. Prediction of water⁃flowing height in fractured zone based on distributed optical fiber and multi⁃attribute fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1200-1210.
[15]	Peng YU,Yan PIAO. New method for extracting person re-identification attributes based on multi-scale features [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1155-1162.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

方法	目标数量较少（≤35）			目标数量较多（>35）
方法	MOTA	IDF-1	FPS	MOTA	IDF-1	FPS
R-YOLO+Triple	73.6	72.2	15.2	72.6	69.7	10
R-YOLO+IDE	71.9	69.6	15.9	70.6	65.9	11
CenterMap-Net+Triple	74.2	72.5	5.8	73.2	71.9	4.9
CenterMap-Net+IDE	72.1	71.9	5.8	70.8	69.4	4.9
本文	71.7	70.0	20	70.1	66.2	18

方案	方法	MOTA	IDF-1	FPS
独立子任务	DeepSORT^［34］	72.2	70.5	12.4
独立子任务	POI^［35］	73.5	72.0	11.0
联合二阶段	TrackRCNN^［17］	57.4	45.6	17.3
联合单阶段	JDE^［18］	53.7	49.0	22.1
	FairMOT^［19］	61.3	56.8	26.7
	本文	71.3	69.5	19.8

检测分支	表观分支	TPR	MOTA	IDF-1
常规卷积	常规卷积	78.3	67.9	64.5
常规卷积	对齐卷积	80.4	68.1	65.1
对齐卷积	常规卷积	78.3	70.4	65.7
对齐卷积	对齐卷积	80.4	71.3	69.5

表观特征	运动预测	MOTA	IDF-1
×	×	67.3	64.2
√	×	70.1	68.0
×	√	68.9	66.1
√	√	71.3	69.5

Multiple object tracking method based on multi-task joint learning

RICH HTML

PDF (PC)