Journal of Jilin University(Engineering and Technology Edition) ›› 2021, Vol. 51 ›› Issue (6): 2154-2163.doi: 10.13229/j.cnki.jdxbgxb20200671

Previous Articles    

Real⁃time camera planning for dynamic multiple targets considering cinematographic visual properties

Jun-cong LIN(),Jun LEI,Meng CHEN,Shi-hui GUO(),Xing GAO,Ming-hong LIAO   

  1. School of Informatics，Xiamen University，Xiamen 361005，China
  • Received:2020-09-03 Online:2021-11-01 Published:2021-11-15
  • Contact: Shi-hui GUO E-mail:jclin@xmu.edu.cn;guoshihui@xmu.edu.cn

RICH HTML

 1   

PDF (PC)

234

Abstract:

As the existing camera planning schemes are unable to deal with multiple targets and achieve real-time effect, in this paper, focusing on automatically optimizing the camera visual effect, a real-time camera planning algorithm is proposed for dynamic multiple targets. The method first adopts the toric camera space among the multiple targets, then takes the visual properties of the targets as the input of the non-linear optimization function. With the model predictive control, we can optimize the targets visual effect in real time. By analyzing the operation time, efficiency and performance of the experiments, it is verified that this method can effectively optimize the visual effect of the targets in the scene and it promotes the research of the multi-target camera planning algorithm.

Key words: computer application, camera planning, toric camera space for multiple targets, visual properties, non-linear optimization

CLC Number: 

  • TP391

Fig.1

Pipeline of camera planning system"

Fig.2

Chart of camera space"

Fig.3

Camera projection on screen"

Fig.4

Targets occlusion"

Fig.5

Camera and targets path"

Fig.6

Computation curves of non-linear optimization algorithm"

Fig.7

Test of different visual properties"

Table 1

Comparison of camera space generatednumber and precomputation timebetween different methods"

目标数量方法生成相机空间数预计算时间/ms
1文献［5，14］11
本文11
2文献［5，14］13
本文13
3文献［5，14］311
本文13
7文献［5，14］2116
本文14
12文献［5，14］6625
本文15

Fig.8

System experiments results"

Table 2

FPS range of different experiments"

实验编号FPS最小值/帧FPS最大值/帧
一36.042.9
二37.241.3
三38.540.1
四42.446.8
五46.650.2
1 Galvane Q, Lino C, Christie M, et al. Directing cinematographic drones[J]. ACM Transactions on Graphics, 2018, 37(3):1-18.
2 Dong S, Xu K, Zhou Q, et al. Multi-robot collaborative dense scene reconstruction[J]. ACM Transactions on Graphics, 2019, 38(4):1-16.
3 Xu K, Shi Y, Zheng L, et al. 3D attention-driven depth acquisition for object identification[J]. ACM Transactions on Graphics, 2016, 35(6):1-14.
4 Lino C, Christie M. Intuitive and efficient camera control with the toric space[J]. ACM Transactions on Graphics, 2015, 34(4): No.82.
5 Galvane Q, Christie M, Lino C, et al. Camera-on-rails: auto-mated computation of constrained camera paths[C]∥Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games, Paris, France, 2015: 151-157.
6 Assa J, Caspi Y,Cohen-Or D. Action synopsis: pose selection and illustration[J]. ACM Transactions on Graphics, 2005, 24(3): 667-676.
7 任庆东, 王艺萤, 刘贤梅. 培训系统中虚拟相机运动规划[J]. 科学技术与工程, 2012, 12(8):1936-1940.
Ren Qing-dong, Wang Yi-ying, Liu Xian-mei. Virtual camera motion planning in the simulation training system[J]. Science Technology and Engineering, 2012, 12(8): 1936-1940.
8 Yeh I C, Lin W C, Lee T Y, et al. Social-event-driven camera control for multicharacter animations[J]. IEEE Transactions on Visualization & Computer Graphics, 2012, 18(9): 1496-1510.
9 Li T Y, Xiao X Y. An interactive camera planning system for automatic cinematographer[C]∥Proceedings of 11th International Multimedia Modelling Conference, Melbourne, Australia, 2005: 310-315.
10 Nägeli T, Meier L, Domahidi A, et al. Real-time planning for automated multi-view drone cinematography[J]. ACM Transactions on Graphics, 2017, 36(4): 1-10.
11 Nägeli T, Alonso-Mora J, Domahidi A, et al. Real-time motion planning for aerial videography with dynamic obstacle avoidance and viewpoint optimization[J]. IEEE Robotics and Automation Letters, 2017, 2(3): 1696-1703.
12 丛岩峰, 安向京, 陈虹, 等. 基于滚动优化原理的类车机器人路径跟踪控制[J]. 吉林大学学报:工学版, 2012, 42(1): 182-187.
Cong Yan-feng, An Xiang-jing, Chen Hong, et al. Path following control of car-like robot based on rolling windows[J]. Journal of Jilin University(Engineering and Technology Edition), 2012, 42(1): 182-187.
13 张家旭,王欣志,赵健,等. 汽车高速换道避让路径规划及离散滑模跟踪控制[J]. 吉林大学学报:工学版, 2021, 51(3): 1081-1090.
Zhang Jia-xu, Wang Xin-zhi, Zhao Jian, et al. Path planning and discrete sliding mode tracking control for high⁃speed lane changing collision avoidance of vehicle[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1081-1090.
14 Lino C, Christie M. Efficient composition for virtual camera control[C]∥Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Lausanne, Switzerland, 2012: 65-70.
15 Gardner J S, Fowlkes C, Nothelfer C, et al. Exploring aesthetic principles of spatial composition through stock photography[J]. Journal of Vision, 2010, 8(6): 337.
16 Ma Shuang, Fan Yang-yu, Chen Chang-wen. Finding your spot: a photography suggestion system for placing human in the scene[C]∥Proceedings of IEEE International Conference on Image Processing, Paris, France, 2014: 556-560.
[1] Jie CAO,Xue QU,Xiao-xu LI. Few⁃shot image classification method based on sliding feature vectors [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1785-1791.
[2] Xiao-xue SUN,Hui ZHONG,Hai-peng CHEN. Statistical analysis system for students' examination results based on decision tree classification technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1866-1872.
[3] Meng-su ZHANG,Chun-tian LIU,Xi-jin LI,Yong-ping HUANG. Design of fuzzy comprehensive evaluation system for performance appraisal based on K⁃means clustering algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1851-1856.
[4] Yin-di YAO,Jun-jin HE,Yang-li LI,Dang-yuan XIE,Ying LI. ET0 simulation of self⁃constructed improved whale optimized BP neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1798-1807.
[5] Hong-wei ZHAO,Zi-jian ZHANG,Jiao LI,Yuan ZHANG,Huang-shui HU,Xue-bai ZANG. Bi⁃direction segmented anti⁃collision algorithm based on query tree [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1830-1837.
[6] Chun-bo WANG,Xiao-qiang DI. Cloud storage integrity verification audit scheme based on label classification [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1364-1369.
[7] Dan-tong OUYANG,Yang LIU,Jie LIU. Fault diagnosis method based on test set under fault response guidance [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1017-1025.
[8] Rong QIAN,Ru ZHANG,Ke-jun ZHANG,Xin JIN,Shi-liang GE,Sheng JIANG. Capsule graph neural network based on global and local features fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1048-1054.
[9] Xiao-long ZHU,Zhong XIE. Geospatial data extraction algorithm based on machine learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1011-1016.
[10] Bao-feng SUN,Xin-xin REN,Zai-si ZHENG,Guo-yi Li. Multi⁃objective flow shop optimal scheduling considering worker's load [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 900-909.
[11] Yuan-ning LIU,Di WU,Xiao-dong ZHU,Qi-xian ZHANG,Shuang-shuang LI,Shu-jun GUO,Chao WANG. User interface components detection algorithm based on improved YOLOv3 [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1026-1033.
[12] Guang-qiu CHEN,Yu-cun CHEN,Jia-yue LI,Guang-wen LIU. Infrared and visible image fusion based on discrete nonseparable shearlet transform and convolutional sparse representation [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 996-1010.
[13] Shu-min WANG,Wei CHEN. Algorithm for identifying abnormal behavior in underground mines based on continuous density hidden Markov model [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1067-1072.
[14] Shu-tao SHEN,Zha-xi NIMA. Double chaos identifiable tampering image encryption method based on blockchain technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1055-1059.
[15] Bing-hai ZHOU,Zhao-xu HE. Static semi⁃kitting strategy⁃based multi⁃objective just⁃in⁃time material distribution scheduling [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 910-916.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd