Journal of Jilin University(Engineering and Technology Edition) ›› 2024, Vol. 54 ›› Issue (5): 1435-1442.doi: 10.13229/j.cnki.jdxbgxb.20220782

Previous Articles    

Forward-looking visual field reproduction for vehicle screen-displayed closed cockpit using monocular vision

Hai-tao WANG1(),Hui-zhuo LIU1(),Xue-yong ZHANG1,Jian WEI2,Xiao-yuan GUO2,Jun-zhe XIAO2   

  1. 1.Shenyang Aircraft Design and Research Institute，Aviation Industry Corporation of China Limited，Shenyang 110035，China
    2.College of Communication Engineering，Jilin University，Changchun 130012，China
  • Received:2022-06-22 Online:2024-05-01 Published:2024-06-11
  • Contact: Hui-zhuo LIU E-mail:wht0110@sina.com;605541492@qq.com

RICH HTML

 0   

PDF (PC)

144

Abstract:

A monocular-vision-based method to reproduce the forward-looking visual field of the vehicle closed cockpit was proposed. Accurate off-line calibration and on-line monocular image acquisition， forward-looking image processing， and perspective image display are performed in a multi-threaded parallel framework. So forward-looking images can be reproduced with low distortion and high fluency on the screen. This paper also built a hardware system of vehicle screen-displayed closed cockpit. Through laboratory and field tests， the effectiveness and efficiency of our method are verified from the aspects of system delay， forward-looking angle of view， image distortion， high-speed straight driving， turning， obstacle avoidance and driving experience. This method provides a good theoretical and practical basis for the subsequent improvement of the vehicle closed cockpit.

Key words: signal and information processing, closed cockpit, visual field reproduction, monocular vision, perspective switching

CLC Number: 

  • TP391.7

Fig.1

Forward-looking visual field reproduction flowchart"

Fig.2

Vehicle screen-displayed closed cockpit test system"

Table 1

Test result of the system’s total delay"

搜索数字系统总延时/ms搜索数字系统总延时/ms
49345833
84332833
12347432
90337631
95338033

Fig.3

Diagram of the forward-looking angle of view test"

Fig.4

Subjective test result of the image distortion"

Fig.5

Camera mounted on rearview mirror and GPS dial"

Fig.6

High-speed straight driving test"

Fig.7

Turning and obstacle avoidance test"

Table 2

Questions in the system usability and driver workload scales"

序号系统可用性驾驶员工作负荷
1我认为我愿意经常使用本系统脑力需求（需要多少认知活动，如思考、决策、计算、记忆、注视、搜索等）
2我认为使用这个系统很不习惯体力需求（如推、拉、转身、动作控制、进行活动的程度等）
3我认为该系统容易使用时间压力（是缓慢并使人感到从容不迫，还是快速而令人感到匆忙慌乱）
4我认为需技术人员支持才能使用该系统绩效满意度（对于预先设置的任务目标，认为自己成功完成了吗）
5我发现该系统的不同功能整合得很好努力程度（包括脑力及体力）
6我认为透视驾驶与传统驾驶相差太大舒适程度（如长时间目视显示屏，是否出现视觉疲劳、眩晕、恶心等不适）
7我认为大部分用户会很快学会用该系统安全感（相对传统驾驶方式）
8该系统使用起来非常笨拙，难以使用精神压力（只注视显示屏）
9我使用该系统时得心应手—
10我认为使用该系统前需学习很多东西—

Table 3

Evaluation result of system usability and driver workload （scores corresponding to all questions）"

驾驶员系统可用性驾驶员工作负荷
Q1Q2Q3Q4Q5Q6Q7Q8Q9Q10得分Q1Q2Q3Q4Q5Q6Q7Q8得分
平均分82.081.3
1325242424275.01124134378.1
2315142524185.01125133281.3
3425242514282.51124134281.3
4335142515185.01125133184.4
5325142524182.51114133281.3
1 谭兆一,陈白帆,苏苇,等. 基于第一视角的车辆盲区视野重现[J]. 软件导刊, 2020, 19(9): 196-200.
Tan Zhao-yi, Chen Bai-fan, Su Wei, et al. Vehicle blind spot reproduction system in first-person view[J]. Software Guide, 2020, 19(9): 196-200.
2 蒙杨超,杨汉飞. 流媒体全视屏后视镜的设计[J]. 时代汽车, 2020 (12): 78-80.
Meng Yang-chao, Yang Han-fei. Design of streaming media full screen rearview mirror[J]. Auto Time, 2020 (12): 78-80.
3 魏小冈. 基于电子透明理念的A柱盲区消除系统[J]. 时代汽车, 2022 (8): 138-139.
Wei Xiao-gang. A-pillar blind area elimination system based on electronic transparency concept[J]. Auto Time, 2022(8): 138-139.
4 Zhang Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
5 谭华春,夏红卫,李琴,等. 车载摄像机的立体标定方法[J]. 吉林大学学报:工学版, 2013, 43(1): 352-356.
Tan Hua-chun, Xia Hong-wei, Li Qin, et al. Three-dimensional calibration for on-board camera[J]. Journal of Jilin University (Engineering and Technology Edition), 2013, 43(1): 352-356.
6 迟德霞,王洋,宁立群,等. 张正友法的摄像机标定试验[J]. 中国农机化学报, 2015, 36(2): 287-289, 337.
Chi De-xia, Wang Yang, Ning Li-qun, et al. Experimental research of camera calibration based on Zhang's method[J]. Journal of Chinese Agricultural Mechanization, 2015, 36(2): 287-289, 337.
7 Brown M, Lowe D. Automatic panoramic image stitching using invariant features[J]. International Journal of Computer Vision, 2007, 74(1): 59-73.
8 徐明,刁燕,罗华,等. 基于改进的 SURF 算子和透视变换模型的图像配准[J]. 计算机与数字工程, 2020, 48(3): 671-677.
Xu Ming, Diao Yan, Luo Hua, et al. Image registration based on improved SURF operator and perspective transform model[J]. Computer & Digital Engineering, 2020, 48(3): 671-677.
9 兰凤崇,李继文,陈吉清. 面向动态场景复合深度学习与并行计算的DG-SLAM算法[J]. 吉林大学学报:工学版, 2021, 51(4): 1437-1446.
Lan Feng-chong, Li Ji-wen, Chen Ji-qing. DG-SLAM algorithm for dynamic scene compound deep learning and parallel computing[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(4): 1437-1446.
10 Chen S, Pu Z, Fan X, et al. Fixing defect of photometric loss for self-supervised monocular depth estimation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32(3): 1328-1338.
11 高振海,李扬,张慧,等. 不同车速下驾驶员变换车道前视行为特征规律[J]. 吉林大学学报:工学版, 2016, 46(5): 1385-1390.
Gao Zhen-hai, Li Yang, Zhang Hui, et al. Driver's forward-looking behavior analysis during lane change under different speed[J]. Journal of Jilin University (Engineering and Technology Edition), 2016, 46(5): 1385-1390.
12 Kwon Y, Park M. Predicting future frames using retrospective cycle GAN[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, US, 2019.
[1] Chun-yang WANG,Wen-qian QIU,Xue-lian LIU,Bo XIAO,Chun-hao SHI. Accurate segmentation method of ground point cloud based on plane fitting [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 933-940.
[2] Li-bin ZHANG,Shi-yuan FENG,Hong-ying SHAN,Guan-ran WANG. Braking timing detection of tractor⁃trailer⁃train based on tire slip rate and monocular vision [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(12): 3415-3423.
[3] Xue-mei LI,Chun-yang WANG,Xue-lian LIU,Chun-hao SHI,Guo-rui LI. Point cloud registration method based on supervoxel bidirectional nearest neighbor distance ratio [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1918-1925.
[4] Xue-mei LI,Chun-yang WANG,Xue-lian LIU,Da XIE. Time delay estimation of linear frequency-modulated continuous-wave lidar signals via SESTH [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 950-958.
[5] Le-ping LIN,Zeng-tong LU,Ning OUYANG. Face reconstruction and recognition in non⁃cooperative scenes [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2941-2946.
[6] Hui-jing DOU,Gang DING,Jia GAO,Xiao LIANG. Wideband signal direction of arrival estimation based on compressed sensing theory [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2237-2245.
[7] Xin-yu JIN,Mu-han XIE, SUN-Bin. Grain information compressed sensing based on semi-tensor product approach [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 379-385.
[8] Li⁃min GUO,Xin CHEN,Tao CHEN. Radar signal modulation type recognition based on AlexNet model [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 1000-1008.
[9] LI Lin-hui, LUN Zhi-mei, LIAN Jing, YUAN Lu-shan, ZHOU Ya-fu, MA Xiao-yi. Convolution neural network-based vehicle detection method [J]. 吉林大学学报(工学版), 2017, 47(2): 384-391.
[10] Li Mao-hai，Hong Bing-rong，Luo Rong-hua . Improved RaoBlackwellized particle filters for mobile robot
simultaneous localization and mapping [J]. 吉林大学学报(工学版), 2007, 37(02): 401-0406.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd