基于自监督学习的单目图像深度估计算法

doi:10.13229/j.cnki.jdxbgxb.20220126

摘要/Abstract

摘要：

针对现有单目图像深度估计方法中存在估计效果较差等问题，为提升单目图像深度估计的有效性，设计一种基于自监督学习的单目图像深度估计算法。首先获取单目图像绝对深度特征、相对深度特征以及位置特征，采用深度差的对数作为损失函数，得到图像点对点的关联特征；然后采用表面法线损失函数处理深度图表面法线，降低图像表面波动，计算图像光度损失函数与两个图像之间的相似度；最后将单目图像输入到自监督学习网络，计算深度幅值，完成基于自监督学习的单目图像深度估计。实验主要分为客观评价和主观评价，在客观评价部分验证了本文估计算法深度预测的准确性达到了91.2%，深度图边缘误差仅为3%，该方法具备较高的准确性；在主观评价中验证了本文算法能够真实预测图形特征，有效提高图像深度估计效果。

关键词: 软件工程, 自监督学习, 单目图像, 深度估计, 特征提取, 损失函数

Abstract:

In order to improve the effectiveness of monocular image depth estimation， a monocular image depth estimation algorithm based on self supervised learning is designed. The absolute depth feature， relative depth feature and position feature of monocular image are obtained， and the logarithm of depth difference is used as the loss function to obtain the point-to-point correlation feature of image； The surface normal loss function is used to deal with the surface normal of the depth map to reduce the surface fluctuation of the image， and the similarity between the image photometric loss function and the two images is calculated； The monocular image is input into the self supervised learning network， the depth amplitude is calculated， and the monocular image depth estimation based on self supervised learning is completed. The experiment is mainly divided into objective evaluation and subjective evaluation. In the objective evaluation part， the accuracy of depth prediction of the proposed estimation algorithm reaches 91.2%， and the edge error of depth map is only 3%. This method has high accuracy. In subjective evaluation， the proposed algorithm is verified to be able to predict the image features and improve the image depth estimation effect effectively.

Key words: software engineering, self-supervised learning, monocular image, depth estimation, feature extraction, loss function

中图分类号:

TP311

白琳,刘林军,李轩昂,吴沙,刘汝庆. 基于自监督学习的单目图像深度估计算法[J]. 吉林大学学报(工学版), 2023, 53(4): 1139-1145.

Lin BAI,Lin-jun LIU,Xuan-ang LI,Sha WU,Ru-qing LIU. Depth estimation algorithm of monocular image based on self-supervised learning[J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1139-1145.

图/表 7

图1

图2

图3

图4

图5

图6

图7

参考文献 15

1	霍智勇, 乔璐. 基于结构化损失的单目深度估计算法研究[J]. 电子科技大学学报, 2021, 50(5): 728-733.
	Huo Zhi-yong, Qiao Lu. Research on monocular depth estimation algorithm based on structured loss[J]. Journal of University of Electronic Science and Technology of China, 2021, 50(5): 728-733.
2	王亚群, 戴华林, 王丽, 等. 基于密集卷积网络的单目图像深度估计方法[J]. 计算机工程, 2021, 47(11): 262-267, 291.
	Wang Ya-qun, Dai Hua-lin, Wang Li, et al. Method for estimating monocular image depth based on dense convolutional network[J]. Computer Engineering, 2021, 47(11): 262-267, 291.
3	王泉德, 张松涛. 基于多尺度特征融合的单目图像深度估计[J]. 华中科技大学学报:自然科学版, 2020, 48(5): 7-12.
	Wang Quan-de, Zhang Song-tao. Monocular depth estimation with multi-scale feature fusion[J]. Journal of Huazhong University of Science and Technology (Nature Science Edition), 2020, 48(5): 7-12.
4	于恒, 梅红岩, 许晓明, 等. 基于深度学习的图像压缩算法研究综述[J]. 计算机工程与应用, 2020, 56(15): 15-23.
	Yu Heng, Mei Hong-yan, Xu Xiao-ming, et al. Survey of image compression algorithm based on deep learning[J]. Computer Engineering and Applications, 2020, 56(15): 15-23.
5	刘杰平, 温竣文, 梁亚玲. 基于多尺度注意力导向网络的单目图像深度估计[J]. 华南理工大学学报: 自然科学版, 2020, 48(12): 52-62.
	Liu Jie-ping, Wen Jun-wen, Liang Ya-ling. Monocular image depth estimation based on multi-scale attention oriented network[J]. Monocular Image Depth Estimation Based on Multi-Scale Attention Oriented Network, 2020, 48(12): 52-62.
6	李旭, 丁萌, 魏东辉, 等. VDAS中基于单目红外图像的深度估计方法[J]. 系统工程与电子技术, 2021, 43(5): 1210-1217.
	Li Xu, Ding Meng, Wei Dong-hui, et al. Depth estimation method based on monocular infrared image in VDAS[J]. Systems Engineering and Electronics, 2021, 43(5): 1210-1217.
7	赵栓峰, 黄涛, 许倩, 等. 面向无人机自主飞行的无监督单目视觉深度估计[J]. 激光与光电子学进展, 2020, 57(2): 145-154.
	Zhao Shuan-feng, Huang Tao, Xu Qian, et al. Unsupervised monocular depth estimation for autonomous flight of drones[J]. Laser & Optoelectronics Progress, 2020, 57(2): 145-154.
8	王倩倩, 赵海涛. 基于深度CRF网络的单目红外场景深度估计[J]. 红外技术, 2020, 42(6): 580-588.
	Wang Qian-qian, Zhao Hai-tao. Depth estimation of monocular infrared scene based on deep crf network[J]. Infrared Technology, 2020, 42(6): 580-588.
9	邹承明, 胡佑璞. 引入生成对抗网络的室外场景单目深度估计[J]. 计算机工程与应用, 2021, 57(6): 176-183.
	Zou Cheng-ming, Hu You-pu. Monocular depth estimation in outdoor scene with generative adversarial network[J]. Computer Engineering and Applications, 2021, 57(6): 176-183.
10	张建博, 袁亮, 何丽, 等. 基于深度预测的单目SLAM绝对尺度估计[J]. 计算机工程与设计, 2021, 42(6): 1749-1755.
	Zhang Jian-bo, Yuan Liang, He Li, et al. Absolute scale estimation of monocular SLAM based on depth prediction[J]. Computer Engineering and Design, 2021, 42(6): 1749-1755.
11	陈莹, 王一良. 基于密集特征融合的无监督单目深度估计[J]. 电子与信息学报, 2021, 43(10): 2976-2984.
	Chen Ying, Wang Yi-liang. Unsupervised monocular depth estimation based on dense feature fusion[J]. Journal of Electronics & Information Technology, 2021, 43(10): 2976-2984.
12	孟昭宇, 张宁宁. 基于聚焦线索的数字图像误匹配点剔除仿真[J]. 计算机仿真, 2020, 37(1): 402-406.
	Meng Zhao-yu, Zhang Ning-ning. Digital image mismatching point culling simulation based on focus cues[J]. Computer Simulation, 2020, 37(1): 402-406.
13	王欣盛, 张桂玲. 基于卷积神经网络的单目深度估计[J]. 计算机工程与应用, 2020, 56(13): 143-149.
	Wang Xin-sheng, Zhang Gui-ling. Monocular depth estimation based on convolutional neural network[J]. Computer Engineering and Applications, 2020, 56(13): 143-149.
14	陈莹, 沈栎. 基于特征回归的单目深度图无标记人体姿态估计[J]. 系统仿真学报, 2020, 32(2): 269-277.
	Chen Ying, Shen Li. Monocular depth image mark-less pose estimation based on feature regression[J]. Journal of System Simulation, 2020, 32(2): 269-277.
15	周云成, 邓寒冰, 许童羽, 等. 基于稠密自编码器的无监督番茄植株图像深度估计模型[J]. 农业工程学报, 2020, 36(11): 182-192.
	Zhou Yun-cheng, Deng Han-bing, Xu tong-yu, et al. Unsupervised deep estimation modeling for tomato plant image based on dense convolutional auto-encoder[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(11): 182-192.

相关文章 15

[1]	时小虎,吴佳琦,吴春国,程石,翁小辉,常志勇. 基于残差网络的弯道增强车道线检测方法[J]. 吉林大学学报(工学版), 2023, 53(2): 584-592.
[2]	赵宏伟,张健荣,朱隽平,李海. 基于对比自监督学习的图像分类框架[J]. 吉林大学学报(工学版), 2022, 52(8): 1850-1856.
[3]	周怡娜,董宏丽,张勇,路敬祎. 基于VMD去噪和散布熵的管道信号特征提取方法[J]. 吉林大学学报(工学版), 2022, 52(4): 959-969.
[4]	李国发,王彦博,何佳龙,王继利. 机电装备健康状态评估研究进展及发展趋势[J]. 吉林大学学报(工学版), 2022, 52(2): 267-279.
[5]	陈晓雷,孙永峰,李策,林冬梅. 基于卷积神经网络和双向长短期记忆的稳定抗噪声滚动轴承故障诊断[J]. 吉林大学学报(工学版), 2022, 52(2): 296-309.
[6]	杜先君,贾亮亮. 基于优化堆叠降噪自编码器的滚动轴承故障诊断[J]. 吉林大学学报(工学版), 2022, 52(12): 2827-2838.
[7]	戴礼灿,代翔,崔莹,魏永超. 基于深度集成学习的社交网络异常数据挖掘算法[J]. 吉林大学学报(工学版), 2022, 52(11): 2712-2717.
[8]	曹洁,何智栋,余萍,王进花. 数据不平衡分布下轴承故障诊断方法[J]. 吉林大学学报(工学版), 2022, 52(11): 2523-2531.
[9]	周大可,张超,杨欣. 基于多尺度特征融合及双重注意力机制的自监督三维人脸重建[J]. 吉林大学学报(工学版), 2022, 52(10): 2428-2437.
[10]	黎才茂,陈少凡,林成蓉,候玉权,李浩. 基于循环知识图谱的虚拟社区知识动态推荐方法[J]. 吉林大学学报(工学版), 2022, 52(10): 2385-2390.
[11]	欧阳宁,李祖锋,林乐平. 基于多层次空⁃谱融合网络的高光谱图像分类[J]. 吉林大学学报(工学版), 2022, 52(10): 2438-2446.
[12]	刘桂霞,裴志尧,宋佳智. 基于深度学习的蛋白质⁃ATP结合位点预测[J]. 吉林大学学报(工学版), 2022, 52(1): 187-194.
[13]	许鸿奎,姜彤彤,李鑫,姜斌祥,王永雷. 结合降噪自编码与极限学习机的LTE上行干扰分析[J]. 吉林大学学报(工学版), 2022, 52(1): 195-203.
[14]	刘远红,郭攀攀,张彦生,李鑫. 基于黎曼流形的稀疏图保持投影的特征提取[J]. 吉林大学学报(工学版), 2021, 51(6): 2268-2279.
[15]	钟辉,康恒,吕颖达,李振建,李红,欧阳若川. 基于注意力卷积神经网络的图像篡改定位算法[J]. 吉林大学学报(工学版), 2021, 51(5): 1838-1844.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed