Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (4): 1139-1145.doi: 10.13229/j.cnki.jdxbgxb.20220126

Previous Articles    

Depth estimation algorithm of monocular image based on self-supervised learning

Lin BAI1(),Lin-jun LIU1,Xuan-ang LI1,Sha WU1,Ru-qing LIU2   

  1. 1.School of Computer，Electronics and Information，Guangxi University，Nanning 530004，China
    2.College of Electronic Engineering，Guangxi Normal University，Guilin 541004，China
  • Received:2022-02-11 Online:2023-04-01 Published:2023-04-20

RICH HTML

 20   

PDF (PC)

409

Abstract:

In order to improve the effectiveness of monocular image depth estimation， a monocular image depth estimation algorithm based on self supervised learning is designed. The absolute depth feature， relative depth feature and position feature of monocular image are obtained， and the logarithm of depth difference is used as the loss function to obtain the point-to-point correlation feature of image； The surface normal loss function is used to deal with the surface normal of the depth map to reduce the surface fluctuation of the image， and the similarity between the image photometric loss function and the two images is calculated； The monocular image is input into the self supervised learning network， the depth amplitude is calculated， and the monocular image depth estimation based on self supervised learning is completed. The experiment is mainly divided into objective evaluation and subjective evaluation. In the objective evaluation part， the accuracy of depth prediction of the proposed estimation algorithm reaches 91.2%， and the edge error of depth map is only 3%. This method has high accuracy. In subjective evaluation， the proposed algorithm is verified to be able to predict the image features and improve the image depth estimation effect effectively.

Key words: software engineering, self-supervised learning, monocular image, depth estimation, feature extraction, loss function

CLC Number: 

  • TP311

Fig.1

Monocular image depth estimation processbased on self supervised learning"

Fig.2

Self supervised learning network trainingprocess"

Fig.3

Experimental image"

Fig.4

Comparison of depth prediction accuracy"

Fig.5

Comparison of accuracy of depth image edgeestimation"

Fig.6

Image 1 estimation results"

Fig.7

Image 2 estimation results"

1 霍智勇, 乔璐. 基于结构化损失的单目深度估计算法研究[J]. 电子科技大学学报, 2021, 50(5): 728-733.
Huo Zhi-yong, Qiao Lu. Research on monocular depth estimation algorithm based on structured loss[J]. Journal of University of Electronic Science and Technology of China, 2021, 50(5): 728-733.
2 王亚群, 戴华林, 王丽, 等. 基于密集卷积网络的单目图像深度估计方法[J]. 计算机工程, 2021, 47(11): 262-267, 291.
Wang Ya-qun, Dai Hua-lin, Wang Li, et al. Method for estimating monocular image depth based on dense convolutional network[J]. Computer Engineering, 2021, 47(11): 262-267, 291.
3 王泉德, 张松涛. 基于多尺度特征融合的单目图像深度估计[J]. 华中科技大学学报:自然科学版, 2020, 48(5): 7-12.
Wang Quan-de, Zhang Song-tao. Monocular depth estimation with multi-scale feature fusion[J]. Journal of Huazhong University of Science and Technology (Nature Science Edition), 2020, 48(5): 7-12.
4 于恒, 梅红岩, 许晓明, 等. 基于深度学习的图像压缩算法研究综述[J]. 计算机工程与应用, 2020, 56(15): 15-23.
Yu Heng, Mei Hong-yan, Xu Xiao-ming, et al. Survey of image compression algorithm based on deep learning[J]. Computer Engineering and Applications, 2020, 56(15): 15-23.
5 刘杰平, 温竣文, 梁亚玲. 基于多尺度注意力导向网络的单目图像深度估计[J]. 华南理工大学学报: 自然科学版, 2020, 48(12): 52-62.
Liu Jie-ping, Wen Jun-wen, Liang Ya-ling. Monocular image depth estimation based on multi-scale attention oriented network[J]. Monocular Image Depth Estimation Based on Multi-Scale Attention Oriented Network, 2020, 48(12): 52-62.
6 李旭, 丁萌, 魏东辉, 等. VDAS中基于单目红外图像的深度估计方法[J]. 系统工程与电子技术, 2021, 43(5): 1210-1217.
Li Xu, Ding Meng, Wei Dong-hui, et al. Depth estimation method based on monocular infrared image in VDAS[J]. Systems Engineering and Electronics, 2021, 43(5): 1210-1217.
7 赵栓峰, 黄涛, 许倩, 等. 面向无人机自主飞行的无监督单目视觉深度估计[J]. 激光与光电子学进展, 2020, 57(2): 145-154.
Zhao Shuan-feng, Huang Tao, Xu Qian, et al. Unsupervised monocular depth estimation for autonomous flight of drones[J]. Laser & Optoelectronics Progress, 2020, 57(2): 145-154.
8 王倩倩, 赵海涛. 基于深度CRF网络的单目红外场景深度估计[J]. 红外技术, 2020, 42(6): 580-588.
Wang Qian-qian, Zhao Hai-tao. Depth estimation of monocular infrared scene based on deep crf network[J]. Infrared Technology, 2020, 42(6): 580-588.
9 邹承明, 胡佑璞. 引入生成对抗网络的室外场景单目深度估计[J]. 计算机工程与应用, 2021, 57(6): 176-183.
Zou Cheng-ming, Hu You-pu. Monocular depth estimation in outdoor scene with generative adversarial network[J]. Computer Engineering and Applications, 2021, 57(6): 176-183.
10 张建博, 袁亮, 何丽, 等. 基于深度预测的单目SLAM绝对尺度估计[J]. 计算机工程与设计, 2021, 42(6): 1749-1755.
Zhang Jian-bo, Yuan Liang, He Li, et al. Absolute scale estimation of monocular SLAM based on depth prediction[J]. Computer Engineering and Design, 2021, 42(6): 1749-1755.
11 陈莹, 王一良. 基于密集特征融合的无监督单目深度估计[J]. 电子与信息学报, 2021, 43(10): 2976-2984.
Chen Ying, Wang Yi-liang. Unsupervised monocular depth estimation based on dense feature fusion[J]. Journal of Electronics & Information Technology, 2021, 43(10): 2976-2984.
12 孟昭宇, 张宁宁. 基于聚焦线索的数字图像误匹配点剔除仿真[J]. 计算机仿真, 2020, 37(1): 402-406.
Meng Zhao-yu, Zhang Ning-ning. Digital image mismatching point culling simulation based on focus cues[J]. Computer Simulation, 2020, 37(1): 402-406.
13 王欣盛, 张桂玲. 基于卷积神经网络的单目深度估计[J]. 计算机工程与应用, 2020, 56(13): 143-149.
Wang Xin-sheng, Zhang Gui-ling. Monocular depth estimation based on convolutional neural network[J]. Computer Engineering and Applications, 2020, 56(13): 143-149.
14 陈莹, 沈栎. 基于特征回归的单目深度图无标记人体姿态估计[J]. 系统仿真学报, 2020, 32(2): 269-277.
Chen Ying, Shen Li. Monocular depth image mark-less pose estimation based on feature regression[J]. Journal of System Simulation, 2020, 32(2): 269-277.
15 周云成, 邓寒冰, 许童羽, 等. 基于稠密自编码器的无监督番茄植株图像深度估计模型[J]. 农业工程学报, 2020, 36(11): 182-192.
Zhou Yun-cheng, Deng Han-bing, Xu tong-yu, et al. Unsupervised deep estimation modeling for tomato plant image based on dense convolutional auto-encoder[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(11): 182-192.
[1] Xiao-hu SHI,Jia-qi WU,Chun-guo WU,Shi CHENG,Xiao-hui WENG,Zhi-yong CHANG. Residual network based curve enhanced lane detection method [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 584-592.
[2] Hong-wei ZHAO,Jian-rong ZHANG,Jun-ping ZHU,Hai LI. Image classification framework based on contrastive self⁃supervised learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1850-1856.
[3] Yi-na ZHOU,Hong-li DONG,Yong ZHANG,Jing-yi LU. Feature extraction method of pipeline signals based on VMD de-noising and dispersion entropy [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 959-969.
[4] Guo-fa LI,Yan-bo WANG,Jia-long HE,Ji-li WANG. Research progress and development trend of health assessment of electromechanical equipment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(2): 267-279.
[5] Xiao⁃lei CHEN,Yong⁃feng SUN,Ce LI,Dong⁃mei LIN. Stable anti⁃noise fault diagnosis of rolling bearing based on CNN⁃BiLSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(2): 296-309.
[6] Xian-jun DU,Liang-liang JIA. Fault diagnosis of rolling bearing based on optimized stacked denoising auto encoders [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2827-2838.
[7] Li-can DAI,Xiang DAI,Ying CUI,Yong-chao WEI. Anomaly data mining algorithm in social network based on deep integrated learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2712-2717.
[8] Jie CAO,Zhi-Dong HE,Ping YU,Jin-hua WANG. Bearing fault diagnosis method under unbalanced data distribution [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2523-2531.
[9] Da-ke ZHOU,Chao ZHANG,Xin YANG. Self-supervised 3D face reconstruction based on multi-scale feature fusion and dual attention mechanism [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(10): 2428-2437.
[10] Cai-mao LI,Shao-fan CHEN,Cheng-rong LIN,Yu-quan HOU,Hao LI. Dynamic recommendation method of virtual community knowledge based on circular knowledge map [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(10): 2385-2390.
[11] Gui-xia LIU,Zhi-yao PEI,Jia-zhi SONG. Prediction of protein-ATP binding site based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 187-194.
[12] Hong-kui XU,Tong-tong JIANG,Xin LI,Bin-xiang JIANG,Yong-lei WANG. LTE uplink interference analysis combined with denoising autoencoder and extreme learning machine [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 195-203.
[13] Yuan-hong LIU,Pan-pan GUO,Yan-sheng ZHANG,Xin LI. Feature extraction of sparse graph preserving projection based on Riemannian manifold [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2268-2279.
[14] Hui ZHONG,Heng KANG,Ying-da LYU,Zhen-jian LI,Hong LI,Ruo-chuan OUYANG. Image manipulation localization algorithm based on channel attention convolutional neural networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1838-1844.
[15] Xue-yun CHEN,Tao XU,Xiao-qiao HUANG. Detection method of medical cell image generation based on conditional generative adversarial network [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1414-1419.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd