GAN数据增强下路面裂缝语义分割算法

doi:10.13229/j.cnki.jdxbgxb.20220003

Abstract

Abstract:

In view of the problem that the number of pavement crack images cannot meet basic needs for deep learning， according to using generative adversary network to expand the data-set， a pavement segmentation algorithm based on the U-Net network is proposed. Firstly， the data-set was initially expanded by traditional image generation， according to the principle of generative adversary network， a algorithm of pavement crack segmentation based on semantic segmentation was proposed， which was used to expand the data-set again. Secondly， based on the U-Net， an algorithm of pavement crack segmentation based on semantic segmentation was proposed， which increased the number of network layers and added Batch Normalization and dropout layer. Finally， the semantic segmentation model of pavement cracks was used to extract cracks in the expanded data image， and compared with the traditional detection algorithm and the existing mainstream segmentation algorithm FCN. The results show that the segmentation accuracy of the algorithm is better than other two algorithms， which more precisely segments pavement crack images and avoids error detection when background pixel is complicated. The mean pixel accuracy and mean intersection over union of the algorithm are 92.43% and 83.43%， respectively. In the practical scene application， it has better detection effect and stronger generalization performance.

Key words: road engineering, pavement crack detection, semantic segmentation, data augmentation, U-Net network, generative adversary network

CLC Number:

U416.2

Yun QUE,Xue JI,Zi-ping JIANG,Yi DAI,Ye-fei WANG,Jia CHEN. Semantic segmentation algorithm of pavement cracks based on GAN data augmentation[J].Journal of Jilin University(Engineering and Technology Edition), 2023, 53(11): 3166-3175.

Figures/Tables 11

Table 1

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 2

Fig.6

Table 3

Fig. 7

Table 4

References 31

1	Zhang D J, Li Q Q. A review of pavement high speed detection technology[J]. Journal of Geomatics, 2015, 40(1): 1-8.
2	Shi Y, Cui L M, Qi Z Q, et al. Automatic road crack detection using random structured forest[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434-3445.
3	Xu W, Tang Z M, Lv J Y. Pavement crack detection based on image saliency[J]. Journal of Image and Graphics, 2013, 18(1): 69-77.
4	Hendrickson C T. Applications of advanced technologies in transportation engineering[J]. Journal of Transportation Engineering, 2004, 130(3): 272-273.
5	伯绍波, 闫茂德, 孙国军, 等. 沥青路面裂缝检测图像处理算法研究[J]. 微计算机信息, 2007, 23(15): 280-282.
	Bo Shao-bo, Yan Mao-de, Sun Guo-jun, et al. Researched on crack detection image algorithm for asphalt pavement surface[J]. Microcomputer Information, 2007, 23(15): 280-282.
6	Li Q, Liu X. Novel approach to pavement image segmentation based on neighboring difference histogram method[J]. IEEE Computer Society, 2008, 2: 792-796.
7	Kirschke K R, Velinsky S A. Histogram-based approach for automated pavement-crack sensing[J]. Journal of Transportation Engineering, 1992, 118(5): 700-710.
8	Velinsky S A, Kirschke K R. Design considerations for automated pavement crack sealing machinery[C]∥Application of Advanced Technologies in Transportation Engineering, Beijing, China, 2015: 76-80.
9	徐志刚, 赵祥模, 宋焕生, 等. 基于直方图估计和形状分析的沥青路面裂缝识别算法[J]. 仪器仪表学报, 2010, 31(10): 2260-2266.
	Xu Zhi-gang, Zhao Xiang-mo, Song Huan-sheng, et al. Asphalt pavement crack recognition algorithm based on histogram estimation and shape analysis[J]. Chinese Journal of Scientific Instrument, 2010, 31(10): 2260-2266.
10	刘娜, 宋伟东, 赵泉华. 形态学和最大熵图像分割的城市路面裂缝检测[J]. 辽宁工程技术大学学报: 自然科学版, 2015, 34(1): 57-61.
	Liu Na, Song Wei-dong, Zhao Quan-hua. Morphology and maximum entropy image segmentation based urban pavement cracks detection[J]. Journal of Liaoning Technical University (Natural Science Edition), 2015, 34(1): 57-61.
11	杨国俊, 齐亚辉, 石秀名. 基于数字图像技术的桥梁裂缝检测综述[J/OL]. [2022-01-04]. DOI: 10.13229/j.cnki.jdxbgxb.20221475 doi: 10.13229/j.cnki.jdxbgxb.20221475
12	Cha Y J, Choi W, Buyukozturk O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrasturcture Engineering, 2017, 32(5): 361-378.
13	Long J, Shelamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
14	张祥甫, 刘健, 石章松, 等. 基于深度学习的语义分割问题研究综述[J]. 激光与光电子学进展, 2019, 56(15): 20-34.
	Zhang Xiang-fu, Liu Jian, Shi Zhang-song, et al. Review of deep learning based semantic segmentation[J]. Laser & Optoelectronics Progress, 2019, 56(15): 20-34.
15	王森, 伍星, 张印辉, 等. 基于深度学习的全卷积网络图像裂纹检测[J]. 计算机辅助设计与图形学学报, 2018, 30(5): 115-123.
	Wang Sen, Wu Xing, Zhang Yin-hui, et al. Image crack detection with fully convolutional network based on deep learning[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(5): 115-123.
16	张振海, 季坤, 党建武. 基于桥梁裂缝识别模型的桥梁裂缝病害识别方法[J]. 吉林大学学报: 工学版, 2023, 53(5): 1418-1426.
	Zhang Zhen-hai, Ji Kun, Dang Jian-wu. Crack identification method for bridge based on BCEM model[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(5): 1418-1426.
17	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]∥International Conference on Medical Image Computing and Computer-Assisted Intervention, Paris, France, 2015: 234-241.
18	Wang G. A semidefinite relaxation method for energy-based source localization in sensor network[J]. IEEE Transactions on Vehicular Technology, 2011, 60(5): 2293-2301.
19	Zhang Z, Liu Q, Wang Y. Road extraction by deep residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.
20	Liu Z, Cao Y, Wang Y, et al. Computer vision-based concrete crack detection using U-Net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139.
21	朱苏雅, 杜建超, 李云松, 等. 采用U-Net卷积神经网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019, 46(6): 1-8.
	Zhu Su-ya, Du Jian-chao, Li Yun-song, et al. Method for bridge crack detection based on the U-Net convolutional networks[J]. Journal of Xidian University, 2019, 46(6): 1-8.
22	Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional enconder-deconder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
23	Ren Y, Huang J, Hong Z, et al. Image-based concrete crack detection in tunnels using deep fully convolutional networks[J]. Construction and Building Materials, 2020, 234: 117-127.
24	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]∥Proceedings of the 2014 Conference on Advances in Neural Information Processing Systems 27, Montreal, Canada, 2014: 2672-2680.
25	Zhang K, Zhang Y, Cheng H D, et al. CrackGAN: a labor-light crack detection approach using industrial learning[J]. ArXiv: Computer Vision and Pattern Recognition, 2019(1909): 08216.
26	李良福, 孙瑞赟. 复杂背景下基于图像处理的桥梁裂缝检测算法[J]. 激光与光电子学进展, 2019, 56(6): 112-122.
	Li Liang-fu, Sun Rui-yun. Bridge crack detection algorithm based on image processing under complex background[J]. Laser & Optoelectronics Progress, 2019, 56(6): 112-122.
27	Perez L, Wang J. The effectiveness of data augmentation in image classification using deep learning[J]. ArXiv: Computer Uision and Pattern Recognition, 2017(1712): 04621.
28	梁俊杰, 韦舰晶, 蒋正锋. 生成对抗网络GAN综述[J]. 计算机科学与探索, 2020, 14(1): 1-17.
	Liang Jun-jie, Wei Jian-jing, Jiang Zheng-feng. Generative adversarial networks GAN overview[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(1): 1-17.
29	Zhu W, Miao J, Qing L, et al. Unsupervised representation learning with deep convolutional generative adversarial networks computer science[J]. Computer Science, 2015(11): 331-347.
30	谢钇. 基于U-net神经网络的混凝土路面图像裂缝识别算法研究[D]. 重庆: 重庆邮电大学软件工程学院, 2020.
	Xie Yi. Research on image crack recognition of concrete pavement based on U-net[D]. Chongqing: School of Software Engineering, Chongqing University of Posts and Telecommunications, 2020.
31	李瀚超, 蔡毅, 王岭雪. 全局特征提取的全卷积网络图像语义分割算法[J]. 红外技术, 2019, 41(7): 595-599.
	Li Han-chao, Cai Yi, Wang Ling-xue. Image semantic segmentation based on fully convoluted network with global feature extraction[J]. Infrared Technology, 2019, 41(7): 595-599.

Related Articles 15

[1]	Sheng-qian ZHAO,Zhuo-hong CONG,Qing-long YOU,Yuan LI. Adhesion and raveling property between asphalt and aggregate： a review [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2437-2464.
[2]	Tao MA,Yuan MA,Xiao-ming HUANG. Optimal combination of key parameters of intelligent compaction based on multiple nonlinear regression [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2067-2077.
[3]	Liu YANG,Chuang-ye WANG,Meng-yan WANG,Yang CHENG. Traffic flow characteristics of six⁃lane freeways with a dedicated lane for automatic cars [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2043-2052.
[4]	Zheng-feng ZHOU,Xiao-tao YU,Ya-le TAO,Mao ZHENG,Chuan-qi YAN. High-temperature performance evaluation of resin and elastomer high viscosity asphalt based on grey correlation analysis [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2078-2088.
[5]	Qing-xia ZHANG,Ji-lin HOU,Xin-hao AN,Xiao-yang HU,Zhong-dong DUAN. Road roughness identification method based on vehicle impulse response [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1765-1772.
[6]	Ping JIANG,Ye-wen CHEN,Xian-hua CHEN,Wei-qing ZHANG,Na LI,Wei WANG. Unconfined compression behavior of modified lime stabilized soil under dry wet and freeze⁃thaw cycles [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1809-1818.
[7]	Chun-di SI,Ya-ning CUI,Zhong-yin XU,Tao-tao FAN. Meso⁃mechanical behavior analysis of asphalt bridge deck pavement after interlayer bonding failure [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1719-1728.
[8]	Yan LI,Jiu-peng ZHANG,Zi-xuan CHEN,Guo-jing HUANG,Pei WANG. Evaluation of asphalt pavement performance based on PCA⁃PSO⁃SVM [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1729-1735.
[9]	Xiao-kang ZHAO,Zhe HU,Jiu-peng ZHANG,Jian-zhong PEI,Ning SHI. Research progress in intelligent monitoring of pavement icing based on optical fiber sensing technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1566-1579.
[10]	Bing HUI,Xin-yi YANG,Le-yang ZHANG,Yang LI. Influence of detecting track offset on calculation error of asphalt pavement wearing [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1756-1764.
[11]	Jue LI,An-shun ZHANG,Jun-hui ZHANG,Jun-feng QIAN. Model testing and numerical analysis of dynamic response of graded crushed rock base structure [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1782-1789.
[12]	Bo LI,Xin LI,Hong RUI,Yuan LIANG. Displacement prediction of tunnel entrance slope based on variational modal decomposition and grey wolf optimized extreme learning machine [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1853-1860.
[13]	Zhuang-zhuang LIU,Wen-qing ZHENG,Jian ZHENG,Yi-zheng LI,Peng-yu JI,Ai-min SHA. Pavement surface temperature monitoring method based on gridding approach [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1746-1755.
[14]	Ning WANG,Tao MA,Feng CHEN,Yong-qiang FU. Key factors affecting smart aggregate perception and data analysis methods [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1799-1808.
[15]	Zhe ZHANG,Wei FU,Jun-hui ZHANG,Chao HUANG. Long⁃term characterising plastic behavior of thawed subgrade clay under cyclic loads [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1790-1798.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	LI Shoutao, LI Yuanchun. Autonomous Mobile Robot Control Algorithm Based on Hierarchical Fuzzy Behaviors in Unknown Environments[J]. 吉林大学学报(工学版), 2005, 35(04): 391 -397 .
[2]	Liu Qing-min，Wang Long-shan，Chen Xiang-wei，Li Guo-fa. Ball nut detection by machine vision[J]. 吉林大学学报(工学版), 2006, 36(04): 534 -538 .
[3]	Li Hong-ying; Shi Wei-guang;Gan Shu-cai. Electromagnetic properties and microwave absorbing property of Z type hexaferrite Ba_3-xLa_xCo₂Fe₂₄O₄₁[J]. 吉林大学学报(工学版), 2006, 36(06): 856 -0860 .
[4]	Zhang Quan-fa，Li Ming-zhe，Sun Gang，Ge Xin . Comparison between flexible and rigid blank-holding in multi-point forming[J]. 吉林大学学报(工学版), 2007, 37(01): 25 -30 .
[5]	Yang Shu-kai, Song Chuan-xue, An Xiao-juan, Cai Zhang-lin . Analyzing effects of suspension bushing elasticity on vehicle yaw response character with virtual prototype method[J]. 吉林大学学报(工学版), 2007, 37(05): 994 -0999 .
[6]	. [J]. 吉林大学学报(工学版), 2007, 37(06): 1284 -1287 .
[7]	Che Xiang-jiu，Liu Da-you，Wang Zheng-xuan . Construction of joining surface with G¹ continuity for two NURBS surfaces[J]. 吉林大学学报(工学版), 2007, 37(04): 838 -841 .
[8]	Liu Han-bing, Jiao Yu-ling, Liang Chun-yu,Qin Wei-jun . Effect of shape function on computing precision in meshless methods[J]. 吉林大学学报(工学版), 2007, 37(03): 715 -0720 .
[9]	. [J]. 吉林大学学报(工学版), 2007, 37(04): 0 .
[10]	Li Yue-ying，Liu Yong-bing，Chen Hua . Surface hardening and tribological properties of a cam materials[J]. 吉林大学学报(工学版), 2007, 37(05): 1064 -1068 .

处理方法	原始图像	处理图像
处理方法	原始图像	a	b	c
翻转与旋转
裁剪
高斯噪声
色彩抖动

参数配置	参数值
基础网络	U-Net
图片尺寸/像素	512×512
批处理量	8
GPU	Nvidia GeForce GTX 1650
求解器	Adam
Max-iter	2000
训练集	分割数据集-train
学习率	1e-4
显存/GB	64 GB
动量	0.95

算法模型	PA	MPA	MIoU	Running time/s
U-Net	0.9663	0.9125	0.7874	338
FCN	0.9644	0.9117	0.7865	647
改进型U-Net	0.9778	0.9243	0.8343	341

图像编号	改进型U-Net		U-Net		FCN
图像编号	PA	IoU	PA	IoU	PA	IoU
平均值	0.9342	0.8235	0.9238	0.8128	0.9214	0.8095
1	0.9505	0.8793	0.9378	0.8647	0.9382	0.8661
2	0.9151	0.7494	0.9073	0.7262	0.9071	0.7263
3	0.9206	0.8301	0.9191	0.7262	0.9204	0.8202
4	0.9465	0.8498	0.9353	0.8463	0.9347	0.8447
5	0.9671	0.9056	0.9663	0.9051	0.9613	0.9002
6	0.9395	0.8414	0.9259	0.8225	0.9248	0.8173
7	0.8858	0.7453	0.8664	0.6761	0.8543	0.6665
8	0.9487	0.8588	0.9321	0.8396	0.9304	0.8346

Semantic segmentation algorithm of pavement cracks based on GAN data augmentation

RICH HTML

PDF (PC)