GAN数据增强下路面裂缝语义分割算法

doi:10.13229/j.cnki.jdxbgxb.20220003

摘要/Abstract

摘要：

为解决现实情况下路面裂缝图像采集数量无法满足深度学习样本数量要求的问题，基于生成式对抗网络的数据扩增方法，提出一种以改进型U-Net网络模型为基础的路面裂缝语义分割算法。首先，基于传统图像处理方法将采集到的样本数据进行初次扩充，根据生成式对抗网络原理，实现样本数据再次扩充；其次，通过增加网络层数、添加归一化层、添加Dropout层提出一种基于改进型U-Net网络模型的路面裂缝语义分割算法；最后，利用基于改进型U-Net网络的路面裂缝语义分割模型提取扩增数据图像中的裂缝，在同等条件下与传统U-Net网络模型检测算法、现有主流分割算法FCN作实验对比研究。结果表明：该改进型算法的分割精度均优于其他两种算法，能较为准确地分割出路面裂缝，在背景像素较为复杂的情况下能较好地避免误检的情况，其平均像素精度、平均交并比分别达到了92.43%、83.43%，在实际场景应用中具有较好的检测效果及较强的泛化性能。

关键词: 道路工程, 沥青路面裂缝, 语义分割, 数据增强, U-Net网络, 生成式对抗网络

Abstract:

In view of the problem that the number of pavement crack images cannot meet basic needs for deep learning， according to using generative adversary network to expand the data-set， a pavement segmentation algorithm based on the U-Net network is proposed. Firstly， the data-set was initially expanded by traditional image generation， according to the principle of generative adversary network， a algorithm of pavement crack segmentation based on semantic segmentation was proposed， which was used to expand the data-set again. Secondly， based on the U-Net， an algorithm of pavement crack segmentation based on semantic segmentation was proposed， which increased the number of network layers and added Batch Normalization and dropout layer. Finally， the semantic segmentation model of pavement cracks was used to extract cracks in the expanded data image， and compared with the traditional detection algorithm and the existing mainstream segmentation algorithm FCN. The results show that the segmentation accuracy of the algorithm is better than other two algorithms， which more precisely segments pavement crack images and avoids error detection when background pixel is complicated. The mean pixel accuracy and mean intersection over union of the algorithm are 92.43% and 83.43%， respectively. In the practical scene application， it has better detection effect and stronger generalization performance.

Key words: road engineering, pavement crack detection, semantic segmentation, data augmentation, U-Net network, generative adversary network

中图分类号:

U416.2

阙云,季雪,蒋子平,戴伊,王叶飞,陈嘉. GAN数据增强下路面裂缝语义分割算法[J]. 吉林大学学报(工学版), 2023, 53(11): 3166-3175.

Yun QUE,Xue JI,Zi-ping JIANG,Yi DAI,Ye-fei WANG,Jia CHEN. Semantic segmentation algorithm of pavement cracks based on GAN data augmentation[J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(11): 3166-3175.

图/表 11

表1

图1

图2

图3

图4

图5

表2

图6

表3

图7

表4

参考文献 31

1	Zhang D J, Li Q Q. A review of pavement high speed detection technology[J]. Journal of Geomatics, 2015, 40(1): 1-8.
2	Shi Y, Cui L M, Qi Z Q, et al. Automatic road crack detection using random structured forest[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434-3445.
3	Xu W, Tang Z M, Lv J Y. Pavement crack detection based on image saliency[J]. Journal of Image and Graphics, 2013, 18(1): 69-77.
4	Hendrickson C T. Applications of advanced technologies in transportation engineering[J]. Journal of Transportation Engineering, 2004, 130(3): 272-273.
5	伯绍波, 闫茂德, 孙国军, 等. 沥青路面裂缝检测图像处理算法研究[J]. 微计算机信息, 2007, 23(15): 280-282.
	Bo Shao-bo, Yan Mao-de, Sun Guo-jun, et al. Researched on crack detection image algorithm for asphalt pavement surface[J]. Microcomputer Information, 2007, 23(15): 280-282.
6	Li Q, Liu X. Novel approach to pavement image segmentation based on neighboring difference histogram method[J]. IEEE Computer Society, 2008, 2: 792-796.
7	Kirschke K R, Velinsky S A. Histogram-based approach for automated pavement-crack sensing[J]. Journal of Transportation Engineering, 1992, 118(5): 700-710.
8	Velinsky S A, Kirschke K R. Design considerations for automated pavement crack sealing machinery[C]∥Application of Advanced Technologies in Transportation Engineering, Beijing, China, 2015: 76-80.
9	徐志刚, 赵祥模, 宋焕生, 等. 基于直方图估计和形状分析的沥青路面裂缝识别算法[J]. 仪器仪表学报, 2010, 31(10): 2260-2266.
	Xu Zhi-gang, Zhao Xiang-mo, Song Huan-sheng, et al. Asphalt pavement crack recognition algorithm based on histogram estimation and shape analysis[J]. Chinese Journal of Scientific Instrument, 2010, 31(10): 2260-2266.
10	刘娜, 宋伟东, 赵泉华. 形态学和最大熵图像分割的城市路面裂缝检测[J]. 辽宁工程技术大学学报: 自然科学版, 2015, 34(1): 57-61.
	Liu Na, Song Wei-dong, Zhao Quan-hua. Morphology and maximum entropy image segmentation based urban pavement cracks detection[J]. Journal of Liaoning Technical University (Natural Science Edition), 2015, 34(1): 57-61.
11	杨国俊, 齐亚辉, 石秀名. 基于数字图像技术的桥梁裂缝检测综述[J/OL]. [2022-01-04]. DOI: 10.13229/j.cnki.jdxbgxb.20221475 doi: 10.13229/j.cnki.jdxbgxb.20221475
12	Cha Y J, Choi W, Buyukozturk O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrasturcture Engineering, 2017, 32(5): 361-378.
13	Long J, Shelamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
14	张祥甫, 刘健, 石章松, 等. 基于深度学习的语义分割问题研究综述[J]. 激光与光电子学进展, 2019, 56(15): 20-34.
	Zhang Xiang-fu, Liu Jian, Shi Zhang-song, et al. Review of deep learning based semantic segmentation[J]. Laser & Optoelectronics Progress, 2019, 56(15): 20-34.
15	王森, 伍星, 张印辉, 等. 基于深度学习的全卷积网络图像裂纹检测[J]. 计算机辅助设计与图形学学报, 2018, 30(5): 115-123.
	Wang Sen, Wu Xing, Zhang Yin-hui, et al. Image crack detection with fully convolutional network based on deep learning[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(5): 115-123.
16	张振海, 季坤, 党建武. 基于桥梁裂缝识别模型的桥梁裂缝病害识别方法[J]. 吉林大学学报: 工学版, 2023, 53(5): 1418-1426.
	Zhang Zhen-hai, Ji Kun, Dang Jian-wu. Crack identification method for bridge based on BCEM model[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(5): 1418-1426.
17	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]∥International Conference on Medical Image Computing and Computer-Assisted Intervention, Paris, France, 2015: 234-241.
18	Wang G. A semidefinite relaxation method for energy-based source localization in sensor network[J]. IEEE Transactions on Vehicular Technology, 2011, 60(5): 2293-2301.
19	Zhang Z, Liu Q, Wang Y. Road extraction by deep residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.
20	Liu Z, Cao Y, Wang Y, et al. Computer vision-based concrete crack detection using U-Net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139.
21	朱苏雅, 杜建超, 李云松, 等. 采用U-Net卷积神经网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019, 46(6): 1-8.
	Zhu Su-ya, Du Jian-chao, Li Yun-song, et al. Method for bridge crack detection based on the U-Net convolutional networks[J]. Journal of Xidian University, 2019, 46(6): 1-8.
22	Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional enconder-deconder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
23	Ren Y, Huang J, Hong Z, et al. Image-based concrete crack detection in tunnels using deep fully convolutional networks[J]. Construction and Building Materials, 2020, 234: 117-127.
24	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]∥Proceedings of the 2014 Conference on Advances in Neural Information Processing Systems 27, Montreal, Canada, 2014: 2672-2680.
25	Zhang K, Zhang Y, Cheng H D, et al. CrackGAN: a labor-light crack detection approach using industrial learning[J]. ArXiv: Computer Vision and Pattern Recognition, 2019(1909): 08216.
26	李良福, 孙瑞赟. 复杂背景下基于图像处理的桥梁裂缝检测算法[J]. 激光与光电子学进展, 2019, 56(6): 112-122.
	Li Liang-fu, Sun Rui-yun. Bridge crack detection algorithm based on image processing under complex background[J]. Laser & Optoelectronics Progress, 2019, 56(6): 112-122.
27	Perez L, Wang J. The effectiveness of data augmentation in image classification using deep learning[J]. ArXiv: Computer Uision and Pattern Recognition, 2017(1712): 04621.
28	梁俊杰, 韦舰晶, 蒋正锋. 生成对抗网络GAN综述[J]. 计算机科学与探索, 2020, 14(1): 1-17.
	Liang Jun-jie, Wei Jian-jing, Jiang Zheng-feng. Generative adversarial networks GAN overview[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(1): 1-17.
29	Zhu W, Miao J, Qing L, et al. Unsupervised representation learning with deep convolutional generative adversarial networks computer science[J]. Computer Science, 2015(11): 331-347.
30	谢钇. 基于U-net神经网络的混凝土路面图像裂缝识别算法研究[D]. 重庆: 重庆邮电大学软件工程学院, 2020.
	Xie Yi. Research on image crack recognition of concrete pavement based on U-net[D]. Chongqing: School of Software Engineering, Chongqing University of Posts and Telecommunications, 2020.
31	李瀚超, 蔡毅, 王岭雪. 全局特征提取的全卷积网络图像语义分割算法[J]. 红外技术, 2019, 41(7): 595-599.
	Li Han-chao, Cai Yi, Wang Ling-xue. Image semantic segmentation based on fully convoluted network with global feature extraction[J]. Infrared Technology, 2019, 41(7): 595-599.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

处理方法	原始图像	处理图像
处理方法	原始图像	a	b	c
翻转与旋转
裁剪
高斯噪声
色彩抖动

参数配置	参数值
基础网络	U-Net
图片尺寸/像素	512×512
批处理量	8
GPU	Nvidia GeForce GTX 1650
求解器	Adam
Max-iter	2000
训练集	分割数据集-train
学习率	1e-4
显存/GB	64 GB
动量	0.95

算法模型	PA	MPA	MIoU	Running time/s
U-Net	0.9663	0.9125	0.7874	338
FCN	0.9644	0.9117	0.7865	647
改进型U-Net	0.9778	0.9243	0.8343	341

图像编号	改进型U-Net		U-Net		FCN
图像编号	PA	IoU	PA	IoU	PA	IoU
平均值	0.9342	0.8235	0.9238	0.8128	0.9214	0.8095
1	0.9505	0.8793	0.9378	0.8647	0.9382	0.8661
2	0.9151	0.7494	0.9073	0.7262	0.9071	0.7263
3	0.9206	0.8301	0.9191	0.7262	0.9204	0.8202
4	0.9465	0.8498	0.9353	0.8463	0.9347	0.8447
5	0.9671	0.9056	0.9663	0.9051	0.9613	0.9002
6	0.9395	0.8414	0.9259	0.8225	0.9248	0.8173
7	0.8858	0.7453	0.8664	0.6761	0.8543	0.6665
8	0.9487	0.8588	0.9321	0.8396	0.9304	0.8346

[1]	赵胜前,丛卓红,游庆龙,李源. 沥青-集料黏附和剥落研究进展[J]. 吉林大学学报(工学版), 2023, 53(9): 2437-2464.
[2]	马涛,马源,黄晓明. 基于多元非线性回归的智能压实关键参数最优解[J]. 吉林大学学报(工学版), 2023, 53(7): 2067-2077.
[3]	杨柳,王创业,王梦言,程阳. 设置自动驾驶小客车专用车道的六车道高速公路交通流特性[J]. 吉林大学学报(工学版), 2023, 53(7): 2043-2052.
[4]	周正峰,于晓涛,陶雅乐,郑茂,颜川奇. 基于灰色关联分析的树脂与弹性体高黏沥青高温性能评价[J]. 吉林大学学报(工学版), 2023, 53(7): 2078-2088.
[5]	张青霞,侯吉林,安新好,胡晓阳,段忠东. 基于车辆脉冲响应的路面不平度识别方法[J]. 吉林大学学报(工学版), 2023, 53(6): 1765-1772.
[6]	姜屏,陈业文,陈先华,张伟清,李娜,王伟. 改性石灰土在干湿和冻融循环下的无侧限抗压性能[J]. 吉林大学学报(工学版), 2023, 53(6): 1809-1818.
[7]	司春棣,崔亚宁,许忠印,凡涛涛. 层间粘结失效后桥面沥青铺装层细观力学行为分析[J]. 吉林大学学报(工学版), 2023, 53(6): 1719-1728.
[8]	李岩,张久鹏,陈子璇,黄果敬,王培. 基于PCA-PSO-SVM的沥青路面使用性能评价[J]. 吉林大学学报(工学版), 2023, 53(6): 1729-1735.
[9]	赵晓康,胡哲,张久鹏,裴建中,石宁. 基于光纤传感技术的路面结冰智能监测研究进展[J]. 吉林大学学报(工学版), 2023, 53(6): 1566-1579.
[10]	惠冰,杨心怡,张乐扬,李扬. 检测车轨迹偏移对沥青路面磨耗计算误差的影响[J]. 吉林大学学报(工学版), 2023, 53(6): 1756-1764.
[11]	李崛,张安顺,张军辉,钱俊峰. 级配碎石基层结构动力响应模型测试及数值分析[J]. 吉林大学学报(工学版), 2023, 53(6): 1782-1789.
[12]	李博,李欣,芮红,梁媛. 基于变分模态分解和灰狼优化极限学习机的隧道口边坡位移预测[J]. 吉林大学学报(工学版), 2023, 53(6): 1853-1860.
[13]	刘状壮,郑文清,郑健,李轶峥,季鹏宇,沙爱民. 基于网格化的路表温度感知技术[J]. 吉林大学学报(工学版), 2023, 53(6): 1746-1755.
[14]	王宁,马涛,陈丰,付永强. 影响智能骨料感知的关键因素及数据分析方法[J]. 吉林大学学报(工学版), 2023, 53(6): 1799-1808.
[15]	黄晓明,赵润民. 道路交通基础设施韧性研究现状及展望[J]. 吉林大学学报(工学版), 2023, 53(6): 1529-1549.