基于旋转自注意力改进Mask RCNN的桥梁裂缝检测方法

doi:10.13229/j.cnki.jdxbgxb.20240122

摘要/Abstract

摘要：

针对现有桥梁裂缝检测方法对桥梁裂缝旋转特征提取不充分，检测分割精度低的问题，提出了一种基于旋转自注意力改进Mask RCNN的桥梁裂缝检测方法。首先，在Mask R-CNN实例分割网络的基础上，采用基于Transformer学习的ViTAE网络作为主干特征提取网络，提高对裂缝的检测和分割精度；然后，设计旋转可变窗口自注意力机制融入桥梁裂缝检测网络，提升特征提取网络对裂缝旋转特征的检测能力；最后，通过可变形卷积进一步拟合裂缝不规则几何形体，强化对裂缝特征信息的识别能力。实验结果表明：本文方法相比于原始Mask R-CNN检测分割方法准确率提高了4.85%，召回率提高了13.95%、F₁-score可达91.66%。本文方法能够更加充分地提取裂缝特征，实现了更加准确的裂缝检测，在主客观评价方面均优于对比方法。

关键词: 桥梁工程, 混凝土桥梁裂缝, 裂缝病害检测, 旋转可变窗口自注意力, Transformer学习

Abstract:

Aiming at the problem that the existing bridge crack detection methods do not fully extract the rotation feature of bridge cracks and have low detection and segmentation accuracy， a bridge crack detection method based on improved Mask RCNN with rotation self-attention was proposed. Firstly， on the basis of the Mask R-CNN instance segmentation network， the ViTAE network based on Transformer learning is used as the backbone feature extraction network to improve the detection and segmentation accuracy of cracks. Then， a rotating variable window self-attention mechanism was designed to integrate into the bridge crack detection network to improve the detection ability of the feature extraction network for crack rotation features. Finally， the deformable convolution was used to further fit the irregular geometry of cracks to strengthen the recognition ability of crack feature information. Experimental results show that compared with the original Mask R-CNN detection and segmentation method， the accuracy of the proposed method is improved by 4.85%， the recall rate is increased by 13.95%， and the F₁-score can reach 91.66%. The proposed method can extract crack features more fully， achieve more accuratecrack detection， and is superior to the comparison methods in subjective and objective evaluation.

Key words: bridge engineering, concrete bridge crack, crack detection of diseases, rotational variable window attention, transformer learning

中图分类号:

U446.3

陈永,安卓奥博,张娇娇. 基于旋转自注意力改进Mask RCNN的桥梁裂缝检测方法[J]. 吉林大学学报(工学版), 2025, 55(11): 3660-3672.

Yong CHEN,Ao-bo ANZHUO,Jiao-jiao ZHANG. Bridge crack detection method based on rotation self-attention improved Mask RCNN[J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(11): 3660-3672.

图/表 16

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

表2

图11

图12

图13

图14

参考文献 23

[1]	杨国俊, 齐亚辉,石秀名. 基于数字图像技术的桥梁裂缝检测综述[J]. 吉林大学学报: 工学版, 2024, 54(2): 313-332.
	Yang Guo-jun, Qi Ya-hui, Shi Xiu-ming. Review of bridge crack detection based on digital image technology[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(2): 313-332.
[2]	徐铖基, 王小虎, 戴雨晴, 等. 混凝土劈裂损伤和毛细传输过程的原位表征[J]. 土木工程学报, 2023, 56(8): 1-11.
	Xu Cheng-ji, Wang Xiao-hu, Dai Yu-qing, et al. In-situ characterization of concrete splitting damage and capillary transport processes[J]. China Civil Engineering Journal, 2023, 56(8): 1-11.
[3]	Zoubir H, Rguig M, Aroussi M E, et al. Concrete bridge crack image classification using histograms of oriented gradients, uniform local binary patterns, and kernel principal component analysis[J]. Electronics, 2022, 11(20): 1-11.
[4]	Vivekananthan V, Vignesh R, Vasanthaseelan S, et al. Concrete bridge crack detection by image processing technique by using the improved OTSU method[J]. Materials Today: Proceedings, 2023, 7: 1002-1007.
[5]	卢印举, 马芳, 戴曙光, 等. 融合多尺度特征的马尔可夫随机场路面裂缝分割算法[J]. 计算机辅助设计与图形学学报, 2022, 34(5): 711-721.
	Lu Yin-ju, Ma Fang, Dai Shu-guang, et al. Markov random field road crack image segmentation algorithm integrating multi-scale features[J]. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(5): 711-721.
[6]	Avendaño J C, Leander J, Karoumi R. Image-based concrete crack detection method using the median absolute deviation[J]. Sensor, 2024, 24(9): 2736.
[7]	Wan C F, Xiong X B, Wen B, et al. Crack detection for concrete bridges with imaged based deep learning[J]. Science Progress, 2022, 105(4): 1-21.
[8]	朱苏雅, 杜建超, 李云松, 等. 采用U-Net卷积网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019, 46(4): 35-42.
	Zhu Su-ya, Du Jian-chao, Li Yun-song, et al. Method for bridge crack detection based on the U-Net convolutional networks[J]. Journal of Xidian University, 2019, 46(4): 35-42.
[9]	谭国金, 欧吉, 艾永明, 等. 基于改进DeepLabv3+模型的桥梁裂缝图像分割方法[J]. 吉林大学学报: 工学版, 2024, 54(1): 173-179.
	Tan Guo-jin, Ji Ou, Ai Yong-ming, et al. Bridge crack image segmentation method based on improved DeepLabv3+model[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(1): 173-179.
[10]	Xiao R Q. YOLOv5s-GTB:light-weighted and improved YOLOv5s for bridge crack detection[DB/OL]. [2024-01-30].
[11]	Deng L, Chu H H, Shi P, et al. Region-based CNN method with deformable modules for visually classifying concrete cracks[J]. Applied Sciences. 2020, 10(7): 2528.
[12]	Liu H J, Yang J, Miao X Y, et al. Crackformer network for pavement crack segmentation[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 24(9): 9240-9252.
[13]	He K M, Gkioxari G, Dollar P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 42(2): 386-397.
[14]	Zhang Q, Xu Y, Zhang J, et al. ViTAEv2: vision transformer advanced by exploring inductive bias for image recognition and beyond[J]. International Journal of Computer Vision, 2023, 131: 1141-1162.
[15]	Worrall D E, Garbin S J, Turmukhambetov D, et al. Harmonic networks: deep translation and rotation equivariance[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 7168-7177.
[16]	Wang D, Zhang Q M, Xu Y F, et al. Advancing plain vision transformer towards remote sensing foundation model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-15.
[17]	Shish V, Noam S, Niki P, et al. Attention is all you need[DB/OL].[2024-01-30].
[18]	余家勇, 李锋,薛现凯, 等. 基于无人机及Mask R-CNN的桥梁结构裂缝智能识别[J]. 中国公路学报, 2021, 34(12): 80-90.
	Yu Jia-yong, Li Feng, Xue Xian-kai, et al. Intelligent identification of bridge structural cracks based on unmanned aerial vehicle and Mask R-CNN[J]. China Journal of Highway and Transport, 2021, 34(12): 80-90.
[19]	Zhu X, Hu H, Lin S, et al. Deformable convnets v2: more deformable, better results[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition, Los Angeles,USA, 2019: 9300-9308.
[20]	李良福, 马卫飞, 李丽, 等. 基于深度学习的桥梁裂缝检测算法研究[J]. 自动化学报, 2019, 45(9): 1727-1742.
	Li Liang-fu, Ma Wei-fei, Li Li, et al. Research on detection algorithm for bridge cracks based on deep learning[J]. Acta Automatica Sinica, 2019, 45(9): 1727-1742.
[21]	Liu Z, Yutong L, Yue C, et al. Swin-Transformer: hierarchical vision transformer using shifted windows[C]∥IEEE International Conference on Computer Vision, Montreal, Canada, 2021: 9992-10002.
[22]	Wang W H, Dai J F, Chen Z, et al. InternImage: exploring large-scale vision foundation models with deformable convolutions[DB/OL]. [2023-8-14].
[23]	Bowen C, Ishan M, Alexander G S, et al. Masked-attention mask transformer for universal image segmentation[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 1280-1289.

相关文章 15

[1]	范亮,曾文,文强,赵富裕,徐英铭. 集束群钉装配式钢-混组合梁桥自振特性与车桥耦合分析[J]. 吉林大学学报(工学版), 2025, 55(7): 2354-2364.
[2]	周勇军,穆丰睿,蔡诚,杨帆. 基于正交试验法的悬索桥索夹螺杆紧固力损失影响因素[J]. 吉林大学学报(工学版), 2025, 55(4): 1188-1196.
[3]	周敉,田兴旺,朱国强,马雷. 装配式桥墩UHPC湿接缝的剪切强度计算方法[J]. 吉林大学学报(工学版), 2025, 55(12): 3928-3941.
[4]	王瑞正,张元海. 设置柔性横隔板的曲线波形钢腹板钢箱组合梁畸变效应[J]. 吉林大学学报(工学版), 2025, 55(11): 3641-3652.
[5]	薛宇欣,周勇军,王业路,范凯翔,赵煜. 基于悬锤系统的简支梁桥冲击系数测试方法适用性[J]. 吉林大学学报(工学版), 2024, 54(9): 2557-2567.
[6]	孙永新,蔺鹏臻,杨子江,冀伟. 考虑黏结-滑移效应的UHPC梁裂缝宽度计算方法[J]. 吉林大学学报(工学版), 2024, 54(9): 2600-2608.
[7]	郭雪莲,韩万水,王涛,周恺,张修石,张书颖. 大件车通行弯桥抗倾覆稳定安全系数评估方法[J]. 吉林大学学报(工学版), 2024, 54(8): 2229-2237.
[8]	肖林,魏欢博,卫星,康志锐. 钢混组合梁栓钉锈胀下混凝土板开裂行为数值分析[J]. 吉林大学学报(工学版), 2024, 54(7): 1958-1965.
[9]	张彦玲,贾云飞,贾晓远,郑旺,李运生. 装配式小箱梁桥内力横向分布系数建议公式[J]. 吉林大学学报(工学版), 2024, 54(6): 1688-1700.
[10]	张春雷,邵长宇,苏庆田,戴昌源. 球扁钢肋钢纤维混凝土组合桥面板正弯矩受力性能试验[J]. 吉林大学学报(工学版), 2024, 54(6): 1634-1642.
[11]	黄汉辉,陈康明,吴庆雄. 钢管混凝土桁式弦杆组合连续梁抗弯性能[J]. 吉林大学学报(工学版), 2024, 54(6): 1665-1676.
[12]	邵长江,崔皓蒙,漆启明,庄卫林. 近断层大跨RC轻柔拱桥纵向阻尼器减震研究[J]. 吉林大学学报(工学版), 2024, 54(5): 1355-1367.
[13]	赵秋,陈鹏,赵煜炜,余澳. 台后设置拱形结构的无缝桥梁整体受力性能[J]. 吉林大学学报(工学版), 2024, 54(4): 1016-1027.
[14]	张洪,朱志伟,胡天宇,龚燕峰,周建庭. 基于改进YOLOv5s的桥梁螺栓缺陷识别方法[J]. 吉林大学学报(工学版), 2024, 54(3): 749-760.
[15]	杨国俊,齐亚辉,石秀名. 基于数字图像技术的桥梁裂缝检测综述[J]. 吉林大学学报(工学版), 2024, 54(2): 313-332.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基线模型	ViTAE	RVSA	DCN	mAP_0.5/%	mAP_0.75/%
Mask-RCNN				84.15	57.31
	√			84.90	65.30
	√	√		88.60	69.80
	√	√	√	89.00	72.60

检测方法	AP/%	AR/%	F₁-score/%
U-Net	71.00	79.30	74.92
Mask R-CNN	84.15	80.54	82.30
Swin-Transformer	86.20	90.34	88.83
Intern-Image	87.40	91.23	89.27
Mask2former	87.29	90.27	88.75
本文网络模型	89.00	94.49	91.66