吉林大学学报(工学版) ›› 2024, Vol. 54 ›› Issue (3): 749-760.doi: 10.13229/j.cnki.jdxbgxb.20230945

• 交通运输工程·土木工程 • 上一篇    

基于改进YOLOv5s的桥梁螺栓缺陷识别方法

张洪1,2(),朱志伟2,胡天宇1,龚燕峰3,周建庭1()   

  1. 1.重庆交通大学 省部共建山区桥梁及隧道工程国家重点实验室，重庆 400074
    2.重庆交通大学 信息科学与工程学院，重庆 400074
    3.重庆交通大学 航运与船舶工程学院，重庆 400074
  • 收稿日期:2023-09-06 出版日期:2024-03-01 发布日期:2024-04-18
  • 通讯作者: 周建庭 E-mail:hongzhang@cqjtu.edu.cn;jtzhou@cqjtu.edu.cn
  • 作者简介:张洪（1987-），男，教授，博士.研究方向：桥梁健康监测及无损检测.E-mail：hongzhang@cqjtu.edu.cn
  • 基金资助:
    国家自然科学基金项目(52278291);重庆市自然科学基金项目(CSTB2022NSCQ-LZX0006);重庆交通大学研究生科研创新项目(2023S0083)

Bridge bolt defect identification method based on improved YOLOv5s

Hong ZHANG1,2(),Zhi-wei ZHU2,Tian-yu HU1,Yan-feng GONG3,Jian-ting ZHOU1()   

  1. 1.State Key Laboratory of Mountain Bridge and Tunnel Engineering，Chongqing Jiaotong University，Chongqing 400074，China
    2.School of Information Science and Engineering，Chongqing Jiaotong University，Chongqing 400074，China
    3.School of Shipping and Naval Architecture，Chongqing Jiaotong University，Chongqing 400074，China
  • Received:2023-09-06 Online:2024-03-01 Published:2024-04-18
  • Contact: Jian-ting ZHOU E-mail:hongzhang@cqjtu.edu.cn;jtzhou@cqjtu.edu.cn

RICH HTML

 3   

PDF (PC)

256

摘要：

针对现有算法在检测桥梁螺栓缺陷时因螺栓背景复杂和尺寸较小而导致的特征提取不充分、目标定位不精确问题，提出了一种基于改进YOLOv5s的桥梁螺栓缺陷识别方法。该方法在骨干网络中引入注意力机制以提升模型对螺栓特征的提取能力并加深对螺栓全局特征的关注度；优化空间金字塔池化结构以减少螺栓特征信息流失；采用MPDIoU作为边界框回归损失函数，提高螺栓边界框的回归精度；将YOLO检测头解耦以消除目标检测中分类任务和回归任务共享检测头对边界框位置回归的负面影响。在螺栓锈蚀、螺栓松动、螺栓脱落和螺母脱落4类典型缺陷螺栓以及正常螺栓的3810张自制螺栓图像数据集上进行训练和测试，实验结果表明：本文算法对螺栓缺陷的检测精度达到90.8%，相较于YOLOv5s提升了3%，均值平均精度达到92.6%，相较于YOLOv5s提升了4.3%，可以应用于桥梁螺栓的缺陷智能识别。

关键词: 桥梁工程, 螺栓缺陷识别, YOLOv5s, 桥梁螺栓

Abstract:

To address the issue of insufficient feature extraction and imprecise target localization in existing algorithms for detecting bridge bolt defects due to the complexity of bolt backgrounds and their small size， a bridge bolt defect recognition method based on enhanced YOLOv5s was proposed. Attention mechanisms in the backbone network was introduced to enhance the model's ability to extract bolt features and deepen its focus on global bolt characteristics. The spatial pyramid pooling structure was optimized to reduce the loss of bolt feature information. MPDIoU was employed as the bounding box regression loss function to improve the accuracy of bolt bounding box regression. The YOLO detection head was decoupled to eliminate the adverse effects of shared detection head in target detection on the regression of bounding box positions. Training and testing were conducted on 3810 self-made bolt image datasets of four typical defects： bolt rusting， bolt loosening， bolt detachment， and nut detachment， as well as normal bolts. Experimental results show that the algorithm achieves a detection accuracy of 90.8% for bolt defects， which is a 3% improvement over YOLOv5s， and a mean average precision of 92.6%， representing a 4.3% improvement over YOLOv5s. This method can be applied for intelligent recognition of bolt defects in bridges.

Key words: bridge engineering, bolt defect identification, YOLOv5s, bridge bolts

中图分类号: 

  • U448.14

图1

ECA模块"

图2

GAM模块"

图3

C3EGA模块"

图4

SPPFCSPC结构"

图5

MPDIoU计算方式"

图6

解耦头"

图7

螺栓缺陷识别模型"

图8

图像超分辨率修复"

图9

数据增强"

图10

数据集中样本框属性"

图11

数据集中缺陷螺栓数量"

表1

消融实验结果"

模型C3EGASPPFCSPCMPDIoU解耦头P/%R/%mF1/%mAP@0.5/%
YOLOv5s----87.881.084.388.3
A√---88.484.386.390.3
B-√--89.883.486.590.2
C--√-88.382.685.490.6
D---√87.684.386.090.2
E√√--89.286.187.691.2
F√√√-89.684.887.192.0
G√√√√90.886.388.592.6

图12

改进模型前、后热力图"

图13

改进模型前、后特征图"

表2

不同检测算法实验结果"

模型mAP@0.5/%模型大小/M
Faster-RCNN69.81024
SSD72.392.6
Centernet78.677.1
DETR81.4474
YOLOv5s88.314.4
YOLOv7-Tiny87.112.3
本文方法92.634.6

图14

不同检测算法效果"

1 王有志, 赵文帅, 刘金樟, 等. 斜拉体系加固桥梁桥下连接件力学性能[J]. 吉林大学学报: 工学版, 2022, 52(10): 2376-2384.
Wang You-zhi, Zhao Wen-shuai, Liu Jin-zhang, et al. Mechanical properties of sub-bridge connectors for bridges reinforced by diagonal tension system[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(10): 2376-2384.
2 Zhou J, Huo L. Computer vision-based detection for delayed fracture of bolts in steel bridges[J]. Journal of Sensors, 2021, 2021: 1-12.
3 Park J, Kim T, Kim J. Image-based bolt-loosening detection technique of bolt joint in steel bridges[J/OL]. [2023-08-22].
4 Ramana L, Choi W, Cha Y J. Fully automated vision-based loosened bolt detection using the Viola-Jones algorithm[J]. Structural Health Monitoring, 2019, 18(2): 422-434.
5 Pan Y, Ma Y, Dong Y, et al. A vision-based monitoring method for the looseness of high-strength bolt[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14.
6 王保宪, 欧丙泽, 赵维刚, 等. 钢桥密集螺栓异常状态视觉识别方法[J]. 中国铁道科学, 2023, 44(05): 81-93.
Wang Bao-xian, Bing-ze Ou, Zhao Wei-gang, et al. Abnormal state visual recognition method for dense bolts of steel bridge[J]. China Railway Science, 2023, 44(5): 81-93.
7 Yang X, Gao Y, Fang C, et al. Deep learning-based bolt loosening detection for wind turbine towers[J]. Structural Control and Health Monitoring, 2022, 29(6): No. e2943.
8 李星霖, 周洋, 孙鑫垚, 等. 非接触式螺栓松动在线检测方法研究[J]. 机械设计与制造, 2023(11): 50-53.
Li Xing-lin, Zhou Yang, Sun Xin-yao, et al. Research on non-contact bolt loosening on-line detection method[J]. Machinery Design & Manufacture, 2023(11): 50-53.
9 Yang Z, Zhao Y, Xu C. Detection of missing bolts for engineering structures in natural environment using machine vision and deep learning[J]. Sensors, 2023, 23(12): No. 5655.
10 戚银城, 武学良, 赵振兵, 等. 嵌入双注意力机制的Faster R-CNN航拍输电线路螺栓缺陷检测[J]. 中国图象图形学报, 2021, 26(11):2594-2604.
Qi Yin-cheng, Wu Xue-liang, Zhao Zhen-bing, et al. Bolt defect detection for Transmission lines using Faster R-CNN aerial photography embedded with dual attention Mechanism[J]. Chinese Journal of Image and Graphics, 2021, 26(11): 2594-2604.
11 郝帅, 杨磊, 马旭, 等. 基于注意力机制与跨尺度特征融合的YOLOv5输电线路故障检测[J].中国电机工程学报, 2023, 43(6): 2319-2331.
Hao Shuai, Yang Lei, Ma Xu, et al. Fault detection of YOLOv5 transmission lines based on attention mechanism and cross-scale feature fusion[J]. Proceedings of the CSEE, 2019, 43(6): 2319-2331.
12 Liu L, Zhao J, Chen Z, et al. A new bolt defect identification method incorporating attention mechanism and wide residual networks[J]. Sensors, 2022, 22(19): No. 7416.
13 鞠晓臣, 赵欣欣, 钱胜胜. 基于自注意力机制的桥梁螺栓检测算法[J]. 浙江大学学报: 工学版, 2022, 56(5): 901-908.
Ju Xiao-chen, Zhao Xin-xin, Qian Sheng-sheng. Self-attention mechanism based bridge bolt detection algorithm[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(5): 901-908.
14 李刚,张运涛,汪文凯,等. 采用DETR与先验知识融合的输电线路螺栓缺陷检测方法[J]. 图学学报, 2023, 44(3): 438-447.
Li Gang, Zhang Yun-tao, Wang Wen-kai, et al. Defect detection method of transmission line bolts based on DETR and prior knowledge fusion[J]. Journal of Graphics, 2023, 44(3): 438-447.
15 Wang Q, Wu B, Zhu P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[J/OL]. [2023-08-25].
16 Liu Y, Shao Z, Hoffmann N. Global attention mechanism: retain information to enhance channel-spatial interactions[J/OL]. [2023-08-25].
17 Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J/OL]. [2023-08-25].
18 Ma S L, Xu Y. MPDIoU: a loss for efficient and accurate bounding box regression[J/OL]. [2023-08-28].
19 Ge Z, Liu S, Wang F, et al. YOLOX: exceeding YOLO series in 2021[J/OL]. [2023-09-01].
20 Wang X T, Xie L B, Dong C, et al. RealESRGAN: training real-world blind super-resolution with pure synthetic data[J/OL]. [2023-09-01].
21 Chen C, Liu M Y, Tuzel O, et al. R-CNN for small object detection[C]∥13th Asian Conference on Computer Vision, Taipei, China, 2016: 214-230.
[1] 杨国俊,齐亚辉,石秀名. 基于数字图像技术的桥梁裂缝检测综述[J]. 吉林大学学报(工学版), 2024, 54(2): 313-332.
[2] 韩智强,谢刚,卓亚娟,骆佐龙,李华腾. 基于车轮-桥面相干激励的大跨连续梁桥振动响应[J]. 吉林大学学报(工学版), 2024, 54(2): 436-444.
[3] 谭国金,欧吉,艾永明,杨润超. 基于改进DeepLabv3+模型的桥梁裂缝图像分割方法[J]. 吉林大学学报(工学版), 2024, 54(1): 173-179.
[4] 卫星,高亚杰,康志锐,刘宇辰,赵骏铭,肖林. 低温环境下栓钉环焊缝焊接残余应力场数值模拟[J]. 吉林大学学报(工学版), 2024, 54(1): 198-208.
[5] 龙关旭,张修石,辛公锋,王涛,杨干. 融合机器视觉的桥梁动态称重方法[J]. 吉林大学学报(工学版), 2024, 54(1): 188-197.
[6] 左新黛,张劲泉,赵尚传. 在役混凝土T梁疲劳刚度退化及寿命预测方法[J]. 吉林大学学报(工学版), 2023, 53(9): 2563-2572.
[7] 安然,王有志. 剪力钉连接件拉剪共同作用抗剪性能[J]. 吉林大学学报(工学版), 2023, 53(9): 2554-2562.
[8] 江辉,李新,白晓宇. 桥梁抗震结构体系发展述评：从延性到韧性[J]. 吉林大学学报(工学版), 2023, 53(6): 1550-1565.
[9] 吴春利,黄诗茗,李魁,顾正伟,黄晓明,张炳涛,杨润超. 基于数值仿真和统计分析的洪水作用下桥墩作用效应分析[J]. 吉林大学学报(工学版), 2023, 53(6): 1612-1620.
[10] 谭国金,孔庆雯,何昕,张攀,杨润超,朝阳军,杨忠. 基于动力特性和改进粒子群优化算法的桥梁冲刷深度识别[J]. 吉林大学学报(工学版), 2023, 53(6): 1592-1600.
[11] 刘子玉,陈士通,支墨墨,黄晓明,陈哲心. 可“临-永”转换抢修钢墩应急使用极限承载力[J]. 吉林大学学报(工学版), 2023, 53(6): 1601-1611.
[12] 冯宇,郝键铭,王峰,张久鹏,黄晓明. 非平稳极端风作用下大跨桥梁瞬态风致效应分析[J]. 吉林大学学报(工学版), 2023, 53(6): 1638-1649.
[13] 王华,王龙林,张子墨,何昕. 基于裂缝宽度变化的连续刚构桥安全性预警技术[J]. 吉林大学学报(工学版), 2023, 53(6): 1650-1657.
[14] 王俊,李加武,王峰,张久鹏,黄晓明. 简化U形峡谷风速分布及其对悬索桥抖振响应的影响[J]. 吉林大学学报(工学版), 2023, 53(6): 1658-1668.
[15] 王峰,刘双瑞,王佳盈,宋佳玲,王俊,张久鹏,黄晓明. 尺寸和形状效应对多孔结构风阻系数的影响[J]. 吉林大学学报(工学版), 2023, 53(6): 1677-1685.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 《吉林大学学报(工学版)》编辑部
地址：长春市人民大街5988号　电话：+86-431-85095297　传真：+86-431-85094074　E-mail: xbgxb@jlu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发