Journal of Jilin University(Engineering and Technology Edition) ›› 2024, Vol. 54 ›› Issue (3): 749-760.doi: 10.13229/j.cnki.jdxbgxb.20230945

Previous Articles    

Bridge bolt defect identification method based on improved YOLOv5s

Hong ZHANG1,2(),Zhi-wei ZHU2,Tian-yu HU1,Yan-feng GONG3,Jian-ting ZHOU1()   

  1. 1.State Key Laboratory of Mountain Bridge and Tunnel Engineering,Chongqing Jiaotong University,Chongqing 400074,China
    2.School of Information Science and Engineering,Chongqing Jiaotong University,Chongqing 400074,China
    3.School of Shipping and Naval Architecture,Chongqing Jiaotong University,Chongqing 400074,China
  • Received:2023-09-06 Online:2024-03-01 Published:2024-04-18
  • Contact: Jian-ting ZHOU E-mail:hongzhang@cqjtu.edu.cn;jtzhou@cqjtu.edu.cn

Abstract:

To address the issue of insufficient feature extraction and imprecise target localization in existing algorithms for detecting bridge bolt defects due to the complexity of bolt backgrounds and their small size, a bridge bolt defect recognition method based on enhanced YOLOv5s was proposed. Attention mechanisms in the backbone network was introduced to enhance the model's ability to extract bolt features and deepen its focus on global bolt characteristics. The spatial pyramid pooling structure was optimized to reduce the loss of bolt feature information. MPDIoU was employed as the bounding box regression loss function to improve the accuracy of bolt bounding box regression. The YOLO detection head was decoupled to eliminate the adverse effects of shared detection head in target detection on the regression of bounding box positions. Training and testing were conducted on 3810 self-made bolt image datasets of four typical defects: bolt rusting, bolt loosening, bolt detachment, and nut detachment, as well as normal bolts. Experimental results show that the algorithm achieves a detection accuracy of 90.8% for bolt defects, which is a 3% improvement over YOLOv5s, and a mean average precision of 92.6%, representing a 4.3% improvement over YOLOv5s. This method can be applied for intelligent recognition of bolt defects in bridges.

Key words: bridge engineering, bolt defect identification, YOLOv5s, bridge bolts

CLC Number: 

  • U448.14

Fig.1

ECA module"

Fig.2

GAM module"

Fig.3

C3EGA module"

Fig.4

SPPFCSPC structure"

Fig.5

MPDIoU calculation"

Fig.6

Decoupling head"

Fig.7

Bolt defect identification model"

Fig.8

Image super-resolution restoration"

Fig.9

Data enhancement"

Fig.10

Bounding box attributes in the dataset"

Fig.11

Number of defective bolts in the dataset"

Table 1

Results of ablation experiments"

模型C3EGASPPFCSPCMPDIoU解耦头P/%R/%mF1/%mAP@0.5/%
YOLOv5s----87.881.084.388.3
A---88.484.386.390.3
B---89.883.486.590.2
C---88.382.685.490.6
D---87.684.386.090.2
E--89.286.187.691.2
F-89.684.887.192.0
G90.886.388.592.6

Fig.12

Heatmaps before and after model improvement"

Fig.13

Feature maps before and after model improvement"

Table 2

Experimental results of different detection algorithms"

模型mAP@0.5/%模型大小/M
Faster-RCNN69.81024
SSD72.392.6
Centernet78.677.1
DETR81.4474
YOLOv5s88.314.4
YOLOv7-Tiny87.112.3
本文方法92.634.6

Fig.14

Different detection algorithm performances"

1 王有志, 赵文帅, 刘金樟, 等. 斜拉体系加固桥梁桥下连接件力学性能[J]. 吉林大学学报: 工学版, 2022, 52(10): 2376-2384.
Wang You-zhi, Zhao Wen-shuai, Liu Jin-zhang, et al. Mechanical properties of sub-bridge connectors for bridges reinforced by diagonal tension system[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(10): 2376-2384.
2 Zhou J, Huo L. Computer vision-based detection for delayed fracture of bolts in steel bridges[J]. Journal of Sensors, 2021, 2021: 1-12.
3 Park J, Kim T, Kim J. Image-based bolt-loosening detection technique of bolt joint in steel bridges[J/OL]. [2023-08-22].
4 Ramana L, Choi W, Cha Y J. Fully automated vision-based loosened bolt detection using the Viola-Jones algorithm[J]. Structural Health Monitoring, 2019, 18(2): 422-434.
5 Pan Y, Ma Y, Dong Y, et al. A vision-based monitoring method for the looseness of high-strength bolt[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14.
6 王保宪, 欧丙泽, 赵维刚, 等. 钢桥密集螺栓异常状态视觉识别方法[J]. 中国铁道科学, 2023, 44(05): 81-93.
Wang Bao-xian, Bing-ze Ou, Zhao Wei-gang, et al. Abnormal state visual recognition method for dense bolts of steel bridge[J]. China Railway Science, 2023, 44(5): 81-93.
7 Yang X, Gao Y, Fang C, et al. Deep learning-based bolt loosening detection for wind turbine towers[J]. Structural Control and Health Monitoring, 2022, 29(6): No. e2943.
8 李星霖, 周洋, 孙鑫垚, 等. 非接触式螺栓松动在线检测方法研究[J]. 机械设计与制造, 2023(11): 50-53.
Li Xing-lin, Zhou Yang, Sun Xin-yao, et al. Research on non-contact bolt loosening on-line detection method[J]. Machinery Design & Manufacture, 2023(11): 50-53.
9 Yang Z, Zhao Y, Xu C. Detection of missing bolts for engineering structures in natural environment using machine vision and deep learning[J]. Sensors, 2023, 23(12): No. 5655.
10 戚银城, 武学良, 赵振兵, 等. 嵌入双注意力机制的Faster R-CNN航拍输电线路螺栓缺陷检测[J]. 中国图象图形学报, 2021, 26(11):2594-2604.
Qi Yin-cheng, Wu Xue-liang, Zhao Zhen-bing, et al. Bolt defect detection for Transmission lines using Faster R-CNN aerial photography embedded with dual attention Mechanism[J]. Chinese Journal of Image and Graphics, 2021, 26(11): 2594-2604.
11 郝帅, 杨磊, 马旭, 等. 基于注意力机制与跨尺度特征融合的YOLOv5输电线路故障检测[J].中国电机工程学报, 2023, 43(6): 2319-2331.
Hao Shuai, Yang Lei, Ma Xu, et al. Fault detection of YOLOv5 transmission lines based on attention mechanism and cross-scale feature fusion[J]. Proceedings of the CSEE, 2019, 43(6): 2319-2331.
12 Liu L, Zhao J, Chen Z, et al. A new bolt defect identification method incorporating attention mechanism and wide residual networks[J]. Sensors, 2022, 22(19): No. 7416.
13 鞠晓臣, 赵欣欣, 钱胜胜. 基于自注意力机制的桥梁螺栓检测算法[J]. 浙江大学学报: 工学版, 2022, 56(5): 901-908.
Ju Xiao-chen, Zhao Xin-xin, Qian Sheng-sheng. Self-attention mechanism based bridge bolt detection algorithm[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(5): 901-908.
14 李刚,张运涛,汪文凯,等. 采用DETR与先验知识融合的输电线路螺栓缺陷检测方法[J]. 图学学报, 2023, 44(3): 438-447.
Li Gang, Zhang Yun-tao, Wang Wen-kai, et al. Defect detection method of transmission line bolts based on DETR and prior knowledge fusion[J]. Journal of Graphics, 2023, 44(3): 438-447.
15 Wang Q, Wu B, Zhu P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[J/OL]. [2023-08-25].
16 Liu Y, Shao Z, Hoffmann N. Global attention mechanism: retain information to enhance channel-spatial interactions[J/OL]. [2023-08-25].
17 Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J/OL]. [2023-08-25].
18 Ma S L, Xu Y. MPDIoU: a loss for efficient and accurate bounding box regression[J/OL]. [2023-08-28].
19 Ge Z, Liu S, Wang F, et al. YOLOX: exceeding YOLO series in 2021[J/OL]. [2023-09-01].
20 Wang X T, Xie L B, Dong C, et al. RealESRGAN: training real-world blind super-resolution with pure synthetic data[J/OL]. [2023-09-01].
21 Chen C, Liu M Y, Tuzel O, et al. R-CNN for small object detection[C]∥13th Asian Conference on Computer Vision, Taipei, China, 2016: 214-230.
[1] Guo-jun YANG,Ya-hui QI,Xiu-ming SHI. Review of bridge crack detection based on digital image technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 313-332.
[2] Zhi-qiang HAN,Gang XIE,Ya-juan ZHUO,Zuo-long LUO,Hua-teng LI. Vibration response of continuous girder bridge based on wheel⁃deck coherent excitation [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 436-444.
[3] Guo-jin TAN,Ji OU,Yong-ming AI,Run-chao YANG. Bridge crack image segmentation method based on improved DeepLabv3+ model [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 173-179.
[4] Xing WEI,Ya-jie GAO,Zhi-rui KANG,Yu-chen LIU,Jun-ming ZHAO,Lin XIAO. Numerical simulation of residual stress field of stud girth weld in low temperature environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 198-208.
[5] Guan-xu LONG,Xiu-shi ZHANG,Gong-feng XIN,Tao WANG,Gan YANG. Bridge weigh-in-motion combined with machine version [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 188-197.
[6] Xin-dai ZUO,Jin-quan ZHANG,Shang-chuan ZHAO. Fatigue stiffness degradation and life prediction method of in⁃service concrete T⁃beams [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2563-2572.
[7] Ran AN,You-zhi WANG. Shear properties of shear stud connectors under combined tension and shear loading [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2554-2562.
[8] Hui JIANG,Xin LI,Xiao-yu BAI. Review on development of bridge seismic structural systems: from ductility to resilience [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1550-1565.
[9] Chun-li WU,Shi-ming HUANG,Kui LI,Zheng-wei GU,Xiao-ming HUANG,Bing-tao ZHANG,Run-chao YANG. Analysis of pier action effect under flood based on numerical simulation and statistical analysis [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1612-1620.
[10] Guo-jin TAN,Qing-wen KONG,Xin HE,Pan ZHANG,Run-chao YANG,Yang-jun CHAO,Zhong YANG. Bridge scour depth identification based on dynamic characteristics and improved particle swarm optimization algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1592-1600.
[11] Zi-yu LIU,Shi-tong CHEN,Mo-mo ZHI,Xiao-ming HUANG,Zhe-xin CHEN. Ultimate bearing capacity of temporary⁃permanent conversion rushrepair steel pier for emergency use [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1601-1611.
[12] Yu FENG,Jian-ming HAO,Feng WANF,Jiu-peng ZHANG,Xiao-ming HUANG. Analysis of transient wind⁃induced response of long⁃span bridge under nonstationary wind field [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1638-1649.
[13] Hua WANG,Long-lin WANG,Zi-mo ZHANG,Xin HE. Safety early warning technology of continuous rigid frame bridges based on crack width variation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1650-1657.
[14] Jun WANG,Jia-wu LI,Feng WANG,Jiu-peng ZHANG,Xiao-ming HUANG. Wind speed distribution in simplified U⁃shaped valley and its effect on buffeting response of long⁃span suspension bridge [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1658-1668.
[15] Feng WANG,Shuang-rui LIU,Jia-ying WANG,Jia-ling SONG,Jun WANG,Jiu-peng ZHANG,Xiao-ming HUANG. Size and shape effects of wind drag coefficients for porous structures [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(6): 1677-1685.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!