Journal of Jilin University(Engineering and Technology Edition) ›› 2022, Vol. 52 ›› Issue (3): 640-647.doi: 10.13229/j.cnki.jdxbgxb20211274

Previous Articles    

Medical image segmentation based on multi⁃scale context⁃aware and semantic adaptor

Xue WANG1,2(),Zhan-shan LI1,2,Ying-da LYU3()   

  1. 1.College of Computer Science and Technology，Jilin University，Changchun 130012，China
    2.Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education，Jilin University，Changchun 130012，China
    3.Center for Computer Fundamental Education，Jilin University，Changchun 130012，China
  • Received:2021-11-25 Online:2022-03-01 Published:2022-03-08
  • Contact: Ying-da LYU E-mail:wxue19@mails.jlu.edu.cn;ydlv@jlu.edu.cn

RICH HTML

 15   

PDF (PC)

924

Abstract:

Due to the complex characteristics of medical images， for example， the lesion region possesses an irregular shape and its scale can greatly vary， the intensity of surrounding tissues is inhomogeneous and the boundary is blurred， which reduce the accuracy of medical image segmentation， a medical image segmentation algorithm based on multi-scale context-aware and semantic adaptor is proposed. In order to improve the representation ability of feature learning， the multi-scale context-aware module is utilized to learn rich context information from multiple receptive fields， and dynamically assign the weight of semantic features at different scales according to the size of the target region. The multi-level semantic adaptor module is adopted to aggregate multi-level abstract semantic features and spatial details to refine the boundary of the target region and reduce the feature gaps between encoders and decoders. The algorithm proposed in this paper is compared with other algorithms quantitatively and qualitatively on three public medical image datasets of different modalities. The experimental results show that the proposed algorithm is superior to other algorithms in various complex scenarios of medical image segmentation tasks.

Key words: computer application, medical image segmentation, multi-scale context-aware, semantic adaptor, context information

CLC Number: 

  • TP391

Fig.1

Architecture of multi-scale context-aware and semantic adaptor encoder-decoder network"

Fig.2

Architecture of multi-scale context-aware module"

Fig.3

Architecture of multi-level semanticadaptor module"

Table 1

Detailed description of datasets"

数据集图片数量图片大小模态
ISIC 20182594变化尺寸皮肤镜图像
Kvasir-SEG［22］1000变化尺寸结肠镜图像
2018 Data Science Bowl670256 × 256混合模态

Table 2

Experimental comparison results on ISIC 2018 dataset"

方法F1RecSpecAcc
U-Net［12］0.81630.81920.97410.9391
BCDU-Net（d=1）［13］0.84700.78300.98000.9360
BCDU-Net（d=3）［13］0.85100.78500.98200.9370
FANet［23］0.87310.86500.96110.9351
本文0.86960.88370.97830.9591

Fig.4

Visualized results on ISIC 2018 skinlesion segmentation"

Table 3

Comparison results on Kvasir-SEG dataset"

方法DiceMIoURecPrec
U-Net［12］0.81160.72170.79490.8726
ResUNet-mod［25］0.79090.42870.69090.8713
ResUNet++［24］0.81330.79270.70640.8774
本文0.86300.87130.85130.9053

Fig.5

Visualized results on Kvasir-SEGpolyp segmentation"

Table 4

Comparison results on 2018 Data Science Bowl dataset"

方法DiceMIoURecPrec
U-Net［12］0.90980.83720.89040.9164
DoubleU-Net［19］0.91330.84070.64070.9496
FANet［23］0.91760.85690.92220.9194
本文0.92370.91800.90220.9589

Fig.6

Visualized results on 2018 Data ScienceBowl nuclear segmentation"

1 宋杰,肖亮,练智超,等. 基于深度学习的数字病理图像分割综述与展望[J]. 软件学报, 2021, 32(5): 1427-1460.
Song Jie, Xiao Liang, Lian Zhi-chao, et al. Overview and prospect of deep learning for image segmentation in digital pathology[J]. Journal of Software, 2021, 32(5): 1427-1460.
2 Mahmud T, Paul B, Fattah S A. PolypSegNet: a modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images[J]. Computers in Biology and Medicine, 2021, 128:No.104119.
3 Ibtehaz N, Rahman M S. MultiResUNet: rethinking the U-net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121:74-87.
4 Zhang Yan, Lu Yao, Chen Wan-kun, et al. MSMANet: a multi-scale mesh aggregation network for brain tumor segmentation[J]. Applied Soft Computing, 2021, 110: No.107733.
5 Tang Yu-cheng, Gao Ri-qiang, Lee H, et al. Pancreas CT segmentation by predictive phenotyping[C]∥International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 2021: 25-35.
6 秦俊. 基于启发式算法的医学图像阈值分割方法研究[D]. 长春: 吉林大学计算机科学与技术学院, 2019.
Qin Jun. Research on heuristic algorithms based medical image threshold segmentation[D]. Changchun: College of Computer Science and Technology, Jilin University, 2019.
7 周显国, 陈大可, 苑森淼. 基于改进模糊聚类分析的医学脑部MRI图像分割[J]. 吉林大学学报: 工学版, 2009, 39(2): 381-385.
Zhou Xian-guo, Chen Da-ke, Yuan Sen-miao. Medical brain MRI images segmentation by improved fuzzy C-means clustering analysis[J]. Journal of Jilin University (Engineering and Technology Edition), 2009, 39(2): 381-385.
8 肖晓尧, 李雄飞, 张小利, 等. 基于多尺度的区域生长的图像分割算法[J]. 吉林大学学报: 工学版, 2017, 47(5): 1591-1597.
Xiao Xiao-yao, Li Xiong-fei, Zhang Xiao-li, et al. Medical image segmentation algorithm based on multi-scale region growing[J]. Journal of Jilin University (Engineering and Technology Edition), 2017, 47(5): 1591-1597.
9 Zhou San-ping, Wang Jin-jun, Zhang Meng-meng, et al. Correntropy-based level set method for medical image segmentation and bias correction[J]. Neurocomputing, 2017, 234: 216-229.
10 Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 3431-3440.
11 Brandao P, Mazomenos E B, Ciuti G, et al. Fully convolutional neural networks for polyp segmentation in colonoscopy[C]∥Medical Imaging 2017: Computer-Aided Diagnosis, Orlando, Florida, USA, 2017: No.101340F.
12 Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[C]∥International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015: 234-241.
13 Azad R, Asadi-Aghbolaghi M, Fathy M, et al. Bi-directional convLSTM U-Net with densley connected convolutions[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Korea, 2019: 406-415.
14 Gu Ran, Wang Guo-tai, Song Tao, et al. CA-Net: comprehensive attention convolutional neural networks for explainable medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2021, 40(2): 699-711.
15 Wang Cai-yong, Wang Yun-long, Liu Yun-fan, et al. ScleraSegNet: an attention assisted U-Net model for accurate sclera segmentation[J]. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2020, 2(1): 40-54.
16 Gu Zai-wang, Cheng Jun, Fu Hua-zhu, et al. CE-Net: context encoder network for 2D medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2019, 38(10): 2281-2292.
17 Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. UNet++:a nested U-Net architecture for medical image segmentation[C]∥Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Cham, 2018:3-11.
18 Huang Hui-min, Lin Lan-fen, Tong Ruo-feng, et al. UNet 3+: a full-scale connected unet for medical image segmentation[C]∥IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, Spain, 2020: 1055-1059.
19 Jha D, Riegler M A, Johansen D, et al. DoubleU-Net: a deep convolutional neural network for medical image segmentation[C]∥IEEE 33rd International Symposium on Computer-Based Medical Systems, Rochester, MN, USA, 2020: 558-564.
20 Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[C]∥Thirty-first AAAI Conference on Artificial Intelligence, San Francisco, California, USA, 2017: 4278-4284.
21 Yu Chang-qian, Wang Jing-bo, Peng Chao, et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation[C]∥Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 325-341.
22 Jha D, Smedsrud P H, Riegler M A, et al. Kvasir-SEG: a segmented polyp dataset[C]∥International Conference on Multimedia Modeling, Daejeon, South Korea, 2020: 451-462.
23 Tomar N K, Jha D, Riegler M A, et al. FANet: a feedback attention network for improved biomedical image segmentation[J/OL]. [2020-12-25]. .
24 Jha D, Smedsrud P H, Riegler M A, et al. ResUNet++: an advanced architecture for medical image segmentation[C]∥IEEE International Symposium on Multimedia, San Diego, CA, USA, 2019: 225-230.
25 Zhang Zheng-xin, Liu Qing-jie, Wang Yun-hong. Road extraction by deep residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.
[1] Su-ming KANG,Ye-e ZHANG. Hadoop⁃based local timing link prediction algorithm across social networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(3): 626-632.
[2] You QU,Wen-hui LI. Single-stage rotated object detection network based on anchor transformation [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 162-173.
[3] Hong-wei ZHAO,Dong-sheng HUO,Jie WANG,Xiao-ning LI. Image classification of insect pests based on saliency detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2174-2181.
[4] Zhou-zhou LIU,Qian-yun ZHANG,Xin-hua MA,Han PENG. Compressed sensing signal reconstruction based on optimized discrete differential evolution algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2246-2252.
[5] Dong-ming SUN,Liang HU,Yong-heng XING,Feng WANG. Text fusion based internet of things service recommendation for trigger⁃action programming pattern [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2182-2189.
[6] Sheng-sheng WANG,Jing-yu CHEN,Yi-nan LU. COVID⁃19 chest CT image segmentation based on federated learning and blockchain [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2164-2173.
[7] Jun-cong LIN,Jun LEI,Meng CHEN,Shi-hui GUO,Xing GAO,Ming-hong LIAO. Real⁃time camera planning for dynamic multiple targets considering cinematographic visual properties [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2154-2163.
[8] Li-li REN,Zhi-jun WANG,Dong-mei YAN. Improved multi⁃verse algorithm with combined slime mould foraging behavior [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2190-2197.
[9] Yin-di YAO,Jun-jin HE,Yang-li LI,Dang-yuan XIE,Ying LI. ET0 simulation of self⁃constructed improved whale optimized BP neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1798-1807.
[10] Hong-wei ZHAO,Zi-jian ZHANG,Jiao LI,Yuan ZHANG,Huang-shui HU,Xue-bai ZANG. Bi⁃direction segmented anti⁃collision algorithm based on query tree [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1830-1837.
[11] Jie CAO,Xue QU,Xiao-xu LI. Few⁃shot image classification method based on sliding feature vectors [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1785-1791.
[12] Xiao-xue SUN,Hui ZHONG,Hai-peng CHEN. Statistical analysis system for students' examination results based on decision tree classification technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1866-1872.
[13] Meng-su ZHANG,Chun-tian LIU,Xi-jin LI,Yong-ping HUANG. Design of fuzzy comprehensive evaluation system for performance appraisal based on K⁃means clustering algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1851-1856.
[14] Chun-bo WANG,Xiao-qiang DI. Cloud storage integrity verification audit scheme based on label classification [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1364-1369.
[15] Dan-tong OUYANG,Yang LIU,Jie LIU. Fault diagnosis method based on test set under fault response guidance [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1017-1025.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd