Journal of Jilin University(Engineering and Technology Edition) ›› 2022, Vol. 52 ›› Issue (6): 1413-1421.doi: 10.13229/j.cnki.jdxbgxb20210027

Previous Articles    

Simplified residual structure and fast deep residual networks

Huai-jiang YANG1,2(),Er-shuai WANG1,3,Yong-xin SUI1,2,Feng YAN1,2,Yue ZHOU1,2   

  1. 1.Changchun Institute of Optics，Fine Mechanics and Physics，Chinese Academy of Sciences，Changchun 130033，China
    2.Changchun National Extreme Precision Optics Co. ，Ltd. ，Changchun 130033，China
    3.Daheng College，University of Chinese Academy of Sciences，Beijing 100049，China
  • Received:2021-01-13 Online:2022-06-01 Published:2022-06-02

RICH HTML

 6   

PDF (PC)

115

Abstract:

In order to address the problem of slow training of the current deep ResNets model, a novel residual structure is designed. Compared with the typical residual structure， the structure only contains a Batch Normalization and ReLU， which reduces training time and improves the training speed by reducing the amount of calculation in the network training process. The comparative experiments are carried out on the CIFAR10/100 image classification database. The classification error rate of 110 layers networks constructed by this method on CIFAR10 and CIFAR100 is 5.29% and 24.80%， respectively. The classification error rate of 110-ResNet is 5.75% and 26.02%， respectively. Training the network takes 133.47 （this method） and 208.26 （ResNet） seconds per epoch， increased by 35.91%. The results show that the network structure greatly improves the training speed while ensuring the classification performance， and has better practical value.

Key words: image processing, image recognition, image classification, convolutional neural networks, deep residual networks

CLC Number: 

  • TP183

Fig.1

Basic and bottleneck residual structure"

Fig.2

Schematic diagram of ReLU"

Fig.3

Two kinds of simplified residual structure"

Fig.4

Degradation model of the second simplified residual structure"

Fig.5

Schematic diagram of fast residual network with 110 layers"

Table 1

Comparison of classification error rateand training time per epoch of threeresidual structures"

残差网络分类错误率/%周期耗时/s
基本型7.62±0.1530.07
第1类残差结构7.68±0.2022.08
第2类残差结构10.04±0.1822.10

Fig.6

Test classification error cure of the threeresidual structures on CIFAR10"

Table 2

Comparison of basic ResNet and fast deep residual networks（FastResNet）"

残差网络CIFAR10错误率/%CIFAR100错误率/%周期耗时/s
56层ResNet6.3127.9384.22
56层FastResNet5.8126.9158.84
110层ResNet5.7526.02208.26
110层FastResNet5.2924.80133.47

Fig.7

Classification error curve of 110-ResNetand 110-FastResNet on CIFAR10"

Fig.8

Classification error curve of 110-ResNetand 110-FastResNet on CIFAR100"

1 He Kai-ming, Zhang Xiang-yu, Ren Shao-qing, et al. Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, LasVegas, NV, USA, 2016: 770-778.
2 He Kai-ming, Zhang Xiang-yu, Ren Shao-qing, et al. Identity mappings in deep residual networks[C]∥European Conference on Computer Vision, Cham, 2016: 630-645.
3 Targ S, Almeid A D, Lyman K. Resnet in resnet: Generalizing residual architectures[EB/OL]. [2016-03-25].
4 Zagoruyko S, Komodakis N. Wide residual networks[EB/OL]. [2017-06-14].
5 Veit A, Wilber M, Belongie S. Residual networks behave like ensembles of relatively shallow networks[C]∥Proceedings of the 30th International Conference on Neural Information Processing Systems. Newyork: Curran Associates Inc, 2016:550-558.
6 Wu Z F, Shen C H, van den Hengel A. Wider or deeper: revisiting the resnet model for visual recognition[EB/OL]. [2016-11-30].
7 王若瑜. 基于Resnet-50的智能驾驶红绿灯分类研究[J]. 电子测试,2019, 25():143-145.
Wang Ruo-yu. Research on classification of intelligent driving traffic light based on Resnet-50[J]. Electronic Test, 2019, 25(Sup.1):143-145.
8 宋倩, 黄昶, 余慧瑶. 基于TensorFlow的交通标志形状识别[J]. 信息通信, 2017, 32(12): 286-288.
Song Qian, Huang Chang, Yu Hui-yao. Traffic sign shape recognition based on tensorflow[J]. Information & Communications, 2017, 32(12): 286-288.
9 王文成, 蒋慧, 乔倩, 等. 基于ResNet50网络的十种鱼类图像分类识别研究[J]. 农村经济与科技, 2019, 30(19): 60-62.
Wang Wen-cheng, Jiang Hui, Qiao Qian, et al. Research on classification and recognition of ten fish images based on Resnet-50[J]. Rural Economy and Science-Technology, 2019, 30(19): 60-62.
10 常川. 基于ResNet深度网络的人类蛋白质图谱图像分类方法研究[J]. 医学信息学, 2019, 40(7): 45-49.
Chang Chuan. Study of image classification method of human protein atlas based on resnet deep network[J]. Journal of Medical Intelligence, 2019, 40(7): 45-49.
11 王恒,李霞,刘晓芳, 等. 基于ResNet50网络的乳腺癌病理图像分类研究[J]. 中国计量大学学报, 2019, 30(1): 72-77.
Wang Heng, Li Xia, Liu Xiao-fang, et al. Classification of breast cancer histopathological images based on ResNet50 network[J]. Journal of China Jiliang University, 2019, 30(1): 72-77.
12 张枫, 田联房, 杜启亮. 基于残差网络与中心损失的人脸识别[J].计算机工程与设计, 2019, 40(6): 1689-1695.
Zhang Feng, Tian Lian-fang, Du Qi-liang. Face recognition based on Resnet and center loss[J]. Computer Engineering and Design, 2019, 40(6): 1689-1695.
13 刘媛. 基于深度学习的说话人识别研究[D]. 上海: 上海交通大学信息与通信工程学院,2015.
Liu Yuan. Research on deep leaning based speaker recognition[D]. Shanghai: School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, 2015.
14 朱超平, 杨艺. 基于YOLO2和ResNet算法的监控视频中的人脸检测与识别[J]. 重庆理工大学学报: 自然科学, 2018, 32(8): 170-175.
Zhu Chao-ping, Yang Yi. Face Detection and recognition in monitoring video based on YOLO2 and ResNet algorithm[J]. Journal of Chongqing University of Technology(Natural Science), 2018, 32(8): 170-175.
15 熊才华,巩言丽,廉华, 等. 基于ResNet-50改进的Faster R-CNN手势识别算法[J]. 计算机时代, 2019, 36(9): 1-4.
Xiong Cai-hua, Gong Yan-li, Lian Hua, et al. An improved Faster R-CNN hand gesture recognition algorithm based on ResNet-50[J]. Computer Era, 2019, 36(9):1-4.
16 刘子毅. 基于图谱特征分析的农业虫害检测方法研究[D]. 杭州: 浙江大学生物系统工程与食品科学学院, 2017.
Liu Zi-yi. Detection of agricultural pest insects based on imaging and spectral feature analysis[D]. Hangzhou: College of Biosystems Engineering and Food Science, Zhejiang University, 2017.
17 张银松,赵银娣,袁慕策. 基于改进Faster-RCNN模型的粘虫板图像昆虫识别与计数[J]. 中国农业大学学报, 2019, 24(5): 115-122.
Zhang Yin-song, Zhao Yin-di, Yuan Mu-ce. Insect identification and counting based on an improved Faster-RCNN model of the sticky board image[J]. Journal of China Agricultural University, 2019, 24(5): 115-122.
18 Huang Gao, Sun Yu, Liu Zhuang, et al. Deep networks with stochastic depth[C]∥European Conference on Computer Vision, Cham, 2016: 646-661.
19 Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[C]∥Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Canada, 2010:249-256.
20 Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
21 Lin Min, Chen Qiang, Yan Shui-cheng. Network in network[EB/OL]. [2014-03-04].
22 Springenberg J, Dosovitskiy A, Brox T, et al. Striving for simplicity: the all convolutional net[EB/OL]. [2015-04-13].
23 Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, 2015: 1-9.
24 Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[EB/OL]. [2015-03-02].
25 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2015-04-10].
26 Santurkar S, Tsipras D, Ilyas A, et al. How does batch normalization help optimization?[EB/OL]. [2021-01-02].
[1] Ming-hua GAO,Can YANG. Traffic target detection method based on improved convolution neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1353-1361.
[2] Hai-yang JIA,Rui XIA,An-qi LYU,Ceng-xuan GUAN,Juan CHEN,Lei WANG. Panoramic mosaic approach of ultrasound medical images based on template fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 916-924.
[3] Xiang-jun LI,Jie-ying TU,Zhi-bin ZHAO. Validity classification of melting curve based on multi⁃scale fusion convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(3): 633-639.
[4] Mao-yue LI,Shuo LIU,Shuai TIAN,Gui-feng XIAO. Image feature extraction and recognition of milling chatter of thin walled parts [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(2): 425-432.
[5] Liang DUAN,Chun-yuan SONG,Chao LIU,Wei WEI,Cheng-ji LYU. State recognition in bearing temperature of high-speed train based on machine learning algorithms [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 53-62.
[6] Hui ZHONG,Heng KANG,Ying-da LYU,Zhen-jian LI,Hong LI,Ruo-chuan OUYANG. Image manipulation localization algorithm based on channel attention convolutional neural networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1838-1844.
[7] Hou⁃jie LI,Fa⁃sheng WANG,Jian⁃jun HE,Yu ZHOU,Wei LI,Yu⁃xuan DOU. Pseudo sample regularization Faster R⁃CNN for traffic sign detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1251-1260.
[8] Kang WANG,Meng YAO,Li-ben LI,Jian-qiao LI,Xiang-jin DENG,Meng ZOU,Long XUE. Mechanical performance identification for lunar soil in lunar surface sampling [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1146-1152.
[9] Qian-yi XU,Gui-he QIN,Ming-hui SUN,Cheng-xun MENG. Classification of drivers' head status based on improved ResNeSt [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(2): 704-711.
[10] Jian LI,Kong-yu LIU,Xian-sheng REN,Qi XIONG,Xue-feng DOU. Application of canny algorithm based on adaptive threshold in MR Image edge detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(2): 712-719.
[11] Fu LIU,Lu LIU,Tao HOU,Yun LIU. Night road image enhancement method based on optimized MSR [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 323-330.
[12] Xiang-jiu CHE,Hua-luo LIU,Qing-bin SHAO. Fabric defect recognition algorithm based onimproved Fast RCNN [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2038-2044.
[13] LIU Zhe, XU Tao, SONG Yu-qing, XU Chun-yan. Image fusion technology based on NSCT and robust principal component analysis model with similar information [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(5): 1614-1620.
[14] CHE Xiang-jiu, WANG Li, GUO Xiao-xin. Improved boundary detection based on multi-scale cues fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(5): 1621-1628.
[15] CHEN Mian-shu, SU Yue, SANG Ai-jun, LI Pei-peng. Image classification methods based on space vector model [J]. 吉林大学学报(工学版), 2018, 48(3): 943-951.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd