吉林大学学报(工学版) ›› 2023, Vol. 53 ›› Issue (8): 2421-2429.doi: 10.13229/j.cnki.jdxbgxb.20211070

• 农业工程·仿生工程 • 上一篇    

基于深度学习与图像处理的蔬菜苗期杂草识别方法

金小俊1(),孙艳霞2,于佳琳3,陈勇1()   

  1. 1.南京林业大学 机械电子工程学院, 南京 210037
    2.南京交通职业技术学院 轨道交通学院, 南京 211188
    3.北京大学 现代农业研究院, 山东 潍坊 261325
  • 收稿日期:2021-10-18 出版日期:2023-08-01 发布日期:2023-08-21
  • 通讯作者: 陈勇 E-mail:xiaojun.jin@outlook.com;chenyongjsnj@163.com
  • 作者简介:金小俊（1987-），男，工程师，博士研究生.研究方向：机器视觉与人工智能技术.E-mail：xiaojun.jin@outlook.com
  • 基金资助:
    国家自然科学基金项目(32072498);江苏省重点研发计划项目(BE2021016);江苏省农业科技自主创新项目（CX（21）3184）

Weed recognition in vegetable at seedling stage based on deep learning and image processing

Xiao-jun JIN1(),Yan-xia SUN2,Jia-lin YU3,Yong CHEN1()   

  1. 1.College of Mechanical and Electronic Engineering，Nanjing Forestry University，Nanjing 210037，China
    2.School of Rail Transportation，Nanjing Vocational Institute of Transport Technology，Nanjing 211188，China
    3.Institute of Advanced Agricultural Sciences，Peking University，Weifang 261325，China
  • Received:2021-10-18 Online:2023-08-01 Published:2023-08-21
  • Contact: Yong CHEN E-mail:xiaojun.jin@outlook.com;chenyongjsnj@163.com

RICH HTML

 20   

PDF (PC)

524

摘要：

对苗期青菜及其伴生杂草进行识别试验，提出了一种基于识别蔬菜进而间接识别杂草的独特方法，该方法结合深度学习和图像处理技术，可以有效降低杂草识别的复杂度，同时提高识别的精度和鲁棒性。首先，采用神经网络模型对青菜进行识别，并标记边框；然后，将青菜边框之外的绿色目标视为杂草，利用颜色特征将其分割，并通过面积滤波得到滤除噪点后的杂草区域；为探究不同深度学习模型对青菜识别的效果，选取SSD模型、RetinaNet模型和FCOS模型，以F1值、平均精度和检测速度3个评价指标进行对比分析。结果表明，SSD模型为青菜识别最优模型，拥有最高的检测速度和较优的识别率，其在测试集的F1值、平均精度和检测速度分别为95.4%、98.1%和31.0 f/s；改进后的MExG因子能有效识别杂草，分割后的杂草形态完整且轮廓清晰。本文提出的蔬菜田杂草识别方法具有高度的可行性和极佳的应用效果，可为相似作物田杂草识别提供技术参考。

关键词: 农业工程, 蔬菜识别, 杂草识别, 深度学习, 图像处理, 颜色特征

Abstract:

In this study， the recognition test of bok choy and its associated weeds at the seedling stage was carried out， and a novel method based on recognizing vegetables and then indirectly recognizing weeds was proposed. By combining deep learning and image processing technology， this method can effectively reduce the complexity of weed recognition， and at the same time improving the accuracy and robustness of weed recognition. First， a neural network model was used for detecting the bok choy and drawing bounding boxes. The green targets outside the bok choy bounding boxes were marked as weeds， and color features were used to segment them. Besides， an area filter was used for eliminating the noises and extracting weed regions. In order to explore the effects of different deep learning models on bok choy recognition， SSD model， RetinaNet model and FCOS model were selected， and three evaluation metrics of F1 value， average accuracy and detection speed were used for comparative analysis. The SSD model was the best model for bok choy recognition， with the highest detection speed and excellent recognition rate. Its F1 value， average accuracy and detection speed in the test set were 95.4%， 98.1% and 31.0 f/s， respectively. The improved MExG index can effectively recognize weeds， and the segmented weeds have complete shapes and clear outlines. Experiment results show that the proposed method for recognizing weeds in vegetable fields is highly feasible and has excellent application effects， which can also provide technical reference for weed recognition in similar crop fields.

Key words: agricultural engineering, vegetable recognition, weed recognition, deep learning, image processing, color feature

中图分类号: 

  • TP391.41

图1

杂草识别流程示意图"

表1

不同模型超参设置"

模型批尺寸动量初始学习率/10-3优化器衰减值/10-4训练周期
SSD40.92SGD524
RetinaNet40.910SGD124
FCOS40.910SGD124

表2

验证集不同置信度阈值的评价数据"

模型置信度精度召回率F1值
SSD0.90.9800.840.904
0.80.9710.890.929
0.70.9660.910.937
0.60.9640.940.952
0.50.9600.950.955
0.40.9590.960.960
0.30.9410.970.955
0.20.9220.980.950
0.10.8950.990.940
RetinaNet0.90.9670.950.958
0.80.9580.960.959
0.70.9580.960.959
0.60.9400.970.955
0.50.9240.980.951
0.40.9240.980.951
0.30.9240.980.951
0.20.8870.990.936
0.10.8870.990.936
FCOS0.90.0000.000.000
0.81.0000.060.113
0.71.0000.280.438
0.60.9980.510.675
0.50.9910.740.847
0.40.9820.870.922
0.30.9700.930.949
0.20.9110.960.935
0.10.8500.970.906

表3

测试集最优置信度阈值的评价数据"

模型置信度精度召回率F1 值平均精度检测速度/（f·s-1）
SSD0.40.9380.970.9540.98131.0
RetinaNet0.7/0.80.943/0.9470.98/0.970.961/0.9590.98417.5
FCOS0.30.9700.940.9550.96617.9

图2

不同场景下的原图及SSD模型青菜检测效果图"

图3

漏检和误检场景图"

图4

杂草分割效果图"

图5

面积滤波及最终杂草识别效果图"

1 金月, 肖宏儒, 曹光乔, 等. 我国叶类蔬菜机械化水平现状与评价方法研究[J]. 中国农机化学报, 41(12): 196-201.
Jin Yue, Xiao Hong-ru, Cao Guang-qiao, et al. Research on status and evaluation methods of leafy vegetable mechanization production level in China[J]. Journal of Chinese Agricultural Mechanization, 41(12): 196-201.
2 Ryder E. World vegetable industry: production, breeding, trends[J]. Horticultural Reviews, 2011, 38: 299-356.
3 Han J, Luo Y, Yang L, et al. Acidification and salinization of soils with different initial pH under greenhouse vegetable cultivation[J]. Journal of Soils and Sediments, 2014, 14(10): 1683-1692.
4 陆海涛, 吕建强, 金伟, 等. 我国叶类蔬菜机械化收获技术的发展现状[J]. 农机化研究, 2018, 40(6): 261-268.
Lu Hai-tao, Lv Jian-qiang, Jin Wei, et al. The current situation of the mechanized harvesting technology development of leaf vegetable in China[J]. Journal of Agricultural Mechanization Research, 2018, 40(6): 261-268.
5 徐艳蕾, 何润, 翟钰婷, 等. 基于轻量卷积网络的田间自然环境杂草识别方法[J]. 吉林大学学报:工学版, 2021, 51(6): 2304-2312.
Xu Yan-lei, He Run, Zhai Yu-ting, et al. Weed identification method based on deep transfer learning in field natural environment[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(6): 2304-2312.
6 王刚, 刘慧力, 贾洪雷, 等. 触碰定位式玉米行间除草装置的设计与试验[J]. 吉林大学学报:工学版, 2021, 51(4): 1518-1527.
Wang Gang, Liu Hui-li, Jia Hong-lei, et al. Design and experiment of touching-positioning weeding device for inter-row maize (Zea Mays L.)[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(4): 1518-1527.
7 洪晓玮, 陈勇, 杨超淞, 等. 有机蔬菜大棚除草机器人研制[J]. 制造业自动化, 2021, 43(5): 33-36.
Hong Xiao-wei, Chen Yong, Yang Chao-song, et al. Development of a weeding robot for organic vegetable greenhouse[J]. Manufacturing Automation, 2021, 43(5): 33-36.
8 Dai X, Xu Y, Zheng J, et al. Comparison of image-based methods for determining the inline mixing uniformity of pesticides in direct nozzle injection systems[J]. Biosystems Engineering, 2020, 190: 157-175.
9 Lanini W, Strange M. Low-input management of weeds in vegetable fields[J]. California Agriculture, 1991, 45(1): 11-13.
10 金小俊, 陈勇, 孙艳霞. 农田杂草识别方法研究进展[J]. 农机化研究, 2011, 33(7): 23-27, 33.
Jin Xiao-jun, Chen Yong, Sun Yan-xia. Research advances of weed identification in agricultural fields[J]. Journal of Agricultural Mechanization Research, 2011, 33(7): 23-27, 33.
11 毛文华, 张银桥, 王辉, 等. 杂草信息实时获取技术与设备研究进展[J]. 农业机械学报, 2013, 44(1): 190-195.
Mao Wen-hua, Zhang Yin-qiao, Wang Hui, et al. Advance techniques and equipments for real-time weed detection[J]. Transactions of the Chinese Society for Agricultural Machinery, 2013, 44(1): 190-195.
12 金小俊, 陈勇, 侯学贵, 等. 基于机器视觉的除草机器人杂草识别[J]. 山东科技大学学报:自然科学版, 2012, 31(2): 104-108.
Jin Xiao-jun, Chen Yong, Hou Xue-gui, et al. Weed recognition of the machine vision based weeding robot[J]. Journal of Shandong University of Science and Technology (Natural Science), 2012, 31(2): 104-108.
13 陈良宵, 王斌. 基于形状特征的叶片图像识别算法比较研究[J]. 计算机工程与应用, 2017 (9): 17-25.
Chen Liang-xiao, Wang Bin. Comparative study of leaf image recognition algorithm based on shape feature[J]. Computer Engineering and Applications, 2017 (9): 17-25.
14 Rojas C P, Guzmán L, Toledo N V. Weed recognition by SVM texture feature classification in outdoor vegetable crops images[J]. Ingeniería E Investigación, 2017, 37(1): 68-74.
15 Bakhshipour A, Jafari A, Nassiri S M, et al. Weed segmentation using texture features extracted from wavelet sub-images[J]. Biosystems Engineering, 2017, 157: 1-12.
16 仇裕淇, 黄振楠, 阮昭, 等. 机器视觉技术在农业生产智能化中的应用综述[J]. 机械研究与应用, 2019, 32(16): 202-206.
Qiu Yu-qi, Huang Zhen-nan, Ruan Zhao, et al. Review on application of machine vision in intelligent agricultural production[J]. Mechanical Research & Application, 2019, 32(16): 202-206.
17 Liakos K G, Busato P, Moshou D, et al. Machine learning in agriculture: a review[J]. Sensors, 2018, 18(8): 18082674.
18 Wang A, Zhang W, Wei X. A review on weed detection using ground-based machine vision and image processing techniques[J]. Computers and Electronics in Agriculture, 2019, 158: 226-240.
19 Jordan M I, Mitchell T M. Machine learning: trends, perspectives, and prospects[J]. Science, 2015, 349(6245): 255-260.
20 Gu J, Wang Z, Kuen J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377.
21 Schmidhuber J. Deep learning in neural networks: an overview[J]. Neural Networks, 2015, 61: 85-117.
22 Shi J, Li Z, Zhu T, et al. Defect detection of industry wood veneer based on NAS and multi-channel mask R-CNN[J]. Sensors, 2020, 20(16): 20164398.
23 Yu Y, Liu Y, Chen J, et al. Detection method for bolted connection looseness at small angles of timber structures based on deep learning[J]. Sensors, 2021, 21(9): 20193106.
24 孙红, 李松, 李民赞, 等. 农业信息成像感知与深度学习应用研究进展[J]. 农业机械学报, 2020, 51(5): 1-17.
Sun Hong, Li Song, Li Min-zan, et al. Research progress of image sensing and deep learning in agriculture[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(5): 1-17.
25 邓向武, 马旭, 齐龙, 等. 基于卷积神经网络与迁移学习的稻田苗期杂草识别[J]. 农机化研究, 2021, 43(10): 167-171.
Deng Xiang-wu, Ma Xu, Qi Long, et al. Recognition of weeds at seedling stage in paddy fields using convolutional neural network and transfer Learning[J]. Journal of Agricultural Mechanization Research, 2021, 43(10): 167-171.
26 Yu J, Schumann A W, Sharpe S M, et al. Detection of grassy weeds in bermudagrass with deep convolutional neural networks[J]. Weed Science, 2020, 68(5): 545-552.
27 Hasan A M, Sohel F, Diepeveen D, et al. A survey of deep learning techniques for weed detection from images[J]. Computers and Electronics in Agriculture, 2021, 184: 106067.
28 董亮, 雷良育, 李雪原, 等. 基于改进型人工神经网络的温室大棚蔬菜作物苗期杂草识别技术[J]. 北方园艺, 2017 (22): 79-82.
Dong Liang, Lei Liang-yu, Li Xue-yuan, et al. Weed identification technology of greenhouse vegetable crops in greenhouse based on improved artificial neural network[J]. Northern Horticulture, 2017 (22): 79-82.
29 孙俊, 谭文军, 武小红, 等. 多通道深度可分离卷积模型实时识别复杂背景下甜菜与杂草[J]. 农业工程学报, 2019, 35(12): 184-190.
Sun Jun, Tan Wen-jun, Wu Xiao-hong, et al. Real-time recognition of sugar beet and weeds in complex backgrounds using multi-channel depth-wise separable convolution model[J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(12): 184-190.
30 Osorio K, Puerto A, Pedraza C, et al. A deep learning approach for weed detection in lettuce crops using multispectral images[J]. AgriEngineering, 2020, 2(3): 471-488.
31 Elstone L, How K Y, Brodie S, et al. High speed crop and weed identification in lettuce fields for precision weeding[J]. Sensors, 2020, 20(2): 455-469.
32 Chen K, Wang J, Pang J, et al. MMDetection: open mmlab detection toolbox and benchmark[J]. arXiv preprint arXiv:.
33 岑海燕, 朱月明, 孙大伟, 等. 深度学习在植物表型研究中的应用现状与展望[J]. 农业工程学报, 2020, 36(9): 1-16.
Cen Hai-yan, Zhu Yue-ming, Sun Da-wei, et al. Current status and future perspective of the application of deep learning in plant phenotype research[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(9): 1-16.
34 马志艳, 朱熠, 杨磊. 基于视觉的苗期作物株间除草关键技术研究现状[J]. 中国农机化学报, 2020, 41(2): 32-38.
Ma Zhi-yan, Zhu Yi, Yang Lei. Research status of key techniques of inter-plant weeding in seedling crops based on vision[J]. Journal of Chinese Agricultural Mechanization, 2020, 41(2): 32-38.
35 Liu W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector[C]∥Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 2016:46448.
36 Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]∥Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017:318-327.
37 Tian Z, Shen C, Chen H, et al. Fcos: fully convolutional one-stage object detection[C]∥Proceedings of the Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea,2019:01355.
38 Burgos-Artizzu X P, Ribeiro A, Guijarro M, et al. Real-time image processing for crop/weed discrimination in maize fields[J]. Computers and Electronics in Agriculture, 2011, 75(2): 337-346.
[1] 王永烁,康建明,彭强吉,陈英凯,方会敏,牛萌萌,王少伟. 果树株间避障除草机设计与试验[J]. 吉林大学学报(工学版), 2023, 53(8): 2410-2420.
[2] 巫威眺,曾坤,周伟,李鹏,靳文舟. 基于多源数据和响应面优化的公交客流预测深度学习方法[J]. 吉林大学学报(工学版), 2023, 53(7): 2001-2015.
[3] 吕锋,李念,冯壮壮,张杨航. 面向用户的个性化产品服务系统协同过滤推介方法[J]. 吉林大学学报(工学版), 2023, 53(7): 1935-1942.
[4] 张振海,季坤,党建武. 基于桥梁裂缝识别模型的桥梁裂缝病害识别方法[J]. 吉林大学学报(工学版), 2023, 53(5): 1418-1426.
[5] 吴飞,农皓业,马晨浩. 基于粒子群优化算法⁃长短时记忆模型的刀具磨损预测方法[J]. 吉林大学学报(工学版), 2023, 53(4): 989-997.
[6] 何科,丁海涛,赖宣淇,许男,郭孔辉. 基于Transformer的轮式里程计误差预测模型[J]. 吉林大学学报(工学版), 2023, 53(3): 653-662.
[7] 高金武,贾志桓,王向阳,邢浩. 基于PSO-LSTM的质子交换膜燃料电池退化趋势预测[J]. 吉林大学学报(工学版), 2022, 52(9): 2192-2202.
[8] 李晓英,杨名,全睿,谭保华. 基于深度学习的不均衡文本分类方法[J]. 吉林大学学报(工学版), 2022, 52(8): 1889-1895.
[9] 申铉京,张雪峰,王玉,金玉波. 像素级卷积神经网络多聚焦图像融合算法[J]. 吉林大学学报(工学版), 2022, 52(8): 1857-1864.
[10] 胡丹,孟新. 基于时变网格的对地观测卫星搜索海上船舶方法[J]. 吉林大学学报(工学版), 2022, 52(8): 1896-1903.
[11] 王斌,何丙辉,林娜,王伟,李天阳. 基于随机森林特征选择的茶园遥感提取[J]. 吉林大学学报(工学版), 2022, 52(7): 1719-1732.
[12] 高明华,杨璨. 基于改进卷积神经网络的交通目标检测方法[J]. 吉林大学学报(工学版), 2022, 52(6): 1353-1361.
[13] 杨怀江,王二帅,隋永新,闫丰,周跃. 简化型残差结构和快速深度残差网络[J]. 吉林大学学报(工学版), 2022, 52(6): 1413-1421.
[14] 贾海洋,夏睿,吕安祺,管噌璇,陈娟,王雷. 基于模板融合的超声医学影像全景拼接方法[J]. 吉林大学学报(工学版), 2022, 52(4): 916-924.
[15] 欧阳继红,郭泽琪,刘思光. 糖尿病视网膜病变分期双分支混合注意力决策网络[J]. 吉林大学学报(工学版), 2022, 52(3): 648-656.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 《吉林大学学报(工学版)》编辑部
地址：长春市人民大街5988号　电话：+86-431-85095297　传真：+86-431-85094074　E-mail: xbgxb@jlu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发