基于深度学习与图像处理的蔬菜苗期杂草识别方法

doi:10.13229/j.cnki.jdxbgxb.20211070

Abstract

Abstract:

In this study， the recognition test of bok choy and its associated weeds at the seedling stage was carried out， and a novel method based on recognizing vegetables and then indirectly recognizing weeds was proposed. By combining deep learning and image processing technology， this method can effectively reduce the complexity of weed recognition， and at the same time improving the accuracy and robustness of weed recognition. First， a neural network model was used for detecting the bok choy and drawing bounding boxes. The green targets outside the bok choy bounding boxes were marked as weeds， and color features were used to segment them. Besides， an area filter was used for eliminating the noises and extracting weed regions. In order to explore the effects of different deep learning models on bok choy recognition， SSD model， RetinaNet model and FCOS model were selected， and three evaluation metrics of F₁ value， average accuracy and detection speed were used for comparative analysis. The SSD model was the best model for bok choy recognition， with the highest detection speed and excellent recognition rate. Its F₁ value， average accuracy and detection speed in the test set were 95.4%， 98.1% and 31.0 f/s， respectively. The improved MExG index can effectively recognize weeds， and the segmented weeds have complete shapes and clear outlines. Experiment results show that the proposed method for recognizing weeds in vegetable fields is highly feasible and has excellent application effects， which can also provide technical reference for weed recognition in similar crop fields.

Key words: agricultural engineering, vegetable recognition, weed recognition, deep learning, image processing, color feature

CLC Number:

TP391.41

Xiao-jun JIN,Yan-xia SUN,Jia-lin YU,Yong CHEN. Weed recognition in vegetable at seedling stage based on deep learning and image processing[J].Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2421-2429.

Figures/Tables 8

Fig.1

Table 1

Table 2

Evaluation matrix with different confidence score in val dataset"

模型	置信度	精度	召回率	$F 1$ 值
SSD	0.9	0.980	0.84	0.904
	0.8	0.971	0.89	0.929
	0.7	0.966	0.91	0.937
	0.6	0.964	0.94	0.952
	0.5	0.960	0.95	0.955
	0.4	0.959	0.96	0.960
	0.3	0.941	0.97	0.955
	0.2	0.922	0.98	0.950
	0.1	0.895	0.99	0.940
RetinaNet	0.9	0.967	0.95	0.958
	0.8	0.958	0.96	0.959
	0.7	0.958	0.96	0.959
	0.6	0.940	0.97	0.955
	0.5	0.924	0.98	0.951
	0.4	0.924	0.98	0.951
	0.3	0.924	0.98	0.951
	0.2	0.887	0.99	0.936
	0.1	0.887	0.99	0.936
FCOS	0.9	0.000	0.00	0.000
	0.8	1.000	0.06	0.113
	0.7	1.000	0.28	0.438
	0.6	0.998	0.51	0.675
	0.5	0.991	0.74	0.847
	0.4	0.982	0.87	0.922
	0.3	0.970	0.93	0.949
	0.2	0.911	0.96	0.935
	0.1	0.850	0.97	0.906

Table 2

Table 3

Evaluation matrix with best confidence score in test dataset"

模型	置信度	精度	召回率	$F 1$ 值	平均精度	检测速度/（f·s^-1）
SSD	0.4	0.938	0.97	0.954	0.981	31.0
RetinaNet	0.7/0.8	0.943/0.947	0.98/0.97	0.961/0.959	0.984	17.5
FCOS	0.3	0.970	0.94	0.955	0.966	17.9

Table 3

Fig.2

Fig.3

Fig.4

Fig.5

References 38

1	金月, 肖宏儒, 曹光乔, 等. 我国叶类蔬菜机械化水平现状与评价方法研究[J]. 中国农机化学报, 41(12): 196-201.
	Jin Yue, Xiao Hong-ru, Cao Guang-qiao, et al. Research on status and evaluation methods of leafy vegetable mechanization production level in China[J]. Journal of Chinese Agricultural Mechanization, 41(12): 196-201.
2	Ryder E. World vegetable industry: production, breeding, trends[J]. Horticultural Reviews, 2011, 38: 299-356.
3	Han J, Luo Y, Yang L, et al. Acidification and salinization of soils with different initial pH under greenhouse vegetable cultivation[J]. Journal of Soils and Sediments, 2014, 14(10): 1683-1692.
4	陆海涛, 吕建强, 金伟, 等. 我国叶类蔬菜机械化收获技术的发展现状[J]. 农机化研究, 2018, 40(6): 261-268.
	Lu Hai-tao, Lv Jian-qiang, Jin Wei, et al. The current situation of the mechanized harvesting technology development of leaf vegetable in China[J]. Journal of Agricultural Mechanization Research, 2018, 40(6): 261-268.
5	徐艳蕾, 何润, 翟钰婷, 等. 基于轻量卷积网络的田间自然环境杂草识别方法[J]. 吉林大学学报:工学版, 2021, 51(6): 2304-2312.
	Xu Yan-lei, He Run, Zhai Yu-ting, et al. Weed identification method based on deep transfer learning in field natural environment[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(6): 2304-2312.
6	王刚, 刘慧力, 贾洪雷, 等. 触碰定位式玉米行间除草装置的设计与试验[J]. 吉林大学学报:工学版, 2021, 51(4): 1518-1527.
	Wang Gang, Liu Hui-li, Jia Hong-lei, et al. Design and experiment of touching-positioning weeding device for inter-row maize (Zea Mays L.)[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(4): 1518-1527.
7	洪晓玮, 陈勇, 杨超淞, 等. 有机蔬菜大棚除草机器人研制[J]. 制造业自动化, 2021, 43(5): 33-36.
	Hong Xiao-wei, Chen Yong, Yang Chao-song, et al. Development of a weeding robot for organic vegetable greenhouse[J]. Manufacturing Automation, 2021, 43(5): 33-36.
8	Dai X, Xu Y, Zheng J, et al. Comparison of image-based methods for determining the inline mixing uniformity of pesticides in direct nozzle injection systems[J]. Biosystems Engineering, 2020, 190: 157-175.
9	Lanini W, Strange M. Low-input management of weeds in vegetable fields[J]. California Agriculture, 1991, 45(1): 11-13.
10	金小俊, 陈勇, 孙艳霞. 农田杂草识别方法研究进展[J]. 农机化研究, 2011, 33(7): 23-27, 33.
	Jin Xiao-jun, Chen Yong, Sun Yan-xia. Research advances of weed identification in agricultural fields[J]. Journal of Agricultural Mechanization Research, 2011, 33(7): 23-27, 33.
11	毛文华, 张银桥, 王辉, 等. 杂草信息实时获取技术与设备研究进展[J]. 农业机械学报, 2013, 44(1): 190-195.
	Mao Wen-hua, Zhang Yin-qiao, Wang Hui, et al. Advance techniques and equipments for real-time weed detection[J]. Transactions of the Chinese Society for Agricultural Machinery, 2013, 44(1): 190-195.
12	金小俊, 陈勇, 侯学贵, 等. 基于机器视觉的除草机器人杂草识别[J]. 山东科技大学学报:自然科学版, 2012, 31(2): 104-108.
	Jin Xiao-jun, Chen Yong, Hou Xue-gui, et al. Weed recognition of the machine vision based weeding robot[J]. Journal of Shandong University of Science and Technology (Natural Science), 2012, 31(2): 104-108.
13	陈良宵, 王斌. 基于形状特征的叶片图像识别算法比较研究[J]. 计算机工程与应用, 2017 (9): 17-25.
	Chen Liang-xiao, Wang Bin. Comparative study of leaf image recognition algorithm based on shape feature[J]. Computer Engineering and Applications, 2017 (9): 17-25.
14	Rojas C P, Guzmán L, Toledo N V. Weed recognition by SVM texture feature classification in outdoor vegetable crops images[J]. Ingeniería E Investigación, 2017, 37(1): 68-74.
15	Bakhshipour A, Jafari A, Nassiri S M, et al. Weed segmentation using texture features extracted from wavelet sub-images[J]. Biosystems Engineering, 2017, 157: 1-12.
16	仇裕淇, 黄振楠, 阮昭, 等. 机器视觉技术在农业生产智能化中的应用综述[J]. 机械研究与应用, 2019, 32(16): 202-206.
	Qiu Yu-qi, Huang Zhen-nan, Ruan Zhao, et al. Review on application of machine vision in intelligent agricultural production[J]. Mechanical Research & Application, 2019, 32(16): 202-206.
17	Liakos K G, Busato P, Moshou D, et al. Machine learning in agriculture: a review[J]. Sensors, 2018, 18(8): 18082674.
18	Wang A, Zhang W, Wei X. A review on weed detection using ground-based machine vision and image processing techniques[J]. Computers and Electronics in Agriculture, 2019, 158: 226-240.
19	Jordan M I, Mitchell T M. Machine learning: trends, perspectives, and prospects[J]. Science, 2015, 349(6245): 255-260.
20	Gu J, Wang Z, Kuen J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377.
21	Schmidhuber J. Deep learning in neural networks: an overview[J]. Neural Networks, 2015, 61: 85-117.
22	Shi J, Li Z, Zhu T, et al. Defect detection of industry wood veneer based on NAS and multi-channel mask R-CNN[J]. Sensors, 2020, 20(16): 20164398.
23	Yu Y, Liu Y, Chen J, et al. Detection method for bolted connection looseness at small angles of timber structures based on deep learning[J]. Sensors, 2021, 21(9): 20193106.
24	孙红, 李松, 李民赞, 等. 农业信息成像感知与深度学习应用研究进展[J]. 农业机械学报, 2020, 51(5): 1-17.
	Sun Hong, Li Song, Li Min-zan, et al. Research progress of image sensing and deep learning in agriculture[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(5): 1-17.
25	邓向武, 马旭, 齐龙, 等. 基于卷积神经网络与迁移学习的稻田苗期杂草识别[J]. 农机化研究, 2021, 43(10): 167-171.
	Deng Xiang-wu, Ma Xu, Qi Long, et al. Recognition of weeds at seedling stage in paddy fields using convolutional neural network and transfer Learning[J]. Journal of Agricultural Mechanization Research, 2021, 43(10): 167-171.
26	Yu J, Schumann A W, Sharpe S M, et al. Detection of grassy weeds in bermudagrass with deep convolutional neural networks[J]. Weed Science, 2020, 68(5): 545-552.
27	Hasan A M, Sohel F, Diepeveen D, et al. A survey of deep learning techniques for weed detection from images[J]. Computers and Electronics in Agriculture, 2021, 184: 106067.
28	董亮, 雷良育, 李雪原, 等. 基于改进型人工神经网络的温室大棚蔬菜作物苗期杂草识别技术[J]. 北方园艺, 2017 (22): 79-82.
	Dong Liang, Lei Liang-yu, Li Xue-yuan, et al. Weed identification technology of greenhouse vegetable crops in greenhouse based on improved artificial neural network[J]. Northern Horticulture, 2017 (22): 79-82.
29	孙俊, 谭文军, 武小红, 等. 多通道深度可分离卷积模型实时识别复杂背景下甜菜与杂草[J]. 农业工程学报, 2019, 35(12): 184-190.
	Sun Jun, Tan Wen-jun, Wu Xiao-hong, et al. Real-time recognition of sugar beet and weeds in complex backgrounds using multi-channel depth-wise separable convolution model[J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(12): 184-190.
30	Osorio K, Puerto A, Pedraza C, et al. A deep learning approach for weed detection in lettuce crops using multispectral images[J]. AgriEngineering, 2020, 2(3): 471-488.
31	Elstone L, How K Y, Brodie S, et al. High speed crop and weed identification in lettuce fields for precision weeding[J]. Sensors, 2020, 20(2): 455-469.
32	Chen K, Wang J, Pang J, et al. MMDetection: open mmlab detection toolbox and benchmark[J]. arXiv preprint arXiv:.
33	岑海燕, 朱月明, 孙大伟, 等. 深度学习在植物表型研究中的应用现状与展望[J]. 农业工程学报, 2020, 36(9): 1-16.
	Cen Hai-yan, Zhu Yue-ming, Sun Da-wei, et al. Current status and future perspective of the application of deep learning in plant phenotype research[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(9): 1-16.
34	马志艳, 朱熠, 杨磊. 基于视觉的苗期作物株间除草关键技术研究现状[J]. 中国农机化学报, 2020, 41(2): 32-38.
	Ma Zhi-yan, Zhu Yi, Yang Lei. Research status of key techniques of inter-plant weeding in seedling crops based on vision[J]. Journal of Chinese Agricultural Mechanization, 2020, 41(2): 32-38.
35	Liu W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector[C]∥Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 2016:46448.
36	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]∥Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017:318-327.
37	Tian Z, Shen C, Chen H, et al. Fcos: fully convolutional one-stage object detection[C]∥Proceedings of the Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea,2019:01355.
38	Burgos-Artizzu X P, Ribeiro A, Guijarro M, et al. Real-time image processing for crop/weed discrimination in maize fields[J]. Computers and Electronics in Agriculture, 2011, 75(2): 337-346.

Related Articles 15

[1]	Yong-shuo WANG,Jian-ming KANG,Qiang-ji PENG,Ying-kai CHEN,Hui-min FANG,Meng-meng NIU,Shao-wei WANG. Design and experiment of obstacle avoidance weeding machine for fruit trees [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2410-2420.
[2]	Wei-tiao WU,Kun ZENG,Wei ZHOU,Peng LI,Wen-zhou JIN. Deep learning method for bus passenger flow prediction based on multi-source data and surrogate-based optimization [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2001-2015.
[3]	Feng LYU,Nian LI,Zhuang-zhuang FENG,Yang-hang ZHANG. Method of collaborative filtering recommendation of personalized product-service system based on user [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 1935-1942.
[4]	Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
[5]	Fei WU,Hao-ye NONG,Chen-hao MA. Tool wear prediction method based on particle swarm optimizationlong and short time memory model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 989-997.
[6]	Wen-li JI,Zhong TIAN,Jing CHAI,Ding-ding ZHANG,Bin WANG. Prediction of water⁃flowing height in fractured zone based on distributed optical fiber and multi⁃attribute fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1200-1210.
[7]	Ke HE,Hai-tao DING,Xuan-qi LAI,Nan XU,Kong-hui GUO. Wheel odometry error prediction model based on transformer [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 653-662.
[8]	Jin-wu GAO,Zhi-huan JIA,Xiang-yang WANG,Hao XING. Degradation trend prediction of proton exchange membrane fuel cell based on PSO⁃LSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(9): 2192-2202.
[9]	Xiao-ying LI,Ming YANG,Rui QUAN,Bao-hua TAN. Unbalanced text classification method based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1889-1895.
[10]	Xuan-jing SHEN,Xue-feng ZHANG,Yu WANG,Yu-bo JIN. Multi⁃focus image fusion algorithm based on pixel⁃level convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1857-1864.
[11]	Dan HU,Xin MENG. Vessel search method by earth observation satellite based on time⁃varying grid [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1896-1903.
[12]	Bin WANG,Bing-hui HE,Na LIN,Wei WANG,Tian-yang LI. Tea plantation remote sensing extraction based on random forest feature selection [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1719-1732.
[13]	Ming-hua GAO,Can YANG. Traffic target detection method based on improved convolution neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1353-1361.
[14]	Huai-jiang YANG,Er-shuai WANG,Yong-xin SUI,Feng YAN,Yue ZHOU. Simplified residual structure and fast deep residual networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1413-1421.
[15]	Hai-yang JIA,Rui XIA,An-qi LYU,Ceng-xuan GUAN,Juan CHEN,Lei WANG. Panoramic mosaic approach of ultrasound medical images based on template fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 916-924.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Weed recognition in vegetable at seedling stage based on deep learning and image processing

RICH HTML

PDF (PC)