基于多尺度级联卷积神经网络的高光谱图像分析

doi:10.13229/j.cnki.jdxbgxb.20220096

Abstract

Abstract:

In object-level hyperspectral image modeling with limited leaf samples， a multi-scale three dimensional-one dimensional cascaded convolution neural network model was proposed. Firstly， in 3D convolution neural network （3D-CNN）， the dilated convolution was embedded to increase the receptive field of convolution kernel， and a multi-scale 3D-CNN network was constructed to extract and fuse spectral-spatial joint features of different scales， to improve the model performance without increasing the number of network parameters. Then， the one-dimensional convolutional neural network （1D-CNN） was cascaded to the optimal multi-scale 3D-CNN network， to further reduce the computational complexity and overfitting degree of model. Finally， on two datasets of chlorophyll content regression for basil leaf and drought stress recognition for pepper leaf， the optimal network architecture was explored， with comparison to a series of baseline CNN models. Experimental results showed that the proposed model can effectively improve the generalization performance and reduce the computational complexity for both regression and classification tasks of leaf hyperspectral images under the condition of small samples.

Key words: agricultural electrification and automation, hyperspectral images, chemometrics, multi-scale cascaded convolutional neural network, dilated convolution, plant phenotyping

CLC Number:

Feng-le ZHU,Yi LIU,Xin QIAO,Meng-zhu HE,Zeng-wei ZHENG,Lin SUN. Analysis of hyperspectral image based on multi-scale cascaded convolutional neural network[J].Journal of Jilin University(Engineering and Technology Edition), 2023, 53(12): 3547-3557.

Figures/Tables 13

Table 1

Fig.1

Fig.2

Fig.3

Fig.4

Table 2

Table 3

Table 4

Table 5

Table 6

Fig.5

Table 7

Fig.6

References 26

1	Yu Ke-qiang, Zhao Yan-ru, Zhu Feng-le, et al. Mapping of chlorophyll and SPAD distribution in pepper leaves during leaf senescence using visible and near-infrared hyperspectral imaging[J]. Transactions of the ASABE, 2016, 59(1): 13-24.
2	Zhang Chu, Wu Wen-yan, Zhou Lei, et al. Developing deep learning based regression approaches for determination of chemical compositions in dry black goji berries (Lycium ruthenicum Murr.) using near-infrared hyperspectral imaging[J]. Food Chemistry, 2020, 319: No. 126536.
3	Hadji I, Wildes R P. What do we understand about convolutional networks?[J/OL]. [2022-01-02].
4	Zhu Su-su, Zhou Lei, Gao Pan, et al. Near-infrared hyperspectral imaging combined with deep learning to identify cotton seed varieties[J]. Molecules, 2019, 24: No. 3268.
5	Yang Wei, Yang Ce, Hao Zi-yuan, et al. Diagnosis of plant cold damage based on hyperspectral imaging and convolutional neural network[J]. IEEE Access, 2019, 7: 118239-118248.
6	Rehman T U, Ma D, Wang L, et al. Predictive spectral analysis using an end-to-end deep model from hyperspectral images for high-throughput plant phenotyping[J]. Computers and Electronics in Agriculture, 2020, 177: No. 105713.
7	He Wei-wen, He Hong-yuan, Wang Fang-lin, et al. Non-Destructive detection and recognition of pesticide residues on garlic chive (Allium Tuberosum) leaves based on short wave infrared hyperspectral imaging and one-dimensional convolutional neural network[J]. Journal of Food Measurement and Characterization, 2021, 15: 4497-4507.
8	Park K, Hong Y K, Kim G H, et al. Classification of apple leaf conditions in hyper-spectral images for diagnosis of Marssonina blotch using mRMR and deep neural network[J]. Computers and Electronics in Agriculture, 2018, 148: 179-187.
9	Zhang Jin-nuo, Yang Yong, Feng Xu-ping, et al. Identification of bacterial blight resistant rice seeds using terahertz imaging and hyperspectral imaging combined with convolutional neural network[J]. Frontiers in Plant Science, 2020, 11: No.821.
10	Polder G, Blok P M, de Villiers H A C, et al. Potato virus Y detection in seed potatoes using deep learning on hyperspectral images[J]. Frontiers in Plant Science, 2019, 10: No.209.
11	Pang Lei, Sen Men, Yan Lei, et al. Rapid vitality estimation and prediction of corn seeds based on spectra and images using deep learning and hyperspectral imaging techniques[J]. IEEE Access, 2020, 8: 123026-123036.
12	Nagasubramanian K, Jones S, Singh A K, et al. Plant disease identification using explainable 3D deep learning on hyperspectral images[J]. Plant Methods, 2019, 15: No. 98.
13	Yan Tian-ying, Xu Wei, Lin Jiao, et al. Combining multi-dimensional convolutional neural network (CNN) with visualization method for detection of Aphis gossypii glover infection in cotton leaves using hyperspectral imaging[J]. Frontiers in Plant Science, 2021, 12: No. 604510.
14	Kamilaris A, Prenafeta-Boldú F X. Deep learning in agriculture: a survey[J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
15	Hara K, Kataoka H, Satoh Y. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 6546-6555.
16	Roy S K, Krishna G, Dubey S R, et al. HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2): 277-281.
17	张祥东, 王腾军, 朱劭俊, 等. 基于扩张卷积注意力神经网络的高光谱图像分类[J]. 光学学报, 2021, 41(3): 43-53.
	Zhang Xiang-dong, Wang Teng-jun, Zhu Shao-jun, et al. Hyperspectral image classification based on dilated convolutional attention neural network[J]. Acta Optica Sinica, 2021, 41(3): 43-53.
18	李运丽. 光强对罗勒叶中花青素含量及光合特性的影响[D]. 南京: 南京农业大学园艺学院, 2011.
	Li Yun-li. Effects of light intensity on anthocyanins contents and photosynthetic characteristics in Ocimum basilicum L . leaves[D]. Nanjing: College of Horticulture, Nanjing Agricultural University, 2011.
19	张德炎. 水分胁迫对辣椒叶片生理活性的影响[J]. 湖北农业科学, 2010, 49(4): 819-821.
	Zhang De-yan. Effect of water stress on some physiology indices of pepper leaves[J]. Hubei Agricultural Sciences, 2010, 49(4): 819-821.
20	Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3D convolutional networks[C]∥Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 4489-4497.
21	Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[J/OL]. [2022-01-05].
22	He Kai-ming, Zhang Xiang-yu, Ren Shao-qing, et al. Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016: 770-778.
23	Cruz A, Ampatzidis Y, Pierro R, et al. Detection of grapevine yellows symptoms in Vitis vinifera L. with artificial intelligence[J]. Computers and Electronics in Agriculture, 2019, 157: 63-76.
24	Ollinger S V. Sources of variability in canopy reflectance and the convergent properties of plants[J]. New Phytologist, 2011, 189: 375-394.
25	蔡倩, 白伟, 郑家明, 等. 水分胁迫对春玉米光合特性及水分利用效率的影响[J]. 辽宁农业科学, 2021, 5: 1-6.
	Cai Qian, Bai Wei, Zheng Jia-ming, et al. Effect of water stress on photosynthetic characteristics and water use efficiency of spring maize[J]. Liaoning Agricultural Sciences, 2021, 5: 1-6.
26	汤东, 程平, 杨建军, 等. 天山北坡山前植物对干旱胁迫的生理响应[J]. 干旱区研究, 2021, 38(6): 1683-1694.
	Tang Dong, Cheng Ping, Yang Jian-jun, et al. Physiological responses of plants to drought stress in the Northern Piedmont, Tianshan Mountains[J]. Arid Zone Research, 2021, 38(6): 1683-1694.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

样本集	样本数	最大值	最小值	均值	标准差
训练集	380	47.00	6.80	31.39	6.97
验证集	80	46.70	8.10	31.08	6.89
测试集	80	45.50	11.10	30.24	6.94

模型	训练方式	验证集		测试集
模型	训练方式	R²	RMSE	R²	RMSE
1D-CNN	手动提取特征	0.7529	3.3902	0.7432	3.4993
2D-CNN	端到端	0.8606	2.5969	0.8415	2.6789
3D-CNN	端到端	0.9433	1.5981	0.8726	2.4204

模型	训练方式	验证集		测试集
模型	训练方式	Accuracy	F₁-score	Accuracy	F₁-score
1D-CNN	手动提取特征	0.6231	0.6411	0.6001	0.6299
2D-CNN	端到端	0.6900	0.6988	0.6617	0.6794
3D-CNN	端到端	0.7792	0.8023	0.7111	0.7457

模型	嵌入的扩张3D卷积核	验证集		测试集
模型	嵌入的扩张3D卷积核	R²	RMSE	R²	RMSE
基准3D-CNN （3×3×3，d=1）		0.9433	1.5981	0.8726	2.4204
多尺度3D-CNN	3×3×3 （d=2）	0.9501	1.4005	0.8922	2.2675
	3×3×3 （d=3）	0.9498	1.4193	0.8868	2.3239
	3×3×3 （d=4）	0.9398	1.6821	0.8768	2.4139

模型	嵌入的扩张 3D卷积核	验证集		测试集
模型	嵌入的扩张 3D卷积核	Accuracy	F₁-score	Accuracy	F₁-score
基准3D-CNN （3×3×3，d=1）		0.7792	0.8023	0.7111	0.7457
多尺度 3D-CNN	3×3×3 （d=2）	0.8031	0.8145	0.7409	0.7522
	3×3×3 （d=3）	0.7900	0.8019	0.7316	0.7389
	3×3×3 （d=4）	0.7821	0.7892	0.7198	0.7272

Analysis of hyperspectral image based on multi-scale cascaded convolutional neural network

RICH HTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 13

References 26

Related Articles 2

Metrics

Comments

Recommended 0

不同转换位点的多尺度3D-1D-CNN模型	训练时长（s·epoch^-1）	验证集		测试集
不同转换位点的多尺度3D-1D-CNN模型	训练时长（s·epoch^-1）	R²	RMSE	R²	RMSE
①（即基准 1D-CNN）	9	0.7529	3.3902	0.7432	3.4993
②	42	0.7783	3.2274	0.7699	3.2999
③	43	0.8014	3.0190	0.7935	3.1171
④	45	0.8188	2.9661	0.8156	2.9744
⑤	47	0.8722	2.4213	0.8563	2.7080
⑥	50	0.9045	2.1165	0.8849	2.3793
⑦	55	0.9316	1.8211	0.9101	2.0468
⑧	63	0.9385	1.7036	0.9031	2.1150
⑨	78	0.9502	1.4113	0.8998	2.1987
⑩（即最佳多尺度3D-CNN）	80	0.9501	1.4005	0.8922	2.2675

[1]	Lin MAO,Feng-zhi REN,Da-wei YANG,Ru-bo ZHANG. Two⁃way feature pyramid network for panoptic segmentation [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(3): 657-665.
[2]	WANG Ke-yan, LI Yun-song, SONG Juan, LIAO Hui-lin, WU Xian-yun. Spatial-spectral lossless compression of hyperspectral images using local edge based prediction [J]. 吉林大学学报(工学版), 2017, 47(2): 677-685.

不同转换位点的多尺度3D-1D-CNN模型	训练时长（s·epoch^-1）	验证集		测试集
不同转换位点的多尺度3D-1D-CNN模型	训练时长（s·epoch^-1）	Accuracy	F₁-score	Accuracy	F₁-score
①（即基准 1D-CNN）	10	0.6231	0.6411	0.6001	0.6299
②	39	0.6337	0.6344	0.6297	0.6123
③	42	0.6552	0.6781	0.6535	0.6588
④	43	0.6885	0.6957	0.6781	0.6973
⑤	47	0.7537	0.7423	0.7324	0.7232
⑥	51	0.7699	0.7771	0.7480	0.7569
⑦	56	0.7916	0.8002	0.7715	0.7791
⑧	63	0.7926	0.7855	0.7689	0.7542
⑨	81	0.7995	0.7912	0.7534	0.7628
⑩（即最佳多尺度3D-CNN）	82	0.8031	0.8145	0.7409	0.7522