Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (5): 1465-1473.doi: 10.13229/j.cnki.jdxbgxb.20230274

Previous Articles    

Robust discrimination method for tea varieties based on spectral transformation and high⁃order Sparsity⁃aided Hodrick⁃Prescott decomposition

Xiu-zhi ZHAO1,2(),Jing-ming NING3,De-hong XIE4()   

  1. 1.College of Artificial Intelligence，Zhejiang Industry & Trade Vocational College，Wenzhou 325002，China
    2.School of Computer and Artificial Intelligence，Wenzhou University，Wenzhou 325002，China
    3.State Key Laboratory of Tea Plant Biology and Utilization，Anhui Agricultural University，Hefei 230036，China
    4.College of Information Science and Technology，Nanjing Forestry University，Nanjing 210037，China
  • Received:2023-03-27 Online:2023-05-01 Published:2023-05-25
  • Contact: De-hong XIE E-mail:cassyzxz@126.com;dehong.xie@gmail.com

RICH HTML

 1   

PDF (PC)

48

Abstract:

In order to improve the accuracy of qualitative analysis of visible near infrared spectroscopy， noise reduction pretreatment is needed. Aiming at the problem that it is easy to produce additional small spectral peaks which deteriorate the accuracy of qualitative analysis during noise reduction， a noise reduction method based on spectral transformation and high-order sparse Hodrick-Prescott decomposition is proposed. In the optimization model of this method， it is assumed that the visible near infrared spectroscopy is composed of low-pass basic waveform spectrum， band-pass characteristic waveform spectrum and noise. In this method， the L2 norm of the residual between the noisy spectroscopy， the basic waveform spectroscopy and the band-pass characteristic waveform spectroscopy is taken as the residual term to ensure that the estimated value is close to the real value. According to the sparsity of the characteristic waveform spectrum， taking the L1 norm of the second-order difference of it as the regularization term， it is constrained to estimate that the characteristic waveform spectrum has a certain sparsity to characterize absorption peaks of the important chemical components in tea. In this method， the low-pass and band-pass zero phase filter matrices are obtained by using the spectral transformation of the filter to help decompose the basic waveform spectroscopy and the characteristic waveform spectroscopy， and the optimal regularization parameters in the optimization equation are obtained by using the L-curve method. The experiment takes the visible near infrared spectroscopy of six kinds of tea as the basic experimental data. In the experiment， our method compares with the wavelet decomposition method， improved Hodrick-Prescott method and savitzky-Golay method by the signal-to-noise ratio，root mean square difference， and the accuracy of the classification model of qualitative analysis of tea varieties as the measures. The experimental results show that our method has the highest signal-to-noise ratio in synthetic spectra containing Gaussian noise and synthetic spectra containing Gaussian pulse mixed noise. For synthetic spectral data sets and real spectral data sets， the test accuracy of the classification model is higher than that of the above three methods， and much higher than the results obtained using their noisy spectral data sets. Therefore， this method has been proved to have advantages in noise reduction of visible near infrared spectroscopy， and can be applied to the pretreatment of the qualitative detection of tea varieties based on visible near infrared spectroscopy.

Key words: visible near infrared spectroscopy, tea, variety, noise reduction, sparsity

CLC Number: 

  • O657.3

Fig.1

Image plane corresponding to six different tea samples （from left to right： oolong， black， green， yellow， Pu'er and white）"

Fig.2

VNIR spectroscopy with simulated Gaussian noises and its denoised VNIR spectroscopy"

Fig.3

Denoised VNIR spectroscopy with simulated mixed noises and its denoised VNIR spectroscopy"

Table 1

Statistical value of signal-to-noise ratio and root mean square error between before and after noise reduction of VNIR spectral data"

指标VNIR数据集DWTMHPS-G本文方法
AveStdAveStdAveStdAveStd
SNR （dB）高斯噪声24.424.4933.203.1132.883.4236.252.04
混合噪声24.315.4130.404.4831.674.7235.123.02
RMSE高斯噪声0.441 40.120 50.168 60.057 00.175 90.063 80.066 10.020 2
混合噪声0.423 00.294 20.192 20.111 640.188 50.121 60.070 70.031 9

Table 2

Test accuracy of tea varieties identification model before and after noise reduction of VNIR spectral data and trained by SVM"

VNIR数据集方 法
含噪DWTMHPS-G本文方法
高斯噪声（合成）61.4268.6678.4875.1287.73
混合噪声（合成）57.5664.3277.5676.3887.95
含噪（真实）51.2365.5675.9677.1683.54

Table 3

Test accuracy of tea varieties identification model before and after noise reduction of VNIR spectral data and trained by KNN"

VNIR数据集方 法
含噪DWTMHPS-G本方法
高斯噪声（合成）54.2261.5977.1274.1588.09
混合噪声（合成）51.7557.4876.5475.9288.40
含噪（真实）48.7959.1674.8776.2485.43
1 Ren G, Liu Y, Ning J, et al. Assessing black tea quality based on visible-near infrared spectra and kernel-based methods[J]. Journal of Food Composition and Analysis, 2021, 98: 103810.
2 Liu Y, Peng Q W, Yu J C, et al. Identification of tea based on CARS-SWR variable optimization of visible/near-infrared spectrum[J]. Journal of the Science of Food and Agriculture, 2020, 100(1): 371-375.
3 Liu T, Liu H, Li Y F, et al. Flexible FTIR spectral imaging enhancement for industrial robot infrared vision sensing[J]. IEEE Transactions on Industrial Informatics, 2020, 16(1): 544-554.
4 Xie B, Xiong Z, Wang Z, et al. Gamma spectrum denoising method based on improved wavelet threshold[J]. Nuclear Engineering and Technology, 2020, 52(8): 1771-1776.
5 Xi Y, Li Y, Duan Z, et al. A novel pre-processing algorithm based on the wavelet transform for raman spectrum[J]. Applied Spectroscopy, 2018, 72(12): 1752-1763.
6 沈毅, 张敏, 张淼. 基于小波阈值降噪和经验模态分解的高光谱图像分类算法[J]. 宇航学报, 2012, 33(4): 471-477.
Shen Yi, Zhang Min, Zhang Miao. Hyperspectral image classification algorithm based on wavelet threshold noise reduction and empirical mode decomposition[J]. Journal of Astronautics, 2012, 33(4): 471-477.
7 周风波, 李长庚, 朱红求. 基于提升小波变换的阈值改进去噪算法在紫外可见光谱中的研究[J]. 光谱学与光谱分析, 2018, 38(2): 506-510.
Zhou Feng-bo, Li Chang-geng, Zhu Hong-qiu. Research on improved denoising algorithm in UV-VIS spectrum based on threshold boosted wavelet transform[J]. Spectroscopy and Spectral Analysis, 2018, 38(2): 506-510.
8 Zheng H M, Dang C L, Gu S M, et al. A quantified self-adaptive filtering method: effective IMFs selection based on CEEMD[J]. Measurement Science and Technology, 2018, 29(8): 085701.
9 Huang H, Hu S, Sun Y A. A discrete curvature estimation based low-distortion adaptive savitzky-golay filter for ECG denoising[J]. Sensors(Basel), 2019, 19(7): 1424-1617.
10 谢德红, 李俊锋, 刘菂, 等. 基于改进Hodrick-Prescott分解模型的近红外自适应降噪方法[J]. 光谱学与光谱分析, 2020, 40(5): 1650-1655.
Xie De-hong, Li Jun-feng, Liu Jing, et al. Near-infrared adaptive noise reduction method based on improved Hodrick-Prescott decomposition model[J]. Spectroscopy and Spectral Analysis, 2020, 40(5): 1650-1655.
11 Yao Z, Su H, Yao J, et al. Yield-adjusted operation for convolution filter denoising[J]. Analytical Chemistry, 2021, 93(49): 16489-16503.
12 Czarnecki M A. Resolution enhancement in second-derivative spectra[J]. Applied Spectroscopy, 2015, 69(1): 67-74.
13 Wu Z, Huang N E. Ensemble empirical mode decomposition: a noise-assisted data analysis method[J]. Advances in Adaptive Data Analysis, 2009, 1(1): 1-41.
14 Torres M E, Colominas M A, Schlotthauer G, et al. A complete ensemble empirical mode decomposition with adaptive noise[C]∥IEEE International Conference on Acoustics, Speech and Signal Processing,Prague, Czech, 2011: 4144-4147.
15 Ouahilal M, Mohajir M E, Chahhou M, et al. A novel hybrid model based on Hodrick-Prescott filter and support vector regression algorithm for optimizing stock market price prediction[J]. Journal of Big Data, 2017, 4(1): 40357.
16 Gustafsson F. Determining the initial states in forward-backward filtering[J]. IEEE Transactions on Signal Processing, 1996, 44(4): 988-992.
17 Soares S F C, Gomes A A, Araujo M C U, et al. The successive projections algorithm[J]. Trac Trends in Analytical Chemistry, 2013, 42: 84-98.
18 Bogawar P S, Bhoyar K K. An improved multiclass support vector machine classifier using reduced hyper-plane with skewed binary tree[J]. Appled Intellence, 2018, 48(11): 4382-4391.
19 周志华. 机器学习[M]. 北京: 清华大学出版社, 2016.
20 Zhang L Z, Dai H M, Zhang J L, et al. A study on origin traceability of white tea(white peony) based on near-infrared spectroscopy and machine learning algorithms[J]. Foods, 2023, 12(3): 499-523.
21 He Y C, Jiang H, Chen Q S. High-precision identification of the actual storage periods of edible oil by FT-NIR spectroscopy combined with chemometric methods[J]. Analytical Methods, 2020, 12(29): 3722-3728.
22 Mishra P, Karami A, Nordon A, et al. Automatic de-noising of close-range hyperspectral images with a wavelength-specific shearlet-based image noise reduction method[J]. Sensors and Actuators B: Chemical, 2019, 281: 1034-1044.
[1] Bin WANG,Bing-hui HE,Na LIN,Wei WANG,Tian-yang LI. Tea plantation remote sensing extraction based on random forest feature selection [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1719-1732.
[2] Wen-jun WANG,Yin-feng YU. Automatic completion algorithm for missing links in nowledge graph considering data sparsity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1428-1433.
[3] Zhou-zhou LIU,Qian-yun ZHANG,Xin-hua MA,Han PENG. Compressed sensing signal reconstruction based on optimized discrete differential evolution algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2246-2252.
[4] Meng-su ZHANG,Chun-tian LIU,Xi-jin LI,Yong-ping HUANG. Design of fuzzy comprehensive evaluation system for performance appraisal based on K⁃means clustering algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1851-1856.
[5] Jian YANG,Qi XIA,Hai-chao ZHOU,Guo-lin WANG. Noise reduction mechanism of truck radial tire based on modified carcass string contour design [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1198-1203.
[6] Lu-shen WU,Wei CHENG,Yun HU. Image segmentation of multilevel threshold based on improved cuckoo search algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 358-369.
[7] Yi-ming BIE,Ru-ru TANG,Yun-hao WANG,Bin WEN,Tian-jun FENG,Lin-hong WANG. Saturation flow rate estimation method of approaching lanes at signalized intersections [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1459-1464.
[8] Yin⁃juan FU,Yong LI,Li⁃qin XU,Kun⁃hui ZHANG. Design and analysis of NLFM⁃Costas RF stealth radar signal [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 994-999.
[9] SHAN Ze-biao,LIU Xiao-song,SHI Hong-wei,WANG Chun-yang,SHI Yao-wu. DOA tracking algorithm using dynamic compressed sensing [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1938-1944.
[10] XI Lei, XU Liang, GAO Jian-min, ZHAO Zhen, WANG Ming-sen. Flow and heat transfer characteristics of steam in thick-wall rectangular ribbed channel [J]. 吉林大学学报(工学版), 2018, 48(3): 752-759.
[11] QIAN Zhi-hui, ZHOU Liang, REN Lei, REN Lu-quan. Completely passive walking machine with bionic subtalar joint and matatarsal phalangeal joint [J]. 吉林大学学报(工学版), 2018, 48(1): 205-211.
[12] CUI Wen-shi, YANG Zhi-gang, WANG Guo-jun, ZHOU Hua. Unsteady flow analysis for wakes of three-dimensional vehicles with different rear slant angles [J]. 吉林大学学报(工学版), 2017, 47(3): 717-724.
[13] GE Chang-jiang, YE Hui, HU Xing-jun, YU Zheng-lei. Prediction and control of trailing edge noise on owl wings [J]. 吉林大学学报(工学版), 2016, 46(6): 1981-1986.
[14] XIAO Zhi-feng, LE Jian-bo, WU Nan-xing, LIU Xiang-dong. Effect of operation pressure on superheated steam fluidized bed drying [J]. 吉林大学学报(工学版), 2015, 45(4): 1375-1380.
[15] SI Wei, MA Biao, WANG Hai-nian, HU Jian. Flexural tensile characteristics of asphalt mixture under freeze-thaw cycles [J]. 吉林大学学报(工学版), 2013, 43(04): 885-890.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd