Journal of Jilin University(Engineering and Technology Edition) ›› 2025, Vol. 55 ›› Issue (4): 1374-1383.doi: 10.13229/j.cnki.jdxbgxb.20230690

Previous Articles     Next Articles

EMD-LSTM-based group prediction algorithm of container resource load in preprocessing molecular spectral line data

Xin-chen YE1,2,3(),Hai-long ZHANG1,2,3,4(),Jie WANG1,3,Da-lei LI1,Meng ZHANG5,Ya-zhou ZHANG1,2,Xu DU1,2,Jia LI1,Wan-qiong WANG1   

  1. 1.Xinjiang Astronomical Observatory，Chinese Academy of Sciences，Urumqi 830011，China
    2.University of Chinese Academy of Sciences，Beijing 100049，China
    3.National Astronomical Data Center，Beijing 100101，China
    4.Key Laboratory of Radio Astronomy，Chinese Academy of Sciences，Nanjing 210008，China
    5.School of Cyber Science and Engineering，Qufu Normal University，Qufu 273165，China
  • Received:2023-07-03 Online:2025-04-01 Published:2025-06-19
  • Contact: Hai-long ZHANG E-mail:yexinchen@xao.ac.cn;zhanghailong@xao.ac.cn

RICH HTML

 1   

PDF (PC)

219

Abstract:

Unequal allocation of container resources in a cluster environment is currently a pressing issue. In response to container load prediction and resource allocation strategies， this paper proposes an empirical mode decomposition-long short-term memory （EMD-LSTM）-based algorithm for predicting container resource load groups in the preprocessing of astronomical data. An adaptive recommendation value generation algorithm based on load prediction information is introduced， which automatically allocates container computing resources according to the degree of load fluctuation. The accuracy of load prediction was verified using simulated data and real astronomical observation data. Experimental results demonstrate that the proposed algorithm outperforms the triple exponential smoothing method and a single LSTM network model in terms of prediction accuracy. In real-time preprocessing of astronomical data， compared to the default strategy， the recommended value generation algorithm proposed in this paper effectively improves the utilization efficiency of computing resources.

Key words: astroinformatics, container, load prediction, long short-term memory, load balancing

CLC Number: 

  • TP301.6

Fig.1

VPA architecture diagram"

Fig.2

LSTM unit"

Fig.3

EMD-LSTM group prediction algorithm architecture"

Table 1

Python library information"

编号库名称功能
1Numpy数据处理
2PyEMDEMD分解
3Keras构建LSTM预测模块

Fig.4

Revised VPA architecture diagram"

Fig.5

Comparison of predicted simulated load data"

Table 2

Correlation coefficients between predicted information and original data"

编号方法相关系数
1三重指数平滑0.988 59
2LSTM0.992 00
3基于LSTM的分组0.997 34

Table 3

Node hardware configuration"

名 称信息
中央处理器Intel i9-10 900 K
内存64 GB
软件TEMPO2 2020.04.1
OSDebian 10.10.0
网络接口10 GbE
硬盘1 TB NVMe SSD

Table 4

NSRT hydrogen molecule sky survey data information"

参数值
望远镜NSRT
文件格式Fits
VEGAS终端模式MODE9
基带带宽450 MHz
量化精度8 bits

Fig.6

Comparison of predicted CPU load data"

Fig.7

Comparison of predicted memory load data"

Table 5

Correlation coefficients between CPU load predicted information and actual load"

编号方法相关系数
1二重指数平滑0.819 87
2LSTM0.838 94
3基于EMD-LSTM的分组0.850 09

Table 6

Correlation coefficients between memory load predicted information and actual load"

编号方法相关系数
1二重指数平滑0.604 30
2LSTM0.654 67
3基于EMD-LSTM的分组0.721 10

Fig.8

Comparison of CPU resource scaling"

Fig.9

Comparison of memory resource scaling"

[1] 陈红松, 陈京九. 基于统计的物联网分布式拒绝服务攻击检测[J]. 吉林大学学报: 工学版, 2020, 50(5): 1894-1904.
Chen Hong-song, Chen Jing-jiu. Statistical based distributed denial of service attack detection research in internet of things[J]. Journal of Jilin University (Engineering and Technology Edition), 2020, 50(5): 1894-1904.
[2] Morris D, Voutsinas S, Hambly N C, et al. Use of Docker for deployment and testing of astronomy software[J]. Astronomy and Computing, 2017, 20: 105-119.
[3] Molenaar G, Makhathini S, Girard J N, et al. Kliko—the scientific compute container format[J]. Astronomy and Computing, 2018, 25: 1-9.
[4] Herwig F, Andrassy R, Annau N, et al. Cyberhubs: virtual research environments for astronomy[J]. The Astrophysical Journal Supplement Series, 2018, 236(1): 2.
[5] Truyen E, van Landuyt D, Preuveneers D, et al. A comprehensive feature comparison study of open-source container orchestration frameworks[J]. Applied Sciences, 2019, 9(5): 931.
[6] Niu J R, Zhu W W, Zhang B, et al. FAST observations of an extremely active episode of FRB 20201124A. Ⅳ. Spin Period Search[J]. Research in Astronomy and Astrophysics, 2022, 22(12): No.124004.
[7] Wang Y B, Wen Z G, Yuen R, et al. The multiple images of the plasma lensing FRB[J]. Research in Astronomy and Astrophysics, 2022, 22(6): No. 065017.
[8] Chen Z H, You S P, Yu X H, et al. An RFI mitigation pipeline for CRAFTS multi-beam data based on signal cross-correlation function and sum threshold algorithm[J]. Research in Astronomy and Astrophysics, 2023, 23(5): No.055014.
[9] Dai V D, Kim Y H. Predictive approach for vertical autoscaling in Kubernetes[J]. Proceedings of the Korean Society of Telecommunications, Korean,2021: 896-897.
[10] Xie Y, Jin M, Zou Z, et al. Real-time prediction of docker container resource load based on a hybrid model of ARIMA and triple exponential smoothing[J]. IEEE Transactions on Cloud Computing, 2020, 10(2): 1386-1401.
[11] Rzadca K, Findeisen P, Swiderski J, et al. Autopilot: workload autoscaling at google[C]∥Proceedings of the Fifteenth European Conference on Computer Systems, Heraklion, Greece, 2020: 1-16.
[12] Shanmugam A S. Docker container reactive scalability and prediction of CPU utilization based on proactive modelling[D]. Dublin: National College of Ireland, 2017.
[13] Bandara K, Bergmeir C, Smyl S. Forecasting across time series databases using recurrent neural networks on groups of similar series: a clustering approach[J]. Expert Systems with Applications, 2020, 140: No.112896.
[14] 冯金巧, 杨兆升, 张林, 等. 一种自适应指数平滑动态预测模型[J]. 吉林大学学报: 工学版, 2007, 37(6): 1284-1287.
Feng Jin-qiao, Yang Zhao-sheng, Zhao Lin, et al. Adaptive exponential smoothing model for dynamic prediction[J]. Journal of Jilin University(Engineering and Technology Edition), 2007, 37(6): 1284-1287.
[1] Hai-long ZHANG,Xu DU,Meng ZHANG,Ya-zhou ZHANG,Jie WANG,Xin-chen YE,Wan-qiong WANG,Jia LI,Han WU,Ting ZHANG. Filter design and performance analysis based on vitis HLS [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(3): 1072-1081.
[2] Yin-fei DAI,Xiu-zhen ZHOU,Yu-bao LIU,Zhi-yuan LIU. In⁃vehicle network intrusion detection system based on CAN bus data [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(3): 857-865.
[3] Hua-zhen FANG,Li LIU,Qing GU,Xiao-feng XIAO,Yu MENG. Driving intention recognition based on trajectory prediction and extreme gradient boosting [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 623-630.
[4] Jin ZHU,Qi HUANG. Automated terminal horizontal transportation scheduling and route planning under network resource allocation [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2245-2255.
[5] Ping YU,Kang ZHAO,Jie CAO. Rolling bearing fault diagnosis based on optimized A-BiLSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2156-2166.
[6] Xiao-hong TANG,Yong-jian GONG. Trajectory planning algorithm for grasping manipulator considering energy consumption and load factors [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 1862-1868.
[7] Qiu-zhan ZHOU,Ze-yu JI,Cong WANG,Jing RONG. Non-intrusive load monitoring via online compression and reconstruction [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1796-1806.
[8] Ling HUANG,Zuan CUI,Feng YOU,Pei-xin HONG,Hao-chuan ZHONG,Yi-xuan ZENG. Vehicle trajectory prediction model for multi-vehicle interaction scenario [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(5): 1188-1195.
[9] Xian-feng REN,Wen-wen YUAN,Xue-qiang WU,Yan-ru SHI,Meng-meng YAO,Kai-xuan ZHANG,Rui-xin YANG,Yue PAN. Multi-dimensional aging diagnosis of lithium-ion battery with a long short-term memory neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(11): 3135-3147.
[10] Hai-long ZHANG,Meng ZHANG,Ya-zhou ZHANG,Jie WANG,Xin-chen YE,Wan-qiong WANG,Jia LI,Xu DU,Ting ZHANG. Channelization of wideband signal based on critical sampling polyphase filter banks [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2388-2394.
[11] Rong-han YAO,Wen-tao XU,Wei-wei GUO. Drivers' takeover behavior and intention recognition based on factor and long short⁃term memory [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 758-771.
[12] Jian-cheng WENG,Rui-cong WEI,Han-mei HE,Hai-hui XU,Jing-jing WANG. Urban road network short-term traffic flow prediction model based on associated road chain group [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(11): 3104-3112.
[13] Qing-tian GENG,Yang ZHAO,Qing-liang LI,Fan-hua YU,Xiao-ning LI. Integrated LSTM and ARIMA method based on attention model for soil temperature [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(10): 2973-2981.
[14] Jin-wu GAO,Zhi-huan JIA,Xiang-yang WANG,Hao XING. Degradation trend prediction of proton exchange membrane fuel cell based on PSO⁃LSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(9): 2192-2202.
[15] Da-xiang LI,Meng-si CHEN,Ying LIU. Spontaneous micro-expression recognition based on STA-LSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 897-909.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] LI Shoutao, LI Yuanchun. Autonomous Mobile Robot Control Algorithm Based on Hierarchical Fuzzy Behaviors in Unknown Environments[J]. 吉林大学学报(工学版), 2005, 35(04): 391 -397 .
[2] Li Hong-ying; Shi Wei-guang;Gan Shu-cai. Electromagnetic properties and microwave absorbing property
of Z type hexaferrite Ba3-xLaxCo2Fe24O41[J]. 吉林大学学报(工学版), 2006, 36(06): 856 -0860 .
[3] Zhang Quan-fa，Li Ming-zhe，Sun Gang，Ge Xin . Comparison between flexible and rigid blank-holding in multi-point forming[J]. 吉林大学学报(工学版), 2007, 37(01): 25 -30 .
[4] Yang Shu-kai, Song Chuan-xue, An Xiao-juan, Cai Zhang-lin . Analyzing effects of suspension bushing elasticity
on vehicle yaw response character with virtual prototype method[J]. 吉林大学学报(工学版), 2007, 37(05): 994 -0999 .
[5] . [J]. 吉林大学学报(工学版), 2007, 37(06): 1284 -1287 .
[6] Che Xiang-jiu，Liu Da-you，Wang Zheng-xuan . Construction of joining surface with G1 continuity for two NURBS surfaces[J]. 吉林大学学报(工学版), 2007, 37(04): 838 -841 .
[7] Liu Han-bing, Jiao Yu-ling, Liang Chun-yu,Qin Wei-jun . Effect of shape function on computing precision in meshless methods[J]. 吉林大学学报(工学版), 2007, 37(03): 715 -0720 .
[8] . [J]. 吉林大学学报(工学版), 2007, 37(04): 0 .
[9] Li Yue-ying，Liu Yong-bing，Chen Hua . Surface hardening and tribological properties of a cam materials[J]. 吉林大学学报(工学版), 2007, 37(05): 1064 -1068 .
[10] Feng Hao，Xi Jian-feng，Jiao Cheng-wu . Placement of roadside traffic signs based on visibility distance[J]. 吉林大学学报(工学版), 2007, 37(04): 782 -785 .
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd