关键词: 流数据处理, 资源分配, 弹性调度, 组件实现

Abstract: Aiming at the problem of  static resource allocation of the Storm platform, we proposed a distributed adaptive elastic resource allocation strategy,  which could  optimally meet the resource requirements of applications. Based on this strategy, combined with the resource allocation mechanism, application programming interface and user interface parameters of Storm, an elastic resource allocation component deployed in Storm was implemented to support adaptive and dynamic adjustment of application resources. The experimental results show that on the real stream data set, compared with the middle-value dynamic resource allocation strategy and the static resource allocation strategy of Storm, this distributed optimal strategy has advantages in throughput, loss rate and resource utilization. Meanwhile, this adaptive elastic resource allocation component can well interact with the Storm system, providing a reference solution for the development of other elastic resource scheduling components.

Key words: stream data processing, resource allocation, elastic scheduling, component implementation