关键词: 气象业务, 高维, 相关性, 缺失数据, 填补方法, 条件生成对抗网络

Abstract: Meteorological business data contain temporal, spatial and multivariable dimensions. The increase in dimensions leads to an increase in data sparsity, presenting different correlation patterns at different temporal/ spatial scales. It is difficult to map the correlation between meteorological elements and data, resulting in poor structural similarity of the filling results. Therefore, a high-dimensional correlation deficiency data block-filling algorithm for meteorological services is proposed. Combined with the mutual information algorithm, the probability distribution of continuous meteorological variables is approximated based on KDE(Kernel Density Estimation). The nonlinear statistical dependence between variables is quantified through the mutual information formula to generate a symmetric mutual information matrix and capture the local correlation of meteorological elements. The normalized mutual information matrix is transformed into a similarity matrix, and the strong correlation is strengthened and the weak correlation is weakened through exponential function mapping. The Laplacian matrix is constructed, its eigenvectors are calculated, and the k-means algorithm is used to cluster the eigenvectors to achieve attribute blocking. The meteorological data is divided into strongly correlated sub-blocks through block processing, and an independent CGAN(Conditional Generative Adversarial Network) is designed for each sub-block. By designing the loss function and training the conditional generative adversarial network, the model can generate imputed values consistent with the distribution of real meteorological data. The experimental results show that when the proposed method is used for block filling of missing data, the structural similarity of the filling results is stable at 0. 92, indicating that this method has an ideal filling effect.

Key words: meteorological operations, high-dimensional, correlation, missing data, filling method, conditional generative adversarial network(CGAN)