Abstract: Aiming at the problem of poor fusion performance in traditional multimodal data feature fusion algorithms, the author proposed a
 multimodal data feature fusion algorithm based on deep learning and D-S theory.  Firstly, within the framework of deep learning, a restricted Boltzmann machine (RBM) was used to train multimodal data. Based on the characteristics of the data and task requirements, an RBM model structure was constructed for multimodal data feature selection. Secondly, based on the selected features, the author calculated the distance between similar modal data, determined the trust function, and set a threshold to remove abnormal data, achieving preliminary fusion of similar modal data. Finally, by calculating the distance between heterogeneous modal data and feartures of different levels, the author determined the trust function of heterogeneous data, and combined with D-S theory, multimodal data feature fusion was achieved. The experimental results show that the purity of the proposed algorithm can reach up to 1.0, and the standardized mutual information can reach up to 0.3, indicating that the proposed algorithm can obtain accurate multimodal data feature fusion results.

Key words: deep learning, D-S theory, multimodal data feature, fuse