Abstract: Aiming at the limitations of extracting retinal features from single-mode retinal images, we proposed a multi-modal retinal disease diagnosis model based on multi-level and multi-scale attention fusion network. Firstly, the multi-level attention network and multi-scale attention network were designed for color retinal images  and retinal optical coherence tomography respectively, and the fusion features were obtained by merging at the feature layer. Secondly, the weighted loss function of the two modes and the loss function of the fusion features were added to extract the unique and complementary information of the two modes in order to  improve the accuracy of retinal disease diagnosis. The results of evaluation experiments  on the MMC-AMD dataset and GAMMA dataset show that the proposed model outperforms  the current mainstream models and has superior diagnostic effect.

Key words: medical image classification, retinal disease diagnosis model, multi-modal classification, attention mechanism