Abstract: Aiming at the problem that the interpretable method based on class activation mapping (CAM) was disturbed by features unrelated to the target class, which led to more noise in the visualization results and lower localization accuracy of target objects, we proposed a convolutional neural network (CNN) visualization method based on ablation analysis. Firstly, the correlation between deep network features and target classes was investigated and feature fusion weights were calculated through ablation experiments. Secondly,  the feature fusion weights were corrected by ReLU or Softmax functions to reduce the interference of irrelevant features and obtain  class activation map with higher localization accuracy, so as to make an effective description of network decisions. A variety of evaluation metrics were used for verification on the ILSVRC 2012 validation set, the experimental results show that the method achieves better model interpretation capability in all indicators.

Key words: interpretability, convolutional neural network, ablation analysis, deep learning