Abstract: To address the issue of catastrophic forgetting in decision-making for autonomous driving in multi-city scenarios, a framework based on continual reinforcement learning is proposed. This framework is built upon the IMPALA( Importance Weighted Actor-Learner Architecture) algorithm architecture. First, a co-attentive awareness module is combined to extract critical environmental representations through cross-scenario feature interaction. Second, a self-activating neural ensemble architecture is built to enable autonomous activation of knowledge modules. Finally, a replay mechanism is applied to relieve the problem of forgetting old knowledge by combining scenario-specific features with historical trajectory experience replay. Off-policy behavior cloning and on-policy learning are employed concurrently to maintain the plasticity and stability of the decision-making algorithm. Whether to use old modules or generate new ones is determined based on the requirements of different autonomous driving scenarios and tasks, and the issue of excessive memory usage is addressed through module fusion. Ablation experiments and comparative ones are conducted in two different groups of multiple city scenarios. The performance of the proposed method is validated by comparing path completion rates and cumulative rewards. Experimental results demonstrate that the average completion rate reaches approximately 85% in the first sequential scenario, and it reaches 81. 93% in the second sequential scenario. The proposed scheme can effectively relieve the issue of catastrophic forgetting in multi-scenario continual decision-making and achieve better stable driving performance. 

Key words: autonomous driving, continual reinforcement learning, replay mechanism, self-activating neural ensembles, catastrophic forgetting