Progress and Trends of Brain-Inspired Continual Learning

Continual learning aims to enable artificial intelligence systems to continuously master new tasks and knowledge while effectively avoiding catastrophic forgetting of previously learned content, thereby striking a balance between plasticity and stability. Currently, neural networks based on backpropagation typically rely on global parameter updates, making them prone to catastrophic forgetting in non-stationary data streams. In recent years, large models have demonstrated a form of “soft continual learning” through mechanisms such as retrieval augmentation, parameter-efficient fine-tuning, mixture of experts, and extended long contexts, showcasing advantages over traditional continual learning methods in terms of in-context learning and external memory retrieval. However, such methods still face challenges, including the high resource consumption of processing ultra-long contexts and the conflicts and forgetting that arise within internal pre-trained parameters during continual training. Meanwhile, brain-inspired continual learning methods have attracted considerable attention. These methods draw on mechanisms such as local synaptic plasticity, the distinct functional areas and nuclear structures of the human brain, and the synergistic interaction between hierarchical memory systems (e.g., short-term/long-term memory, episodic/semantic memory), aiming to achieve capabilities akin to humans’ long-term, sustained interaction and learning with the real world. This paper discusses the following 3 aspects: (1) the root causes of catastrophic forgetting, quantitative evaluation metrics, classical continual learning methods, and current research progress in continual learning for large models; (2) the behavioral characteristics of human continual learning, the neural mechanisms underlying human continual learning, and the application of brain-inspired algorithms in continual learning, including the interaction between the hippocampus and neocortex in short-term/long-term memory based on the complementary learning systems theory, as well as mechanisms such as memory consolidation and replay; (3) the open challenges currently facing the field of continual learning, along with suggestions for future evaluation frameworks.