4 papers
On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
Chongyang Zhao, Mingsong Li, Haodong Lu +1
Multimodal Continual Instruction Tuning aims to continually enhance Large Vision Language Models (LVLMs) by learning from new data without forgetting previously acquired knowledge.…
Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts
Haodong Lu, Chongyang Zhao, Minhui Xue +3
Continual learning (CL) with large pre-trained models aims to incrementally acquire knowledge without catastrophic forgetting. Existing LoRA-based Mixture-of-Experts (MoE) methods…
Take Only What You Need: Rank Minimization as an Implicit Forgetting Regularizer in Continual Learning
Haodong Lu, Chongyang Zhao, Minhui Xue +3
The central tension in continual learning (CL) is the trade-off between plasticity (acquiring new knowledge) and stability (retaining prior knowledge). We study how a pre-trained b…
Learning Mamba as a Continual Learner: Meta-Learning Selective State Space Models for Continual Learning
Chongyang Zhao, Dong Gong
Continual learning (CL) learns from a non-stationary data stream without storing or re-training on all seen samples. Meta-continual learning (MCL) casts CL as sequence prediction a…