4 papers
Functional Subnetwork Localization for Efficient Continual Learning in Large Language Models
Shuoran Jiang, Wentao Zhu, Yang Xiang +8
Biological intelligence continually adapts to new information while retaining prior skills through localized plasticity. In contrast, continual learning in large language models (L…
Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning
Xiang Tan, Run He, Yawen Cui +6
Class-Incremental Learning (CIL) with pre-trained models (PTMs) aims to sequentially adapt PTMs to new categories without forgetting old knowledge. Built upon PTMs, existing adapte…
Model-agnostic Selective Labeling with Provable Statistical Guarantees
Huipeng Huang, Wenbo Liao, Huajun Xi +3
Obtaining high-quality labels for large datasets is expensive, requiring massive annotations from human experts. While AI models offer a cost-effective alternative by predicting la…
GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay
Yunan Zhang, Shuoran Jiang, Mengchen Zhao +4
The continual learning capability of large language models (LLMs) is crucial for advancing artificial general intelligence. However, continual fine-tuning LLMs across various domai…