Publications (13)
Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model
Kai Tong, Kang Pan, Xiao Zhang +5
Large Language Models (LLMs) possess encompassing capabilities that can process diverse language-related tasks. However, finetuning on LLMs will diminish this general skills and co…
APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squares
Kejia Fan, Jianheng Tang, Zhirui Yang +8
Personalized Federated Learning (PFL) has presented a significant challenge to deliver personalized models to individual clients through collaborative training. Existing PFL method…
AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data
Jianheng Tang, Huiping Zhuang, Jingyu He +10
Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing F…
REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning
Run He, Di Fang, Yizhu Chen +5
Exemplar-free class-incremental learning (EFCIL) aims to mitigate catastrophic forgetting in class-incremental learning (CIL) without available historical training samples as exemp…
Non-Forgetting Knowledge Allocation with Bi-level Competition for Class-Incremental Learning
Xiang Tan, Run He, Yawen Cui +6
Class-Incremental Learning (CIL) with pre-trained models (PTMs) aims to sequentially adapt PTMs to new categories without forgetting old knowledge. Built upon PTMs, existing adapte…
F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning
Huiping Zhuang, Yuchen Liu, Run He +5
Online Class Incremental Learning (OCIL) aims to train models incrementally, where data arrive in mini-batches, and previous data are not accessible. A major challenge in OCIL is C…
Rethinking Adapter Placement: A Dominant Adaptation Module Perspective
Suoxin Zhang, Run He, Di Fang +3
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method that places trainable low-rank adapters into frozen pre-trained models. Recent studies show that…
DeepAFL: Deep Analytic Federated Learning
Jianheng Tang, Yajiang Huang, Kejia Fan +8
Federated Learning (FL) is a popular distributed learning paradigm to break down data silo. Traditional FL approaches largely rely on gradient-based updates, facing significant iss…
DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning
Huiping Zhuang, Run He, Kai Tong +3
Class-incremental learning (CIL) under an exemplar-free constraint has presented a significant challenge. Existing methods adhering to this constraint are prone to catastrophic for…
AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models
Run He, Kai Tong, Di Fang +5
In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) with pre-…
L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Xiang Zhang, Run He, Jiao Chen +5
Class-incremental learning (CIL) enables models to learn new classes continually without forgetting previously acquired knowledge. Multi-label CIL (MLCIL) extends CIL to a real-wor…
GACL: Exemplar-Free Generalized Analytic Continual Learning
Huiping Zhuang, Yizhu Chen, Di Fang +5
Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose pre…
Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
Run He, Di Fang, Yicheng Xu +5
Exemplar-Free Class-Incremental Learning (EFCIL) aims to sequentially learn from distinct categories without retaining exemplars but easily suffers from catastrophic forgetting of…