4 papers · 1 filter
Energy-Structured Low-Rank Adaptation for Continual Learning
Longhua Li, Lei Qi, Qi Tian +1
While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compac…
Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregation
Longhua Li, Lei Qi, Xin Geng +1
Model merging enables training-free multi-task learning by aggregating knowledge from models fine-tuned from a shared checkpoint, but their task updates can interfere. Using proxy…
Model Merging in the Essential Subspace
Longhua Li, Lei Qi, Qi Tian +1
Model merging aims to integrate multiple task-specific fine-tuned models derived from a shared pre-trained checkpoint into a single multi-task model without additional training. De…
Stratified Knowledge-Density Super-Network for Scalable Vision Transformers
Longhua Li, Lei Qi, Xin Geng
Training and deploying multiple vision transformer (ViT) models for different resource constraints is costly and inefficient. To address this, we propose transforming a pre-trained…