7 papers
When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging
Shangge Liu, Yuehan Yin, Yinghuan Shi +2
Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by catastrophic forgetting and we…
Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Borui Kang, Jinrui Gu, Junhan Lv +3
Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of…
Understanding and Enforcing Weight Disentanglement in Task Arithmetic
Shangge Liu, Yuehan Yin, Lei Wang +5
Task arithmetic provides an efficient, training-free way to edit pre-trained models, yet lacks a fundamental theoretical explanation for its success. The existing concept of ``weig…
LibContinual: A Comprehensive Library towards Realistic Continual Learning
Wenbin Li, Shangge Liu, Borui Kang +7
A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones. While the field has evolved wi…
Robust Dataset Distillation by Matching Adversarial Trajectories
Wei Lai, Tianyu Ding, ren dongdong +4
Dataset distillation synthesizes compact datasets that enable models to achieve performance comparable to training on the original large-scale datasets. However, existing distillat…
ONNXPruner: ONNX-Based General Model Pruning Adapter
Dongdong Ren, Wenbin Li, Tianyu Ding +5
Recent advancements in model pruning have focused on developing new algorithms and improving upon benchmarks. However, the practical application of these algorithms across various…
Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation
Wenxiao Deng, Wenbin Li, Tianyu Ding +5
Dataset distillation has emerged as a promising approach in deep learning, enabling efficient training with small synthetic datasets derived from larger real ones. Particularly, di…