4 papers
Routing-Aware Expert Calibration for Machine Unlearning in Mixture-of-Experts Language Models
Jingyi Xie, Yijun Lin, Yinjiang Xiong +2
Machine unlearning is increasingly important for large language models, yet unlearning in Mixture-of-Experts (MoE) architectures remains underexplored. Unlike dense models, MoE arc…
Efficient machine unlearning with minimax optimality
Jingyi Xie, Linjun Zhang, Sai Li
There is a growing demand for efficient data removal to comply with regulations like the GDPR and to mitigate the influence of biased or corrupted data. This has motivated the fiel…
HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning
Liyuan Wang, Jingyi Xie, Xingxing Zhang +2
The deployment of pre-trained models (PTMs) has greatly advanced the field of continual learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgettin…
Towards a General Framework for Continual Learning with Pre-training
Liyuan Wang, Jingyi Xie, Xingxing Zhang +2
In this work, we present a general framework for continual learning of sequentially arrived tasks with the use of pre-training, which has emerged as a promising direction for artif…