2 papers
cs.LG2026
Decoupling Token Roles in Autoregressive Pretraining
Suqin Yuan, Runqi Lin, Kevin Qinghong Lin +4
Autoregressive pretraining increasingly draws on heterogeneous data, making it important to understand how a model learns from an individual token. The next-token prediction object…
cs.LG2026
Robust Dynamic Expansion for Continual Learning under Backdoor Attacks via Purification and Selective Recovery
Keyu Lin, Fei Ye, Qihe Liu +2
Continual learning (CL) enables models to acquire new knowledge from sequentially arriving tasks while retaining previously learned knowledge. However, in practical scenarios, task…