3 papers
cs.LG2025
Demystifying Language Model Forgetting with Low-rank Example Associations
Xisen Jin, Xiang Ren
Large language models (LLMs) suffer from forgetting of upstream knowledge when fine-tuned. Despite efforts on mitigating forgetting, few have investigated how forgotten upstream ex…
cs.CL2025
Dataless Knowledge Fusion by Merging Weights of Language Models
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro +1
Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training…
cs.LG2024
What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement
Xisen Jin, Xiang Ren
Language models deployed in the wild make errors. However, simply updating the model with the corrected error instances causes catastrophic forgetting -- the updated model makes er…