59 citations · 178 across the 12 of their papers we have counts for
12 papers
Demystifying Language Model Forgetting with Low-rank Example Associations
Xisen Jin, Xiang Ren
Large language models (LLMs) suffer from forgetting of upstream knowledge when fine-tuned. Despite efforts on mitigating forgetting, few have investigated how forgotten upstream ex…
What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement
Xisen Jin, Xiang Ren
Language models deployed in the wild make errors. However, simply updating the model with the corrected error instances causes catastrophic forgetting -- the updated model makes er…
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning
Genta Indra Winata, Lingjue Xie, Karthik Radhakrishnan +5
Real-life multilingual systems should be able to efficiently incorporate new languages as data distributions fed to the system evolve and shift over time. To do this, systems need…
Dataless Knowledge Fusion by Merging Weights of Language Models
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro +1
Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training…
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora
Xisen Jin, Dejiao Zhang, Henghui Zhu +5
Pretrained language models (PTLMs) are typically learned over a large, static corpus and further fine-tuned for various downstream tasks. However, when deployed in the real world,…
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning
Xisen Jin, Bill Yuchen Lin, Mohammad Rostami +1
The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pur…
Refining Language Models with Compositional Explanations
Huihan Yao, Ying Chen, Qinyuan Ye +2
Pre-trained language models have been successful on text classification tasks, but are prone to learning spurious correlations from biased datasets, and are thus vulnerable when ma…
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning
Xisen Jin, Arka Sadhu, Junyi Du +1
We explore task-free continual learning (CL), in which a model is trained to avoid catastrophic forgetting in the absence of explicit task boundaries or identities. Among many effo…
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani +2
Hate speech classifiers trained on imbalanced datasets struggle to determine if group identifiers like "gay" or "black" are used in offensive or prejudiced ways. Such biases manife…
Visually Grounded Continual Learning of Compositional Phrases
Xisen Jin, Junyi Du, Arka Sadhu +2
Humans acquire language continually with much more limited access to data samples at a time, as compared to contemporary NLP systems. To study this human-like language acquisition…
Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models
Xisen Jin, Zhongyu Wei, Junyi Du +2
The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the…
Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation
Xisen Jin, Wenqiang Lei, Zhaochun Ren +4
The task of dialogue generation aims to automatically provide responses given previous utterances. Tracking dialogue states is an important ingredient in dialogue generation for es…