4 papers
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
Wanru Zhao, Yihong Chen, Yuzhi Tang +6
Data curation is a critical yet under-explored area in large language model (LLM) training. Existing methods, such as data selection and mixing, operate in an offline paradigm, det…
LLM Unlearning via Neural Activation Redirection
William F. Shen, Xinchi Qiu, Meghdad Kurmanji +5
The ability to selectively remove knowledge from LLMs is highly desirable. However, existing methods often struggle with balancing unlearning efficacy and retain model utility, and…
Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages
Wanru Zhao, Yihong Chen, Royson Lee +4
Pre-trained large language models (LLMs) have become a cornerstone of modern natural language processing, with their capabilities extending across a wide range of applications and…
How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective
Xinchi Qiu, William F. Shen, Yihong Chen +4
While unlearning knowledge from large language models (LLMs) is receiving increasing attention, one important aspect remains unexplored. Existing approaches and benchmarks assume d…