6 papers
EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation
Jinghan Jia, Hadi Reisizadeh, Chongyu Fan +3
Large language models (LLMs) have shown remarkable reasoning capabilities when trained with chain-of-thought (CoT) supervision. However, the long and verbose CoT traces, especially…
BLUR: A Bi-Level Optimization Approach for LLM Unlearning
Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu +6
Enabling large language models (LLMs) to unlearn knowledge and capabilities acquired during training has proven vital for ensuring compliance with data regulations and promoting et…
A Doubly Stochastically Perturbed Algorithm for Linearly Constrained Bilevel Optimization
Prashant Khanduri, Ioannis Tsaknakis, Yihua Zhang +2
In this work, we develop analysis and algorithms for a class of (stochastic) bilevel optimization problems whose lower-level (LL) problem is strongly convex and linearly constraine…
SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models
Anil Ramakrishna, Yixin Wan, Xiaomeng Jin +6
We introduce SemEval-2025 Task 4: unlearning sensitive content from Large Language Models (LLMs). The task features 3 subtasks for LLM unlearning spanning different use cases: (1)…
LUME: LLM Unlearning with Multitask Evaluations
Anil Ramakrishna, Yixin Wan, Xiaomeng Jin +6
Unlearning aims to remove copyrighted, sensitive, or private content from large language models (LLMs) without a full retraining. In this work, we develop a multi-task unlearning b…
Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
Chongyu Fan, Jinghan Jia, Yihua Zhang +3
The LLM unlearning technique has recently been introduced to comply with data regulations and address the safety and ethical concerns of LLMs by removing the undesired data-model i…