4 papers · 1 filter
Unlearning Deceptive Behaviors in LLMs with Contrastive Forget Sets
Haoran Tang, Rajiv Khanna
Large language models often know the truth and say otherwise: a model that answers correctly when asked neutrally will affirm a user's mistaken belief, or misstate a fact its syste…
Unmerge: Efficient Machine Unlearning via Task Arithmetic
Haoran Tang, Andrew Tan, Rajiv Khanna
Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hy…
Contrastive Representation Shaping for LLM Unlearning
Haoran Tang, Rajiv Khanna
Most LLM unlearning methods aim to approximate retrain-from-scratch behaviors with minimal distribution shift, often via alignment-style objectives defined in the prediction space.…
Sharpness-Aware Machine Unlearning
Haoran Tang, Rajiv Khanna
We characterize the effectiveness of Sharpness-aware minimization (SAM) under machine unlearning scheme, where unlearning forget signals interferes with learning retain signals. Wh…