5 papers
Learning What to Forget: Distributional Unlearning for LLM Representation Spaces
Pinaki Mohanty, Haoran Tang, Maggie Makar +1
Machine learning systems increasingly face the need to remove the influence of entire data domains, such as toxic language, harmful behavior, or topical content, rather than isolat…
Unlearning Deceptive Behaviors in LLMs with Contrastive Forget Sets
Haoran Tang, Rajiv Khanna
Large language models often know the truth and say otherwise: a model that answers correctly when asked neutrally will affirm a user's mistaken belief, or misstate a fact its syste…
Unmerge: Efficient Machine Unlearning via Task Arithmetic
Haoran Tang, Andrew Tan, Rajiv Khanna
Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hy…
Contrastive Representation Shaping for LLM Unlearning
Haoran Tang, Rajiv Khanna
Most LLM unlearning methods aim to approximate retrain-from-scratch behaviors with minimal distribution shift, often via alignment-style objectives defined in the prediction space.…
Sharpness-Aware Machine Unlearning
Haoran Tang, Rajiv Khanna
We characterize the effectiveness of Sharpness-aware minimization (SAM) under machine unlearning scheme, where unlearning forget signals interferes with learning retain signals. Wh…