7 papers · 1 filter
Detecting Functional Memorization in Code Language Models
Matthieu Meeus, Anil Ramakrishna, Shengyuan Hu +3
Large language models (LLMs) are increasingly used to generate code at scale. Meanwhile, prior work has investigated whether training data may be recoverable from model outputs, by…
From Narrow Unlearning to Emergent Misalignment: Causes, Consequences, and Containment in LLMs
Erum Mushtaq, Anil Ramakrishna, Satyapriya Krishna +5
Recent work has shown that fine-tuning on insecure code data can trigger an emergent misalignment (EMA) phenomenon, where models generate malicious responses even to prompts unrela…
BLUR: A Bi-Level Optimization Approach for LLM Unlearning
Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu +6
Enabling large language models (LLMs) to unlearn knowledge and capabilities acquired during training has proven vital for ensuring compliance with data regulations and promoting et…
Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
Chongyu Fan, Jinghan Jia, Yihua Zhang +3
The LLM unlearning technique has recently been introduced to comply with data regulations and address the safety and ethical concerns of LLMs by removing the undesired data-model i…
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
Zhiqi Bu, Xiaomeng Jin, Bhanukiran Vinzamuri +4
Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspectiv…
K-Edit: Language Model Editing with Contextual Knowledge Awareness
Elan Markowitz, Anil Ramakrishna, Ninareh Mehrabi +4
As the world changes, we need to be able to update our models and correct false information without costly retraining. Knowledge-based model editing enables precise modifications t…