4 papers · 1 filter
Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failures
Sangyeon Yoon, Hyesoo Hong, Wonje Jeung +1
Machine unlearning aims to remove specific content from trained models while preserving overall performance. However, the phenomenon of benign relearning, in which forgotten inform…
Information-Theoretic Discrete Diffusion
Moongyu Jeon, Sangwoo Shin, Dongjae Jeon +1
We present an information-theoretic framework for discrete diffusion models that yields principled estimators of log-likelihood using score-matching losses. Inspired by the I-MMSE…
Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition
Yoonjun Cho, Soeun Kim, Dongjae Jeon +3
Decomposing weight matrices into quantization and low-rank components () is a widely used technique for compressing large lang…
Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
Dongjae Jeon, Dueun Kim, Albert No
In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a prev…