8 papers · 1 filter
Information-Theoretic Discrete Diffusion
Moongyu Jeon, Sangwoo Shin, Dongjae Jeon +1
We present an information-theoretic framework for discrete diffusion models that yields principled estimators of log-likelihood using score-matching losses. Inspired by the I-MMSE…
Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs
Bumjun Kim, Dongjae Jeon, Dueun Kim +2
Diffusion large language models (dLLMs) have emerged as a promising alternative to autoregressive models, offering flexible generation orders and strong performance on complex reas…
A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
Wonje Jeung, Sangyeon Yoon, Yoonjun Cho +4
Diffusion large language models (dLLMs) enable any-order generation, but this flexibility enlarges the attack surface: harmful spans may appear at arbitrary positions, and template…
Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition
Yoonjun Cho, Soeun Kim, Dongjae Jeon +3
Decomposing weight matrices into quantization and low-rank components () is a widely used technique for compressing large lang…
DUSK: Do Not Unlearn Shared Knowledge
Wonje Jeung, Sangyeon Yoon, Hyesoo Hong +4
Large language models (LLMs) are increasingly deployed in real-world applications, raising concerns about the unauthorized use of copyrighted or sensitive data. Machine unlearning…
R-TOFU: Unlearning in Large Reasoning Models
Sangyeon Yoon, Wonje Jeung, Albert No
Large Reasoning Models (LRMs) embed private or copyrighted information not only in their final answers but also throughout multi-step chain-of-thought (CoT) traces, making reliable…