8 papers · 1 filter
DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs
Bumjun Kim, Dongjae Jeon, Moongyu Jeon +1
Parallel decoding for Diffusion LLMs (dLLMs) is difficult because each denoising step provides only token-wise marginal distributions, while unmasking multiple tokens simultaneousl…
Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning
Shin So, Kyelim Lee, Albert No
Critical-data-size accounts of grokking suggest a natural post-threshold intuition: once training data is sufficient to identify the underlying rule, additional data should acceler…
Preserve-Then-Quantize: Balancing Rank Budgets for Quantization Error Reconstruction in LLMs
Yoonjun Cho, Dongjae Jeon, Soeun Kim +2
Quantization Error Reconstruction (QER) reduces accuracy loss in Post-Training Quantization (PTQ) by approximating weights as …
Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failures
Sangyeon Yoon, Hyesoo Hong, Wonje Jeung +1
Machine unlearning aims to remove specific content from trained models while preserving overall performance. However, the phenomenon of benign relearning, in which forgotten inform…
An Information Theoretic Evaluation Metric For Strong Unlearning
Dongjae Jeon, Wonje Jeung, Taeheon Kim +2
Machine unlearning (MU) aims to remove the influence of specific data from trained models, addressing privacy concerns and ensuring compliance with regulations such as the ``right…
Information-Theoretic Discrete Diffusion
Moongyu Jeon, Sangwoo Shin, Dongjae Jeon +1
We present an information-theoretic framework for discrete diffusion models that yields principled estimators of log-likelihood using score-matching losses. Inspired by the I-MMSE…