81 citations · 414 across the 42 of their papers we have counts for
10 papers · 1 filter
On-Policy Distillation with Negative-Policy Rollouts
Jaehui Hwang, Dongyoon Han, Sangdoo Yun +1
On-policy distillation (OPD) has been widely studied as a post-training method in which a student model obtains token-level supervision from a stronger teacher on its own rollouts.…
DLoop: Looped Speculative Decoding
Geonmo Gu, Byeongho Heo, HeeJae Jun +4
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequ…
Isotropic Embedding Perturbations for Robust Vision Language Encoders
Hyesong Choi, Daeun Kim, Song Park +5
Data augmentation is fundamental to training modern deep vision and multimodal models. While individual methods, such as RandAug, CutMix, Mixup, RandErase, and DropPath, offer stro…
Revisiting Complete Reasoning Traces for Post-Training
Jaehui Hwang, Sangdoo Yun, Byeongho Heo +1
Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex,…
Verification-Aware Training for Speculative Decoding
Geonmo Gu, Byeongho Heo, HeeJae Jun +4
Speculative decoding accelerates large language model inference by using a draft model to generate candidate tokens, which are verified by the target model in a single forward pass…
On-Policy Delta Distillation for Multilingual Math Reasoning
Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1
On-Policy Distillation (OPD) is emerging as a promising alternative to reinforcement learning for LLM post-training, yet its effectiveness in multilingual settings remains underexp…