24 papers
Speculative Forcing for Efficient Autoregressive Video Generation
Hyunwook Choi, Seungryong Kim, Dongyoon Han +1
Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find…
Learning Transferable World-Action Models from Task-Paired Human Videos
Jeongeun Park, Taekyung Kim, Dongyoon Han +1
World-action models inherit broad visual and physical priors from video pre-training. However, standard within-video next-frame prediction allows them to rely on appearance shared…
On-Policy Distillation with Negative-Policy Rollouts
Jaehui Hwang, Dongyoon Han, Sangdoo Yun +1
On-policy distillation (OPD) has been widely studied as a post-training method in which a student model obtains token-level supervision from a stronger teacher on its own rollouts.…
DLoop: Looped Speculative Decoding
Geonmo Gu, Byeongho Heo, HeeJae Jun +4
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequ…
Isotropic Embedding Perturbations for Robust Vision Language Encoders
Hyesong Choi, Daeun Kim, Song Park +5
Data augmentation is fundamental to training modern deep vision and multimodal models. While individual methods, such as RandAug, CutMix, Mixup, RandErase, and DropPath, offer stro…
Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training
Jaedeok Lee, Keonwoo Kim, Dongyoon Han +3
Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training inherits a different situatio…
Verification-Aware Training for Speculative Decoding
Geonmo Gu, Byeongho Heo, HeeJae Jun +4
Speculative decoding accelerates large language model inference by using a draft model to generate candidate tokens, which are verified by the target model in a single forward pass…
On-Policy Delta Distillation for Multilingual Math Reasoning
Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1
On-Policy Distillation (OPD) is emerging as a promising alternative to reinforcement learning for LLM post-training, yet its effectiveness in multilingual settings remains underexp…
Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs
Seunghyun Lee, Dongyoon Han, Sangdoo Yun
In a dual-use field, hazardous knowledge and the benign expertise around it share vocabulary, concepts, and causal structure. That overlap makes selective refusal hard, and existin…
On-Policy Delta Distillation
Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1
On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervisio…
Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time
Jeongeun Park, Juhan Park, Taekyung Kim +3
Extending a vision-language-action (VLA) policy to a new task typically requires task-specific teleoperated demonstrations and per-task fine-tuning, making adaptation costly in bot…
WorldKV: Efficient World Memory with World Retrieval and Compression
Jung Yi, Minjae Kim, Paul Hyunbin Cho +3
Autoregressive video diffusion models have enabled real-time, action-conditioned world generation. However, sustaining a persistent world, where revisiting a previously seen viewpo…
Gaze Attention: Query-Adaptive Visual Routing for Efficient Multimodal LLMs
Junha Song, Byeongho Heo, Geonmo Gu +3
When humans describe a visual scene, they do not process the entire image uniformly; instead, they selectively fixate on regions relevant to their intended description. In contrast…
MEME: Multi-entity & Evolving Memory Evaluation
Seokwon Jung, Alexander Rubinstein, Arnas Uselis +2
LLM-based agents increasingly operate in persistent environments where they must store, update, and reason over information across many sessions. While prior benchmarks evaluate on…
MuCo: Multi-turn Contrastive Learning for Multimodal Embedding Model
Geonmo Gu, Byeongho Heo, Jaemyung Yu +7
Universal Multimodal embedding models built on Multimodal Large Language Models (MLLMs) have traditionally employed contrastive learning, which aligns representations of query-targ…
Grounding World Simulation Models in a Real-World Metropolis
Junyoung Seo, Hyunwook Choi, Minkyung Kwon +10
What if a world simulation model could render not an imagined environment but a city that actually exists? Prior generative world models synthesize visually plausible yet artificia…
Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction
Jang-Hyun Kim, Dongyoon Han, Sangdoo Yun
Efficient key-value (KV) cache management is crucial for the practical deployment of large language models (LLMs), yet existing compression techniques often incur a trade-off betwe…
Oops, Wait: Discourse Tokens Matter in Reasoning Model
Jaehui Hwang, Byeongho Heo, Sangdoo Yun +1
Recent studies suggest that even data-efficient training with (1K) reasoning trajectories can induce non-trivial reasoning capabilities in large language models through pos…
Hierarchical Vision Language Action Model Using Success and Failure Demonstrations
Jeongeun Park, Jihwan Yoon, Byungwoo Jeon +6
Prior Vision-Language-Action (VLA) models are typically trained on teleoperated successful demonstrations, while discarding numerous failed attempts that occur naturally during dat…
What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluation
Heejin Do, Jaehui Hwang, Dongyoon Han +2
Evaluating large language models (LLMs) on final-answer correctness is the dominant paradigm. This approach, however, provides a coarse signal for model improvement and overlooks t…
RL makes MLLMs see better than SFT
Junha Song, Sangdoo Yun, Dongyoon Han +2
A dominant assumption in Multimodal Language Model (MLLM) research is that its performance is largely inherited from the LLM backbone, given its immense parameter scale and remarka…
Token Bottleneck: One Token to Remember Dynamics
Taekyung Kim, Dongyoon Han, Byeongho Heo +2
Deriving compact and temporally aware visual representations from dynamic scenes is essential for successful execution of sequential scene understanding tasks such as visual tracki…
Aligned Novel View Image and Geometry Synthesis via Cross-modal Attention Instillation
Min-Seop Kwak, Junho Kim, Sangdoo Yun +4
We introduce a diffusion-based framework that performs aligned novel view image and geometry generation via a warping-and-inpainting methodology. Unlike prior methods that require…
A Unified Framework for Motion Reasoning and Generation in Human Interaction
Jeongeun Park, Sungjoon Choi, Sangdoo Yun
Recent advancements in large language models (LLMs) have significantly improved their ability to generate natural and contextually relevant text, enabling more human-like AI intera…