8 papers
One-Step Generation via Riemannian Wasserstein Gradient Flows
David Li, Chanhyuk Lee, Jaehoon Yoo +7
Recently, Drifting Models and Wasserstein Gradient Flows have attracted substantial attention because they move iterative distributional refinement to training and amortize it into…
PatchKV: Weight-Space Compensation of KV Cache
Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi +2
Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compression methods reduce the cache t…
Self-conditioned Flow Map Language Models via Fixed-point Flows
Jaehoon Yoo, Wonjung Kim, Floor Eijkelboom +4
Self-conditioning is a core technique that enhances continuous flow-based language models, where the model learns to denoise generated text by conditioning on its own denoising est…
Posterior Refinement: Fast Language Generation via Any-Order Flow Maps
Manan Agarwal, Sheel Shah, Chanhyuk Lee +6
Non-autoregressive generation offers a powerful paradigm for iterative refinement, allowing models to recursively critique, erase and regenerate arbitrary subsets of tokens. Howeve…
Infinite Mask Diffusion for Few-Step Distillation
Jaehoon Yoo, Wonjung Kim, Chanhyuk Lee +1
Masked Diffusion Models (MDMs) have emerged as a promising alternative to autoregressive models in language modeling, offering the advantages of parallel decoding and bidirectional…
Flow Map Language Models: One-step Language Modeling via Continuous Denoising
Chanhyuk Lee, Jaehoon Yoo, Manan Agarwal +6
Language models based on discrete diffusion have attracted widespread interest for their potential to provide faster generation than autoregressive models. Despite their promise, t…
AdaRank: Adaptive Rank Pruning for Enhanced Model Merging
Chanhyuk Lee, Jiho Choi, Chanryeol Lee +2
Model merging has emerged as a promising approach for unifying independently fine-tuned models into an integrated framework, significantly enhancing computational efficiency in mul…
Revisiting Weight Averaging for Model Merging
Jiho Choi, Donggyun Kim, Chanhyuk Lee +1
Model merging aims to build a multi-task learner by combining the parameters of individually fine-tuned models without additional training. While a straightforward approach is to a…