4 papers
Dynamic Chunking for Diffusion Language Models
Yichen Zhu, Xiaoming Shi, Peng Zhao +3
Block discrete diffusion language models factorize a sequence autoregressively over fixed-size positional blocks, decoupling within-block parallel denoising from across-block condi…
Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instruction
Yifei Wang, Weimin Bai, Colin Zhang +3
In this paper, we unify more than 10 existing one-step diffusion distillation approaches, such as Diff-Instruct, DMD, SIM, SiD, -distill, etc, inside a theory-driven framework w…
David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training
Weijian Luo, Colin Zhang, Debing Zhang +1
We propose Diff-Instruct* (DI*), a data-efficient post-training approach for one-step text-to-image generative models to improve its human preferences without requiring image data.…
Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
Junfeng Tian, Da Zheng, Yang Cheng +3
Large language models (LLM) have prioritized expanding the context window from which models can incorporate more information. However, training models to handle long contexts prese…