9 papers
From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
Yuchuan Tian, Yuchen Liang, Shuo Zhang +10
Diffusion Language Models (DLMs) enable fast generation, yet training large DLMs from scratch is costly. As a practical shortcut, adapting off-the-shelf Auto-Regressive (AR) model…
Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants
Yunhe Wang, Kai Han, Huiling Zhen +13
The paradigm of Large Language Models (LLMs) is currently defined by auto-regressive (AR) architectures, which generate text through a sequential ``brick-by-brick'' process. Despit…
Deferred Commitment Decoding for Diffusion Language Models
Yingte Shu, Yuchuan Tian, Chao Xu +2
Diffusion language models (DLMs) have recently emerged as a strong alternative to autoregressive models by enabling parallel text generation. To improve inference efficiency and KV…
Nexus: Higher-Order Attention Mechanisms in Transformers
Hanting Chen, Chong Zhu, Kai Han +6
Transformers have achieved significant success across various domains, relying on self-attention to capture dependencies. However, the standard first-order attention mechanism is o…
Revealing the Temporally Stable Bimodal Energy Distribution of FRB 20121102A with a Tripled Burst Set from AI Detections
Yidan Wang, Jing Han, Pei Wang +34
Active repeating Fast Radio Bursts (FRBs), with their large number of bursts, burst energy distribution, and their potential energy evolution, offer critical insights into the FRBs…
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
Ning Ding, Jing Han, Yuchuan Tian +3
Diffusion Transformer (DiT) has now become the preferred choice for building image generation models due to its great generation capability. Unlike previous convolution-based UNet…