10 papers
Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
Yao Tang, Li Dong, Yaru Hao +3
Large language models often solve complex reasoning tasks more effectively with Chain-of-Thought (CoT), but at the cost of long, low-bandwidth token sequences. Humans, by contrast,…
DenseAnnotate: Enabling Scalable Dense Caption Collection for Images and 3D Scenes via Spoken Descriptions
Xiaoyu Lin, Aniket Ghorpade, Hansheng Zhu +9
With the rapid adoption of multimodal large language models (MLLMs) across diverse applications, there is a pressing need for task-centered, high-quality training data. A key limit…
Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
Huangjie Zheng, Shansan Gong, Ruixiang Zhang +5
Standard discrete diffusion models treat all unobserved states identically by mapping them to an absorbing [MASK] token. This creates an 'information void' where semantic informati…
Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows
Ruixiang Zhang, Shuangfei Zhai, Jiatao Gu +6
Autoregressive models have driven remarkable progress in language modeling. Their foundational reliance on discrete tokens, unidirectional context, and single-pass decoding, while…
DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation
Shansan Gong, Ruixiang Zhang, Huangjie Zheng +4
Diffusion large language models (dLLMs) are compelling alternatives to autoregressive (AR) models because their denoising models operate over the entire sequence. The global planni…
TADA: Improved Diffusion Sampling with Training-free Augmented Dynamics
Tianrong Chen, Huangjie Zheng, David Berthelot +3
Diffusion models have demonstrated exceptional capabilities in generating high-fidelity images but typically suffer from inefficient sampling. Many solver designs and noise schedul…