11 papers
Less Uniform Discrete Diffusion is More Powerful and Scalable
Kaibo Wang, Ding Ding, Fangyu Ding +5
Although uniform diffusion language models (UDLMs) represent a promising diffusion paradigm, scaling them remains challenging. We identify the core obstacle as an over-uniform trai…
Exploring More to Solve More: Boosting Diversity in Text Diffusion Models via Entropy-Based Guidance
Jingwei Zhang, Haoyu Lei, Zijin Feng +2
Although diffusion models have revolutionized continuous domains like image synthesis through high quality generations and controllable guidance mechanisms, bringing this controlla…
Stabilizing Reinforcement Learning for Diffusion Language Models
Jianyuan Zhong, Kaibo Wang, Ding Ding +5
Group Relative Policy Optimization (GRPO) is highly effective for post-training autoregressive (AR) language models, yet its direct application to diffusion large language models (…
Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
Fangyu Ding, Ding Ding, Sijin Chen +8
While Masked Diffusion Language Models (MDLMs) relying on token masking and unmasking have shown promise in language modeling, their computational efficiency and generation flexibi…
HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs
Azim Ospanov, Zijin Feng, Jiacheng Sun +3
Informal mathematics has been central to modern large language model (LLM) reasoning, offering flexibility and efficient construction of arguments. However, purely informal reasoni…
ProofFlow: A Dependency Graph Approach to Faithful Proof Autoformalization
Rafael Cabral, Tuan Manh Do, Xuejun Yu +3
Proof autoformalization, the task of translating natural language theorems and proofs into machine-verifiable code, is a critical step for integrating large language models into ri…
Masked Diffusion Models as Energy Minimization
Sitong Chen, Shen Nie, Jiacheng Sun +4
We present a systematic theoretical framework that interprets masked diffusion models (MDMs) as solutions to energy minimization problems in discrete optimal transport. Specificall…
Counterspeech for Mitigating the Influence of Media Bias: Comparing Human and LLM-Generated Responses
Luyang Lin, Zijin Feng, Lingzhi Wang +1
Biased news contributes to societal polarization and is often reinforced by hostile reader comments, constituting a vital yet often overlooked aspect of news dissemination. Our stu…
Mathesis: Towards Formal Theorem Proving from Natural Languages
Yu Xuejun, Jianyuan Zhong, Zijin Feng +17
Recent advances in large language models show strong promise for formal reasoning. However, most LLM-based theorem provers have long been constrained by the need for expert-written…
Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
Shuchen Xue, Tianyu Xie, Tianyang Hu +5
Efficiently scaling Large Language Models (LLMs) necessitates exploring alternatives to dominant autoregressive (AR) methods, with Masked Diffusion Models (MDMs) emerging as candid…
Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling
Tianyu Xie, Shuchen Xue, Zijin Feng +4
Discrete diffusion models have recently shown great promise for modeling complex discrete data, with masked diffusion models (MDMs) offering a compelling trade-off between quality…