9 papers
Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
Keyue Jiang, Yuxiang Wang, Yanan Zhao +7
Diffusion Language Models (DLMs) have demonstrated strong scaling capacity as alternatives to autoregressive language models. However, their performance is highly sensitive to the…
Reinforcement Learning from Denoising Feedback
Qi He, Huan Chen, Ya Guo +3
Policy loss estimation remains a fundamental and long-standing challenge in reinforcement learning (RL) for diffusion language models (DLMs). We introduce Reinforcement Learning fr…
On the Trainability of Masked Diffusion Language Models via Blockwise Locality
Yuxiang Wang, Yu Xiang, Baojian Zhou +4
Masked diffusion language models (MDMs) have recently emerged as a promising alternative to standard autoregressive large language models (AR-LLMs), yet their optimization can be s…
Logics-STEM: Empowering LLM Reasoning via Failure-Driven Post-Training and Document Knowledge Enhancement
Mingyu Xu, Cheng Fang, Keyue Jiang +16
We present Logics-STEM, a state-of-the-art reasoning model fine-tuned on Logics-STEM-SFT-Dataset, a high-quality and diverse dataset at 10M scale that represents one of the largest…
Accelerated Evolving Set Processes for Local PageRank Computation
Binbin Huang, Luo Luo, Yanghua Xiao +2
This work proposes a novel framework based on nested evolving set processes to accelerate Personalized PageRank (PPR) computation. At each stage of the process, we employ a localiz…
From Text to Trajectories: GPT-2 as an ODE Solver via In-Context
Ziyang Ma, Baojian Zhou, Deqing Yang +1
In-Context Learning (ICL) has emerged as a new paradigm in large language models (LLMs), enabling them to perform novel tasks by conditioning on a few examples embedded in the prom…