5 papers
Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
Keyue Jiang, Yuxiang Wang, Yanan Zhao +7
Diffusion Language Models (DLMs) have demonstrated strong scaling capacity as alternatives to autoregressive language models. However, their performance is highly sensitive to the…
GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
Xiaohang Tang, Keyue Jiang, Che Liu +4
Reinforcement learning (RL) can be used to improve the policy (denoiser) of diffusion large language models (dLLMs), while being hindered by the intractability of the policy likeli…
On the Trainability of Masked Diffusion Language Models via Blockwise Locality
Yuxiang Wang, Yu Xiang, Baojian Zhou +4
Masked diffusion language models (MDMs) have recently emerged as a promising alternative to standard autoregressive large language models (AR-LLMs), yet their optimization can be s…
Logics-STEM: Empowering LLM Reasoning via Failure-Driven Post-Training and Document Knowledge Enhancement
Mingyu Xu, Cheng Fang, Keyue Jiang +16
We present Logics-STEM, a state-of-the-art reasoning model fine-tuned on Logics-STEM-SFT-Dataset, a high-quality and diverse dataset at 10M scale that represents one of the largest…
GraphGPT: Generative Pre-trained Graph Eulerian Transformer
Qifang Zhao, Weidong Ren, Tianyu Li +3
We introduceGraphGPT, a novel self-supervised generative pre-trained model for graph learning based on the Graph Eulerian Transformer (GET). First, we propose GET, which combines a…