14 citations · 65 across the 30 of their papers we have counts for
5 papers · 1 filter
Improved Large Language Diffusion Models
Shen Nie, Qiyang Min, Shaoxuan Xu +7
Modern large language models are predominantly trained with autoregressive factorization and causal attention. We present \emph{iLLaDA}, an 8B masked diffusion language model train…
Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model
Chenghao Fan, Wen Heng, Bo Li +6
Diffusion-based language models (DLLMs) offer non-sequential, block-wise generation and richer data reuse compared to autoregressive (AR) models, but existing code DLLMs still lag…
Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
Yuxuan Song, Zheng Zhang, Cheng Luo +19
We present Seed Diffusion Preview, a large-scale language model based on discrete-state diffusion, offering remarkably fast inference speed. Thanks to non-sequential, parallel gene…
-PO: Generalizing Preference Optimization with -divergence Minimization
Jiaqi Han, Mingjian Jiang, Yuxuan Song +2
Preference optimization has made significant progress recently, with numerous methods developed to align language models with human preferences. This paper introduces -divergenc…
Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods
Ning Miao, Yuxuan Song, Hao Zhou +1
It has been a common approach to pre-train a language model on a large corpus and fine-tune it on task-specific data. In practice, we observe that fine-tuning a pre-trained model o…