12 papers
DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models
Zirui Wu, Lin Zheng, Jiacheng Ye +5
Block diffusion language models accelerate decoding through parallel block-wise denoising, yet whether they can be reliably scaled for long chain-of-thought (CoT) reasoning remains…
Proxy Compression for Language Modeling
Lin Zheng, Xinyu Li, Qian Liu +2
Modern language models are trained almost exclusively on token sequences produced by a fixed tokenizer, an external lossless compressor often over UTF-8 byte sequences, thereby cou…
Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Models
Lin Zheng, Vasilisa Bashlovkina, Timothy Dozat +3
Tokenizer-free language models eliminate the tokenizer step of the language modeling pipeline by operating directly on bytes; patch-based variants further aggregate contiguous byte…
DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas
Zirui Wu, Lin Zheng, Zhihui Xie +8
Diffusion Language Models (DLMs) present a compelling alternative to autoregressive models, offering flexible, any-order infilling without specialized prompting design. However, th…
Dream-Coder 7B: An Open Diffusion Language Model for Code
Zhihui Xie, Jiacheng Ye, Lin Zheng +8
We present Dream-Coder 7B, an open-source discrete diffusion language model for code generation that exhibits emergent any-order generation capabilities. Unlike traditional autoreg…
Dream 7B: Diffusion Large Language Models
Jiacheng Ye, Zhihui Xie, Lin Zheng +5
We introduce Dream 7B, the most powerful open diffusion large language model to date. Unlike autoregressive (AR) models that generate tokens sequentially, Dream 7B employs discrete…