CoBit: Language Modeling with Bitstream Diffusion

Georgios Batzolis, Mark Girolami, Luca Ambrogioni

Diffusion language models (DLMs) promise parallel, order-agnostic generation, but on standard benchmarks they have historically lagged behind autoregressive models in sample qualit…