Continued Training of Continuous Language Flows with Token Identity Squared Supervision

Jiawei Jiang, Zhaoxiang Zang, Zhengtao Yu

Diffusion and flow models have recently been extended to language modeling, generating text through continuous denoising in token embedding spaces. Pretrained models provide token…