6 papers
Off-Manifold Collapse in Guided Protein Language Models
Shuibai Zhang, Xinchi Liu, Fred Zhangzhi Peng +4
Protein language models are widely used priors for protein sequence design, and a growing body of work controls them at inference time as an alternative to fine-tuning. Such guidan…
Coupling Models for One-Step Discrete Generation
Fred Zhangzhi Peng, Avishek Joey Bose, Anru R. Zhang +1
Generative modeling over discrete structures underpins applications across deep learning, from biological sequence design and code generation to large language models, yet generati…
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1
Diffusion language models (DLMs) have recently demonstrated capabilities that complement standard autoregressive (AR) models, particularly in non-sequential generation and bidirect…
Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models
Shuibai Zhang, Caspian Zhuang, Chihan Cui +8
Diffusion language models (DLMs) enable parallel, non-autoregressive text generation, yet existing DLM mixture-of-experts (MoE) models inherit token-choice (TC) routing from autore…
Corrective Diffusion Language Models
Shuibai Zhang, Fred Zhangzhi Peng, Yiheng Zhang +2
While Diffusion Language Models (DLMs) are theoretically well-suited for iterative refinement due to their non-causal structure, they often fail to reliably revise incorrect tokens…
Planner Aware Path Learning in Diffusion Language Models Training
Fred Zhangzhi Peng, Zachary Bezemek, Jarrid Rector-Brooks +5
Diffusion language models have emerged as a powerful alternative to autoregressive models, enabling fast inference through more flexible and parallel generation paths. This flexibi…