13 papers
Embarrassingly Simple Self-Distillation Improves Code Generation
Ruixiang Zhang, Richard He Bai, Huangjie Zheng +3
Can a large language model (LLM) improve at code generation using only its own raw outputs, without a verifier, a teacher model, or reinforcement learning? We answer in the affirma…
Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling
Yizhu Jiao, Ruixiang Zhang, Richard Bai +3
Code generation is typically trained in the primal space of programs: a model produces a candidate solution and receives sparse execution feedback, often a single pass/fail bit. Te…
Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling
Huangjie Zheng, Shansan Gong, Ruixiang Zhang +5
Standard discrete diffusion models treat all unobserved states identically by mapping them to an absorbing [MASK] token. This creates an 'information void' where semantic informati…
ChipChat: Low-Latency Cascaded Conversational Agent in MLX
Tatiana Likhomanenko, Luke Carlson, Richard He Bai +8
The emergence of large language models (LLMs) has transformed spoken dialog systems, yet the optimal architecture for real-time on-device voice agents remains an open question. Whi…
Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows
Ruixiang Zhang, Shuangfei Zhai, Jiatao Gu +6
Autoregressive models have driven remarkable progress in language modeling. Their foundational reliance on discrete tokens, unidirectional context, and single-pass decoding, while…
What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?
Yizhe Zhang, Richard Bai, Zijin Gu +5
Language models usually use left-to-right (L2R) autoregressive factorization. However, L2R factorization may not always be the best inductive bias. Therefore, we investigate whethe…