6 papers
Athanor: An Agent-Native System for Autonomous Post-Training Research
Yufa Zhou, Yichen Wang, Jiakun Fan +5
Recursive self-improvement, in which AI systems build stronger AI systems, is one promising route to superintelligence. We study a weaker form that can be measured today: frontier…
Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One
Fred Zhangzhi Peng, Kaiwen Zheng, Anru R. Zhang
Language generation is almost universally treated as a sequential process: autoregressive models emit one token at a time, while diffusion language models replace token-level seria…
Coupling Models for One-Step Discrete Generation
Fred Zhangzhi Peng, Avishek Joey Bose, Anru R. Zhang +1
Generative modeling over discrete structures underpins applications across deep learning, from biological sequence design and code generation to large language models, yet generati…
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1
Diffusion language models (DLMs) have recently demonstrated capabilities that complement standard autoregressive (AR) models, particularly in non-sequential generation and bidirect…
Why Do Transformers Fail to Forecast Time Series In-Context?
Yufa Zhou, Yixiao Wang, Surbhi Goel +1
Time series forecasting (TSF) remains a challenging and largely unsolved problem in machine learning, despite significant recent efforts leveraging Large Language Models (LLMs), wh…
The Geometry of Reasoning: Flowing Logics in Representation Space
Yufa Zhou, Yixiao Wang, Xunjian Yin +2
We study how large language models (LLMs) ``think'' through their representation space. We propose a novel geometric framework that models an LLM's reasoning as flows -- embedding…