11 papers
Parallax: Parameterized Local Linear Attention for Language Modeling
Yifei Zuo, Dhruv Pai, Zhichen Zeng +3
Large Language Models (LLMs) have become the central paradigm in artificial intelligence, yet the core computational primitive of attention has remained structurally unchanged. Loc…
CreFlow: Corrective Reflow for Sparse-Reward Embodied Video Diffusion RL
Zhenyang Ni, Yijiang Li, Ruochen Jiao +7
Video generation models trained on heterogeneous data with likelihood-surrogate objectives can produce visually plausible rollouts that violate physical constraints in embodied man…
All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs
Xi Chen, Mingyu Jin, Jingcheng Niu +7
In this paper, we present empirical and theoretical evidence against a central but largely implicit assumption in circuit and sheaf discovery (CSD), which we term the Functional An…
Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM Reasoning
Shenao Zhang, Yaqing Wang, Yinxiao Liu +5
Large Language Models (LLMs) trained via Reinforcement Learning (RL) have exhibited strong reasoning capabilities and emergent reflective behaviors, such as rethinking and error co…
Learning to Reason as Action Abstractions with Scalable Mid-Training RL
Shenao Zhang, Donghan Yu, Yihao Feng +4
Large language models excel with reinforcement learning (RL), but fully unlocking this potential requires a mid-training stage. An effective mid-training phase should identify a co…
Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
Yutong Yin, Zhaoran Wang
Humans exhibit remarkable compositional reasoning by integrating knowledge from various sources. For example, if someone learns ( B = f(A) ) from one source and ( C = g(B) ) from a…