6 papers
Latent Thought Credit: Multi-Answer Credit Assignment for Latent Reasoning
Xuyang Zhao, Liting Zhang, Zichen Xu +4
Latent reasoning allows language models to carry out intermediate reasoning in continuous latent representations rather than fully externalizing it as discrete chains of thought. H…
Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning
Xuyang Zhao, Liting Zhang, Zichen Xu +4
On-policy self-distillation (OPSD) improves reasoning by using a privileged view of a model conditioned on reference solutions to supervise a student view that observes only the qu…
TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization
Liting Zhang, Shiwan Zhao, Xuyang Zhao +3
Latent reasoning has emerged as a promising alternative to discrete Chain-of-Thought (CoT) in large language models (LLMs), enabling more expressive reasoning by operating over con…
Efficient Low Rank Attention for Long-Context Inference in Large Language Models
Tenghui Li, Guoxu Zhou, Xuyang Zhao +2
As the length of input text increases, the key-value (KV) cache in LLMs imposes prohibitive GPU memory costs and limits long-context inference on resource constrained devices. Exis…
AgentCDM: Enhancing Multi-Agent Collaborative Decision-Making via ACH-Inspired Structured Reasoning
Xuyang Zhao, Shiwan Zhao, Hualong Yu +2
Multi-agent systems (MAS) powered by large language models (LLMs) hold significant promise for solving complex decision-making tasks. However, the core process of collaborative dec…
A Statistical Theory of Regularization-Based Continual Learning
Xuyang Zhao, Huiyuan Wang, Weiran Huang +1
We provide a statistical analysis of regularization-based continual learning on a sequence of linear regression tasks, with emphasis on how different regularization terms affect th…