10 papers · 1 filter
How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws
Ziheng Cheng, Yixiao Huang, Hanlin Zhu +1
Recent studies on reinforcement learning (RL) report seemingly conflicting evidence about large language model (LLM) reasoning. Training on mathematics can improve performance in o…
Transformers Provably Learn to Internalize Chain-of-Thought
Yixiao Huang, Hanlin Zhu, Zixuan Wang +4
Chain-of-Thought (CoT) prompting substantially improves the sample efficiency of transformers, reducing the complexity of tasks like parity learning from exponential to polynomial…
Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learning
Ziheng Cheng, Yixiao Huang, Hanlin Zhu +5
Diffusion models are increasingly used as powerful conditional generators, yet real deployments often involve multiple target distributions arising from different tasks, e.g., dive…
Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought
Hanlin Zhu, Shibo Hao, Zhiting Hu +3
Previous work shows that the chain of continuous thought (continuous CoT) improves the reasoning capability of large language models (LLMs) by enabling implicit parallel thinking,…
Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
Hanlin Zhu, Shibo Hao, Zhiting Hu +3
Large Language Models (LLMs) have demonstrated remarkable performance in many applications, including challenging reasoning problems via chain-of-thoughts (CoTs) techniques that ge…
Safe Learning Under Irreversible Dynamics via Asking for Help
Benjamin Plaut, Juan Liévano-Karim, Hanlin Zhu +1
Most learning algorithms with formal regret guarantees essentially rely on trying all possible behaviors, which is problematic when some errors cannot be recovered from. Instead, w…
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics
Hanlin Zhu, Baihe Huang, Shaolun Zhang +4
Auto-regressive large language models (LLMs) show impressive capacities to solve many complex reasoning tasks while struggling with some simple logical reasoning tasks such as inve…
Avoiding Catastrophe in Online Learning by Asking for Help
Benjamin Plaut, Hanlin Zhu, Stuart Russell
Most learning algorithms with formal regret guarantees assume that all mistakes are recoverable and essentially rely on trying all possible behaviors. This approach is problematic…
Towards Optimal Statistical Watermarking
Baihe Huang, Hanlin Zhu, Banghua Zhu +4
We study statistical watermarking by formulating it as a hypothesis testing problem, a general framework which subsumes all previous statistical watermarking methods. Key to our fo…
On Representation Complexity of Model-based and Model-free Reinforcement Learning
Hanlin Zhu, Baihe Huang, Stuart Russell
We study the representation complexity of model-based and model-free reinforcement learning (RL) in the context of circuit complexity. We prove theoretically that there exists a br…