5 papers
Agentic Transformers Provably Learn to Search via Reinforcement Learning
Tong Yang, Yu Huang, Yingbin Liang +1
Tree search is a central abstraction behind many language-agent reasoning and decision-making tasks: agents must explore actions, remember failures, and backtrack toward promising…
ConvexBench: Can LLMs Recognize Convex Functions?
Yepeng Liu, Yu Huang, Yu-Xiang Wang +2
Convex analysis is a modern branch of mathematics with many applications. As Large Language Models (LLMs) start to automate research-level math and sciences, it is important for LL…
Multi-head Transformers Provably Learn Symbolic Multi-step Reasoning via Gradient Descent
Tong Yang, Yu Huang, Yingbin Liang +1
Transformers have demonstrated remarkable capabilities in multi-step reasoning tasks. However, understandings of the underlying mechanisms by which they acquire these abilities thr…
Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information
Yuan Cheng, Yu Huang, Zhe Xiong +2
Uncovering hidden graph structures underlying real-world data is a critical challenge with broad applications across scientific domains. Recently, transformer-based models leveragi…
A Theoretical Analysis of Self-Supervised Learning for Vision Transformers
Yu Huang, Zixin Wen, Yuejie Chi +1
Self-supervised learning has become a cornerstone in computer vision, primarily divided into reconstruction-based methods like masked autoencoders (MAE) and discriminative methods…