517 citations · 1.1k across the 28 of their papers we have counts for
12 papers · 1 filter
What can a Single Attention Layer Learn? A Study Through the Random Features Lens
Hengyu Fu, Tianyu Guo, Yu Bai +1
Attention layers -- which map a sequence of inputs to a sequence of outputs -- are core building blocks of the Transformer architecture which has achieved significant breakthroughs…
Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection
Yu Bai, Fan Chen, Huan Wang +2
Neural sequence models based on the transformer architecture have demonstrated remarkable \emph{in-context learning} (ICL) abilities, where they can perform new tasks when prompted…
Lower Bounds for Learning in Revealing POMDPs
Fan Chen, Huan Wang, Caiming Xiong +2
This paper studies the fundamental limits of reinforcement learning (RL) in the challenging \emph{partially observable} setting. While it is well-established that learning in Parti…
Partially Observable RL with B-Stability: Unified Structural Condition and Sharp Sample-Efficient Algorithms
Fan Chen, Yu Bai, Song Mei
Partial Observability -- where agents can only observe partial information about the true underlying state of the system -- is ubiquitous in real-world applications of Reinforcemen…
Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games
Ziang Song, Song Mei, Yu Bai
Imperfect-Information Extensive-Form Games (IIEFGs) is a prevalent model for real-world games involving imperfect information and sequential plays. The Extensive-Form Correlated Eq…
Efficient Phi-Regret Minimization in Extensive-Form Games via Online Mirror Descent
Yu Bai, Chi Jin, Song Mei +2
A conceptually appealing approach for learning Extensive-Form Games (EFGs) is to convert them to Normal-Form Games (NFGs). This approach enables us to directly translate state-of-t…
Efficient and Differentiable Conformal Prediction with General Function Classes
Yu Bai, Song Mei, Huan Wang +2
Quantifying the data uncertainty in learning tasks is often done by learning a prediction interval or prediction set of the label given the input. Two commonly desired properties f…
Near-Optimal Learning of Extensive-Form Games with Imperfect Information
Yu Bai, Chi Jin, Song Mei +1
This paper resolves the open question of designing near-optimal algorithms for learning imperfect-information extensive-form games from bandit feedback. We present the first line o…
When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?
Ziang Song, Song Mei, Yu Bai
Multi-agent reinforcement learning has made substantial empirical progresses in solving games with a large number of players. However, theoretically, the best known sample complexi…
Understanding the Under-Coverage Bias in Uncertainty Estimation
Yu Bai, Song Mei, Huan Wang +1
Estimating the data uncertainty in regression tasks is often done by learning a quantile function or a prediction interval of the true label conditioned on the input. It is frequen…
Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models
Zitong Yang, Yu Bai, Song Mei
Recent work showed that there could be a large gap between the classical uniform convergence bound and the actual test error of zero-training-error predictors (interpolators) such…
Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification
Yu Bai, Song Mei, Huan Wang +1
Modern machine learning models with high accuracy are often miscalibrated -- the predicted top probability does not reflect the actual accuracy, and tends to be over-confident. It…