517 citations · 1.1k across the 28 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.LG2023★ 3 cited
What can a Single Attention Layer Learn? A Study Through the Random Features Lens
Hengyu Fu, Tianyu Guo, Yu Bai +1
Attention layers -- which map a sequence of inputs to a sequence of outputs -- are core building blocks of the Transformer architecture which has achieved significant breakthroughs…
cs.LG2023★ 9 cited
Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection
Yu Bai, Fan Chen, Huan Wang +2
Neural sequence models based on the transformer architecture have demonstrated remarkable \emph{in-context learning} (ICL) abilities, where they can perform new tasks when prompted…
cs.LG2023★ 1 cited
Lower Bounds for Learning in Revealing POMDPs
Fan Chen, Huan Wang, Caiming Xiong +2
This paper studies the fundamental limits of reinforcement learning (RL) in the challenging \emph{partially observable} setting. While it is well-established that learning in Parti…