77 citations · 298 across the 18 of their papers we have counts for
3 papers · 1 filter
Implicit Acceleration and Feature Learning in Infinitely Wide Neural Networks with Bottlenecks
Etai Littwin, Omid Saremi, Shuangfei Zhai +4
We analyze the learning dynamics of infinitely wide neural networks with a finite sized bottle-neck. Unlike the neural tangent kernel limit, a bottleneck in an otherwise infinite w…
Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning
Yue Wu, Shuangfei Zhai, Nitish Srivastava +4
Offline Reinforcement Learning promises to learn effective policies from previously-collected, static datasets without the need for exploration. However, existing Q-learning and ac…
An Attention Free Transformer
Shuangfei Zhai, Walter Talbott, Nitish Srivastava +4
We introduce Attention Free Transformer (AFT), an efficient variant of Transformers that eliminates the need for dot product self attention. In an AFT layer, the key and value are…