46 citations · 94 across the 3 of their papers we have counts for
5 papers
Augmenting Self-attention with Persistent Memory
Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample +2
Transformer networks have lead to important progress in language modeling and machine translation. These models include two consecutive modules, a feed-forward layer and a self-att…
Adaptive Attention Span in Transformers
Sainbayar Sukhbaatar, Edouard Grave, Piotr Bojanowski +1
We propose a novel self-attention mechanism that can learn its optimal attention span. This allows us to extend significantly the maximum context size used in Transformer, while ma…
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
Amanpreet Singh, Tushar Jain, Sainbayar Sukhbaatar
Learning when to communicate and doing that effectively is essential in multi-agent tasks. Recent works show that continuous communication allows efficient training with back-propa…
Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning
Sainbayar Sukhbaatar, Emily Denton, Arthur Szlam +1
In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play fr…
Auto-pooling: Learning to Improve Invariance of Image Features from Image Sequences
Sainbayar Sukhbaatar, Takaki Makino, Kazuyuki Aihara
Learning invariant representations from images is one of the hardest challenges facing computer vision. Spatial pooling is widely used to create invariance to spatial shifting, but…