89 citations · 89 across the 2 of their papers we have counts for
6 papers
Differentiable Patch Selection for Image Recognition
Jean-Baptiste Cordonnier, Aravindh Mahendran, Alexey Dosovitskiy +3
Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand…
Group Equivariant Stand-Alone Self-Attention For Vision
David W. Romero, Jean-Baptiste Cordonnier
We provide a general self-attention formulation to impose group equivariance to arbitrary symmetry groups. This is achieved by defining positional encodings that are invariant to t…
Robust Cross-lingual Embeddings from Parallel Sentences
Ali Sabet, Prakhar Gupta, Jean-Baptiste Cordonnier +2
Recent advances in cross-lingual word embeddings have primarily relied on mapping-based methods, which project pretrained word embeddings from different languages into a shared spa…
On the Relationship between Self-Attention and Convolutional Layers
Jean-Baptiste Cordonnier, Andreas Loukas, Martin Jaggi
Recent trends of incorporating attention mechanisms in vision have led researchers to reconsider the supremacy of convolutional layers as a primary building block. Beyond helping C…
Extrapolating paths with graph neural networks
Jean-Baptiste Cordonnier, Andreas Loukas
We consider the problem of path inference: given a path prefix, i.e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix. I…
Sparsified SGD with Memory
Sebastian U. Stich, Jean-Baptiste Cordonnier, Martin Jaggi
Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The…