249 citations · 798 across the 17 of their papers we have counts for
11 papers · 1 filter
Augmenting Convolutional networks with attention-based aggregation
Hugo Touvron, Matthieu Cord, Alaaeldin El-Nouby +4
We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning. We replace the final average pooling by an attention-based aggre…
Watermarking Images in Self-Supervised Latent Spaces
Pierre Fernandez, Alexandre Sablayrolles, Teddy Furon +2
We revisit watermarking techniques based on pre-trained deep networks, in the light of self-supervised approaches. We present a way to embed both marks and binary messages into the…
Nearest neighbor search with compact codes: A decoder perspective
Kenza Amara, Matthijs Douze, Alexandre Sablayrolles +1
Modern approaches for fast retrieval of similar vectors on billion-scaled datasets rely on compressed-domain approaches such as binary sketches or product quantization. These metho…
ResNet strikes back: An improved training procedure in timm
Ross Wightman, Hugo Touvron, Hervé Jégou
The influential Residual Networks designed by He et al. remain the gold-standard architecture in numerous scientific publications. They typically serve as the default architecture…
ResMLP: Feedforward networks for image classification with data-efficient training
Hugo Touvron, Piotr Bojanowski, Mathilde Caron +8
We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification. It is a simple residual network that alternates (i) a linear layer in which…
Emerging Properties in Self-Supervised Vision Transformers
Mathilde Caron, Hugo Touvron, Ishan Misra +4
In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets). Beyond the…
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
Ben Graham, Alaaeldin El-Nouby, Hugo Touvron +4
We design a family of image classification architectures that optimize the trade-off between accuracy and efficiency in a high-speed regime. Our work exploits recent findings in at…
Gradient-based Adversarial Attacks against Text Transformers
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou +1
We propose the first general-purpose gradient-based attack against transformer models. Instead of searching for a single adversarial example, we search for a distribution of advers…
Going deeper with Image Transformers
Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles +2
Transformers have been recently adapted for large scale image classification, achieving high scores shaking up the long supremacy of convolutional neural networks. However the opti…
Training Vision Transformers for Image Retrieval
Alaaeldin El-Nouby, Natalia Neverova, Ivan Laptev +1
Transformers have shown outstanding results for natural language understanding and, more recently, for image classification. We here extend this work and propose a transformer-base…
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze +3
Recently, neural networks purely based on attention were shown to address image understanding tasks such as image classification. However, these visual transformers are pre-trained…