743 citations · 2.1k across the 27 of their papers we have counts for
3 papers · 1 filter
Transformed CNNs: recasting pre-trained convolutional layers with self-attention
Stéphane d'Ascoli, Levent Sagun, Giulio Biroli +1
Vision Transformers (ViT) have recently emerged as a powerful alternative to convolutional networks (CNNs). Although hybrid models attempt to bridge the gap between these two archi…
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
Stéphane d'Ascoli, Hugo Touvron, Matthew Leavitt +3
Convolutional architectures have proven extremely successful for vision tasks. Their hard inductive biases enable sample-efficient learning, but come at the cost of a potentially l…
On the interplay between data structure and loss function in classification problems
Stéphane d'Ascoli, Marylou Gabrié, Levent Sagun +1
One of the central puzzles in modern machine learning is the ability of heavily overparametrized models to generalize well. Although the low-dimensional structure of typical datase…