256 citations · 1k across the 26 of their papers we have counts for
28 papers
ConvNets Match Vision Transformers at Scale
Samuel L. Smith, Andrew Brock, Leonard Berrada +1
Many researchers believe that ConvNets perform well on small or moderately sized datasets, but are not competitive with Vision Transformers when given access to datasets on the web…
Unlocking Accuracy and Fairness in Differentially Private Image Classification
Leonard Berrada, Soham De, Judy Hanwen Shen +6
Privacy-preserving machine learning aims to train models on private data without leaking sensitive information. Differential privacy (DP) is considered the gold standard framework…
Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues
Antonio Orvieto, Soham De, Caglar Gulcehre +2
Deep neural networks based on linear RNNs interleaved with position-wise MLPs are gaining traction as competitive approaches for sequence modeling. Examples of such architectures i…
Resurrecting Recurrent Neural Networks for Long Sequences
Antonio Orvieto, Samuel L Smith, Albert Gu +4
Recurrent Neural Networks (RNNs) offer fast inference on long sequences but are hard to optimize and slow to train. Deep state-space models (SSMs) have recently been shown to perfo…
Differentially Private Diffusion Models Generate Useful Synthetic Images
Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal +7
The ability to generate privacy-preserving synthetic versions of sensitive image datasets could unlock numerous ML applications currently constrained by data availability. Due to t…
Unlocking High-Accuracy Differentially Private Image Classification through Scale
Soham De, Leonard Berrada, Jamie Hayes +2
Differential Privacy (DP) provides a formal privacy guarantee preventing adversaries with access to a machine learning model from extracting information about individual training p…