70 citations · 197 across the 14 of their papers we have counts for
14 papers
Operationalizing Contextual Integrity in Privacy-Conscious Assistants
Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi +10
Advanced AI assistants combine frontier LLMs and tool access to autonomously perform complex tasks on behalf of users. While the helpfulness of such assistants can increase dramati…
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
Aleksandar Botev, Soham De, Samuel L Smith +59
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve ex…
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Soham De, Samuel L. Smith, Anushan Fernando +14
Recurrent neural networks (RNNs) have fast inference and scale efficiently on long sequences, but they are difficult to train and hard to scale. We propose Hawk, an RNN with gated…
ConvNets Match Vision Transformers at Scale
Samuel L. Smith, Andrew Brock, Leonard Berrada +1
Many researchers believe that ConvNets perform well on small or moderately sized datasets, but are not competitive with Vision Transformers when given access to datasets on the web…
Unlocking Accuracy and Fairness in Differentially Private Image Classification
Leonard Berrada, Soham De, Judy Hanwen Shen +6
Privacy-preserving machine learning aims to train models on private data without leaking sensitive information. Differential privacy (DP) is considered the gold standard framework…
Differentially Private Diffusion Models Generate Useful Synthetic Images
Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal +7
The ability to generate privacy-preserving synthetic versions of sensitive image datasets could unlock numerous ML applications currently constrained by data availability. Due to t…
Unlocking High-Accuracy Differentially Private Image Classification through Scale
Soham De, Leonard Berrada, Jamie Hayes +2
Differential Privacy (DP) provides a formal privacy guarantee preventing adversaries with access to a machine learning model from extracting information about individual training p…
A Stochastic Bundle Method for Interpolating Networks
Alasdair Paren, Leonard Berrada, Rudra P. K. Poudel +1
We propose a novel method for training deep neural networks that are capable of interpolation, that is, driving the empirical loss to zero. At each iteration, our method constructs…
Comment on Stochastic Polyak Step-Size: Performance of ALI-G
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
This is a short note on the performance of the ALI-G algorithm (Berrada et al., 2020) as reported in (Loizou et al., 2021). ALI-G (Berrada et al., 2020) and SPS (Loizou et al., 202…
Make Sure You're Unsure: A Framework for Verifying Probabilistic Specifications
Leonard Berrada, Sumanth Dathathri, Krishnamurthy Dvijotham +5
Most real world applications require dealing with stochasticity like sensor noise or predictive uncertainty, where formal specifications of desired behavior are inherently probabil…
Training Neural Networks for and by Interpolation
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
In modern supervised learning, many deep neural networks are able to interpolate the data: the empirical loss can be driven to near zero on all samples simultaneously. In this work…
Deep Frank-Wolfe For Neural Network Optimization
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
Learning a deep neural network requires solving a challenging optimization problem: it is a high-dimensional, non-convex and non-smooth minimization problem with a large number of…
Smooth Loss Functions for Deep Top-k Classification
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
The top-k error is a common measure of performance in machine learning and computer vision. In practice, top-k classification is typically performed with deep neural networks train…
Trusting SVM for Piecewise Linear CNNs
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
We present a novel layerwise optimization algorithm for the learning objective of Piecewise-Linear Convolutional Neural Networks (PL-CNNs), a large class of convolutional neural ne…