75.5k citations · 100.8k across the 160 of their papers we have counts for
7 papers · 1 filter
Perceiver IO: A General Architecture for Structured Inputs & Outputs
Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac +12
A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible. Current architectures, however, cannot be applied…
Comment on Stochastic Polyak Step-Size: Performance of ALI-G
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
This is a short note on the performance of the ALI-G algorithm (Berrada et al., 2020) as reported in (Loizou et al., 2021). ALI-G (Berrada et al., 2020) and SPS (Loizou et al., 202…
Training Neural Networks for and by Interpolation
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
In modern supervised learning, many deep neural networks are able to interpolate the data: the empirical loss can be driven to near zero on all samples simultaneously. In this work…
Deep Frank-Wolfe For Neural Network Optimization
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
Learning a deep neural network requires solving a challenging optimization problem: it is a high-dimensional, non-convex and non-smooth minimization problem with a large number of…
Kickstarting Deep Reinforcement Learning
Simon Schmitt, Jonathan J. Hudson, Augustin Zidek +8
We present a method for using previously-trained 'teacher' agents to kickstart the training of a new 'student' agent. To this end, we leverage ideas from policy distillation and po…
Smooth Loss Functions for Deep Top-k Classification
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
The top-k error is a common measure of performance in machine learning and computer vision. In practice, top-k classification is typically performed with deep neural networks train…
Trusting SVM for Piecewise Linear CNNs
Leonard Berrada, Andrew Zisserman, M. Pawan Kumar
We present a novel layerwise optimization algorithm for the learning objective of Piecewise-Linear Convolutional Neural Networks (PL-CNNs), a large class of convolutional neural ne…