3.5k citations · 6.4k across the 16 of their papers we have counts for
16 papers
ConvNets Match Vision Transformers at Scale
Samuel L. Smith, Andrew Brock, Leonard Berrada +1
Many researchers believe that ConvNets perform well on small or moderately sized datasets, but are not competitive with Vision Transformers when given access to datasets on the web…
Towards Learning Universal Audio Representations
Luyu Wang, Pauline Luc, Yan Wu +8
The ability to learn universal audio representations that can solve diverse speech, music, and environment tasks can spur many applications that require general sound content under…
Perceiver IO: A General Architecture for Structured Inputs & Outputs
Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac +12
A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible. Current architectures, however, cannot be applied…
Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error
Stanislav Fort, Andrew Brock, Razvan Pascanu +2
In computer vision, it is standard practice to draw a single sample from the data augmentation procedure for each unique image in the mini-batch. However recent work has suggested…
Skillful Precipitation Nowcasting using Deep Generative Models of Radar
Suman Ravuri, Karel Lenc, Matthew Willson +17
Precipitation nowcasting, the high-resolution forecasting of precipitation up to two hours ahead, supports the real-world socio-economic needs of many sectors reliant on weather-de…
Perceiver: General Perception with Iterative Attention
Andrew Jaegle, Felix Gimeno, Andrew Brock +3
Biological systems perceive the world by simultaneously processing high-dimensional inputs from modalities as diverse as vision, audition, touch, proprioception, etc. The perceptio…
High-Performance Large-Scale Image Recognition Without Normalization
Andrew Brock, Soham De, Samuel L. Smith +1
Batch normalization is a key component of most image classification models, but it has many undesirable properties stemming from its dependence on the batch size and interactions b…
Characterizing signal propagation to close the performance gap in unnormalized ResNets
Andrew Brock, Soham De, Samuel L. Smith
Batch Normalization is a key component in almost all state-of-the-art image classifiers, but it also introduces practical challenges: it breaks the independence between training ex…
Training Generative Adversarial Networks by Solving Ordinary Differential Equations
Chongli Qin, Yan Wu, Jost Tobias Springenberg +4
The instability of Generative Adversarial Network (GAN) training has frequently been attributed to gradient descent. Consequently, recent methods have aimed to tailor the models an…
BYOL works even without batch statistics
Pierre H. Richemond, Jean-Bastien Grill, Florent Altché +8
Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach for image representation. From an augmented view of an image, BYOL trains an online network to predict a tar…
Evolving Normalization-Activation Layers
Hanxiao Liu, Andrew Brock, Karen Simonyan +1
Normalization layers and activation functions are fundamental components in deep networks and typically co-locate with each other. Here we propose to design them using an automated…
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Andrew Brock, Jeff Donahue, Karen Simonyan
Despite recent progress in generative image modeling, successfully generating high-resolution, diverse samples from complex datasets such as ImageNet remains an elusive goal. To th…
SMASH: One-Shot Model Architecture Search through HyperNetworks
Andrew Brock, Theodore Lim, J. M. Ritchie +1
Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning…
FreezeOut: Accelerate Training by Progressively Freezing Layers
Andrew Brock, Theodore Lim, J. M. Ritchie +1
The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set…
Neural Photo Editing with Introspective Adversarial Networks
Andrew Brock, Theodore Lim, J. M. Ritchie +1
The increasingly photorealistic sample quality of generative image models suggests their feasibility in applications beyond image generation. We present the Neural Photo Editor, an…
Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
Andrew Brock, Theodore Lim, J. M. Ritchie +1
When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in ap…