61 citations · 200 across the 9 of their papers we have counts for
12 papers
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping
James Martens, Andy Ballard, Guillaume Desjardins +4
Using an extended and formalized version of the Q/C map analysis of Poole et al. (2016), along with Neural Tangent Kernel theory, we identify the main pathologies present in deep n…
Gradient Forward-Propagation for Large-Scale Temporal Video Modelling
Mateusz Malinowski, Dimitrios Vytiniotis, Grzegorz Swirszcz +2
How can neural networks be trained on large-volume temporal data efficiently? To compute the gradients required to update parameters, backpropagation blocks computations until the…
Sideways: Depth-Parallel Training of Video Models
Mateusz Malinowski, Grzegorz Swirszcz, Joao Carreira +1
We propose Sideways, an approximate backpropagation scheme for training video models. In standard backpropagation, the gradients and activations at every computation step through t…
Verification of Non-Linear Specifications for Neural Networks
Chongli Qin, Krishnamurthy, Dvijotham +7
Prior work on neural network verification has focused on specifications that are linear functions of the output of the network, e.g., invariance of the classifier output under adve…
Distilling Policy Distillation
Wojciech Marian Czarnecki, Razvan Pascanu, Simon Osindero +3
The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning. This process, referred to as distillation, has been used to great success,…
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
Edward Grefenstette, Robert Stanforth, Brendan O'Donoghue +3
While deep learning has led to remarkable results on a number of challenging problems, researchers have discovered a vulnerability of neural networks in adversarial settings, where…
Sobolev Training for Neural Networks
Wojciech Marian Czarnecki, Simon Osindero, Max Jaderberg +2
At the heart of deep learning we aim to use neural networks as function approximators - training them to produce outputs from inputs in emulation of a ground truth function or data…
Understanding Synthetic Gradients and Decoupled Neural Interfaces
Wojciech Marian Czarnecki, Grzegorz Świrszcz, Max Jaderberg +3
When training neural networks, the use of Synthetic Gradients (SG) allows layers or modules to be trained without update locking - without waiting for a true error gradient to be b…
Local minima in training of neural networks
Grzegorz Swirszcz, Wojciech Marian Czarnecki, Razvan Pascanu
There has been a lot of recent interest in trying to characterize the error surface of deep models. This stems from a long standing question. Given that deep networks are highly no…
Approximation Algorithms for the Joint Replenishment Problem with Deadlines
Marcin Bienkowski, Jaroslaw Byrka, Marek Chrobak +5
The Joint Replenishment Problem (JRP) is a fundamental optimization problem in supply-chain management, concerned with optimizing the flow of goods from a supplier to retailers. Ov…
Dynamics of exponential linear map in functional space
David Gamarnik, Tomasz Nowicki, Grzegorz Swirszcz
We consider the question of existence of a unique invariant probability distribution which satisfies some evolutionary property. The problem arises from the random graph theory but…
Maximum Weight Independent Sets and Matchings in Sparse Random Graphs. Exact Results using the Local Weak Convergence Method
David Gamarnik, Tomasz Nowicki, Grzegorz Swirscsz
Let and be an -node sparse random graph and a sparse random -regular graph, respectively, and let and be the sizes of the…