248 citations · 651 across the 15 of their papers we have counts for
15 papers
Neural Compression of Atmospheric States
Piotr Mirowski, David Warde-Farley, Mihaela Rosca +7
Atmospheric states derived from reanalysis comprise a substantial portion of weather and climate simulation outputs. Many stakeholders -- such as researchers, policy makers, and in…
Implicit regularisation in stochastic gradient descent: from single-objective to two-player games
Mihaela Rosca, Marc Peter Deisenroth
Recent years have seen many insights on deep learning optimisation being brought forward by finding implicit regularisation effects of commonly used gradient-based optimisers. Unde…
Investigating the Edge of Stability Phenomenon in Reinforcement Learning
Rares Iordan, Marc Peter Deisenroth, Mihaela Rosca
Recent progress has been made in understanding optimisation dynamics in neural networks trained with full-batch gradient descent with momentum with the uncovering of the edge of st…
On a continuous time model of gradient descent dynamics and instability in deep learning
Mihaela Rosca, Yan Wu, Chongli Qin +1
The recipe behind the success of deep learning has been the combination of neural networks and gradient-based optimization. Understanding the behavior of gradient descent however,…
Discretization Drift in Two-Player Games
Mihaela Rosca, Yan Wu, Benoit Dherin +1
Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity ori…
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective
Florin Gogianu, Tudor Berariu, Mihaela Rosca +3
Most of the recent deep reinforcement learning advances take an RL-centric perspective and focus on refinements of the training objective. We diverge from this view and show we can…
A case for new neural network smoothness constraints
Mihaela Rosca, Theophane Weber, Arthur Gretton +1
How sensitive should machine learning models be to input changes? We tackle the question of model smoothness and show that it is a useful inductive bias which aids generalization,…
Monte Carlo Gradient Estimation in Machine Learning
Shakir Mohamed, Mihaela Rosca, Michael Figurnov +1
This paper is a broad and accessible survey of the methods we have at our disposal for Monte Carlo gradient estimation in machine learning and across the statistical sciences: the…
Deep Compressed Sensing
Yan Wu, Mihaela Rosca, Timothy Lillicrap
Compressed sensing (CS) provides an elegant framework for recovering sparse signals from compressed measurements. For example, CS can exploit the structure of natural images and re…
Training language GANs from Scratch
Cyprien de Masson d'Autume, Mihaela Rosca, Jack Rae +1
Generative Adversarial Networks (GANs) enjoy great success at image generation, but have proven difficult to train in the domain of natural language. Challenges with gradient estim…
Learning Implicit Generative Models with the Method of Learned Moments
Suman Ravuri, Shakir Mohamed, Mihaela Rosca +1
We propose a method of moments (MoM) algorithm for training large-scale implicit generative models. Moment estimation in this setting encounters two problems: it is often difficult…
Distribution Matching in Variational Inference
Mihaela Rosca, Balaji Lakshminarayanan, Shakir Mohamed
With the increasingly widespread deployment of generative models, there is a mounting need for a deeper understanding of their behaviors and limitations. In this paper, we expose t…
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
William Fedus, Mihaela Rosca, Balaji Lakshminarayanan +3
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is…
Variational Approaches for Auto-Encoding Generative Adversarial Networks
Mihaela Rosca, Balaji Lakshminarayanan, David Warde-Farley +1
Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given…
Sequence-to-sequence neural network models for transliteration
Mihaela Rosca, Thomas Breuel
Transliteration is a key component of machine translation systems and software internationalization. This paper demonstrates that neural sequence-to-sequence models obtain state of…