58 citations · 110 across the 9 of their papers we have counts for
10 papers
On a continuous time model of gradient descent dynamics and instability in deep learning
Mihaela Rosca, Yan Wu, Chongli Qin +1
The recipe behind the success of deep learning has been the combination of neural networks and gradient-based optimization. Understanding the behavior of gradient descent however,…
Towards Learning Universal Audio Representations
Luyu Wang, Pauline Luc, Yan Wu +8
The ability to learn universal audio representations that can solve diverse speech, music, and environment tasks can spur many applications that require general sound content under…
Discretization Drift in Two-Player Games
Mihaela Rosca, Yan Wu, Benoit Dherin +1
Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity ori…
Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory
Jason Ramapuram, Yan Wu, Alexandros Kalousis
Episodic and semantic memory are critical components of the human memory model. The theory of complementary learning systems (McClelland et al., 1995) suggests that the compressed…
Training Generative Adversarial Networks by Solving Ordinary Differential Equations
Chongli Qin, Yan Wu, Jost Tobias Springenberg +4
The instability of Generative Adversarial Network (GAN) training has frequently been attributed to gradient descent. Consequently, recent methods have aimed to tailor the models an…
Shaping Belief States with Generative Environment Models for RL
Karol Gregor, Danilo Jimenez Rezende, Frederic Besse +3
When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressiv…
Deep Compressed Sensing
Yan Wu, Mihaela Rosca, Timothy Lillicrap
Compressed sensing (CS) provides an elegant framework for recovering sparse signals from compressed measurements. For example, CS can exploit the structure of natural images and re…
Learning Attractor Dynamics for Generative Memory
Yan Wu, Greg Wayne, Karol Gregor +1
A central challenge faced by memory systems is the robust retrieval of a stored pattern in the presence of interference due to other stored patterns and noise. A theoretically well…
Optimizing Agent Behavior over Long Time Scales by Transporting Value
Chia-Chun Hung, Timothy Lillicrap, Josh Abramson +5
Humans spend a remarkable fraction of waking life engaged in acts of "mental time travel". We dwell on our actions in the past and experience satisfaction or regret. More than mere…
The Kanerva Machine: A Generative Distributed Memory
Yan Wu, Greg Wayne, Alex Graves +1
We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them. Inspired by Kanerva's sparse distributed memory, it has a robust dis…