538 citations · 1.1k across the 31 of their papers we have counts for
7 papers · 1 filter
Reducing the variance in online optimization by transporting past gradients
Sébastien M. R. Arnold, Pierre-Antoine Manzagol, Reza Babanezhad +2
Most stochastic optimization methods use gradients once before discarding them. While variance reduction methods have shown that reusing past gradients can be beneficial when there…
On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas, Fabian Pedregosa, Bart van Merriënboer +3
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most pre…
Anytime Tail Averaging
Nicolas Le Roux
Tail averaging consists in averaging the last examples in a stream. Common techniques either have a memory requirement which grows with the number of samples to average, are not av…
Distributional reinforcement learning with linear function approximation
Marc G. Bellemare, Nicolas Le Roux, Pablo Samuel Castro +1
Despite many algorithmic advances, our theoretical understanding of practical distributional reinforcement learning methods remains limited. One exception is Rowland et al. (2018)'…
Negative eigenvalues of the Hessian in deep neural networks
Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol
The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landsc…
A Geometric Perspective on Optimal Representations for Reinforcement Learning
Marc G. Bellemare, Will Dabney, Robert Dadashi +6
We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functions. We leverage this perspective to pro…
The Value Function Polytope in Reinforcement Learning
Robert Dadashi, Adrien Ali Taïga, Nicolas Le Roux +2
We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of…