1k citations · 1.9k across the 49 of their papers we have counts for
5 papers · 1 filter
On Empirical Comparisons of Optimizers for Deep Learning
Dami Choi, Christopher J. Shallue, Zachary Nado +3
Selecting an optimizer is a central step in the contemporary deep learning pipeline. In this paper, we demonstrate the sensitivity of optimizer comparisons to the hyperparameter tu…
Direct Policy Gradients: Direct Optimization of Policies in Discrete Action Spaces
Guy Lorberbom, Chris J. Maddison, Nicolas Heess +2
Direct optimization is an appealing framework that replaces integration with optimization of a random objective for approximating gradients in models with discrete random variables…
Hamiltonian descent for composite objectives
Brendan O'Donoghue, Chris J. Maddison
In optimization the duality gap between the primal and the dual problems is a measure of the suboptimality of any primal-dual point. In classical mechanics the equations of motion…
Dual Space Preconditioning for Gradient Descent
Chris J. Maddison, Daniel Paulin, Yee Whye Teh +1
The conditions of relative smoothness and relative strong convexity were recently introduced for the analysis of Bregman gradient methods for convex optimization. We introduce a ge…
Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
Emile Mathieu, Charline Le Lan, Chris J. Maddison +2
The variational auto-encoder (VAE) is a popular method for learning a generative model and embeddings of the data. Many real datasets are hierarchically structured. However, tradit…