43 citations · 74 across the 4 of their papers we have counts for
8 papers
Amortized Proximal Optimization
Juhan Bae, Paul Vicol, Jeff Z. HaoChen +1
We propose a framework for online meta-optimization of parameters that govern optimization, called Amortized Proximal Optimization (APO). We first interpret various existing neural…
Complex Momentum for Optimization in Games
Jonathan Lorraine, David Acuna, Paul Vicol +1
We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving conv…
Cost-Efficient Online Hyperparameter Optimization
Jingkang Wang, Mengye Ren, Ilija Bogunovic +2
Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters. However, these online HPO algorith…
Optimizing Millions of Hyperparameters by Implicit Differentiation
Jonathan Lorraine, Paul Vicol, David Duvenaud
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations.…
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
Matthew MacKay, Paul Vicol, Jon Lorraine +2
Hyperparameter optimization can be formulated as a bilevel optimization problem, where the optimal parameters on the training set depend on the hyperparameters. We aim to adapt reg…
Reversible Recurrent Neural Networks
Matthew MacKay, Paul Vicol, Jimmy Ba +1
Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which…