80 citations · 197 across the 25 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
cs.LG2019★ 43 cited
Optimizing Millions of Hyperparameters by Implicit Differentiation
Jonathan Lorraine, Paul Vicol, David Duvenaud
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations.…
cs.LG2019★ 29 cited
Understanding Neural Architecture Search Techniques
George Adam, Jonathan Lorraine
Automatic methods for generating state-of-the-art neural network architectures without human experts have generated significant attention recently. This is because of the potential…
cs.LG2019★ 29 cited
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
Matthew MacKay, Paul Vicol, Jon Lorraine +2
Hyperparameter optimization can be formulated as a bilevel optimization problem, where the optimal parameters on the training set depend on the hyperparameters. We aim to adapt reg…