565 citations · 905 across the 19 of their papers we have counts for
3 papers · 1 filter
On Implicit Bias in Overparameterized Bilevel Optimization
Paul Vicol, Jonathan Lorraine, Fabian Pedregosa +2
Many problems in machine learning involve bilevel optimization (BLO), including hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems consist of tw…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Amortized Proximal Optimization
Juhan Bae, Paul Vicol, Jeff Z. HaoChen +1
We propose a framework for online meta-optimization of parameters that govern optimization, called Amortized Proximal Optimization (APO). We first interpret various existing neural…