2k citations · 7.2k across the 38 of their papers we have counts for
3 papers · 1 filter
DataRater: Meta-Learned Dataset Curation
Dan A. Calian, Gregory Farquhar, Iurii Kemaev +9
The quality of foundation models depends heavily on their training data. Consequently, great efforts have been put into dataset curation. Yet most approaches rely on manual tuning…
No More Pesky Learning Rates
Tom Schaul, Sixin Zhang, Yann LeCun
The performance of stochastic gradient descent (SGD) depends critically on how learning rates are tuned and decreased over time. We propose a method to automatically adjust multipl…
Natural Evolution Strategies
Daan Wierstra, Tom Schaul, Tobias Glasmachers +2
This paper presents Natural Evolution Strategies (NES), a recent family of algorithms that constitute a more principled approach to black-box optimization than established evolutio…