9.4k citations · 11.7k across the 21 of their papers we have counts for
3 papers · 1 filter
Decoupled Weight Decay Regularization
Ilya Loshchilov, Frank Hutter
L regularization and weight decay regularization are equivalent for standard stochastic gradient descent (when rescaled by the learning rate), but as we demonstrate this is \em…
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter
The original ImageNet dataset is a popular large-scale benchmark for training Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm design, architecture se…
Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization
Ilya Loshchilov, Tobias Glasmachers, Hans-Georg Beyer
The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a popular method to deal with nonconvex and/or stochastic optimization problems when the gradient information is not…