370 citations · 380 across the 4 of their papers we have counts for
6 papers
Weight Norm Control
Ilya Loshchilov
We note that decoupled weight decay regularization is a particular case of weight norm control where the target norm of weights is set to 0. Any optimization method (e.g., Adam) wh…
Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari
Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter
Evolution Strategies (ES) have recently been demonstrated to be a viable alternative to reinforcement learning (RL) algorithms on a set of challenging deep RL problems, including A…
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter
The original ImageNet dataset is a popular large-scale benchmark for training Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm design, architecture se…
Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization
Ilya Loshchilov, Tobias Glasmachers, Hans-Georg Beyer
The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a popular method to deal with nonconvex and/or stochastic optimization problems when the gradient information is not…
Anytime Bi-Objective Optimization with a Hybrid Multi-Objective CMA-ES (HMO-CMA-ES)
Ilya Loshchilov, Tobias Glasmachers
We propose a multi-objective optimization algorithm aimed at achieving good anytime performance over a wide range of problems. Performance is assessed in terms of the hypervolume m…
Alternative Restart Strategies for CMA-ES
Ilya Loshchilov, Marc Schoenauer, Michèle Sebag
This paper focuses on the restart strategy of CMA-ES on multi-modal functions. A first alternative strategy proceeds by decreasing the initial step-size of the mutation while doubl…