Tuning the Scheduling of Distributed Stochastic Gradient Descent with Bayesian Optimization

Valentin Dalibard, Michael Schaarschmidt, Eiko Yoneki

We present an optimizer which uses Bayesian optimization to tune the system parameters of distributed stochastic gradient descent (SGD). Given a specific context, our goal is to qu…