Showing cs.LGShow all
3 papers · 1 filter
cs.LG2021
Neurons learn slower than they think
Ilona Kulikovskikh
Recent studies revealed complex convergence dynamics in gradient-based methods, which has been little understood so far. Changing the step size to balance between high convergence…
cs.LG2021
Painless step size adaptation for SGD
Ilona Kulikovskikh, Tarzan Legović
Convergence and generalization are two crucial aspects of performance in neural networks. When analyzed separately, these properties may lead to contradictory results. Optimizing a…
cs.LG2020
Why to "grow" and "harvest" deep learning models?
Ilona Kulikovskikh, Tarzan Legović
Current expectations from training deep learning models with gradient-based methods include: 1) transparency; 2) high convergence rates; 3) high inductive biases. While the state-o…