996 citations · 2.4k across the 62 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2020★ 1 cited
On regularization of gradient descent, layer imbalance and flat minima
Boris Ginsburg
We analyze the training dynamics for deep linear networks using a new metric - layer imbalance - which defines the flatness of a solution. We demonstrate that different regularizat…
cs.LG2019★ 177 cited
NeMo: a toolkit for building AI applications using Neural Modules
Oleksii Kuchaiev, Jason Li, Huyen Nguyen +11
NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition. NeMo is built around neural modules, c…
cs.LG2019★ 83 cited
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks
Boris Ginsburg, Patrice Castonguay, Oleksii Hrinchuk +7
We propose NovoGrad, an adaptive stochastic gradient descent method with layer-wise gradient normalization and decoupled weight decay. In our experiments on neural networks for ima…