17 citations · 20 across the 7 of their papers we have counts for
7 papers
Unsupervised Process Reward Models
Artyom Gadetsky, Maxim Kodryan, Siba Smarak Panigrahi +2
Process Reward Models (PRMs) are a powerful mechanism for steering large language model reasoning by providing fine-grained, step-level supervision. However, this effectiveness com…
Where Do Large Learning Rates Lead Us?
Ildus Sadrtdinov, Maxim Kodryan, Eduard Pokonechny +2
It is generally accepted that starting neural networks training with large learning rates (LRs) improves generalization. Following a line of research devoted to understanding this…
Large Learning Rates Improve Generalization: But How Large Are We Talking About?
Ekaterina Lobacheva, Eduard Pockonechnyy, Maxim Kodryan +1
Inspired by recent research that recommends starting neural networks training with large learning rates (LRs) to achieve the best generalization, we explore this hypothesis in deta…
Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes
Maxim Kodryan, Ekaterina Lobacheva, Maksim Nakhodnov +1
A fundamental property of deep learning normalization techniques, such as batch normalization, is making the pre-normalization parameters scale invariant. The intrinsic domain of s…
On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay
Ekaterina Lobacheva, Maxim Kodryan, Nadezhda Chirkova +2
Training neural networks with batch normalization and weight decay has become a common practice in recent years. In this work, we show that their combined use may result in a surpr…
On Power Laws in Deep Ensembles
Ekaterina Lobacheva, Nadezhda Chirkova, Maxim Kodryan +1
Ensembles of deep neural networks are known to achieve state-of-the-art performance in uncertainty estimation and lead to accuracy improvement. In this work, we focus on a classifi…
MARS: Masked Automatic Ranks Selection in Tensor Decompositions
Maxim Kodryan, Dmitry Kropotov, Dmitry Vetrov
Tensor decomposition methods have proven effective in various applications, including compression and acceleration of neural networks. At the same time, the problem of determining…