8 citations · 13 across the 9 of their papers we have counts for
4 papers · 1 filter
Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
Alexey Kravatskiy, Ivan Kozyrev, Nikolai Kozlov +3
In this article, we explore the use of various matrix norms for optimizing functions of weight matrices, a crucial problem in deep learning. Moving beyond the spectral norm that un…
Another approach to build Lyapunov functions for the first order methods in the quadratic case
Daniil Merkulov, Ivan Oseledets
Lyapunov functions play a fundamental role in analyzing the stability and convergence properties of optimization methods. In this paper, we propose a novel and straightforward appr…
NAG-GS: A Semi-Implicit Stochastic Optimizer with Large-Step Stability
Valentin Leplat, Daniil Merkulov, Aleksandr Katrutsa +3
Stochastic first-order methods are widely used for large-scale optimization and machine learning, where their performance often depends on the learning rate. We study NAG-GS, a sto…
Follow the bisector: a simple method for multi-objective optimization
Alexandr Katrutsa, Daniil Merkulov, Nurislam Tursynbek +1
This study presents a novel Equiangular Direction Method (EDM) to solve a multi-objective optimization problem. We consider optimization problems, where multiple differentiable los…