91 citations · 113 across the 41 of their papers we have counts for
42 papers
Dual Advantage Fields
Alexey Zemtsov, Maxim Bobrin, Alexander Nikulin +5
Offline goal-conditioned reinforcement learning requires both long-horizon reachability estimates and local action comparisons. Dual goal representations provide value fields that…
Heterogeneous-Horizon Exact-Weight Local SGD
Dmitry Pasechnyuk-Vilensky, Martin Takáč
We study adaptive aggregation for heterogeneous local SGD in convex finite-sum optimization, allowing heterogeneous local horizons, minibatch sizes, gradient noise, and participati…
Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models
Yanda Li, Yuhan Liu, Zirui Song +3
Large audio-language models (LALMs) generalize across speech, sound, and music, but unified decoders can exhibit a \emph{temporal smoothing bias}: transient acoustic cues may be un…
Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
Aleksandr Beznosikov, Georgiy Kormakov, Alexander Grigorievskiy +7
The objective of Vertical Federated Learning (VFL) is to collectively train a model using features available on different devices while sharing the same users. This paper focuses o…
Ethic Duality: A Homological Framework for Primal-Dual Problems
Dmitry Pasechnyuk-Vilensky, Martin Takáč
We develop a homological duality framework based on a contravariant functor with dualizing object . A morphism is called ethic when it satisfies th…
Adaptive Regularized Newton Method with Inexact Hessian
Aleksandr Shestakov, Nail Bashirov, Andrei Semenov +4
Newton's method is the most widespread high-order method, demanding the gradient and the Hessian of the objective function. However, one of the main disadvantages of Newtons method…
Moduli space of optimization algorithms
Dmitry Pasechnyuk-Vilensky, Martin Takáč
We introduce a geometric and operator-theoretic formalism viewing optimization algorithms as discrete connections on a space of update operators. Each iterative method is encoded b…
Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
Andrey Veprikov, Arman Bolatov, Aleksandr Bogdanov +4
Optimization lies at the core of modern deep learning, yet existing methods often face a fundamental trade-off between adapting to problem geometry and leveraging curvature utiliza…
Cubic Regularized Newton Method with Variance Reduction for Finite-sum Non-convex Problems
Dmitry Pasechnyuk-Vilensky, Dmitry Kamzolov, Martin Takáč
We study finite-sum non-convex optimization and analyze a variance-reduced cubic Newton method based on EMA-smoo…
Loss-Transformation Invariance in the Damped Newton Method
Alexander Shestakov, Sushil Bohara, Samuel Horváth +2
The Newton method is a powerful optimization algorithm, valued for its rapid local convergence and elegant geometric properties. However, its theoretical guarantees are usually lim…
Who to Trust? Aggregating Client Predictions in Federated Distillation
Viktor Kovalchuk, Denis Son, Arman Bolatov +6
Under data heterogeneity (e.g., ), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combinatio…
Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
Farshed Abdukhakimov, Cuong Anh Pham, Samuel Horváth +2
The Polyak stepsize for Gradient Descent is known for its fast convergence but requires prior knowledge of the optimal functional value, which is often unavailable in practice. In…
What Scalable Second-Order Information Knows for Pruning at Initialization
Ivo Gollini Navarrete, Nicolás Mauricio Cuadrado Ávila, Martin Takáč +1
Pruning remains an effective strategy for reducing both the costs and environmental impact associated with deploying large neural networks (NNs) while maintaining performance. Clas…
RECALL: Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles
Munachiso Nwadike, Zangir Iklassov, Toluwani Aremu +6
We introduce the concept of the self-referencing causal cycle (abbreviated RECALL) - a mechanism that enables large language models (LLMs) to bypass the limitations of unidirection…
Generalising Battery Control in Net-Zero Buildings via Personalised Federated RL
Nicolas M Cuadrado Avila, Samuel Horváth, Martin Takáč
This work studies the challenge of optimal energy management in building-based microgrids through a collaborative and privacy-preserving framework. We evaluated two common RL algor…
Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under -Smoothness
Aleksandr Lobanov, Alexander Gasnikov, Eduard Gorbunov +1
The gradient descent (GD) method -- is a fundamental and likely the most popular optimization algorithm in machine learning (ML), with a history traced back to a paper in 1847 (Cau…
Methods for Convex -Smooth Optimization: Clipping, Acceleration, and Adaptivity
Eduard Gorbunov, Nazarii Tupitsa, Sayantan Choudhury +4
Due to the non-smoothness of optimization problems in Machine Learning, generalized smoothness assumptions have been gaining a lot of attention in recent years. One of the most pop…
MirrorCheck: Efficient Adversarial Defense for Vision-Language Models
Samar Fares, Klea Ziu, Toluwani Aremu +5
Vision-Language Models (VLMs) are increasingly susceptible to sophisticated adversarial attacks, including adaptive strategies specifically designed to bypass existing defenses. To…
Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
Savelii Chezhegov, Yaroslav Klyukin, Andrei Semenov +5
Methods with adaptive stepsizes, such as AdaGrad and Adam, are essential for training modern Deep Learning models, especially Large Language Models. Typically, the noise in the sto…
Exploring Jacobian Inexactness in Second-Order Methods for Variational Inequalities: Lower Bounds, Optimal Algorithms and Quasi-Newton Approximations
Artem Agafonov, Petr Ostroukhov, Roman Mozhaev +5
Variational inequalities represent a broad class of problems, including minimization and min-max problems, commonly found in machine learning. Existing second-order and high-order…
Newton Method Revisited: Global Convergence Rates up to for Stepsize Schedules and Linesearch Procedures
Slavomír Hanzely, Farshed Abdukhakimov, Martin Takáč
This paper investigates the global convergence of stepsized Newton methods for convex functions with Hölder continuous Hessians or third derivatives. We propose several simple step…
Enhancing Policy Gradient with the Polyak Step-Size Adaption
Yunxiang Li, Rui Yuan, Chen Fan +4
Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other…
Federated Learning Can Find Friends That Are Advantageous
Nazarii Tupitsa, Samuel Horváth, Martin Takáč +1
In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges. While collaboration among clients can significantly e…
Advancing the lower bounds: An accelerated, stochastic, second-order method with optimal adaptation to inexactness
Artem Agafonov, Dmitry Kamzolov, Alexander Gasnikov +4
We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theore…