91 citations · 113 across the 41 of their papers we have counts for
14 papers · 1 filter
Dual Advantage Fields
Alexey Zemtsov, Maxim Bobrin, Alexander Nikulin +5
Offline goal-conditioned reinforcement learning requires both long-horizon reachability estimates and local action comparisons. Dual goal representations provide value fields that…
Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
Andrey Veprikov, Arman Bolatov, Aleksandr Bogdanov +4
Optimization lies at the core of modern deep learning, yet existing methods often face a fundamental trade-off between adapting to problem geometry and leveraging curvature utiliza…
Who to Trust? Aggregating Client Predictions in Federated Distillation
Viktor Kovalchuk, Denis Son, Arman Bolatov +6
Under data heterogeneity (e.g., ), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combinatio…
What Scalable Second-Order Information Knows for Pruning at Initialization
Ivo Gollini Navarrete, Nicolás Mauricio Cuadrado Ávila, Martin Takáč +1
Pruning remains an effective strategy for reducing both the costs and environmental impact associated with deploying large neural networks (NNs) while maintaining performance. Clas…
Generalising Battery Control in Net-Zero Buildings via Personalised Federated RL
Nicolas M Cuadrado Avila, Samuel Horváth, Martin Takáč
This work studies the challenge of optimal energy management in building-based microgrids through a collaborative and privacy-preserving framework. We evaluated two common RL algor…
Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
Savelii Chezhegov, Yaroslav Klyukin, Andrei Semenov +5
Methods with adaptive stepsizes, such as AdaGrad and Adam, are essential for training modern Deep Learning models, especially Large Language Models. Typically, the noise in the sto…
Enhancing Policy Gradient with the Polyak Step-Size Adaption
Yunxiang Li, Rui Yuan, Chen Fan +4
Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other…
Federated Learning Can Find Friends That Are Advantageous
Nazarii Tupitsa, Samuel Horváth, Martin Takáč +1
In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges. While collaboration among clients can significantly e…
SP2: A Second Order Stochastic Polyak Method
Shuang Li, William J. Swartworth, Martin Takáč +2
Recently the "SP" (Stochastic Polyak step size) method has emerged as a competitive adaptive method for setting the step sizes of SGD. SP can be interpreted as a method specialized…
Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information
Majid Jahani, Sergey Rusakov, Zheng Shi +3
We present a novel adaptive optimization algorithm for large-scale machine learning problems. Equipped with a low-cost estimate of local curvature and Lipschitz smoothness, our met…
Decentralized Personalized Federated Learning for Min-Max Problems
Ekaterina Borodich, Aleksandr Beznosikov, Abdurakhmon Sadiev +4
Personalized Federated Learning (PFL) has witnessed remarkable advancements, enabling the development of innovative machine learning applications that preserve the privacy of train…
AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods
Zheng Shi, Abdurakhmon Sadiev, Nicolas Loizou +2
We present AI-SARAH, a practical variant of SARAH. As a variant of SARAH, this algorithm employs the stochastic recursive gradient yet adjusts step-size based on local geometry. AI…
DynNet: Physics-based neural architecture design for linear and nonlinear structural response modeling and prediction
Soheil Sadeghi Eshkevari, Martin Takáč, Shamim N. Pakzad +1
Data-driven models for predicting dynamic responses of linear and nonlinear systems are of great importance due to their wide application from probabilistic analysis to inverse pro…
Distributed Fixed Point Methods with Compressed Iterates
Sélim Chraibi, Ahmed Khaled, Dmitry Kovalev +3
We propose basic and natural assumptions under which iterative optimization methods with compressed iterates can be analyzed. This problem is motivated by the practice of federated…