5 citations · 10 across the 21 of their papers we have counts for
11 papers · 1 filter
Optimal Parameter-Free Second-order Acceleration with Inexact Hessians under Generalized Smoothness
Artem Agafonov, Aslan Ashabokov, Alexander D'yakonov +3
We study second-order convex optimization under generalized Hessian smoothness, where the local Hessian Lipschitz constant may grow linearly with the gradient norm. For inexact Hes…
Certified Residual Quasi-Newton Methods for Distributed Variational Inequalities
Ewsey R. Obzherin, Roman M. Mozhaev, Alexander V. Gasnikov +3
Second-order methods for smooth monotone variational inequalities reach the optimal rate , but a distributed exact Jacobian costs times more communication than an…
Application of Optimal Inexact Second-Order Acceleration to Distributed Stochastic Optimization under Statistical Similarity
Yury A. Sokolov, Maxim K. Mashtaler, Alexander V. Gasnikov +3
We consider distributed stochastic convex optimization with a fixed budget of independent samples split among workers. Sample average approximation reduces the problem to a…
Neither Adversarial Training Nor Purification: Emergent Adversarial Robustness from Oscillatory Predictive Learning
Mohammed-Yassine Habibi, Klea Ziu, Martin Takáč +1
Adversarial robustness in computer vision is still largely achieved through adversarial training or test-time adversarial purification, both of which introduce significant computat…
Chebyshev-Exact Acceleration under Hessian Variation, I: Sine-Jacobi Method
Dmitry Pasechnyuk-Vilensky, Martin Takáč
We study finite-horizon one-gradient realizations with the Chebyshev minimax terminal residual on . Under time-dependent Hessian perturbations, the terminal first variation…
Privacy from Symmetry: Orthogonally Equivariant Transformers for LLM Inference
Alexander Yukhimchuk, Andrey Shulga, Mladen Kolar +1
Running large language models locally is often impractical, pushing inference on sensitive text to third-party providers. Split inference partially mitigates this by keeping tokens…
Decentralized Inexact Cubic Newton Method with Consensus Procedure
Artem Agafonov, Anton Novitskii, Alexander Rogozin +5
Distributed optimization is widely used in large-scale and privacy-preserving machine learning, where each agent stores a local objective and communicates only with its neighbors i…
Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1
Gradient clipping is a standard safeguard for training neural networks under noisy, heavy-tailed stochastic gradients; yet, most clipping rules treat all parameters as vectors and…
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by u…
Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
Artem Riabinin, Andrey Veprikov, Arman Bolatov +2
We study adaptive learning rate scheduling for norm-constrained optimizers (e.g., Muon and Lion). We introduce a generalized smoothness assumption under which local curvature decre…
CORE: Collaborative Reasoning via Cross Teaching
Kshitij Mishra, Mirat Aubakirov, Martin Takac +2
Large language models exhibit complementary reasoning errors: on the same instance, one model may succeed with a particular decomposition while another fails. We propose Collaborat…