5 citations · 10 across the 21 of their papers we have counts for
10 papers · 1 filter
Neither Adversarial Training Nor Purification: Emergent Adversarial Robustness from Oscillatory Predictive Learning
Mohammed-Yassine Habibi, Klea Ziu, Martin Takáč +1
Adversarial robustness in computer vision is still largely achieved through adversarial training or test-time adversarial purification, both of which introduce significant computat…
Privacy from Symmetry: Orthogonally Equivariant Transformers for LLM Inference
Alexander Yukhimchuk, Andrey Shulga, Mladen Kolar +1
Running large language models locally is often impractical, pushing inference on sensitive text to third-party providers. Split inference partially mitigates this by keeping tokens…
Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1
Gradient clipping is a standard safeguard for training neural networks under noisy, heavy-tailed stochastic gradients; yet, most clipping rules treat all parameters as vectors and…
Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
Artem Riabinin, Andrey Veprikov, Arman Bolatov +2
We study adaptive learning rate scheduling for norm-constrained optimizers (e.g., Muon and Lion). We introduce a generalized smoothness assumption under which local curvature decre…
Learning Latent Graph Geometry via Fixed-Point Schrödinger-Type Activation: A Theoretical Study
Dmitry Pasechnyuk-Vilensky, Martin Takáč
We study neural architectures in which each hidden layer is defined by the stationary state of a dissipative Schrödinger-type dynamics on a learned latent graph. On stable branches…
Bant: Byzantine Antidote via Trial Function and Trust Scores
Gleb Molodtsov, Daniil Medyakov, Sergey Skorik +6
Recent advancements in machine learning have improved performance while also increasing computational demands. While federated and distributed setups address these issues, their st…
Thinking like a CHEMIST: Combined Heterogeneous Embedding Model Integrating Structure and Tokens
Nikolai Rekut, Alexey Orlov, Klea Ziu +3
Representing molecular structures effectively in chemistry remains a challenging task. Language models and graph-based models are extensively utilized within this domain, consisten…
DAG: Projected Stochastic Approximation Iteration for DAG Structure Learning
Klea Ziu, Slavomír Hanzely, Loka Li +3
Learning the structure of Directed Acyclic Graphs (DAGs) presents a significant challenge due to the vast combinatorial search space of possible graphs, which scales exponentially…
Local Methods with Adaptivity via Scaling
Savelii Chezhegov, Sergey Skorik, Nikolas Khachaturov +5
The rapid development of machine learning and deep learning has introduced increasingly complex optimization challenges that must be addressed. Indeed, training modern, advanced mo…
SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
Farshed Abdukhakimov, Chulu Xiang, Dmitry Kamzolov +2
Adaptive optimization methods are widely recognized as among the most popular approaches for training Deep Neural Networks (DNNs). Techniques such as Adam, AdaGrad, and AdaHessian…