10 papers
Controlling Refusal Behavior of LLMs via Stiefel-Constrained Rotation Steering
Kirill Bunin, Dmitry Bylinkin, Vladimir Aletov +3
Activation steering has emerged as a lightweight approach for controlling model refusal at inference time. A growing line of research explores trainable rotations of activations to…
Leveraging Association Context Retrieval in Knowledge Edit- ing to Build White-Box Attacks on LLMs
Roman Maksimov, Vladimir Aletov, Vladimir Solodkin +3
As large language models (LLMs) are granted increasing autonomy, it is essential to investigate methods that can induce unsafe behavior. We propose a novel white-box attack inspire…
The Kuramoto Neural Operator: Learning to Solve PDEs via Coupled Oscillator Dynamics
Petr Badolia, Leonid Obukhov, Dmitry Bylinkin +1
Operator learning is a rapidly advancing area of computational science. It is particularly well suited to problems where a partial differential equation (PDE) must be solved repeat…
Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning
Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin +1
Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specifi…
Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin +3
Knowledge editing (KE) provides a lightweight alternative to repeated fine-tuning of LLMs. However, most existing KE methods target dense feed-forward layers, while modern LLMs inc…
Accelerated Methods with Complexity Separation Under Data Similarity for Federated Learning Problems
Dmitry Bylinkin, Sergey Skorik, Dmitriy Bystrov +3
Heterogeneity within data distribution poses a challenge in many modern federated learning tasks. We formalize it as an optimization problem involving a computationally heavy compo…
Communication-Efficient Federated Learning with Adaptive Number of Participants
Sergey Skorik, Vladislav Dorofeev, Gleb Molodtsov +4
Rapid scaling of deep learning models has enabled performance gains across domains, yet it introduced several challenges. Federated Learning (FL) has emerged as a promising framewo…
Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
Dmitry Bylinkin, Mikhail Aleksandrov, Savelii Chezhegov +1
Physics-informed neural networks (PINNs) have gained prominence in recent years and are now effectively used in a number of applications. However, their performance remains unstabl…
Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similarity
Dmitry Bylinkin, Aleksandr Beznosikov
In recent years, as data and problem sizes have increased, distributed learning has become an essential tool for training high-performance models. However, the communication bottle…
Accelerated Stochastic ExtraGradient: Mixing Hessian and Gradient Similarity to Reduce Communication in Distributed and Federated Learning
Dmitry Bylinkin, Kirill Degtyarev, Aleksandr Beznosikov
Modern realities and trends in learning require more and more generalization ability of models, which leads to an increase in both models and training sample size. It is already di…