2 citations · 9 across the 10 of their papers we have counts for
10 papers
Structured Transforms for Low-Overhead Quantization of Language Models
Daria Cherniuk, Alexander Rudikov, Boris Kashin +1
We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient…
Deep Learning for Subspace Regression
Vladimir Fanaskov, Vladislav Trifonov, Alexander Rudikov +2
It is often possible to perform reduced order modelling by specifying linear subspace which accurately captures the dynamics of the system. This approach becomes especially appeali…
Locally Subspace-Informed Neural Operators for Efficient Multiscale PDE Solving
Alexander Rudikov, Vladimir Fanaskov, Sergei Stepanov +4
Neural operators (NOs) struggle with high-contrast multiscale partial differential equations (PDEs), where fine-scale heterogeneities cause large errors. To address this, we use th…
ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equations
Vladislav Trifonov, Alexander Rudikov, Oleg Iliev +3
We present ConDiff, a novel dataset for scientific machine learning. ConDiff focuses on the parametric diffusion equation with space dependent coefficients, a fundamental problem i…
Astral: training physics-informed neural networks with error majorants
Vladimir Fanaskov, Tianchi Yu, Alexander Rudikov +1
The primal approach to physics-informed learning is a residual minimization. We argue that residual is, at best, an indirect measure of the error of approximate solution and propos…
Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers
Vladislav Trifonov, Alexander Rudikov, Oleg Iliev +3
Large linear systems are ubiquitous in modern computational science and engineering. The main recipe for solving them is the use of Krylov subspace iterative methods with well-desi…
Quantization of Large Language Models with an Overdetermined Basis
Daniil Merkulov, Daria Cherniuk, Alexander Rudikov +4
In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation. This approach hinges on decomposing any given vector, matrix, or te…
Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving
Alexander Rudikov, Vladimir Fanaskov, Ekaterina Muravleva +2
Deep learning solvers for partial differential equations typically have limited accuracy. We propose to overcome this problem by using them as preconditioners. More specifically, w…
Neural functional a posteriori error estimates
Vladimir Fanaskov, Alexander Rudikov, Ivan Oseledets
We propose a new loss function for supervised and physics-informed training of neural networks and operators that incorporates a posteriori error estimate. More specifically, durin…
General Covariance Data Augmentation for Neural PDE Solvers
Vladimir Fanaskov, Tianchi Yu, Alexander Rudikov +1
The growing body of research shows how to replace classical partial differential equation (PDE) integrators with neural networks. The popular strategy is to generate the input-outp…