330 citations · 373 across the 33 of their papers we have counts for
8 papers · 1 filter
Inverted Activations: Reducing Memory Footprint in Neural Network Training
Georgii Novikov, Ivan Oseledets
The scaling of neural networks with increasing data and model sizes necessitates the development of more efficient deep learning algorithms. A significant challenge in neural netwo…
Quantization of Large Language Models with an Overdetermined Basis
Daniil Merkulov, Daria Cherniuk, Alexander Rudikov +4
In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation. This approach hinges on decomposing any given vector, matrix, or te…
Black-Box Approximation and Optimization with Hierarchical Tucker Decomposition
Gleb Ryzhakov, Andrei Chertkov, Artem Basharin +1
We develop a new method HTBB for the multidimensional black-box approximation and gradient-free optimization, which is based on the low-rank hierarchical Tucker decomposition with…
TQCompressor: improving tensor decomposition methods in neural networks via permutations
V. Abronin, A. Naumov, D. Mazur +7
We introduce TQCompressor, a novel method for neural network model compression with improved tensor decompositions. We explore the challenges posed by the computational and storage…
Sparse and Transferable Universal Singular Vectors Attack
Kseniia Kuvshinova, Olga Tsymboi, Ivan Oseledets
The research in the field of adversarial attacks and models' vulnerability is one of the fundamental directions in modern machine learning. Recent studies reveal the vulnerability…
Quantization Aware Factorization for Deep Neural Network Compression
Daria Cherniuk, Stanislav Abukhovich, Anh-Huy Phan +3
Tensor decomposition of convolutional and fully-connected layers is an effective way to reduce parameters and FLOP in neural networks. Due to memory and power consumption limitatio…