3 papers
cs.LG2025
Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers
Andrei Chertkov, Artem Basharin, Mikhail Saygin +3
The growing demand for energy-efficient, high-performance AI systems has led to increased attention on alternative computing platforms (e.g., photonic, neuromorphic) due to their p…
cs.LG2024
Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition
Artem Basharin, Andrei Chertkov, Ivan Oseledets
We propose a new model for multi-token prediction in transformers, aiming to enhance sampling efficiency without compromising accuracy. Motivated by recent work that predicts the p…
cs.LG2024
Black-Box Approximation and Optimization with Hierarchical Tucker Decomposition
Gleb Ryzhakov, Andrei Chertkov, Artem Basharin +1
We develop a new method HTBB for the multidimensional black-box approximation and gradient-free optimization, which is based on the low-rank hierarchical Tucker decomposition with…