5 papers
Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers
Andrei Chertkov, Artem Basharin, Mikhail Saygin +3
The growing demand for energy-efficient, high-performance AI systems has led to increased attention on alternative computing platforms (e.g., photonic, neuromorphic) due to their p…
Global Optimization of Atomic Clusters via Physically-Constrained Tensor Train Decomposition
Konstantin Sozykin, Nikita Rybin, Andrei Chertkov +5
The global optimization of atomic clusters represents a fundamental challenge in computational chemistry and materials science due to the exponential growth of local minima with sy…
Tensor Train Decomposition for Adversarial Attacks on Computer Vision Models
Andrei Chertkov, Ivan Oseledets
Deep neural networks (DNNs) are widely used today, but they are vulnerable to adversarial attacks. To develop effective methods of defense, it is important to understand the potent…
High-dimensional Optimization with Low Rank Tensor Sampling and Local Search
Konstantin Sozykin, Andrei Chertkov, Anh-Huy Phan +2
We present a novel method called TESALOCS (TEnsor SAmpling and LOCal Search) for multidimensional optimization, combining the strengths of gradient-free discrete methods and gradie…
Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition
Artem Basharin, Andrei Chertkov, Ivan Oseledets
We propose a new model for multi-token prediction in transformers, aiming to enhance sampling efficiency without compromising accuracy. Motivated by recent work that predicts the p…