1 citations · 1 across the 4 of their papers we have counts for
4 papers
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko +2
Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many mode…
OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models
Ali Aliev, Kamil Garifullin, Nikolay Yudin +5
In a rapidly growing field of model training there is a constant practical interest in parameter-efficient fine-tuning and various techniques that use a small amount of training da…
Optimization on the Extended Tensor-Train Manifold with Shared Factors
Alexander Molozhavenko, Maxim Rakhuba
This paper studies tensors that admit decomposition in the Extended Tensor Train (ETT) format, with a key focus on the case where some decomposition factors are constrained to be e…
LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko +4
This work presents a novel, fully Riemannian framework for Low-Rank Adaptation (LoRA) that geometrically treats low-rank adapters by optimizing them directly on the fixed-rank mani…