8 citations · 15 across the 6 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
On the Spatial Structure of Mixture-of-Experts in Transformers
Daniel Bershatsky, Ivan Oseledets
A common assumption is that MoE routers primarily leverage semantic features for expert selection. However, our study challenges this notion by demonstrating that positional token…
cs.CL2024★ 1 cited
LoTR: Low Tensor Rank Weight Adaptation
Daniel Bershatsky, Daria Cherniuk, Talgat Daulbaev +2
In this paper we generalize and extend an idea of low-rank adaptation (LoRA) of large language models (LLMs) based on Transformer architecture. Widely used LoRA-like methods of fin…