8 citations · 10 across the 7 of their papers we have counts for
3 papers · 1 filter
Structured Transforms for Low-Overhead Quantization of Language Models
Daria Cherniuk, Alexander Rudikov, Boris Kashin +1
We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient…
CoRoVA: Compressed Representations for Vector-Augmented Code Completion
Daria Cherniuk, Nikita Sukhorukov, Danil Gusak +4
Retrieval-augmented generation has emerged as one of the most effective approaches for code completion enhancement, especially when repository-level context is important. However,…
LoTR: Low Tensor Rank Weight Adaptation
Daniel Bershatsky, Daria Cherniuk, Talgat Daulbaev +2
In this paper we generalize and extend an idea of low-rank adaptation (LoRA) of large language models (LLMs) based on Transformer architecture. Widely used LoRA-like methods of fin…