4 papers
CoRoVA: Compressed Representations for Vector-Augmented Code Completion
Daria Cherniuk, Nikita Sukhorukov, Danil Gusak +4
Retrieval-augmented generation has emerged as one of the most effective approaches for code completion enhancement, especially when repository-level context is important. However,…
Cluster Topology-Driven Placement of Experts Reduces Network Traffic in MoE Inference
Danil Sivtsov, Aleksandr Katrutsa, Ivan Oseledets
Efficient deployment of a pre-trained LLM to a cluster with multiple servers is a critical step for providing fast responses to users' queries. The recent success of Mixture-of-Exp…
Diagonal Batching Unlocks Parallelism in Recurrent Memory Transformers for Long Contexts
Danil Sivtsov, Ivan Rodkin, Gleb Kuzmin +2
Transformer models struggle with long-context inference due to their quadratic time and linear memory complexity. Recurrent Memory Transformers (RMTs) offer a solution by reducing…
Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
Alexander Tyurin, Danil Sivtsov
We propose a new unifying framework, Birch SGD, for analyzing and designing distributed SGD methods. The central idea is to represent each method as a weighted directed tree, refer…