2.3k citations · 2.9k across the 31 of their papers we have counts for
4 papers · 1 filter
Contextually Guided Transformers via Low-Rank Adaptation
Andrey Zhmoginov, Jihwan Lee, Max Vladymyrov +1
Large Language Models (LLMs) based on Transformers excel at text processing, but their reliance on prompts for specialized behavior introduces computational overhead. We propose a…
Projectable Models: One-Shot Generation of Small Specialized Transformers from Large Ones
Andrey Zhmoginov, Jihwan Lee, Mark Sandler
Modern Foundation Models (FMs) are typically trained on corpora spanning a wide range of different data modalities, topics and downstream tasks. Utilizing these models can be very…
How new data permeates LLM knowledge and how to dilute it
Chen Sun, Renat Aksitov, Andrey Zhmoginov +5
Large language models learn and continually learn through the accumulation of gradient-based updates, but how individual pieces of new information affect existing knowledge, leadin…
Long Context In-Context Compression by Getting to the Gist of Gisting
Aleksandar Petrov, Mark Sandler, Andrey Zhmoginov +2
Long context processing is critical for the adoption of LLMs, but existing methods often introduce architectural complexity that hinders their practical adoption. Gisting, an in-co…