2 citations · 5 across the 15 of their papers we have counts for
15 papers
Progressive Cramming: Reliable Token Compression and What It Reveals
Dmitrii Tarasov, Timofei Lashukov, Elizaveta Goncharova +1
Token cramming compresses sequences into learned embeddings with near-perfect reconstruction, but fixed token budgets and 99\% accuracy thresholds leave it unclear whether residual…
NoReGeo: Non-Reasoning Geometry Benchmark
Irina Abdullaeva, Anton Vasiliuk, Elizaveta Goncharova +4
We present NoReGeo, a novel benchmark designed to evaluate the intrinsic geometric understanding of large language models (LLMs) without relying on reasoning or algebraic computati…
Simple Vision-Language Math Reasoning via Rendered Text
Matvey Skripkin, Elizaveta Goncharova, Andrey Kuznetsov
We present a lightweight yet effective pipeline for training vision-language models to solve math problems by rendering LaTeX encoded equations into images and pairing them with st…
Sentence-Anchored Gist Compression for Long-Context LLMs
Dmitrii Tarasov, Elizaveta Goncharova, Kuznetsov Andrey
This work investigates context compression for Large Language Models (LLMs) using learned compression tokens to reduce the memory and computational demands of processing long seque…
SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
Nikita Dragunov, Temurbek Rahmatullaev, Elizaveta Goncharova +5
The recently proposed Large Concept Model (LCM) generates text by predicting a sequence of sentence-level embeddings and training with either mean-squared error or diffusion object…
Image Reconstruction as a Tool for Feature Analysis
Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova +1
Vision encoders are increasingly used in modern applications, from vision-only models to multimodal systems such as vision-language models. Despite their remarkable success, it rem…
When Less is Enough: Adaptive Token Reduction for Efficient Image Representation
Eduard Allakhverdov, Elizaveta Goncharova, Andrey Kuznetsov
Vision encoders typically generate a large number of visual tokens, providing information-rich representations but significantly increasing computational demands. This raises the q…
MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing
Matvey Skripkin, Elizaveta Goncharova, Dmitrii Tarasov +1
Multimodal language models (MLMs) integrate visual and textual information by coupling a vision encoder with a large language model through the specific adapter. While existing app…
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4
We introduce methods to quantify how Large Language Models (LLMs) encode and store contextual information, revealing that tokens often seen as minor (e.g., determiners, punctuation…
Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
Viktoriia Chekalina, Anton Razzhigaev, Elizaveta Goncharova +1
In this paper we present an approach to reduce hallucinations in Large Language Models (LLMs) by incorporating Knowledge Graphs (KGs) as an additional modality. Our method involves…
Your Transformer is Secretly Linear
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova +4
This paper reveals a novel linear characteristic exclusive to transformer decoders, including models such as GPT, LLaMA, OPT, BLOOM and others. We analyze embedding transformations…
OmniFusion Technical Report
Elizaveta Goncharova, Anton Razzhigaev, Matvey Mikhalchuk +6
Last year, multimodal architectures served up a revolution in AI-based approaches and solutions, extending the capabilities of large language models (LLM). We propose an \textit{Om…
MERA: A Comprehensive LLM Evaluation in Russian
Alena Fenogenova, Artem Chervyakov, Nikita Martynov +16
Over the past few years, one of the most notable advancements in AI research has been in foundation models (FMs), headlined by the rise of language models (LMs). As the models' siz…
LM-Polygraph: Uncertainty Estimation for Language Models
Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun +9
Recent advancements in the capabilities of large language models (LLMs) have paved the way for a myriad of groundbreaking applications in various fields. However, a significant cha…
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova +3
In this study, we present an investigation into the anisotropy dynamics and intrinsic dimension of embeddings in transformer architectures, focusing on the dichotomy between encode…