10 papers
Progressive Cramming: Reliable Token Compression and What It Reveals
Dmitrii Tarasov, Timofei Lashukov, Elizaveta Goncharova +1
Token cramming compresses sequences into learned embeddings with near-perfect reconstruction, but fixed token budgets and 99\% accuracy thresholds leave it unclear whether residual…
SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
Nikita Dragunov, Temurbek Rahmatullaev, Elizaveta Goncharova +5
The recently proposed Large Concept Model (LCM) generates text by predicting a sequence of sentence-level embeddings and training with either mean-squared error or diffusion object…
NoReGeo: Non-Reasoning Geometry Benchmark
Irina Abdullaeva, Anton Vasiliuk, Elizaveta Goncharova +4
We present NoReGeo, a novel benchmark designed to evaluate the intrinsic geometric understanding of large language models (LLMs) without relying on reasoning or algebraic computati…
Simple Vision-Language Math Reasoning via Rendered Text
Matvey Skripkin, Elizaveta Goncharova, Andrey Kuznetsov
We present a lightweight yet effective pipeline for training vision-language models to solve math problems by rendering LaTeX encoded equations into images and pairing them with st…
Sentence-Anchored Gist Compression for Long-Context LLMs
Dmitrii Tarasov, Elizaveta Goncharova, Kuznetsov Andrey
This work investigates context compression for Large Language Models (LLMs) using learned compression tokens to reduce the memory and computational demands of processing long seque…
Image Reconstruction as a Tool for Feature Analysis
Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova +1
Vision encoders are increasingly used in modern applications, from vision-only models to multimodal systems such as vision-language models. Despite their remarkable success, it rem…