7 papers
Progressive Cramming: Reliable Token Compression and What It Reveals
Dmitrii Tarasov, Timofei Lashukov, Elizaveta Goncharova +1
Token cramming compresses sequences into learned embeddings with near-perfect reconstruction, but fixed token budgets and 99\% accuracy thresholds leave it unclear whether residual…
OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
Maksim Savkin, Mikhail Goncharov, Alexander Gambashidze +7
Recent progress in the development of language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practi…
Towards Robust Speech Deepfake Detection via Human-Inspired Reasoning
Artem Dvirniak, Evgeny Kushnir, Dmitrii Tarasov +5
The modern generative audio models can be used by an adversary in an unlawful manner, specifically, to impersonate other people to gain access to private information. To mitigate t…
Sentence-Anchored Gist Compression for Long-Context LLMs
Dmitrii Tarasov, Elizaveta Goncharova, Kuznetsov Andrey
This work investigates context compression for Large Language Models (LLMs) using learned compression tokens to reduce the memory and computational demands of processing long seque…
Speech-to-LaTeX: New Models and Datasets for Converting Spoken Equations and Sentences
Dmitrii Korzh, Dmitrii Tarasov, Artyom Iudin +6
Conversion of spoken mathematical expressions is a challenging task that involves transcribing speech into a strictly structured symbolic representation while addressing the ambigu…
Image Reconstruction as a Tool for Feature Analysis
Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova +1
Vision encoders are increasingly used in modern applications, from vision-only models to multimodal systems such as vision-language models. Despite their remarkable success, it rem…
MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing
Matvey Skripkin, Elizaveta Goncharova, Dmitrii Tarasov +1
Multimodal language models (MLMs) integrate visual and textual information by coupling a vision encoder with a large language model through the specific adapter. While existing app…