2 citations · 7 across the 12 of their papers we have counts for
13 papers
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk +6
While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams…
Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution
Anton Razzhigaev, Andrei Gritsaev, Andrei Kaznacheev +3
We present Ouroboros, a self-developing agent harness whose tools, prompts, context assembly, and core implementation improve through reviewed commits that become the runtime for l…
MindShift: Analyzing Language Models' Reactions to Psychological Prompts
Anton Vasiliuk, Irina Abdullaeva, Polina Druzhinina +2
Large language models (LLMs) hold the potential to absorb and reflect personality traits and attitudes specified by users. In our study, we investigated this potential using robust…
SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
Nikita Dragunov, Temurbek Rahmatullaev, Elizaveta Goncharova +5
The recently proposed Large Concept Model (LCM) generates text by predicting a sequence of sentence-level embeddings and training with either mean-squared error or diffusion object…
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4
We introduce methods to quantify how Large Language Models (LLMs) encode and store contextual information, revealing that tokens often seen as minor (e.g., determiners, punctuation…
Universal Adversarial Attack on Aligned Multimodal LLMs
Temurbek Rahmatullaev, Polina Druzhinina, Nikita Kurdiukov +3
We propose a universal adversarial attack on multimodal Large Language Models (LLMs) that leverages a single optimized image to override alignment safeguards across diverse queries…
Logit-KL Flow Matching: Non-Autoregressive Text Generation via Sampling-Hybrid Inference
Egor Sevriugov, Nikita Dragunov, Anton Razzhigaev +2
Non-autoregressive (NAR) language models offer notable efficiency in text generation by circumventing the sequential bottleneck of autoregressive decoding. However, accurately mode…
Your Transformer is Secretly Linear
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova +4
This paper reveals a novel linear characteristic exclusive to transformer decoders, including models such as GPT, LLaMA, OPT, BLOOM and others. We analyze embedding transformations…
OmniFusion Technical Report
Elizaveta Goncharova, Anton Razzhigaev, Matvey Mikhalchuk +6
Last year, multimodal architectures served up a revolution in AI-based approaches and solutions, extending the capabilities of large language models (LLM). We propose an \textit{Om…
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova +3
In this study, we present an investigation into the anisotropy dynamics and intrinsic dimension of embeddings in transformer architectures, focusing on the dichotomy between encode…
Answer Candidate Type Selection: Text-to-Text Language Model for Closed Book Question Answering Meets Knowledge Graphs
Mikhail Salnikov, Maria Lysyuk, Pavel Braslavski +3
Pre-trained Text-to-Text Language Models (LMs), such as T5 or BART yield promising results in the Knowledge Graph Question Answering (KGQA) task. However, the capacity of the model…
Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion
Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva +7
Text-to-image generation is a significant domain in modern computer vision and has achieved substantial improvements through the evolution of generative architectures. Among these,…
MEKER: Memory Efficient Knowledge Embedding Representation for Link Prediction and Question Answering
Viktoriia Chekalina, Anton Razzhigaev, Albert Sayapin +2
Knowledge Graphs (KGs) are symbolically structured storages of facts. The KG embedding contains concise data used in NLP tasks requiring implicit information about the real world.…