4 papers · 1 filter
SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
Nikita Dragunov, Temurbek Rahmatullaev, Elizaveta Goncharova +5
The recently proposed Large Concept Model (LCM) generates text by predicting a sequence of sentence-level embeddings and training with either mean-squared error or diffusion object…
Logit-KL Flow Matching: Non-Autoregressive Text Generation via Sampling-Hybrid Inference
Egor Sevriugov, Nikita Dragunov, Anton Razzhigaev +2
Non-autoregressive (NAR) language models offer notable efficiency in text generation by circumventing the sequential bottleneck of autoregressive decoding. However, accurately mode…
MindShift: Analyzing Language Models' Reactions to Psychological Prompts
Anton Vasiliuk, Irina Abdullaeva, Polina Druzhinina +2
Large language models (LLMs) hold the potential to absorb and reflect personality traits and attitudes specified by users. In our study, we investigated this potential using robust…
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4
We introduce methods to quantify how Large Language Models (LLMs) encode and store contextual information, revealing that tokens often seen as minor (e.g., determiners, punctuation…