32 citations · 161 across the 38 of their papers we have counts for
4 papers · 2 filters
Retrieval Capacity of Self-Attention Under Competition
Timur Mudarisov, Mikhail Burtsev, Radu State
How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention. Without retraining, we retain o…
The Geometry of Inference in Transformer Residual Streams
Timur Mudarisov, Mikhail Burtsev, Radu State
Transformer language models build predictions through successive residual updates, but how their representations become specific to an eventual outcome remains unclear. We study th…
Extending LLM Context via Associative Recurrent Memory
Gleb Kuzmin, Ivan Rodkin, Aydar Bulatov +8
Extending the context length of large language models (LLMs) is critical for many real-world applications, yet standard transformers remain constrained by quadratic compute and lin…
Wikontic: Constructing Wikidata-Aligned, Ontology-Aware Knowledge Graphs with Large Language Models
Alla Chepurova, Aydar Bulatov, Mikhail Burtsev +1
Knowledge graphs (KGs) provide structured, verifiable grounding for large language models (LLMs), but current LLM-based systems commonly use KGs as auxiliary structures for text re…