32 citations · 161 across the 38 of their papers we have counts for
6 papers · 1 filter
Learning Elementary Cellular Automata with Transformers
Mikhail Burtsev
Large Language Models demonstrate remarkable mathematical capabilities but at the same time struggle with abstract reasoning and planning. In this study, we explore whether Transfo…
Associative Recurrent Memory Transformer
Ivan Rodkin, Yuri Kuratov, Aydar Bulatov +1
This paper addresses the challenge of creating a neural architecture for very long sequences that requires constant time for processing new information at each time step. Our appro…
AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents
Petr Anokhin, Nikita Semenov, Artyom Sorokin +4
Advancements in the capabilities of Large Language Models (LLMs) have created a promising foundation for developing autonomous agents. With the right tools, these agents could lear…
Complexity of Symbolic Representation in Working Memory of Transformer Correlates with the Complexity of a Task
Alsu Sagirova, Mikhail Burtsev
Even though Transformers are extensively used for Natural Language Processing tasks, especially for machine translation, they lack an explicit memory to store key concepts of proce…
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
Yuri Kuratov, Aydar Bulatov, Petr Anokhin +4
In recent years, the input context sizes of large language models (LLMs) have increased dramatically. However, existing evaluation methods have not kept pace, failing to comprehens…
In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss
Yuri Kuratov, Aydar Bulatov, Petr Anokhin +3
This paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark design…