32 citations · 161 across the 38 of their papers we have counts for
4 papers · 2 filters
Associative Recurrent Memory Transformer
Ivan Rodkin, Yuri Kuratov, Aydar Bulatov +1
This paper addresses the challenge of creating a neural architecture for very long sequences that requires constant time for processing new information at each time step. Our appro…
Complexity of Symbolic Representation in Working Memory of Transformer Correlates with the Complexity of a Task
Alsu Sagirova, Mikhail Burtsev
Even though Transformers are extensively used for Natural Language Processing tasks, especially for machine translation, they lack an explicit memory to store key concepts of proce…
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
Yuri Kuratov, Aydar Bulatov, Petr Anokhin +4
In recent years, the input context sizes of large language models (LLMs) have increased dramatically. However, existing evaluation methods have not kept pace, failing to comprehens…
In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss
Yuri Kuratov, Aydar Bulatov, Petr Anokhin +3
This paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark design…