From the 1 of 5 linked papers with an AI index.
5 papers
Extending LLM Context via Associative Recurrent Memory
Gleb Kuzmin, Ivan Rodkin, Aydar Bulatov +8
The paper introduces the Associative Recurrent Memory Transformer (ARMT) to enable large language models to handle much longer contexts with constant memory usage and reduced compu…
Semantic Chunking and the Entropy of Natural Language
Weishun Zhong, Doron Sivan, Tankut Can +2
The entropy rate of printed English is famously estimated to be about one bit per character, a benchmark that modern large language models (LLMs) have only recently approached. Thi…
Synaptic Theory of Chunking in Working Memory
Weishun Zhong, Mikhail Katkov, Misha Tsodyks
Working memory often appears to exceed its basic span by organizing items into compact representations called chunks. Chunking can be learned over time for familiar inputs; however…
Random Tree Model of Meaningful Memory
Weishun Zhong, Tankut Can, Antonis Georgiou +3
Traditional studies of memory for meaningful narratives focus on specific stories and their semantic structures but do not address common quantitative features of recall across dif…
Large-scale study of human memory for meaningful narratives
Antonios Georgiou, Tankut Can, Mikhail Katkov +1
The statistical study of human memory requires large-scale experiments, involving many stimuli conditions and test subjects. While this approach has proven to be quite fruitful for…