From the 1 of 6 linked papers with an AI index.
4 papers · 1 filter
Extending LLM Context via Associative Recurrent Memory
Gleb Kuzmin, Ivan Rodkin, Aydar Bulatov +8
The paper introduces the Associative Recurrent Memory Transformer (ARMT) to enable large language models to handle much longer contexts with constant memory usage and reduced compu…
Semantic Chunking and the Entropy of Natural Language
Weishun Zhong, Doron Sivan, Tankut Can +2
The entropy rate of printed English is famously estimated to be about one bit per character, a benchmark that modern large language models (LLMs) have only recently approached. Thi…
Large-scale study of human memory for meaningful narratives
Antonios Georgiou, Tankut Can, Mikhail Katkov +1
The statistical study of human memory requires large-scale experiments, involving many stimuli conditions and test subjects. While this approach has proven to be quite fruitful for…
Information Theory of Meaningful Communication
Doron Sivan, Misha Tsodyks
In Shannon's seminal paper, entropy of printed English, treated as a stationary stochastic process, was estimated to be roughly 1 bit per character. However, considered as a means…