47 citations · 61 across the 10 of their papers we have counts for
4 papers · 1 filter
Extending LLM Context via Associative Recurrent Memory
Gleb Kuzmin, Ivan Rodkin, Aydar Bulatov +8
Extending the context length of large language models (LLMs) is critical for many real-world applications, yet standard transformers remain constrained by quadratic compute and lin…
Semantic Chunking and the Entropy of Natural Language
Weishun Zhong, Doron Sivan, Tankut Can +2
Humans and large language models can predict next letter or word from its prior context much better than random guessing, indicating strong redundancy of language viewed as a stoch…
Information Theory of Meaningful Communication
Doron Sivan, Misha Tsodyks
In Shannon's seminal paper, entropy of printed English, treated as a stationary stochastic process, was estimated to be roughly 1 bit per character. However, considered as a means…
Large-scale study of human memory for meaningful narratives
Antonios Georgiou, Tankut Can, Mikhail Katkov +1
The statistical study of human memory requires large-scale experiments, involving many stimuli conditions and test subjects. While this approach has proven to be quite fruitful for…