6 citations · 9 across the 14 of their papers we have counts for
5 papers · 1 filter
Clustering in Causal Attention Masking
Nikita Karagodin, Yury Polyanskiy, Philippe Rigollet
This work presents a modification of the self-attention dynamics proposed by Geshkovski et al. (arXiv:2312.10794) to better reflect the practically relevant, causally masked attent…
Dynamic metastability in the self-attention model
Borjan Geshkovski, Hugo Koubbi, Yury Polyanskiy +1
We consider the self-attention model - an interacting particle system on the unit sphere, which serves as a toy model for Transformers, the deep neural network architecture behind…
Entropy Contractions in Markov Chains: Half-Step, Full-Step and Continuous-Time
Pietro Caputo, Zongchen Chen, Yuzhou Gu +1
This paper considers the speed of convergence (mixing) of a finite Markov kernel with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines ei…
Unsourced Multiple Access: A Coding Paradigm for Massive Random Access
Gianluigi Liva, Yury Polyanskiy
This paper is a tutorial introduction to the field of unsourced multiple access (UMAC) protocols. We first provide a historical survey of the evolution of random access protocols,…
Evolution of the 5G New Radio Two-Step Random Access towards 6G Unsourced MAC
Patrick Agostini, Jean-Francois Chamberland, Federico Clazzer +7
This report summarizes some considerations on possible evolutions of grant-free random access in the next generation of the 3GPP wireless cellular standard. The analysis is carried…