3 papers
cs.AI2026
First-Token Attraction in Mamba Dynamics
Trinh T. Nguyen, Minh-Khoi Nguyen-Nhat, Duy-Tung Pham +3
We study token dynamics in deep Mamba models, with a particular focus on the attracting role of the first token. Unlike attention mechanisms, where token interactions are governed…
cs.LG2026
Quasi-Equivariant Metanetworks
Viet-Hoang Tran, An Nguyen, Benoît Guérand +2
Metanetworks are neural architectures designed to operate directly on pretrained weights to perform downstream tasks. However, the parameter space serves only as a proxy for the un…
cs.LG2025
Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding
Duy-Tung Pham, An The Nguyen, Viet-Hoang Tran +4
This paper investigates the dynamical properties of tokens in pre-trained Transformer models and explores their application to improving Transformers. To this end, we analyze the d…