Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
The Role of Feed-Forward Layers in Transformer Dynamics
Thomas Jacob Maranzatto, Semih Akkoc, Sennur Ulukus
We study the dynamical behavior of tokens in transformers from a control-theoretic perspective. Our model includes the feed-forward layer present after the self-attention mechanism…
cs.LG2026
On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers
Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5
We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems. In this perspective, tokens are modeled as particles…