2 citations · 4 across the 7 of their papers we have counts for
4 papers · 1 filter
The Asymptotic Behavior of Attention in Transformers
Álvaro Rodríguez Abella, João Pedro Silvestre, Paulo Tabuada
The transformer architecture has become the foundation of modern Large Language Models (LLMs), yet its theoretical properties are still not well understood. As with classic neural…
Meanings and Feelings of Large Language Models: Observability of Latent States in Generative AI
Tian Yu Liu, Stefano Soatto, Matteo Marchi +2
We tackle the question of whether Large Language Models (LLMs), viewed as dynamical systems with state evolving in the embedding space of symbolic tokens, are observable. That is,…
Heat Death of Generative Models in Closed-Loop Learning
Matteo Marchi, Stefano Soatto, Pratik Chaudhari +1
Improvement and adoption of generative machine learning models is rapidly accelerating, as exemplified by the popularity of LLMs (Large Language Models) for text, and diffusion mod…
A Framework for Time-Varying Optimization via Derivative Estimation
Matteo Marchi, Jonathan Bunton, João Pedro Silvestre +1
Optimization algorithms have a rich and fundamental relationship with ordinary differential equations given by its continuous-time limit. When the cost function varies with time --…