4 papers · 1 filter
Predictive Geometry of Hidden Trajectories in Transformers
Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1
Decoder-only transformers are trained only through a terminal next-token prediction loss, yet this loss constrains every intermediate hidden state through the fixed downstream comp…
Friction-Augmented Drifting Models for Resource-Efficient Domain Translation
Arkadii Kazanskii, Tatiana Petrova, Andrey Ustyuzhanin +3
Single-step generators promise high-fidelity synthesis at a fraction of the inference and training cost of ordinary differential equation (ODE)-based flow models, a central concern…
Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels
Tatiana Petrova
Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditio…
Limitations of Normalization in Attention Mechanism
Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1
This paper investigates the limitations of the normalization in attention mechanisms. We begin with a theoretical framework that enables the identification of the model's selective…