3 papers
eess.SY2026
Dynamical principles of habituation across substrates and scales
Matthew Smart, Stanislav Y. Shvartsman, Martin Mönnigmann
Habituation is a basic form of learning in which a system's response to repeated stimulation progressively diminishes but eventually recovers when the stimulus is withheld. Long st…
cs.LG2026
Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics
Matthew Smart, Soumya Ganguly, Nilava Metya +2
We study minimal attention-only transformers under all-token corruption and show they admit a two-stage empirical Bayes interpretation. A single attention step computes a kernel-we…
cs.LG2025
In-context denoising with one-layer transformers: connections between attention and associative memory retrieval
Matthew Smart, Alberto Bietti, Anirvan M. Sengupta
We introduce in-context denoising, a task that refines the connection between attention-based architectures and dense associative memory (DAM) networks, also known as modern Hopfie…