language modeling 1positional embeddings 1self-attention 1state estimation 1stochastic differential equations 1
From the 1 of 3 linked papers with an AI index.
3 papers
cs.LG2026
Robust Filter Attention: Self-Attention as Precision-Weighted State Estimation
Peter Racioppo
The paper presents Robust Filter Attention, a new view of self‑attention as a robust state estimator based on linear stochastic differential equations, and shows it improves langua…
cs.LG2026
The Transformer as a Polar State Estimator
Peter Racioppo
We show that the core components of the Transformer---attention, residual connections, and normalization---arise naturally from a single geometric state estimation problem. Modelin…
cs.LG2026
AC-SINDy: Compositional Sparse Identification of Nonlinear Dynamics
Peter Racioppo
We present AC-SINDy, a compositional extension of the Sparse Identification of Nonlinear Dynamics (SINDy) framework that replaces explicit feature libraries with a structured repre…