4 papers · 1 filter
The Role of Symmetry in Optimizing Overparameterized Networks
Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2
Overparameterization is central to the success of deep learning, yet the mechanisms by which it improves optimization remain incompletely understood. We analyze weight-space symmet…
The Expressive Limits of Diagonal SSMs for State-Tracking
Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh +1
State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling tasks while remaining efficient and highly-p…
Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs
Behnoush Khavari, Mehran Shakerinava, Jayesh Khullar +4
Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ran…
Beyond Scalar Rewards: An Axiomatic Framework for Lexicographic MDPs
Mehran Shakerinava, Siamak Ravanbakhsh, Adam Oberman
Recent work has formalized the reward hypothesis through the lens of expected utility theory, by interpreting reward as utility. Hausner's foundational work showed that dropping th…