3 citations · 4 across the 5 of their papers we have counts for
4 papers · 1 filter
Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
Neehal Tumma, Noel Loo, Daniela Rus
To address the increasing long-context compute limitations of softmax attention, several subquadratic recurrent operators have been developed. This work includes models such as Mam…
LFM2 Technical Report
Alexander Amini, Anna Banaszak, Harold Benoit +30
We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under…
Quantifying Memory Utilization with Effective State-Size
Rom N. Parnichkun, Neehal Tumma, Armin W. Thomas +6
The need to develop a general framework for architecture analysis is becoming increasingly important, given the expanding design space of sequence models. To this end, we draw insi…
Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control
Neehal Tumma, Mathias Lechner, Noel Loo +2
Developing autonomous agents that can interact with changing environments is an open challenge in machine learning. Robustness is particularly important in these settings as agents…