activity
20222026
most citedCharacterizing the Spectrum of the NTK via a Power Series Expansion

2 citations · 7 across the 11 of their papers we have counts for

collaborators
Showing 2024Show all

2 papers · 1 filter

cs.CL2024

Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models

Elvis Nunez, Luca Zancato, Benjamin Bowman +3

The "state" of State Space Models (SSMs) represents their memory, which fades exponentially over an unbounded span. By contrast, Attention-based models have "eidetic" (i.e., verbat…

cs.LG2024★ 1 cited

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory

Luca Zancato, Arjun Seshadri, Yonatan Dukler +6

We describe a family of architectures to support transductive inference by allowing memory to grow to a finite but a-priori unknown bound while making efficient use of finite resou…