4 papers
Priming: Hybrid State Space Models From Pre-trained Transformers
Aditya Chattopadhyay, Elvis Nunez, Prannay Kaul +6
Hybrid State-Space models combine Attention with recurrent State-Space Model (SSM) layers, balancing eidetic memory from Attention with compressed fading memory from SSMs. This yie…
Experience-Guided Adaptation of Inference-Time Reasoning Strategies
Adam Stein, Matthew Trager, Benjamin Bowman +4
Enabling agentic AI systems to adapt their problem-solving approaches based on post-training interactions remains a fundamental challenge. While systems that update and maintain a…
Maximally-Informative Retrieval for State Space Model Generation
Evan Becker, Benjamin Bowman, Matthew Trager +4
Given a query and dataset, the optimal way of answering the query is to make use all the information available. Modern LLMs exhibit impressive ability to memorize training data, bu…
Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models
Elvis Nunez, Luca Zancato, Benjamin Bowman +3
The "state" of State Space Models (SSMs) represents their memory, which fades exponentially over an unbounded span. By contrast, Attention-based models have "eidetic" (i.e., verbat…