10 citations · 16 across the 5 of their papers we have counts for
9 papers
Maximally-Informative Retrieval for State Space Model Generation
Evan Becker, Benjamin Bowman, Matthew Trager +4
Given a query and dataset, the optimal way of answering the query is to make use all the information available. Modern LLMs exhibit impressive ability to memorize training data, bu…
PICASO: Permutation-Invariant Context Composition with State Space Models
Tian Yu Liu, Alessandro Achille, Matthew Trager +3
Providing Large Language Models with relevant contextual knowledge at inference time has been shown to greatly improve the quality of their generations. This is often achieved by p…
Descriminative-Generative Custom Tokens for Vision-Language Models
Pramuditha Perera, Matthew Trager, Luca Zancato +2
This paper explores the possibility of learning custom tokens for representing new concepts in Vision-Language Models (VLMs). Our aim is to learn tokens that can be effective for b…
Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models
Elvis Nunez, Luca Zancato, Benjamin Bowman +3
The "state" of State Space Models (SSMs) represents their memory, which fades exponentially over an unbounded span. By contrast, Attention-based models have "eidetic" (i.e., verbat…
Marconi: Prefix Caching for the Era of Hybrid LLMs
Rui Pan, Zhuang Wang, Zhen Jia +5
Hybrid models that combine the language modeling capabilities of Attention layers with the efficiency of Recurrent layers (e.g., State Space Models) have gained traction in practic…
Long-context Protein Language Modeling Using Bidirectional Mamba with Shared Projection Layers
Yingheng Wang, Zichen Wang, Gil Sadeh +4
Self-supervised training of language models (LMs) has seen great success for protein sequences in learning meaningful representations and for generative drug design. Most protein L…