1 citations · 3 across the 17 of their papers we have counts for
4 papers · 1 filter
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato +4
Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval. Continued pretraining on long-context data can help but is…
Maximally-Informative Retrieval for State Space Model Generation
Evan Becker, Benjamin Bowman, Matthew Trager +4
Given a query and dataset, the optimal way of answering the query is to make use all the information available. Modern LLMs exhibit impressive ability to memorize training data, bu…
PICASO: Permutation-Invariant Context Composition with State Space Models
Tian Yu Liu, Alessandro Achille, Matthew Trager +3
Providing Large Language Models with relevant contextual knowledge at inference time has been shown to greatly improve the quality of their generations. This is often achieved by p…
Meaning Representations from Trajectories in Autoregressive Models
Tian Yu Liu, Matthew Trager, Alessandro Achille +3
We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strateg…