46 papers
Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge
Arda Uzunoglu, Benjamin Van Durme, Benjamin van Durme +1
Large language models are increasingly trained and deployed with long contexts that span documents, code repositories, and interaction histories. This scaling reflects the implicit…
Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost
Zixi Huang, Xiheng Wang, Andrew Wang +4
Recently, the practice of augmenting LLM agent capability with skills has gained prevalence. We explore the cost effective adaptation of agents to novel domains by means of learnin…
More Than Efficiency: Embedding Compression Improves Domain Adaptation in Dense Retrieval
Chunsheng Zuo, Daniel Khashabi
The paper shows that applying PCA to compress query embeddings can both speed up dense retrieval and improve its performance on specialized domains, acting as a lightweight form of…
Self-Compacting Language Model Agents
Tianjian Li, Jingyu Zhang, William Jurayj +5
Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaf…
CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu +9
Creativity is often seen as a hallmark of human intelligence. While large language models(LLMs) are increasingly perceived as generating creative text, there is still no cross-doma…
Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG
Dayeon Ki, Marine Carpuat, Paul McNamee +4
Multilingual Retrieval-Augmented Generation (mRAG) systems enable language models to answer knowledge-intensive queries with citation-supported responses across languages. Despite…