6 papers
Self-Compacting Language Model Agents
Tianjian Li, Jingyu Zhang, William Jurayj +5
Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaf…
Conformal Thinking: Risk Control for Reasoning on a Compute Budget
Xi Wang, Anushri Suresh, Alvin Zhang +6
Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases, motivating adaptive reasoning -- spending toke…
Entropy After </Think> for reasoning model early exiting
Xi Wang, James McInerney, Lequn Wang +1
Reasoning LLMs show improved performance with longer chains of thought. However, recent work has highlighted their tendency to overthink, continuing to revise answers even after re…
How to set AdamW's weight decay as you scale model and dataset size
Xi Wang, Laurence Aitchison
The scaling of the optimal AdamW weight decay hyperparameter with model and dataset size is critical as we seek to build larger models, but is poorly understood. We show that weigh…
Are vision language models robust to uncertain inputs?
Xi Wang, Eric Nalisnick
Robustness against uncertain and ambiguous inputs is a critical challenge for deep learning models. While recent advancements in large scale vision language models (VLMs, e.g. GPT4…
KBLaM: Knowledge Base augmented Language Model
Xi Wang, Taketomo Isazawa, Liana Mikaelyan +1
In this paper, we propose Knowledge Base augmented Language Model (KBLaM), a new method for augmenting Large Language Models (LLMs) with external knowledge. KBLaM works with a know…