9 papers
Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training
Shengrui Li, Fei Zhao, Kaiyan Zhao +6
Determining an effective data mixture is a key factor in Large Language Model (LLM) pre-training, where models must balance general competence with proficiency on hard tasks such a…
ItinBench: Benchmarking Planning Across Multiple Cognitive Dimensions with Large Language Models
Tianlong Wang, Pinqiao Wang, Weili Shi +1
Large language models (LLMs) with advanced cognitive capabilities are emerging as agents for various reasoning and planning tasks. Traditional evaluations often focus on specific r…
BEACON: Bayesian Optimal Stopping for Efficient LLM Sampling
Guangya Wan, Zixin Stephen Xu, Sasa Zorc +4
Sampling multiple responses is a common way to improve LLM output quality, but it comes at the cost of additional computation. The key challenge is deciding when to stop generating…
COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context
Guangya Wan, Mingyang Ling, Xiaoqi Ren +3
Long-horizon tasks that require sustained reasoning and multiple tool interactions remain challenging for LLM agents: small errors compound across steps, and even state-of-the-art…
Memory in Large Language Models: Mechanisms, Evaluation and Evolution
Dianxing Zhang, Wendong Li, Kani Song +4
Under a unified operational definition, we define LLM memory as a persistent state written during pretraining, finetuning, or inference that can later be addressed and that stably…
Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Models
Zhawnen Chen, Tianchun Wang, Yizhou Wang +4
Can large multimodal models have a human-like ability for emotional and social reasoning, and if so, how does it work? Recent research has discovered emergent theory-of-mind (ToM)…